topic Re: Recomended file system for SAS Viya 3.4 in Administration and Deployment

Recomended file system for SAS Viya 3.4

syedhisam — Wed, 28 Nov 2018 06:50:44 GMT

I am in process of installing SAS Viya 3.4 on RHEL 7.5, looks like since version 7 RHEL default file system is xfs . Wanted to check what is SAS recommendation for File System xfs or ext4 ? I could not find anything around this in the documentation.

Re: Recomended file system for SAS Viya 3.4

eleven — Wed, 28 Nov 2018 08:14:13 GMT

Hi Syedhisam,

i think this sas note still valid
http://support.sas.com/kb/43/820.html

Re: Recomended file system for SAS Viya 3.4

alexal — Wed, 28 Nov 2018 13:16:21 GMT

@syedhisam,

Please note that SAS supports only local file systems for the CAS cache such as EXT4 and XFS. It does not support network file systems such as GPFS.

Set Up the CAS Cache Directory

Re: Recomended file system for SAS Viya 3.4

MargaretC — Wed, 28 Nov 2018 13:19:40 GMT

What SAS Viya file system are you asking about? Details on the different file systems can be found here: https://support.sas.com/resources/papers/checklist-sas-viya-administration-tasks.pdf and here: https://support.sas.com/documentation/onlinedoc/viya/3.4/AdminDoc34.pdf

* CAS_Disk_Cache
* Location of permanent CAS files? If so, what format are you saving them in - Hadoop, SASHDAT, ??
* SAS WORK for the SAS SPRE node?

XFS is preferred over EXT4 for places where SAS7BDAT files are written. HDFS for places where HADOOP or SASHDAT file are written.

Re: Recomended file system for SAS Viya 3.4

syedhisam — Wed, 28 Nov 2018 13:26:40 GMT

What SAS Viya file system are you asking about?

Do you have still have confusion or is it a Sarcasm ?

Re: Recomended file system for SAS Viya 3.4

MargaretC — Wed, 28 Nov 2018 13:35:37 GMT

For CAS_Disk_Cache, only local file systems can be used. XFS is better than EXT4. No, GPFS can not be used.

Re: Recomended file system for SAS Viya 3.4

MargaretC — Wed, 28 Nov 2018 13:36:52 GMT

I am not being sarcastic. There are lots of different file systems needed for SAS Viya 3.4. Please read the links I sent you.

Re: Recomended file system for SAS Viya 3.4

syedhisam — Wed, 28 Nov 2018 14:38:35 GMT

Thanks a lot Guys @eleven @MargaretC @alexal

Really appreciate information shared, will review and follow-up in case required.

Thanks Again !!

Re: Recomended file system for SAS Viya 3.4

SimonDawson — Thu, 29 Nov 2018 10:34:19 GMT

-- Tried to post this yesterday but the forums went offline for maintenance --

XFS is a great choice. Red Hat have a paper on their site about tuning RHEL for SAS workloads. Their "bake off" put XFS on top in their case.

https://access.redhat.com/articles/1593123

While I do love fast things and you should certainly listen to the advice @MargaretC and @alexal have given you, I'll play devils advocate for a bit. The difference based purely on the file system for most workloads is marginal. For a lot of workloads your users won't notice the difference. Depending on what your requirements are you might want to think about your choice considering the broader picture. What is ubiquitous across your environments, consider not just your SAS deployments. What are your system administrators and application administrators are familiar with? Do you have other constraints on your file system choice? As a simple example some backup solutions only support specific file systems.

The far more important thing to get right is the storage on which the file system resides. Getting your disks and volumes correctly setup to support your SAS deployment is far more important than the file system you pick. Spend some time to ensure you have the correct layout of drives, number of disks and spindles, and appropriate disk attachment method to the systems for the various types of storage you'll need. Getting the storage laid out correctly is where you will get the most bang for your buck.

Re: Recomended file system for SAS Viya 3.4

BigSASFan — Thu, 10 Oct 2019 19:37:11 GMT

I come late to the party at the end of 2018, and I am new to Viya.

That said, since there are so many different part of Viya, I am curious what filesystem you recommmend for each part.

Assuming the OS is Redhat Linux 7.x or 8.x:

I already read in this thread that for CAS_Disk_Cache, XFS is preferred.

How about the other Viya components ? Specifically:

1. What filesystem for controller? How about if we have multiple controllers, should they use shared filesystem like GPFS ?

2. What filesystem for SAS data (currently in SAS 9.4 GRID they are in .sas7bdat format) ? In Viya, which is better SASHDAT or HDFS ? But anyway, it looks that the traditional GPFS shared filesystem doesn't play a role for workers in Viya ?

Thanks in advance,

Lei

Re: Recomended file system for SAS Viya 3.4

MargaretC — Sat, 12 Oct 2019 12:51:32 GMT

Lei,
I believe you have your answer to this question in a separate email thread. Is that correct?
Margaret

Re: Recomended file system for SAS Viya 3.4

BigSASFan — Thu, 31 Oct 2019 07:17:57 GMT

Hi Magaret,

That's true, sorry for the confusions !

Re: Recomended file system for SAS Viya 3.4

RobCollum — Tue, 12 Nov 2019 21:51:28 GMT

CAS_DISK_CACHE is not the only storage location for consideration... but I did just release a post about it in Communities: Provisioning CAS_DISK_CACHE For SAS Viya. And yes as noted above, xfs is preferred and so is ext4. Avoid ext3 and any kind of NFS-based storage for CAS' cache.

You have the option to rely on shared file system solutions for loading data into CAS, like NFS or a SAN or even HDFS. These among other topics are covered in several other posts of mine as well:

4 Approaches for Parallel Data Loading to CAS
https://communities.sas.com/t5/SAS-Communities-Library/4-Approaches-for-Parallel-Data-Loading-to-CAS/ta-p/370955
Contemplating shared file systems for SAS
https://communities.sas.com/t5/SAS-Communities-Library/Contemplating-shared-file-systems-for-SAS/ta-p/468263
Contemplating I/O for SAS
https://communities.sas.com/t5/SAS-Communities-Library/Contemplating-I-O-for-SAS/ta-p/459472

Some of those link to white papers published by Margaret and her team.

HTH,

Rob