topic Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure in Administration and Deployment

Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

MargaretC — Wed, 25 Nov 2020 13:01:34 GMT

SAS recently tested SAS 9.4 on MS Azure instances using Sycomp Storage Fueled by IBM Spectrum Scale (GPFS) file system (https://azuremarketplace.microsoft.com/en-us/marketplace/apps/sycompatechnologycompanyinc1588192103892.sycompstoragefueledbyibmspectrumscalewithrhel?tab=Overview ). The results of the testing were very favorable.

The test infrastructure consisted of the Medium Storage Cluster offering from Sycomp (11 MS Azure instances) defined like this:

Five E64-32ds_v4 instances for the SAS clients – each with six P30 premium drives striped together for local storage.
Six D64s_v3 instances for the Spectrum Scale storage cluster – each with five P60 premium drives.

During the test, we modified some of the IBM Spectrum Scale settings to improve IO throughput:

mmchconfig pagepool=200g,prefetchPct=40,maxFilesToCache=50000,maxMBpS=4000,seqDiscardThreshold=1073741824,workerThreads=768

rhel_iotest results:

The 1-node test gave us 237 MB per second per physical core for WRITEs and 267 MB per second per physical core for READs.

The 2-node test gave us 225 MB per second per physical core for WRITEs and 218 MB per second per physical core for READs.

The 3-node test gave us 167 MB per second per physical core for WRITEs and 150 MB per second per physical core for READs.

The 4-node test gave us 153 MB per second per physical core for WRITEs and 112 MB per second per physical core for READs.

The 5-node test gave us 94 MB per second per physical core for WRITEs and 86 MB per second per physical core for READs.

Please note that SAS recommends a minimum of 100 MB per second per physical for WRITEs and READs.

As you can see, the Medium Storage Cluster can support the minimum IO throughput needs of a 64-core SAS Grid application. Other testing with Sycomp has shown the ability to have even more cores achieve the minimum IO throughput needs of SAS with over 100 cores. Please reach out to Sycomp for more details.

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

JuanS_OCS — Wed, 25 Nov 2020 13:25:34 GMT

Thank you very much for this share @MargaretC , good one. I know many have been waiting for GPFS in Azure.

Are you aware if currently this file system, or any other, support HA and/or data redundancy (multiplication factor > 1) in Azure?

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

MargaretC — Wed, 25 Nov 2020 13:37:57 GMT

Spectrum Scale has the ability to do HA in MS Azure. Not sure about the others.
Margaret

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

JuanS_OCS — Wed, 25 Nov 2020 14:36:41 GMT

Thank you Margaret, more good news.

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

TheJZ — Thu, 03 Dec 2020 04:34:27 GMT

Hi Juan,
I can give you the details on the capabilities of our offer. Email me jz@sycomp.com

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

RamSalinas — Wed, 25 Aug 2021 20:38:39 GMT

Hi again Margaret, do you have the test details in order to reach those numbers, we're working on a new SAS grid environment with Spectrum Scale but with a lot of issues with the I/O

Best!
Ram

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

MargaretC — Wed, 25 Aug 2021 20:44:30 GMT

Is this what you are looking for? Sycomp Storage Fueled by IBM Spectrum Scale: A new... - SAS Support Communities

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

TheJZ — Wed, 25 Aug 2021 20:49:33 GMT

Hi Ram,

Are you using our offer in Azure to deploy the cluster? Would be happy to have a conversation.

email: jz@sycomp.com

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

RamsesSalinas — Wed, 25 Aug 2021 21:15:35 GMT

Noup, this is an on-prem deployment 😞

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

MargaretC — Wed, 25 Aug 2021 21:22:24 GMT

I was confused that you are asking about Spectrum Scale in MS Azure since that is the title of this thread. General information on setting up Spectrum Scale in a non-public cloud infrastructure can be found here: performance-tuning-sas-red-hat-using-ibm-spectrum-scale.pdf

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

RamsesSalinas — Wed, 25 Aug 2021 21:32:56 GMT

Thanks both of you!

Re: Sycomp Storage Fueled by IBM Spectrum Scale: A new shared file system for SAS Grid on MS Azure

TheJZ — Wed, 25 Aug 2021 22:55:11 GMT

Hi Ram,

If you would like our assistance with the on-prem Spectrum Scale tuning, please let me know and we will be happy to assist.