topic Re: Regarding Datapaths and Indexpaths in SPDS Performance. in SAS Programming

Regarding Datapaths and Indexpaths in SPDS Performance.

Bhoopesh — Thu, 18 Oct 2012 14:16:46 GMT

Is there any impact of increasing the datapaths in Linux environment from 8 to 16...?

Is there would be any benefit if we increase the Indexpath from 1 to 2...

Any suggestions...?

Re: Regarding Datapaths and Indexpaths in SPDS Performance.

LinusH — Thu, 18 Oct 2012 14:22:22 GMT

What kind of file system do you use?

If your data is on a SAN, no of paths usually does not affect performance that much. If you have local file systems assigning additional paths to exclusive physical paths could improve performance.

Index path behave different from data path, it's more of spill over paths. SPDS does not spread indexes between those paths. So I believe that having more index paths is not increasing performance.

Re: Regarding Datapaths and Indexpaths in SPDS Performance.

Bhoopesh — Thu, 18 Oct 2012 18:06:47 GMT

Hi Linush,

Data is not on SAN, but yes the new file systems are on SAN, and by increasing the datapaths should increase the performance.

Parrallelism would be achieved in that case. Once all these file systems have been increased, Is there any other way for faster retrieval(access) of data from these datapaths.

Yes, i do agree with your index paths. There it will not improve any performance.

Re: Regarding Datapaths and Indexpaths in SPDS Performance.

LinusH — Thu, 18 Oct 2012 19:19:54 GMT

If you think that it should increase performance, why do you ask?

To take advantage of 16 partitions fully, each spds process needs to launch as many, or even more threads. How many CPU/cores do you have? And aren't there any concurrent jobs/users? They need to share those cores.

To point at some test case (it's not facts, just a test case), where it showed that after having more than 12 data partitions, it didn't gain any performance (and that with a 12 core server).http://support.sas.com/resources/papers/partitiontables.pdf

There are some papers on the subject of optimizing SPDS, not much about SAN specifically, but most concepts could apply anyway.

http://support.sas.com/resources/papers/tnote/tnote_spds.html

You'll probably want to work close with the server/SAN administrators for help on monitoring, the server, connection(s) and disks.

Re: Regarding Datapaths and Indexpaths in SPDS Performance.

Bhoopesh — Thu, 18 Oct 2012 19:50:44 GMT

Thanks for your wonderful comments...

We have 16 Processors\CPU\cores as of now...But planning to move to 24 cores\processors\CPU in next month... Pls advice in that case.

Yes, we do have some concurrent users too that are hitting the server on a regular basis.

Re: Regarding Datapaths and Indexpaths in SPDS Performance.

LinusH — Fri, 19 Oct 2012 07:11:44 GMT

Again, I think that no of data paths is not so important when having data on a SAN. SAN administrators tend to scramble the physical disks into small chunks (LUNs), which they can assign to different usages. This means that your file system is just a virtual one, and that you are probably sharing physical disks with other servers/applications, or, with other file systems on your own server.

Things to focus on should be:

total I/O throughput all the way from physical disks to the CPUs, and don't forget the spdswork.
No of data partitions. With 24 CPUs, 20-100 partitions per table should be an ok interval.