Hi @Eileen1496
There are few fundamental elements you have to pay attention to, and rethink your design/approach accordingly
You are trying to run all concurrent processes (Parent + forked SAS sessions) on your Single Desktop/Server
Your Single Desktop/Server has a fixed amount of CPUs , GB of RAM and Disk space which you can work within
Every forked SAS session would have its own default settings (-memsize, -cpucount, -sortsize) that gets initialized from sas_v9.cfg file or the command line. To find what's your SAS defaults are, run the following in your Parent SAS session
Proc Options group=performance; run;
check the log for settings values
You can manually specify/dictate how much each forked SAS session utilizes by changing your sascmd option. Example:
options sascmd="sas -memsize 4G -sortsize 3G -cpucount 8";
Having said that, you'll need to ensure the total amount of memory (memsize) and cpucount of all your concurrent SAS sessions (Parent + forked/child) never exceeds 80% your machine's resources (#2 from above), otherwise everything will slow down and processing gets queued while your Desktop/Server trying to make resources available!!
Sadly, not every divide & conquer approach guarantees faster processing! Sometimes adjusting the assigned session settings (-memsize, -cpucount, -sortsize, -work, -utilloc ) along with performant advanced coding techniques can process your data in a single SAS session much faster.
You said
The time when it does not work for me is: I try to do an inner join between 2 datasets. I think since both of them require computing (instead of just read into memory, which i believe does not use CPU), they should all be speed up by using SAS/CONNECT.
Loading data into memory still requires compute to perform the search/lookup operation for the join to take place, but it does it much faster than reading from disk. So if you can allocate enough memory (-memsize) to your SAS session to load at least one of your two datasets into memory (Hash Object), this would allow to process your data much faster and more straight forward. It's just another way to look at the issue.
Just my 2 cents, hope this helps,
Ahmed
... View more