08-02-2013 12:53 AM
I gather MP Connect can be used to setup multi-thread processing. Can people point me to documentation and simple examples?
An example problem. I have something like:
set full (where=(var1=1));
We have a Win2008 server with a fast disk system and 32 cores which we use pretty much as a workstation (usually just one job running at a time). If I run this job on a large dataset, it takes about 5 minutes with one cpu core at 100% and the others unused. Is it possible to split the processing across multiple cores?
08-02-2013 01:27 AM
The most easy way to use multithreading is using procedures that support it. That are:
- Proc SQL , Sort
- HP versions of stat
etc SAS(R) 9.4 Language Reference: Concepts (Support for Parallel Processing )
Some SAS options are controlling the limits.
The nature of a sas-data-step (retain lag) knowing the order is blocking a parallelized (threaded) approach.
SQL Queries and Sorting on the other hand are ideal for threading.
When you want to use MP-Connect (or just Conect) you have to plan by yourself wich datasteps can run parallel and wich ones have to wait on another. The waitfor is statement is important for that. The others are rsubmit/endrsubmit and the communcation syslput/sysrput , signon/signoff
SAS/CONNECT(R) 9.2 User's Guide (Example 5: Using MP CONNECT and the WAITFOR Statement)
08-02-2013 01:59 AM
Is there any documentation describing when proc SQL will multithread? If I wrote my simple example in SQL, would it process it with multiple threads?
08-02-2013 02:57 AM
The best thing about proc SQL multithreading.... You don't need to do anything for it, it will work.
Look at your results IF you do not care about the ordening (left out order in the sql) you could notice it are different sets taken out of the Original datasets and than being appended.
By the way Eguide flow is also supporting parallel processing, see:
Your are in the performance and tuning area now. That is measuring (OS and SAS) with different scenarios