MP Connect introduction?

[BruceBrad]

I gather MP Connect can be used to setup multi-thread processing. Can people point me to documentation and simple examples?

An example problem. I have something like:

data subset;

set full (where=(var1=1));

run;

We have a Win2008 server with a fast disk system and 32 cores which we use pretty much as a workstation (usually just one job running at a time). If I run this job on a large dataset, it takes about 5 minutes with one cpu core at 100% and the others unused. Is it possible to split the processing across multiple cores?

[jakarman]

Bruce,

The most easy way to use multithreading is using procedures that support it. That are:

- Proc SQL , Sort

- HP versions of stat

etc SAS(R) 9.4 Language Reference: Concepts (Support for Parallel Processing )

Some SAS options are controlling the limits.

The nature of a sas-data-step (retain lag) knowing the order is blocking a parallelized (threaded) approach.

SQL Queries and Sorting on the other hand are ideal for threading.

When you want to use MP-Connect (or just Conect) you have to plan by yourself wich datasteps can run parallel and wich ones have to wait on another. The waitfor is statement is important for that. The others are rsubmit/endrsubmit and the communcation syslput/sysrput , signon/signoff 

SAS/CONNECT(R) 9.2 User's Guide (Example 5: Using MP CONNECT and the WAITFOR Statement)

Tips and Tricks for Scalability and Performance

[BruceBrad]

Is there any documentation describing when proc SQL will multithread? If I wrote my simple example in SQL, would it process it with multiple threads?

[jakarman]

The best thing about proc SQL multithreading.... You don't need to do anything for it, it will work. 

Look at your results IF you do not care about the ordening (left out order in the sql) you could notice it are different sets taken out of the Original datasets and than being appended.

By the way Eguide flow is also supporting parallel processing, see:

https://communities.sas.com/thread/39741?start=0&tstart=0

Your are in the performance and tuning area now. That is measuring (OS and SAS) with different scenarios