BookmarkSubscribeRSS Feed
BruceBrad
Lapis Lazuli | Level 10

I gather MP Connect can be used to setup multi-thread processing. Can people point me to documentation and simple examples?

An example problem. I have something like:

data subset;

set full (where=(var1=1));

run;

We have a Win2008 server with a fast disk system and 32 cores which we use pretty much as a workstation (usually just one job running at a time). If I run this job on a large dataset, it takes about 5 minutes with one cpu core at 100% and the others unused. Is it possible to split the processing across multiple cores?

3 REPLIES 3
jakarman
Barite | Level 11

Bruce,

The most easy way to use multithreading is using procedures that support it. That are:

- Proc SQL , Sort

- HP versions of stat

etc SAS(R) 9.4 Language Reference: Concepts (Support for Parallel Processing )

Some SAS options are controlling the limits.

The nature of a sas-data-step (retain lag) knowing the order is blocking a parallelized (threaded) approach.

SQL Queries and Sorting on the other hand are ideal for threading.

When you want to use MP-Connect (or just Conect) you have to plan by yourself wich datasteps can run parallel and wich ones have to wait on another. The waitfor is statement is important for that. The others are rsubmit/endrsubmit and the communcation syslput/sysrput , signon/signoff 

SAS/CONNECT(R) 9.2 User's Guide (Example 5: Using MP CONNECT and the WAITFOR Statement)

Tips and Tricks for Scalability and Performance

---->-- ja karman --<-----
BruceBrad
Lapis Lazuli | Level 10

Is there any documentation describing when proc SQL will multithread? If I wrote my simple example in SQL, would it process it with multiple threads?

jakarman
Barite | Level 11

The best thing about proc SQL multithreading.... You don't need to do anything for it, it will work. 

Look at your results IF you do not care about the ordening (left out order in the sql) you could notice it are different sets taken out of the Original datasets and than being appended.

By the way Eguide flow is also supporting parallel processing, see:

https://communities.sas.com/thread/39741?start=0&tstart=0

Your are in the performance and tuning area now. That is measuring (OS and SAS) with different scenarios   

---->-- ja karman --<-----

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 970 views
  • 3 likes
  • 2 in conversation