BookmarkSubscribeRSS Feed
iscgonzalez
Obsidian | Level 7

Hello,

 

I have a program that contains multiple proc sql on macros, something like this:

 

%macro m1000();

proc sql;

create table m1000 as 

select * from test1000;

run;

%mend;

 

I have around 70 macros like this and my question is, how could I run those 70 macros having 5 processes in parallel at the same time to run them (like a queue with only 5 threads) and as soon as one finishes, start running the next. I explored the systask command, but couldn't achieve to do this.

6 REPLIES 6
Tom
Super User Tom
Super User

Not sure why you have phrased the question in terms of running macros.  Macros are just a way to generate SAS code.

 

So what are the programs you want to run in parallel?  What do they do? How are they related to each other?  Are the all totally independent? If they are related how?  Does the results of running one impact the others?  Or are they just related by wanting to start from some common setup ? Or perhaps because you want to aggregate the results when they are done?

 

Do you have SAS/Connect licensed and installed?

Reeza
Super User

How is your SAS installation set up?

Is it just Base SAS? EG? SAS Studio? Viya?

Is it a server installation or desktop?


To check if you have SAS CONNECT run the following:

 

proc product_status;run;

I'm assuming you've already perused the LexJansen site for papers on parallel processing?

 

https://www.lexjansen.com/search/searchresults.php?q=parallel%20processing%20sas

ballardw
Super User

@iscgonzalez wrote:

Hello,

 

I have a program that contains multiple proc sql on macros, something like this:

 

%macro m1000();

proc sql;

create table m1000 as 

select * from test1000;

run;

%mend;

 

I have around 70 macros like this and my question is, how could I run those 70 macros having 5 processes in parallel at the same time to run them (like a queue with only 5 threads) and as soon as one finishes, start running the next. I explored the systask command, but couldn't achieve to do this.


For that specific code you would be better off renaming the data set test1000 to m1000. It would be faster and if these sets are of any size, a whole lot faster, could be done in one call to proc data sets. Example:

data junk1;
   x=3;
run;
data junk2;
  x=4;
run;

proc datasets library=work nolist nodetails;
   change Junk1 = New1
          Junk2 = New2
   ;

run;
quit;

As an aside, the above approach won't let you accidentally overwrite an existing "new" data set if exists, which might be a good thing if this is getting turned loose on unsuspecting data.

Sajid01
Meteorite | Level 14

Hello @iscgonzalez 
As rightly pointed out by @Tom Macros are simply text replacement tokens.

from your question, I see that you want to run the 70 segments of code in parallel.
if they can be run in parallel then SAS grid can be a good candidate.   One should also factor the I/O requirements.

SASKiwi
PROC Star

If you run your programs in batch mode then you you can run as many as you like at the same time as long as each runs independently of each other. No need for fancy splitting or SYSTASK.

sbxkoenk
SAS Super FREQ

Hello,

 

Many ways to run things in parallel.

My favorite V9.4 way is MP CONNECT. See below.

(Let me know if you want to set up // CAS sessions in SAS VIYA 3.5+ and I will post, in this topic thread, some interesting blogs on // CAS sessions).

 

Base SAS + SAS/CONNECT - A simple method to generate load on any number of licensed cores
Posted 04-08-2021 05:46 AM | by SimonWilliams
https://communities.sas.com/t5/SAS-Communities-Library/Base-SAS-SAS-CONNECT-A-simple-method-to-gener...

 

Running SAS programs in parallel using SAS/CONNECT®
By Leonid Batkhan on SAS Users January 13, 2021
https://blogs.sas.com/content/sgf/2021/01/13/running-sas-programs-in-parallel-using-sas-connect/

 

Using SYSTASK and SAS macro loops for massively parallel processing
By Leonid Batkhan on SAS Users June 14, 2021
https://blogs.sas.com/content/sgf/2021/06/14/using-systask-and-sas-macro-loops-for-massively-paralle...

 

Koen

Ready to join fellow brilliant minds for the SAS Hackathon?

Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.

Register today!
How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 6 replies
  • 1270 views
  • 4 likes
  • 7 in conversation