Parallel execution of macro

Accepted Solution Solved
Reply
Contributor
Posts: 44
Accepted Solution

Parallel execution of macro

[ Edited ]
%macro m(country);

data &COUNTRY.;
retain row_num country;
do i = 1 to 1000;
country = "&country.";
row_num = i;
output;
drop i;
end;
run;

%mend m;

data _null_;
   array x[2] $2 ('uk' 'de');
   	do j=1 to dim(x);
		country = x[j];
		call execute ('%m('!!trim(country)!!')');
  	end;
run;

data de_uk;
merge uk (in=a) de (in=b);
by country;
run;

proc sort data=de_uk;
by country;
run;

data de_sum_row_num(keep=country nvar);
set de_uk;
by country;
if first.country then nvar=0;
nvar+row_num;
if last.country then output;
run;

Hello all,

 

 

I would like to run the first part of attached code in parallel (i.e. both countries at once in different programs but it one session). Any suggestion how to amend to code? My version of sas is 5.1 (5.100.0.14281). Thanks


Accepted Solutions
Solution
‎02-23-2017 10:17 AM
Esteemed Advisor
Posts: 6,646

Re: Parallel execution of macro

First of all, you need to have your whole SAS system (server and EG client(s)) configured for parallel execution.

Then your EG can send different nodes in project flows or whole project flows to different server instances.

Within any node, there is no way to "escape" the current server instance, and each instance works in a strictly serial manner. The SAS language as such does not have any construct for parallel execution (as you can find in Occam).

 

I guess that setting up your system for parallel execution just to test it out would be crushing peanuts with an elephant's hoof.

I myself have rarely found benefits in parallel execution; often you just end up with processes competing for resources and being slower in the end than if you had run them one after another.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers

View solution in original post


All Replies
Esteemed Advisor
Esteemed Advisor
Posts: 7,203

Re: Parallel execution of macro

SAS programs are linear.  Processing of datasteps of procedures can be split over multiple cores/machines with the specific software (Grid maybe), but individual programs run one step at a time one ofter each other.

 

I would expect that you would be far better off using by groups.  In your code you create a new dataset for each country, this just increases the processing needed.  Have one dataset and do your code by country:

data country_data (drop=i);
  retain row_num country;
  do country="uk","de";
    do i = 1 to 1000;
      row_num = i;
      output;
    end;
  end;
run;

proc print data=country_data;
  by country;  // see use of by groups!
run;
Contributor
Posts: 44

Re: Parallel execution of macro

Thanks for your reply. I created this code just for demonstration purposes - I would like to learn the way how to construct the code for parallel execution and then apply it on way more complicated codes.

Super User
Super User
Posts: 6,318

Re: Parallel execution of macro

[ Edited ]

Do you have SAS/Connect licensed?  Look at the documentation there.

I suspect that a programmer that understands PROC DS2 might be able to convert a working data step into a parallel execution.

 

Your current problem is trivial and can take advantage of the multi-threaded features of PROC MEANS.

 

* Generate data ;
data de_uk ;
   do country= 'uk','de' ;
      do row_num=1 to 1000;
          output;
      end;
   end;
run;

* Summarize data ;
proc means noprint nway data=de_uk;
   class country ;
   var row_num ;
   output  out=de_dum_row_num(drop=_: ) sum=nvar ;
run;

 

It would probably help if you split yout problem description into parts to show where you are just generating data to have a working set. Or change the problem to use existing SASHELP datasets so that it is clearer what is the part that you feel needs to run in parallel. 

 

 

Contributor
Posts: 44

Re: Parallel execution of macro

Let's assume there would be 10 countries in way more complicated code, I would like to execute the script for all countries at once so it only takes that long as the most complicated country. Hopefully, it is clearer now. Thanks

Esteemed Advisor
Posts: 6,646

Re: Parallel execution of macro

SAS version 5 would place you right in the middle of the 1980's, so I guess you meant Enterprise Guide 5.1. Smiley Wink

EG 5.1 is the first version that allows parallel execution, BUT: parallel execution is handled by EG for tasks and/or process flows. You can't initiate parallel execution within a single code node, as that is sent to one of the workspace server instances as a whole and that's it.

So you need to split your code into two separate code nodes in EG in order to execute them in parallel on multiple workspace server instances that use the same server context.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 44

Re: Parallel execution of macro

Thanks Kurt, I actually meant EG 5.1, you are right. So you say there is no way to proceed? Even to write a script for multiple sessions? Thanks

Solution
‎02-23-2017 10:17 AM
Esteemed Advisor
Posts: 6,646

Re: Parallel execution of macro

First of all, you need to have your whole SAS system (server and EG client(s)) configured for parallel execution.

Then your EG can send different nodes in project flows or whole project flows to different server instances.

Within any node, there is no way to "escape" the current server instance, and each instance works in a strictly serial manner. The SAS language as such does not have any construct for parallel execution (as you can find in Occam).

 

I guess that setting up your system for parallel execution just to test it out would be crushing peanuts with an elephant's hoof.

I myself have rarely found benefits in parallel execution; often you just end up with processes competing for resources and being slower in the end than if you had run them one after another.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Valued Guide
Posts: 2,174

Re: Parallel execution of macro

PROC SCAPROC includes features to indicate at which points a process could benefit from parallel execution.
Worth having a look - in particular - at examples
☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 8 replies
  • 169 views
  • 1 like
  • 5 in conversation