topic Re: Parallel Processing in SAS Programming

Parallel Processing

Saranya_Sub — Sat, 30 Nov 2019 11:55:00 GMT

Hi,

i have around 10 XML file in a folder ,i have to loop through each of the XML and create SAS Dataset , sequentailly am doing it and the output is good.Now i have run the same macro for 10 XML file in parallel.10 sessions has to be created.

am not sure how to do the same. can any one guide me in doing that .

Thanks,

Saran.

Re: Parallel Processing

ed_sas_member — Sat, 30 Nov 2019 11:58:05 GMT

Hi @Saranya_Sub

Could you please show your code so that we can help you?

Re: Parallel Processing

Saranya_Sub — Sat, 30 Nov 2019 12:09:48 GMT

%MACRO XML_Parse(_dir=);
filename dir "&_dir";
data work.dir;
length name $100;
d=dopen("dir");
num=dnum(d);
retain count 0;
do i=1 to num;
name=dread(d,i);
r=index(name,".xml");

if r ne 0 then
do;
call symput('XMLNAME'||left(i), scan(name,1,'.'));
output;
end;
end;

proc sql;

select count(*) INTO :_totalxml from work.dir;
quit;

%LET _totalxml=&_totalxml;

%do _XMLLOOP = 1 %TO &_totalxml;

filename xmlfile "&_dir.&_XMLNAME..xml";
filename mapfile "&_dir.&_XMLNAME..map";
libname xmlfile xmlv2 xmltype=xmlmap xmlmap=mapfile automap=replace;

---CODE TO JOIN ALL THE TABLES USING THE ORIDIANL COLUMN TO BRING TO ONE TABLES

%END;

%MEND;
%XML_Parse(_dir='/ccc/ccc/cc');

This code will read each of the XML file ,when first XML processing is completed it moves to the next one.

now my question is is there any way to run the macro in parallel so that i can read all the process all the XML file in same time

Sequential(above code)

XML 1 Parallel( i want it in this way)

XML 2 --- > XML 1 XML 2 XML 3

XML 3

though there are some document in the web ,am not able to understand clearly.

Please help

Thanks,

Saran

Re: Parallel Processing

unison — Sat, 30 Nov 2019 15:45:04 GMT

Take a look to see if anything in this thread solves your problem: https://communities.sas.com/t5/General-SAS-Programming/Macros-and-Parallel-processing/td-p/430028

-unison

Re: Parallel Processing

yabwon — Sat, 30 Nov 2019 18:21:56 GMT

Hi @Saranya_Sub,

let me share this little step-by-step "quasi" tutorial+macro for parallel processing in SAS I did some time ago (in advance I apologise for any spelling/grammar issues :-). Maybe you will find it useful.

http://www.mini.pw.edu.pl/~bjablons/SASpublic/Parallel-processing-in-BASE-SAS.sas

All the best

Bart

Re: Parallel Processing

SASKiwi — Sun, 01 Dec 2019 00:53:29 GMT

In my experience using SAS/CONNECT is the easiest and most reliable way to do this within SAS code. If you don't have this product then that rules out this option.