Parallel Processing

[Saranya_Sub]

Hi,

  i have around 10 XML file in a folder ,i have to loop through each of the XML and create SAS Dataset , sequentailly am doing it and the output is good.Now i have run the same macro for 10 XML file in parallel.10 sessions has to be created.

am not sure how to do the same. can any one guide me in doing that .

Thanks,

Saran.

[ed_sas_member]

Hi @Saranya_Sub 

Could you please show your code so that we can help you?

[Saranya_Sub]

%MACRO XML_Parse(_dir=);
filename dir "&_dir";
data work.dir;
length name $100;
d=dopen("dir");
num=dnum(d);
retain count 0;
do i=1 to num;
name=dread(d,i);
r=index(name,".xml");

if r ne 0 then
do;
call symput('XMLNAME'||left(i), scan(name,1,'.'));
output;
end;
end;

proc sql;

select count(*) INTO :_totalxml from work.dir;
quit;

%LET _totalxml=&_totalxml;

%do _XMLLOOP = 1 %TO &_totalxml;

filename xmlfile "&_dir.&_XMLNAME..xml";
filename mapfile "&_dir.&_XMLNAME..map";
libname xmlfile xmlv2 xmltype=xmlmap xmlmap=mapfile automap=replace;


---CODE TO JOIN ALL THE TABLES USING THE ORIDIANL COLUMN TO BRING TO ONE TABLES

%END;

%MEND;
%XML_Parse(_dir='/ccc/ccc/cc');

This code will read each of the XML file ,when first XML processing is completed it moves to the next one.

now my question is is there any way to run the macro in parallel so that i can read all the process all the XML file in same time 

Sequential(above code)

XML 1             Parallel( i want it in this way)

XML 2   --- >  XML 1 XML 2 XML 3 

XML 3  

though there are some document in the web ,am not able to understand clearly.

Please help

Thanks,

Saran

[unison]

Take a look to see if anything in this thread solves your problem: https://communities.sas.com/t5/General-SAS-Programming/Macros-and-Parallel-processing/td-p/430028

-unison

[SASKiwi]

In my experience using SAS/CONNECT is the easiest and most reliable way to do this within SAS code. If you don't have this product then that rules out this option.

[yabwon · Accepted solution]

Hi @Saranya_Sub,

let me share this little step-by-step "quasi" tutorial+macro for parallel processing in SAS I did some time ago (in advance I apologise for any spelling/grammar issues :-). Maybe you will find it useful.

http://www.mini.pw.edu.pl/~bjablons/SASpublic/Parallel-processing-in-BASE-SAS.sas 

All the best

Bart