Parallel processing via E.G.

[deleted_user]

I know that Enterprise Guide lacks some straight forward functionality
for an advanced programmer, but do any of you know if
parallel processing is doable in or via Enterprise Guide 4.1 ?

Here's a paper explaining how to do it via SAS 8, I assume with PC SAS:

http://www2.sas.com/proceedings/sugi29/124-29.pdf

Thanks.

[ChrisHemedinger]

You can submit SAS MPCONNECT code from within EG and, provided that your server has the correct infrastructure, it should work as expected. In this scenario, you are using EG to connect to a single workspace SAS session, and then MPCONNECT is farming out the various jobs to other SAS CONNECT servers that you have configured.

If you use EG to run tasks/code on multiple SAS servers that you have configured (ex: SASMain, SASMain2, etc), EG will run tasks in parallel as long as the tasks do not depend on each other's results. In this scenario, EG is simply submitting multiple jobs across the various servers and collecting the results in a single place (your EG project).

More information about SAS parallel processing and scalability is available at:
http://support.sas.com/rnd/scalability/

Chris

[JohnH]

You may be interested in how I have encorporated the SAS Connect code with EGuide. I think you could modify it to achieve your purpose. Go get my papers in Coders Corner from SUGI31 and the followup this time at the SAS Global Forum -- John Hennessey

[deleted_user]

This is a simple example that shows how to do parallel processing, but I'm not sure that it's E.G. specific. I've also tried to tweak it after reading John's paper, but still doesn't run. I do have Integration Technologies module.

I get the following error:

ERROR: The SAS/CONNECT product with which the global statement 'rsubmit' is associated is either not licensed for your system or the product license has expired. Please contact your SAS installation representative.

ERROR: The SAS/CONNECT product with which the global statement 'rsubmit' is associated is not found in your sas registry. Please contact your SAS installation representative.

http://support.sas.com/rnd/scalability/tricks/connect.html#workpath

options autosignon=yes sascmd="sas";

/* create TASK1 SAS session to sort the SALES data */
rsubmit task1 wait=no sysrputsync=yes;
proc sort data=work.zipcode(keep= zip y x) out = zip_task1;
by zip;
run;
/* get TASK1's WORK library path in parent session */
%sysrput pathtask1=%sysfunc(pathname(work));
endrsubmit;

/* create TASK2 SAS session to sort the GOALS data */
rsubmit task2 wait=no sysrputsync=yes;
proc sort data=work.zipcode( drop = y x ) out = zip_task2;
by zip;
run;
/* get TASK2's WORK library path in parent session */
%sysrput pathtask2=%sysfunc(pathname(work));
endrsubmit;

/* wait until both sorts have finished */
waitfor _all_ task1 task2;

/* now we can define a libref to the sorted data sets and */
/* merge them inthe parent session */
libname worktsk1 "&pathtask1";
libname worktsk2 "&pathtask2";

data combine;
merge worktsk1.zip_task1
worktsk2.zip_task2;
by date;
run;

signoff task1;
signoff task2;

Thanks.

[JohnH]

Do you have the SAS/Connect product licensed? If not, you cannot do what I suggest. I do not know how to do it without SAS/Connect.

[ChrisHemedinger]

Yes, that is an important point.

SAS/CONNECT and SAS Integration Technologies are two similar, but distinct, products.

SAS/CONNECT allows you to connect to data and submit jobs on a remote SAS server from another client SAS session. Note that you have SAS sessions at each end point.

SAS Integration Technologies allows you to connect to data and submit jobs on a remote SAS server from any arbitrary client application -- for example SAS Enterprise Guide or even a custom application. In this case, one end point is a SAS session and the other is a client application driving IOM (the object model facilitated by SAS Integration Technologies).

It is possible to use these technologies together -- but they each must be licensed and installed at the appropriate end points.

Chris