Re: Using a macro program inside parallel processing

ctisseuil · Posted 04-01-2021 08:55 AM

Hi,

I am a SAS EG user. I would like to run some process in parallel, where each session uses a macro program that I have defined initially. I have tried the following code without success. Could you highlight me ?

Thanks in advance.

Clement

/* My macro program */
%macro sampsql(filein, fileout,n);
proc Sql outobs=&n;
create table &fileout as
select *,ranuni(99) as nn from &filein
order by nn;
quit;
%mend sampsql;

/* running parallel processes */
options autosignon;
options sascmd="sas";
%syslput _all_ / remote=task1;
%syslput _all_ / remote=task2;

Rsubmit task1 inheritlib=(RES);
%sampsql(filein=sashelp.Cars, fileout=SampleSRS1,n=100);
Endrsubmit;

Rsubmit task2 inheritlib=(RES);
%sampsql(filein=sashelp.Cars, fileout=SampleSRS2,n=1000);
Endrsubmit;

waitfor _all_;
signoff _all_;

PaigeMiller · Posted 04-01-2021 09:01 AM

The macro %SAMPSQL must be defined on the Remote machines.

I have tried the following code without success.

What do you mean "without success". Explain. Show us the LOG.

--
Paige Miller

Tom · Posted 04-01-2021 09:02 AM

You need some way for each of the separate processes to know how to find the macro definition. Compiling the macro in your driver session does not mean that the macro will be available to the separate sessions.

For example if might be as simple as making sure they all use the same autoexec.sas file that includes commands to set SASAUTOS option so that the macro definition can be automatically loaded.

Or you might rsubmit code to each session to compile the macro. Note in that case it would be better to do that via a %INCLUDE instead of trying rsubmit the macro definition as that can in rare cases cause some confusion to the parser. Probably not an issue with your simple little example, but if the text of the macro definition is long and/or complex the risk of parsing issues is greater.

ctisseuil · Posted 04-01-2021 09:28 AM

Hi Tom,

Thanks for your answer.

As far as I understand, I have tried to follow your advice to integrate my macro program inside an autoexec process flow and then setting the SASAUTOS option.

/* running parallel processes */
options SASAUTOS;
options autosignon;
options sascmd="sas";
%syslput _all_ / remote=task1;
%syslput _all_ / remote=task2;

Rsubmit task1 inheritlib=(RES);
%sampsql(filein=sashelp.Cars, fileout=SampleSRS1,n=100);
Endrsubmit;

Rsubmit task2 inheritlib=(RES);
%sampsql(filein=sashelp.Cars, fileout=SampleSRS2,n=1000);
Endrsubmit;

waitfor _all_;
signoff _all_;

However, I get the following error message :

1                                                          Le Système SAS                              14:29 Thursday, April 1, 2021

1          ;*';*";*/;quit;run;
2          OPTIONS PAGENO=MIN;
3          %LET _CLIENTTASKLABEL='Parallelisation : test avec un macro programme';
4          %LET _CLIENTPROCESSFLOWNAME='Foire aux Questions';
5          %LET _CLIENTPROJECTPATH='R:\DMCD\MARK\PERSO\Clement Perso\SAS-BOITE-OUTILS\sas-boite-outils.egp';
6          %LET _CLIENTPROJECTPATHHOST='UC5416';
7          %LET _CLIENTPROJECTNAME='sas-boite-outils.egp';
8          %LET _SASPROGRAMFILE='';
9          %LET _SASPROGRAMFILEHOST='';
10         
11         ODS _ALL_ CLOSE;
12         OPTIONS DEV=PNG;
13         GOPTIONS XPIXELS=0 YPIXELS=0;
14         FILENAME EGSR TEMP;
15         ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR
16             STYLE=HtmlBlue
17             STYLESHEET=(URL="file:///C:/Program%20Files%20(x86)/SASHome/x86/SASEnterpriseGuide/7.1/Styles/HtmlBlue.css")
18             NOGTITLE
19             NOGFOOTNOTE
NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR
20             GPATH=&sasworklocation
21             ENCODING=UTF8
22             options(rolap="on")
23         ;
24         
25         GOPTIONS ACCESSIBLE;
26         
27         /* running parallel processes */
28         
ERROR 26-12: Signe égal plus valeur requis après l'option SAS SASAUTOS.
29         options SASAUTOS;
                   ________
                   26

30         options autosignon;
31         options sascmd="sas";
32         %syslput _all_ / remote=task1;
NOTE: Remote signon to TASK1 commencing (SAS Release 9.03.01M2P081512).
NOTE: Unable to open SASUSER.PROFILE. WORK.PROFILE will be opened instead.
NOTE: All profile changes will be lost at the end of the session.
NOTE: Copyright (c) 2002-2010 by SAS Institute Inc., Cary, NC, USA. 
NOTE: SAS (r) Proprietary Software 9.3 (TS1M2) 
      Licensed to IRP AUTO OFFICE ANALYTICS VM 4 CORES, Site 70238211.
NOTE: This session is executing on the X64_SRV12  platform.



NOTE: Enhanced analytical products:

SAS/STAT 12.1

NOTE: L'initialisation de SAS used:
      real time           0.21 secondes
      cpu time            0.10 secondes
      
NOTE: Remote signon to TASK1 complete.
33         %syslput _all_ / remote=task2;
NOTE: Remote signon to TASK2 commencing (SAS Release 9.03.01M2P081512).
NOTE: Unable to open SASUSER.PROFILE. WORK.PROFILE will be opened instead.
NOTE: All profile changes will be lost at the end of the session.
NOTE: Copyright (c) 2002-2010 by SAS Institute Inc., Cary, NC, USA. 
NOTE: SAS (r) Proprietary Software 9.3 (TS1M2) 
      Licensed to IRP AUTO OFFICE ANALYTICS VM 4 CORES, Site 70238211.
NOTE: This session is executing on the X64_SRV12  platform.



NOTE: Enhanced analytical products:

SAS/STAT 12.1

NOTE: L'initialisation de SAS used:
      real time           0.25 secondes
      cpu time            0.14 secondes
      
NOTE: Remote signon to TASK2 complete.
34         
35         Rsubmit task1 inheritlib=(RES);
2                                                          Le Système SAS                              15:23 Thursday, April 1, 2021

NOTE: Remote submit to TASK1 commencing.
WARNING: Apparent invocation of macro SAMPSQL not resolved.
1    %sampsql(filein=sashelp.Cars, fileout=SampleSRS1,n=100);
     -
     180

ERROR 180-322: Statement is not valid or it is used out of proper order.

NOTE: Remote submit to TASK1 complete.
36         
37         Rsubmit task2 inheritlib=(RES);
3                                                          Le Système SAS                              15:23 Thursday, April 1, 2021

NOTE: Remote submit to TASK2 commencing.
WARNING: Apparent invocation of macro SAMPSQL not resolved.
1    %sampsql(filein=sashelp.Cars, fileout=SampleSRS2,n=1000);
     -
     180

ERROR 180-322: Statement is not valid or it is used out of proper order.

NOTE: Remote submit to TASK2 complete.
ERROR 85-169: Task name expected.
38         
39         waitfor _all_;
                        _
                        85

40         signoff _all_;
NOTE: Remote signoff from TASK1 commencing.
NOTE: SAS Institute Inc., SAS Campus Drive, Cary, NC USA 27513-2414
NOTE: Le Système SAS used:
      real time           0.54 secondes
      cpu time            0.10 secondes
      
NOTE: Remote signoff from TASK1 complete.
NOTE: Remote signoff from TASK2 commencing.
NOTE: SAS Institute Inc., SAS Campus Drive, Cary, NC USA 27513-2414
NOTE: Le Système SAS used:
      real time           0.29 secondes
      cpu time            0.14 secondes
      
NOTE: Remote signoff from TASK2 complete.
41         
42         GOPTIONS NOACCESSIBLE;
43         %LET _CLIENTTASKLABEL=;
44         %LET _CLIENTPROCESSFLOWNAME=;
45         %LET _CLIENTPROJECTPATH=;
46         %LET _CLIENTPROJECTPATHHOST=;
47         %LET _CLIENTPROJECTNAME=;
48         %LET _SASPROGRAMFILE=;
49         %LET _SASPROGRAMFILEHOST=;
50         
51         ;*';*";*/;quit;run;
4                                                          Le Système SAS                              14:29 Thursday, April 1, 2021

52         ODS _ALL_ CLOSE;
53         
54         
55         QUIT; RUN;
56

Tom · Posted 04-01-2021 09:45 AM

You don't seem to understand how autocall macros work. But even so what you tried coudl not work for the same reason you where having trouble before. You changed a setting in the controlling session, but did nothing to change the setting in the remote sessions. From your log it looks like the %SYSLPUT statements is what is triggering the startup of the remote sessions that you have named TASK1 and TASK2.

So after they are started you need to make sure they know about the macro you are trying to call .

Replace the comments I have added below with something that will define %SAMPSQL() . Otherwise those sessions have no idea what to do when they see %SAMPSQL() being called.

%syslput _all_ / remote=task1;
%syslput _all_ / remote=task2;

rsubmit task1 inheritlib=(RES);
/* ADD CODE HERE TO DEFINE THE MACRO SAMPSQL */
%sampsql(filein=sashelp.Cars, fileout=SampleSRS1,n=100);
Endrsubmit;

rsubmit task2 inheritlib=(RES);
/* ADD CODE HERE TO DEFINE THE MACRO SAMPSQL */
%sampsql(filein=sashelp.Cars, fileout=SampleSRS2,n=1000);
Endrsubmit;

waitfor _all_;
signoff _all_;

ctisseuil · Posted 04-01-2021 10:27 AM

Nice,

Thank you, things are getting better.

The only trouble I have is that the resulting tables from the two processes are created but they are empty.

However when I run the macro program in the main SAS session, the tables are correctly created and filled with data...

Any ideas ?

options autosignon;
options sascmd="sas";
%syslput _all_ / remote=task1;
%syslput _all_ / remote=task2;

Rsubmit task1;
%macro sampsql(filein, fileout,n);

	proc Sql outobs=&n;
		create table &fileout as
			select *,ranuni(99) as nn from &filein
				order by nn;
	quit;
%mend sampsql;
%sampsql(filein=RES.Cars, fileout=RES.SampleSRS1,n=10);
Endrsubmit;


Rsubmit task2;
%macro sampsql(filein, fileout,n);
	proc Sql outobs=&n;
		create table &fileout as
			select *,ranuni(99) as nn 
				from &filein 
					order by nn;
	quit;
%mend sampsql;
%sampsql(filein=RES.Cars, fileout=RES.SampleSRS2,n=10);
Endrsubmit;

log file

1                                                          Le Système SAS                              14:29 Thursday, April 1, 2021

1          ;*';*";*/;quit;run;
2          OPTIONS PAGENO=MIN;
3          %LET _CLIENTTASKLABEL='Parallelisation : test avec un macro programme';
4          %LET _CLIENTPROCESSFLOWNAME='Foire aux Questions';
5          %LET _CLIENTPROJECTPATH='R:\DMCD\MARK\PERSO\Clement Perso\SAS-BOITE-OUTILS\sas-boite-outils.egp';
6          %LET _CLIENTPROJECTPATHHOST='UC5416';
7          %LET _CLIENTPROJECTNAME='sas-boite-outils.egp';
8          %LET _SASPROGRAMFILE='';
9          %LET _SASPROGRAMFILEHOST='';
10         
11         ODS _ALL_ CLOSE;
12         OPTIONS DEV=PNG;
13         GOPTIONS XPIXELS=0 YPIXELS=0;
14         FILENAME EGSR TEMP;
15         ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR
16             STYLE=HtmlBlue
17             STYLESHEET=(URL="file:///C:/Program%20Files%20(x86)/SASHome/x86/SASEnterpriseGuide/7.1/Styles/HtmlBlue.css")
18             NOGTITLE
19             NOGFOOTNOTE
20             GPATH=&sasworklocation
21             ENCODING=UTF8
22             options(rolap="on")
23         ;
NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR
24         
25         GOPTIONS ACCESSIBLE;
26         /*options sasautos = (sasautos "&DIR_PGM\macropg.sas");*/
27         /*options append=(sasautos=("&DIR_PGM\macropg.sas") ) ;*/
28         /*-sasautos ("&DIR_PGM\macropg.sas")*/
29         options autosignon;
30         options sascmd="sas";
31         %syslput _all_ / remote=task1;
32         %syslput _all_ / remote=task2;
33         
34         Rsubmit task1;
NOTE: Remote submit to TASK1 commencing.
297  %macro sampsql(filein, fileout,n);
298  
299   proc Sql outobs=&n;
300    create table &fileout as
301     select *,ranuni(99) as nn from &filein
302      order by nn;
303   quit;
304  %mend sampsql;
305  %sampsql(filein=RES.Cars, fileout=RES.SampleSRS1,n=10);
2                                                          Le Système SAS                              15:49 Thursday, April 1, 2021

WARNING: Table RES.SAMPLESRS1 non remplacée car de l'option NOREPLACE.
WARNING: Table RES.SAMPLESRS1 non remplacée car de l'option NOREPLACE.

NOTE: PROCEDURE SQL used (Total process time):
      real time           0.04 secondes
      cpu time            0.00 secondes
      

NOTE: Remote submit to TASK1 complete.
35         
36         
37         Rsubmit task2;
NOTE: Remote submit to TASK2 commencing.
304  %macro sampsql(filein, fileout,n);
305   proc Sql outobs=&n;
306    create table &fileout as
307     select *,ranuni(99) as nn
308      from &filein
309       order by nn;
310   quit;
311  %mend sampsql;
312  %sampsql(filein=RES.Cars, fileout=RES.SampleSRS2,n=10);
WARNING: Table RES.SAMPLESRS2 non remplacée car de l'option NOREPLACE.
WARNING: Table RES.SAMPLESRS2 non remplacée car de l'option NOREPLACE.

NOTE: PROCEDURE SQL used (Total process time):
      real time           0.03 secondes
      cpu time            0.00 secondes
      

NOTE: Remote submit to TASK2 complete.
38         
39         
40         
41         GOPTIONS NOACCESSIBLE;
42         %LET _CLIENTTASKLABEL=;
43         %LET _CLIENTPROCESSFLOWNAME=;
44         %LET _CLIENTPROJECTPATH=;
45         %LET _CLIENTPROJECTPATHHOST=;
46         %LET _CLIENTPROJECTNAME=;
47         %LET _SASPROGRAMFILE=;
48         %LET _SASPROGRAMFILEHOST=;
49         
50         ;*';*";*/;quit;run;
51         ODS _ALL_ CLOSE;
52         
53         
54         QUIT; RUN;
55

Tom · Posted 04-01-2021 10:41 AM

Sounds like you just need to change the setting of the REPLACE option.

But for now test your existing code by using output table name that does not already exist to make sure that is the only issue.

ctisseuil · Posted 04-01-2021 11:18 AM

Well,

After ensuring the tables do not exist yet, the created tables are still empty...

Are there any instructions I have to provide before closing the parallelized sessions to retrieve the results ?

Log :

1                                                          Le Système SAS                              16:45 Thursday, April 1, 2021

1          ;*';*";*/;quit;run;
2          OPTIONS PAGENO=MIN;
3          %LET _CLIENTTASKLABEL='Parallelisation : test avec un macro programme';
4          %LET _CLIENTPROCESSFLOWNAME='Foire aux Questions';
5          %LET _CLIENTPROJECTPATH='R:\DMCD\MARK\PERSO\Clement Perso\SAS-BOITE-OUTILS\sas-boite-outils.egp';
6          %LET _CLIENTPROJECTPATHHOST='UC5416';
7          %LET _CLIENTPROJECTNAME='sas-boite-outils.egp';
8          %LET _SASPROGRAMFILE='';
9          %LET _SASPROGRAMFILEHOST='';
10         
11         ODS _ALL_ CLOSE;
12         OPTIONS DEV=PNG;
13         GOPTIONS XPIXELS=0 YPIXELS=0;
14         FILENAME EGSR TEMP;
15         ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR
16             STYLE=HtmlBlue
17             STYLESHEET=(URL="file:///C:/Program%20Files%20(x86)/SASHome/x86/SASEnterpriseGuide/7.1/Styles/HtmlBlue.css")
18             NOGTITLE
19             NOGFOOTNOTE
NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR
20             GPATH=&sasworklocation
21             ENCODING=UTF8
22             options(rolap="on")
23         ;
24         
25         GOPTIONS ACCESSIBLE;
26         
27         options sascmd="sas";
28         signon task1;
NOTE: A link to TASK1 has been previously established. SIGNON is not necessary.
29         %syslput _all_ / remote=task1;
30         Rsubmit task1 inheritlib=(RES);
31         
32         
33         /* --------------------------------------------------------------------
34         DEUXIEME PROCESSUS
35         -------------------------------------------------------------------- */
36         signon task2;
NOTE: A link to TASK2 has been previously established. SIGNON is not necessary.
37         %syslput _all_ / remote=task2;
38         Rsubmit task2 inheritlib=(RES);
39         
40         GOPTIONS NOACCESSIBLE;
41         %LET _CLIENTTASKLABEL=;
42         %LET _CLIENTPROCESSFLOWNAME=;
43         %LET _CLIENTPROJECTPATH=;
44         %LET _CLIENTPROJECTPATHHOST=;
45         %LET _CLIENTPROJECTNAME=;
46         %LET _SASPROGRAMFILE=;
47         %LET _SASPROGRAMFILEHOST=;
48         
49         ;*';*";*/;quit;run;
50         ODS _ALL_ CLOSE;
51         
52         
53         QUIT; RUN;
54

Tom · Posted 04-01-2021 11:28 AM

This code does not appear to be trying to do anything. In fact it has an opening RSUBMIT without a closing ENDRSUBMIT.

Your original code appeared to be trying to use a shared libref to transfer data. You can also just use PROC DOWNLOAD to move data from the remote session to the local session.

rsubmit task1;
data work.newfile; set sashelp.class; run;
proc download data=work.newfile out=work.newfile status=n; run;
endrsubmit;

Or if you are sure the two sessions are running on the same machine (or at least have access to the same files) you could write the data directly to the target location and avoid the delay of transferring the data between the two sessions.

rsignon task1;
rsubmit;
  libname remote "/some/location";
  data remote.newfile;
    set sashelp.class;
  run;
endrsubmit;

libname local "/some/location" ;
proc print data=local.newfile;
run;

2025 SAS Hackathon: There is still time!

SAS Training: Just a Click Away