Statistical programming, matrix languages, and more

Macro with submit/endsubmit

Reply
Occasional Contributor Tli
Occasional Contributor
Posts: 16

Macro with submit/endsubmit

Good day!
I would like to ask syntax submit/endsubmit in sas iml.
There is an error when I use SAS Macro with submit/endsubmit doing genetic algorithm (GA). Log is as following,

 

ERROR: Submit block cannot be directly placed in a macro. Instead, place the submit block into a file first and then use %include to include the file within a macro definition.

 

I should use Macro to do loop because I need to do the same procedure for 1000 datasets. Does any solution to deal with this problem?
Many thanks!

SAS Super FREQ
Posts: 3,753

Re: Macro with submit/endsubmit

If you post your code, we might be able to make suggestions for how to improve it. What procedure are you calling in the SUBMIT block?  I assume you are doing the GA in SAS/IML, so what is the purpose of the SUBMIT block?

 

You can easily run the same procedure many times from a SAS/IML program by putting a DO loop around the SUBMIT block and passing the name of the data set in as a parameter. No macro is required. See the article "Passing values from PROC IML into SAS procedures."

 

Here's an example of how to call the SUBMIT block in a loop. It is based on the article "Read hundreds of data sets into matrices."

It calls PROC MEANS on 129 different data sets and locates the variable that has the largest skewness. The OK= option on the SUBMIT statement is used to prevent the program from stopping when PROC MEANS encounters data sets that do not have any numerical variables.

 

/* find the variable in the Sashelp library that has the largest skewness */
proc iml;
libref = "Sashelp";                /* read from this library */
dsNames = T( datasets(libref) );   /* 129 data sets */
print "There are " (ncol(dsNames)) " data sets in the " (libref) " library.";
maxSkew = -1e6;
do i = 1 to ncol(dsNames);
   ds = libref + "." + strip(dsNames[i]);
   /* call PROC MEANS to compute skewness of all numerical variables */
   submit ds / OK=isOK;
   proc means data=&ds noprint;
   var _numeric_;
   output out=out(drop=_TYPE_ _FREQ_) skewness=;
   run;
   endsubmit;

   /* protect against PROC errors; not all data sets have numerical variables */
   if isOK then do;  
      /* read skewness for 'ds' data set */
      use out; 
      read all var _ALL_ into skew[c=varNames];
      close out;

      max = max(skew);
      if max > maxSkew then do;
         maxSkew = max;
         maxSkewVarName = varNames[ skew[<:>] ];
         maxSkewDSName = ds;
      end;
   end;
end;

print "The variable " maxSkewVarName 
      " in the data set " maxSkewDSName 
      " has a skewsness of " maxSkew;

Occasional Contributor Tli
Occasional Contributor
Posts: 16

Re: Macro with submit/endsubmit


Yes, I can use iml to do loop instead of Macro. But I can’t find any syntax to create different names of final datasets automatically.
eg.
edit data&i var _all_;
read all var _all_ into data&i;

 

create summary&i from summary;
append from summary;
close summary&i;
Aimed to create files by sas itself called summary1, summary2,…
Is it possible to this by iml without macro?
Thank you.

SAS Super FREQ
Posts: 3,753

Re: Macro with submit/endsubmit

Yes, simply put the name of the data set into a character variable and enclose the variable in parethesese on the USE and CLOSE statements. See "Read data sets that are specified by an array of names"

Occasional Contributor Tli
Occasional Contributor
Posts: 16

Re: Macro with submit/endsubmit

Hello,

Good day!

Sorry for the late response.

I tried in many ways. It didn’t work.

I would like to ask syntax submit/endsubmit in sas iml again.

 

Please see the attached.

 

There is an error when I use SAS Macro with submit/endsubmit doing genetic algorithm (GA). Log is as following,

ERROR: Submit block cannot be directly placed in a macro. Instead, place the submit block into a file first and then use %include to include the file within a macro definition.

Occasional Contributor Tli
Occasional Contributor
Posts: 16

Re: Macro with submit/endsubmit

Sorry!

There are two .csv and one .sas files.

I can't upload to the attachments.

How to some the code and the data to you?

SAS Super FREQ
Posts: 3,753

Re: Macro with submit/endsubmit

When you replay, there is a field underneath the text editor that says Attachments. Click the Browse button and navigate to the files that you want to attach.

Occasional Contributor Tli
Occasional Contributor
Posts: 16

Re: Macro with submit/endsubmit

[ Edited ]

It shows "The contents of the attachment doesn't match its file type."

It didn't work.

So, I post the code is as following,

%GLOBAL name rep1 rep2;
%LET name=GA;
%LET rep1=0;
%LET rep2=0;
/*GA for PLS*/
%MACRO Data;
%DO i=&rep1 %TO &rep2;   /* i=1 to 1000*/
/*
******SECTION 1******
*/
/* GA algorithm */ /* This section is in PROC IML environment. */
PROC IML;
SUBMIT;
PROC IMPORT OUT= WORK.NIR140
            DATAFILE= "D:\cal_data(&i).csv" 
            DBMS=CSV REPLACE;
     GETNAMES=YES;
     DATAROW=2;
PROC IMPORT OUT= WORK.NIR70
            DATAFILE= "D:\val_data(&i).csv" 
            DBMS=CSV REPLACE;
     GETNAMES=YES;
     DATAROW=2;  
RUN;
ENDSUBMIT;
DM LOG 'CLEAR';
DM OUTPUT 'CLEAR';
/*start a "start finish module"*/
start uniform_cross(child1, child2, parent1, parent2);
child1 = parent1;
child2 = parent2;
do i = 1 to ncol(parent1);
r = uniform(1234);
if r<=0.5 then do;
child1[i] = parent2[i];
child2[i] = parent1[i];
end;
end;
finish;
/*end of start finish module*/


/*Start GA*/
start knapsack( x );

/*%SF*/
edit nir140 var _all_;                                                  
read all var _all_ into nir;
X_new=j (140, 351, 0);                                      /*@@@140 40 : number of observations*/
do n=1 to 351; 
    if  x[, n] =1 
    then do; 
    a=n+1;
   X_new[,n]=nir[,a ]; 
   end;
end;

data_new=nir[,1]||X_new;

create data_new from data_new;
append from data_new;
close data_new; /*PROC print data=data_new;  proc print data=new_var; */

SUBMIT;
proc pls data=data_new method=pls  cv=one NOCENTER NOSCALE ;
model COL1= COL2-COL352 / SOLUTION;                          
*ods output ParameterEstimates=coef;              
ods output ResidualSummary=RS;
ods output CVResults=CVResults;
output out=score XSCORE=XSCORE; 
RUN;
ENDSUBMIT;

DM LOG 'CLEAR';
DM OUTPUT 'CLEAR';

EDIT RS var _all_;
READ all var _all_ into RS;
CLOSE RS;

EDIT CVResults var _all_;                                                         
read all var _all_ into CVResults;
CLOSE CVResults;

EDIT score var _all_;
READ all var _all_ into score;
CLOSE score;

/*calculating PRESS*/ 
RMPress=RS[, 2];
create RMPress from RMPress;                        /*P*/
append from RMPress;
close RMPress;                                               /*P*/
/*factor numbers*/   /*x: extract only factor number from CVResults*/
f_no=CVResults[2,1];
create f_no from f_no;                           /*P*/
append from f_no;
close f_no;                                           /*P*/
/*n*/
n=nrow(score);
/*RMSECV*/
/*factor numbers*/   /*x: extract only factor number from CVResults*/
minRMPress=RS[f_no+1, 2];

RMSECV=SQRT(((n-1)*(minRMPress##2))/n);
create RMSECV from RMSECV;                /*P*/        
append from RMSECV;
close RMSECV;                                         /*P*/
/*end of %SF*/
return(RMSECV);
finish;

id = gasetup(2, 351, );
call gasetobj(id, 0, "knapsack" ); /* minimize objective module */   
call gasetcro(id, 0.95, 0,"uniform_cross"); /* user crossover module */                            
call gasetmut(id,
                     0.01, /* mutation probabilty */
                     1);
call gasetsel(id, 2, /* carry 3 over to next generation */                             
                   0, /* dual tournament */
                   2 /* best-player-wins probabilty */
                    );
call gainit(id, 30, repeat({0,1},1, 351)); 
/*
DM LOG 'CLEAR';
DM OUTPUT 'CLEAR';
*/

niter = 500;                                                                                           /*@@@@ now is 5, change niter to 1000*/
/*
DM LOG 'CLEAR';
DM OUTPUT 'CLEAR';
*/

summary = j(niter,2);
mattrib summary [c = {"bestValue", "avgValue"}];
call gagetval(value, id);
summary[1,1] = value[1];
summary[1,2] = value[:];
do i = 1 to niter;
DM LOG 'CLEAR';
DM OUTPUT 'CLEAR';
    call garegen(id);
    call gagetval(value, id);
    summary[i,1] = value[1];
    summary[i,2] = value[:];
DM LOG 'CLEAR';
DM OUTPUT 'CLEAR';
end;
call gagetmem(mem, value, id, 1);
print "best member " mem[f = 1.0 l = ""],
        "best value " value[l = ""];
iteration = t(1:niter);
print iteration summary;
call gaend(id);


create mem&i from mem;
append from mem;
close mem&i;

create value&i from value;
append from value;
close value&i;

create summary&i from summary;
append from summary;
close summary&i;
/* It is the end of GA algorithm. */
/*
DM LOG 'CLEAR';
DM OUTPUT 'CLEAR';
*/
%END;
%MEND;
%Data


QUIT;

 

 

Occasional Contributor Tli
Occasional Contributor
Posts: 16

Re: Macro with submit/endsubmit

This code is no problem before I added Macro.
Occasional Contributor Tli
Occasional Contributor
Posts: 16

Re: Macro with submit/endsubmit

Anyway, my goal is to run the GA code for 1000 times and save the results. If there is no solution in SAS, substiturion will be accepted. Does any resolution to achieve the goal? Thank you.
SAS Super FREQ
Posts: 3,753

Re: Macro with submit/endsubmit

Maybe I'm missing something, but I believe that you already have a solution: Get rid of the macro loop and instead use a DO loop in SAS/IML to repeat the GA algorithm. Is there some reason that this won't accomplish what you want?

Occasional Contributor Tli
Occasional Contributor
Posts: 16

Re: Macro with submit/endsubmit

[ Edited ]

Yes, I can use iml to do loop instead of Macro. But I can’t find any syntax to create different names of final datasets automatically.

 

eg. 

create summary&i from summary;
append from summary;
close summary&i;


Aimed to create files by sas itself called summary1, summary2,… , summary1000  i.e. give specific names in every run.
How to do this even if there is no macro.
Thank you.

SAS Super FREQ
Posts: 3,753

Re: Macro with submit/endsubmit

already answered this question. See my 2nd response (4th in thread).

Occasional Contributor Tli
Occasional Contributor
Posts: 16

Re: Macro with submit/endsubmit

Thank you very much!
I figure it out!!
Ask a Question
Discussion stats
  • 13 replies
  • 958 views
  • 5 likes
  • 2 in conversation