topic Re: Bootstrap, sampling macro & saving the result in a 2 dimensional array in SAS Programming

Bootstrap, sampling macro & saving the result in a 2 dimensional array

Marzi — Sat, 12 Nov 2016 11:03:55 GMT

Dear all;

I wanna doing the linear regression (a customize macro for LM, not the one offers by proc reg) for 1000 time for sample data (with replacement) and save the coefficient of all 1000 time in an array for the next process, this is what I wrote so far, But I can't save get the result of do loop:

%Macro Sample(data);
 proc surveyselect data= &data out=Sample NOPRINT 
     method=urs sampsize=5                          
     outhits;                                      
run;
%mend;


data BLM (drop = j);
   merge xy;
   array time{*} _numeric_;
   do j = 1 to 1000;
      temp = %Sample(xy);
      %LM(temp);
      time{j} = want;
   end;
run;

the %LM macro doing the LM, the "want" is the array which returns the coefficient values .
xy is my original data.

I'll appreciate it if someone tell me what is my mistake.

Thanks

Re: Bootstrap, sampling macro & saving the result in a 2 dimensional array

Shmuel — Sat, 12 Nov 2016 13:32:08 GMT

I see some issues in your code:

data BLM (drop = j);
   merge xy;
   array time{*} _numeric_;
   do j = 1 to 1000;
      temp = %Sample(xy);
      %LM(temp);
      time{j} = want;
   end;
run;

1) Merge is used to join two datasets or more.
What did you mean by xy ? Is this a dataset name ? where is the second one ?

2) You cant run a procedure inside a data-step.

%sample - is macro to run a procedure;

If you like to run the procedure 1000 (or any othen number) times

you need enter it in a macro like:

%macro loop;

%do j=1 %to 1000;

%sample(...);

%end;

%mend loop;

%loop;

3) Your code contains the line:

%LM(temp);

Where is macro LM defined ? is it a saved macro?

Is it coded as part of a datastep or as a separated procedure ?

4) Procedure surveyselect wiil produce output dataset, according to your code, named SAMPLE.

I don't see that you have used this dataset name in your code;

Re: Bootstrap, sampling macro & saving the result in a 2 dimensional array

sbxkoenk — Sat, 12 Nov 2016 13:33:18 GMT

Hello,

You cannot run PROCs inside a datastep.

You can use a "data _NULL_"-step however to steer and control the generation of the code / PROCs you want (this is called: dynamic data-driven code generation using FILE - PUT). The generated program is subsequently called and submitted via a %INCLUDE statement.

I will use a macro though (alternative to "dynamic data-driven code generation") to produce what you want.

By the way: your merge statement is weird. A merge statement should mention multiple data sets.

ods trace off;
options mprint nosymbolgen nomlogic nomstored;

proc datasets library=WORK NoList;
 delete Sample                / memtype=DATA; run;
 delete ParameterEstimates:   / memtype=DATA; run;
 delete ALLParameterEstimates / memtype=DATA; run; 
QUIT;

%MACRO Sample_LM;
%DO i = 1 %TO 25 %BY 1;
%LET seed=%EVAL(&i.*3+15963);

proc surveyselect data=sashelp.class out=work.Sample NOPRINT 
     method=urs sampsize=5 seed=&seed.                          
     outhits;                                      
run;

*ODS OUTPUT ParameterEstimates=work.ParameterEstimates;
PROC REG data=work.Sample outest=work.ParameterEstimates noprint;
 model&i. : model weight = height;
run;
QUIT;

proc append base=work.ALLParameterEstimates data=work.ParameterEstimates;
run;
QUIT;

%END; /* %DO i = 1 %TO 1000 %BY 1; */
%MEND Sample_LM;

%Sample_LM
/* All coefficients for height are in the dataset work.ALLParameterEstimates */

Maybe you prefer working with a real matrix language that supports custom functions and subroutines (modules). In that case I recommend to have a look into these 2 SAS/IML books. They contain multiple in-depth and elaborated examples on bootstrapping and sampling/simulating data.

IML stands for Interactive Matrix Language. You can use this matrix language within

PROC IML; /* put your IML-body here */ ; QUIT;

https://support.sas.com/publishing/authors/wicklin.html

* Statistical Programming with SAS/IML Software – Rick Wicklin – October 29, 2010

* Simulating Data with SAS® – Rick Wicklin – April 2013

Cheers,

Koen

Re: Bootstrap, sampling macro & saving the result in a 2 dimensional array

Reeza — Sat, 12 Nov 2016 13:52:33 GMT

The methods listed in this paper may be of interest

http://www2.sas.com/proceedings/forum2007/183-2007.pdf

Re: Bootstrap, sampling macro & saving the result in a 2 dimensional array

Marzi — Sun, 13 Nov 2016 02:51:00 GMT

@koen
your advice was really helpful, many thanks.

I would like to know if the "data" in "proc append" will be the output of a macro how we can modify it.
like:
proc append base=work.ALLParameterEstimates data=%LM(work.ParameterEstimates);
run;
QUIT;

I know this is technically wrong, but is there anyway to use the macro as data input for "proc append" or maybe I have to use another proc

Re: Bootstrap, sampling macro & saving the result in a 2 dimensional array

Ksharp — Sun, 13 Nov 2016 04:18:03 GMT

Something like this:

%Macro Sample(data);
%do i=1 %to 100;
 proc surveyselect data= &data out=temp NOPRINT 
     method=urs sampsize=5                          
     outhits;                                      
run;
%LM(temp)

proc append base=want data=table_from_LM force;run;
%end;
%mend;


%sample(BLM)

Re: Bootstrap, sampling macro & saving the result in a 2 dimensional array

Reeza — Sun, 13 Nov 2016 04:41:13 GMT

@Marzi in SAS, a macro generates code. That needs to be valid syntax. It DOES NOT return a value, though you can wrangle it, but generally that's better done via PROC FCMP.

If you're going to repeat a section of code multiple times, that's a good indicator that you need a macro to repeat the code. However, the datasets generated need to be managed by you. For example, one parameter in a macro can be the name of the output data set. Then that can be used in a further proc or macro if required.

In general, unless you're dealing with 10's of millions of rows I find using the BY method much easier to navigate in the long run.

Re: Bootstrap, sampling macro & saving the result in a 2 dimensional array

Marzi — Mon, 14 Nov 2016 02:58:40 GMT

@Shmuel, I got the point of my mistake from your comments. thanks