topic Performing a Bootstrapping validation of a Credit Risk Model in SAS Programming

Performing a Bootstrapping validation of a Credit Risk Model

Atulya212 — Wed, 29 Jan 2020 12:54:14 GMT

Dear All,

I have a macro that calculates KS, GINI, ROC of a credit risk model. Using that, I have to do a Bootstrapping validation so first I created a BootSample using the proc surveyselect command. The Bootsample contains 5,000 random samples. Each sample contains around 1500 records. How do I execute the macro for each of the samples in my dataset?

Re: Performing a Bootstrapping validation of a Credit Risk Model

PaigeMiller — Wed, 29 Jan 2020 13:19:58 GMT

Everything you could possibly want to know about how to do a bootstrap in SAS from @Rick_SAS

https://blogs.sas.com/content/iml/2018/12/12/essential-guide-bootstrapping-sas.html

Re: Performing a Bootstrapping validation of a Credit Risk Model

Rick_SAS — Wed, 29 Jan 2020 13:30:39 GMT

In general, running a statistic on each bootstrap sample will be slower than performing a BY group analysis on the output from PROC SURVEYSELECT where the Replicate variable is used as the BY variable. For example, you can see "The bootstrap method in SAS: A t test example"

Did you write the macro yourself? (Or at least understand the contents of the macro?) If so, it would be efficient to rewrite the analysis to directly use the output from SURVEYSELECT.

If you are a beginner and do not understand the macro, then you might be forced to use the slow method. For the slow method, you have a macro loop (say, %DO i = 1 %to 5000) and you use

WHERE REPLICATE=&i;

to subset the bootstrap samples.

Re: Performing a Bootstrapping validation of a Credit Risk Model

Oligolas — Wed, 29 Jan 2020 13:54:39 GMT

one way could be to run your macro with a call execute command.

To guaranty a better debugging I'd rather construct a loop over your macro that will filter your input dataset on the Bootsample you would like to process.

%MACRO doit(BootSampleStart=,BootSampleStop=);

   %DO i=&BootSampleStart. %TO &BootSampleStop.;
      %put NOTE: Processing BootSample &i.;

      /*Your code*/
      DATA _tmp;
         SET have;
         where BootSample=&i.;
      RUN;

      %YourMacro(inputDataset=_tmp <,your parameters>);
      PROC DATASETS lib=work nolist;delete _tmp; RUN;QUIT;
   %END;

%MEND doit;
** call example:;
%doit(BootSampleStart=1526,BootSampleStop=1528);