topic Re: too many conditional statements for large datasets in SAS Programming

too many conditional statements for large datasets

Grumbler — Wed, 05 Jul 2017 05:53:43 GMT

so i have coded thousands of macro variables each containing a if statement that i need to process in a data step.

ideally, i can just make 1 macro and run it. however, it's too big for 1 macro variable to handle.

so i have no choice but to create thousands.

so how do i call each of the macro variable in 1 data step?

right now, i am doing a loop which will call 1 macro at a time but it's just terrible inefficient.

example,

%do i=1 %to &count %by 1;

data step;

set step;

&macro&i;

run;

%end;

something like this.

how can i make this into 1 data step and run all macro variables within?

thanks a bunch.

Re: too many conditional statements for large datasets

Reeza — Wed, 05 Jul 2017 06:02:24 GMT

Can you change your design to use DOSUBL or %include code from a text file?

Can you post a small example of what you're process is so we can make better suggestion for improvement or alternatives?

@Grumbler wrote:

so i have coded thousands of macro variables each containing a if statement that i need to process in a data step.

ideally, i can just make 1 macro and run it. however, it's too big for 1 macro variable to handle.

so i have no choice but to create thousands.

so how do i call each of the macro variable in 1 data step?

right now, i am doing a loop which will call 1 macro at a time but it's just terrible inefficient.

example,

%do i=1 %to &count %by 1;

data step;

set step;

&macro&i;

run;

%end;

something like this.

how can i make this into 1 data step and run all macro variables within?

thanks a bunch.

Re: too many conditional statements for large datasets

BrunoMueller — Wed, 05 Jul 2017 06:04:34 GMT

The ineffciency comes from reading the same data for each of your if statements. You should code it like this:

%macro test;

  data step;
    set step;

    %do i=1 %to &count %by 1;
      &macro&i;
    %end;
  run;

%mend;

%test

In this way the macro will generate all the IF statements in one DATA Step and the data is only red once.

Bruno

Re: too many conditional statements for large datasets

Grumbler — Wed, 05 Jul 2017 06:58:02 GMT

thanks everyone. problem solved. forgot about using macro loops inside the data step. 🙂

Re: too many conditional statements for large datasets

RW9 — Wed, 05 Jul 2017 08:02:25 GMT

I would really be suprised if the use of thousands of macro variables is the best method to process data. Base SAS is the executable language for processing data, and is fully functional, macro does not do anything other than generate this code. If you can provide some examples I can show better methods to get the output. For example there is data modelling, i.e. restructuring data to make coding more efficient - one thing I always see is the "Excel" mentality, having data going across the page which is ok for people to look at, but sub optimal to program with. Next there is lots of functionality within the base langauge to perform various functions against arrays of variables and over numbers of rows.