topic Re: how do you manage hundreds of macro variables in SAS Programming

how do you manage hundreds of macro variables

ZRick — Tue, 19 Mar 2013 16:51:41 GMT

In my current SAS system, which consists of nearly 100 macros, in each macro, there are at least over 10 macro variables in each macro.

It is a mess and makes me headache, just managing these many macro variables.

How do you manage these macro variables in an organized fashion?

organized meaning:

each macro variable is unique in its functionality
different macro variables don't step over each other
the ability of switch on batch of macro variables across the macros

Re: how do you manage hundreds of macro variables

Cynthia_sas — Tue, 19 Mar 2013 18:44:10 GMT

Hi: When you have macro programs (macros) that use macro variables, generally, the macro variables used in the macro programs are LOCAL in scope (exist for the duration of the macro program), unless you do something to make them GLOBAL in scope (exist for the duration of your SAS session). SAS has statements to manage the scope of a macro variable %LOCAL and %GLOBAL. And if you use the CALL SYMPUTX method of creating macro variables, you can explicitly control whether the macro variable is written to a global or local symbol table. Under special circumstances, prompt values on the BI platform can get automatically created as GLOBAL macro variables, so you have to be aware of this fact as you are writing your stored processes and macro programs to use in stored processes.

When I worked in a big shop that had more than 100 macro programs, we were required to have a huge comment section at the top of each macro program that outlined the macro variables that were global and coming from somewhere else and the macro variables that were local and we had to document the use and possible values of each macro variable and outline the error processing/validation that was performed for any macro variables being supplied as parameters on the invocation. We were not allowed to use positional parameters. We had to use keyword parameters.

For some systems, all our macro variables started with _macvarname or for others it was macvarname_A, macvarname_X where the _A meant first in importance, usually global macro variable and the _X meant "working" or "temporary" or used within the macro program for manipulation. Parameters were allowed to be defined without the _A, for example, but then as a first step in our macro program, we'd have some thing like this:

%LET macvar_X = %upcase(parmvar);

or whatever other manipulation we needed to do.

I'm sure that other folks have other ways of working with lots of macro programs and macro variables.

cynthia

Re: how do you manage hundreds of macro variables

ballardw — Tue, 19 Mar 2013 20:25:40 GMT

I have to say that including "the ability of switch on batch of macro variables across the macros" as part of Organized brings chills to my spine. My feeling about this phrase when I first read it was you wanted to change scope of variables within macros conditionally which I think would require oodles of programming and possible not ever quite work.

Maybe we need some expansion of what you really mean by the phrase "the ability of switch on batch of macro variables across the macros" with examples of what you expect to accomplish with this feature.

Re: how do you manage hundreds of macro variables

Tom — Tue, 19 Mar 2013 20:54:16 GMT

You need to describe what the macro variables are doing?

Ideally you would have only local macro variables. So each macro might have a few input parameters (which are by definition local) and define some local macro variables that it uses while it is running.

If you need to pass a value into a macro then you would do it though the parameters.

If you need to pass a value back from a macro to its calling environment then pass the name of the macro variable. Or better still pass the values in a dataset instead.

Re: how do you manage hundreds of macro variables

Astounding — Tue, 19 Mar 2013 21:32:12 GMT

I'm not saying this is necessarily the best solution, but here's something to think about. You could add a parameter to a macro:

%macro A (next_macro_to_run=);

Then at the end of the macro definition, you could add:

%if %length(&next_macro_to_run) %then %&next_macro_to_run;

%mend A;

That would make all the local macro variables that are part of %A available to the next macro that you would like to run. For example, if you specified:

%A (next_macro_to_run=B)

In that case, you would generate %B at the end of running %A. And all of the %A local variables would still exist.

Re: how do you manage hundreds of macro variables

Reeza — Tue, 19 Mar 2013 21:51:39 GMT

I avoid having that many like hell. :smileydevil:

I'd be curious to know why you do have so many, and from there I'm sure people will have suggestions on how to reduce and/or manage them more efficiently.

Re: how do you manage hundreds of macro variables

ZRick — Wed, 20 Mar 2013 02:05:26 GMT

Because we are developing production system using SAS.

We need to have the production system that is:

robust that is less error prone
scalable that can handle small / large volume of data
versatile that meet all kinds of different needs with different sets of parameters
portable that have the design of object oriented rather than procedure oriented

It becomes challenge in every system design/development that have above goals.

That is why we have hundred macros with many macro variables for macros.

In adhoc environment, any data step can do meet all kinds of needs, but once to have a data step truly up to production strength, it is very difficult.

Data t;

set t;

a+b;

run;

it can easily add hundreds of lines of code with macro variable to code the above data step with all different layers to make it really production ready.

I just want to like to hear if some of the experts who handled this this scenario before.

Try to have an overall picture, please. Any suggestions is greatly appreciated.

Re: how do you manage hundreds of macro variables

Tom — Wed, 20 Mar 2013 02:48:55 GMT

Still not clear why you need "hundreds" of macro variables. Many complicated environments have been built using SAS that do not require very many macro variables.

Re: how do you manage hundreds of macro variables

ScottBass — Wed, 20 Mar 2013 11:33:00 GMT

Why don't you acknowledge that "...it's a mess and gives you a headache", and rearchitect your approach?

I'm not saying the below is the best approach - I threw it together in a hurry - but perhaps it gives you some ideas...

options mprint;

* first approach - "wide" dataset ;

%macro test(metadata=sashelp.cars,make=,model=);

data _null_;

set &metadata;

where strip(make)="&make" and strip(model)="&model";

length dummy $1 name $32;

call missing(dummy);

do while (1);

call vnext(name);

if (name in ("model","type")) then continue;

if (name="dummy") then leave;

call symputx("__"||name,vvaluex(name),"L");

end;

run;

%printmvars

%mend;

%macro printmvars;

proc sql;

select name,value from dictionary.macros where name like "^__%" escape "^";

quit;

%mend;

%test(make=Acura, model=MDX);

%test(make=Cadillac, model=Escalade); * the choice of swinging golfers everywhere ;

* second approach - "tall" dataset ;

data metadata;

length macroname name $32 value $200;

input macroname name value;

datalines;

TEST1 FOO 1

TEST1 BAR 2

TEST1 BLAH 3

TEST2 WHATEVER 4

TEST2 IMDONE 5

;

run;

%macro createmvars(metadata=metadata,mname=);

%let mname=%upcase(&mname);

data _null_;

set &metadata;

where upcase(macroname)="&mname";

call symputx(name,value,"L");

run;

%&mname

%mend;

%macro test1;

proc sql;

select * from dictionary.macros where scope in ("CREATEMVARS","&sysmacroname");

quit;

%put &foo &bar &blah;

%mend;

%macro test2;

proc sql;

select * from dictionary.macros where scope in ("CREATEMVARS","&sysmacroname");

quit;

%put &whatever &imdone;

%mend;

%createmvars(mname=test1);

%createmvars(mname=test2);

Re: how do you manage hundreds of macro variables

ballardw — Wed, 20 Mar 2013 14:29:20 GMT

ZRick wrote:

Because we are developing production system using SAS.

We need to have the production system that is:

robust that is less error prone
scalable that can handle small / large volume of data
versatile that meet all kinds of different needs with different sets of parameters
portable that have the design of object oriented rather than procedure oriented

Once upon a time I would have suggested SAS FSP/AF as that product combination is object oriented.