topic When user-written functions call a macro that executes steps, what environment do they execute in? in SAS Programming

When user-written functions call a macro that executes steps, what environment do they execute in?

Quentin — Thu, 28 Jun 2012 19:14:38 GMT

Warning: long.

Hi All,

I was inspired by some recent papers showing how you can use PROC FCMP to write a function or subroutine, and have that function include a macro call which runs several steps, the end result being that you can have a data step, which has a function call in it which runs other data steps. Wow I said that badly. The papers are:

http://support.sas.com/resources/papers/proceedings12/004-2012.pdf

http://support.sas.com/resources/papers/proceedings10/326-2010.pdf

The magic is you can end up with something like:

data a;

set sashelp.class;

x=standardize(age);

run;

And the user-written standardize() function could call a macro which has data steps/proc steps/ etc to compute a standardized age, and returns the value. So the magic is you have one datastep that can call other data steps. Actually the magic of the first paper is you can wrap this in a macro function and do something like: %put %computemean(data=sashelp.class,var=age); and your macro calls a user-written function which calls a macro which executes data steps or procs.

So I got to wondering how this could work, that is, what environment do the called data steps execute in? That is, how is the environment where the called data steps execute different than (and encapsulated from ) the global environment? I haven't been able to find much in the docs about this. I feel like I need to undertstand the environment in order to know whether I should worry about collisions, inheritance, etc. For example, could datasets created in the local environment collide with the global environement? Are system options defined in the global environment inherited by the local environment?

I ran some tests, and was surprised by some of the findings. Soren Lassen was kind enough to develop a better test script, which I have pasted below (slightly modified). The script basically sets options, titles, and global macro variables in the global environment, then uses the PROC FCMP run_macro approach to repoort on the local environment where any data steps written by the macro execute. It doesn't actually change anything in the local environment (though I played with that too).

My findings from running this (on 9.3, results were different for people that ran it on 9.2) are:

1. Some global system options are inherited by the local enviroment, but some are not. See below, the global OBS option is not inherited by the local ("during") environment. The global MERGENOBY option is inherited by the local environment.

2. The global title statement is not inherited by the local environment (I think I have seen this mentioned in papers/posts).

3. It is possible for the local environment to end up with 2 global macro variables with same name, with different values (scary, right? : )

4. The datasets created in the global environment do exist in the local environment (not terribly surprising, but good to know)

Does anyone know how this FCMP run_macro magic works? Is FCMP or run_macro somehow creating an (almost) independent envrionment/session for steps to run in (so gets it's own dictionary.titles, it's own system options, it's own macro symbole tables etc?) Or is there only one environment, and SAS is basically saving (most) options/titles/etc, resetting (some) of them, and then restoring them after the function call?

Script, followed by excerpts from the log. I've tried to color-code the log to show what I'm looking at. Would be greatful for thoughts on this.

Test script:

%macro show(when);   
  %put &when: Options;
  %put ---------------------------------;
  proc options option=obs;run;
  proc options option=mergenoby;run;
  %put ---------------------------------;
  %put;
  %put &when: Macro variables G:;
  %put ---------------------------------;
  data _null_;
    set sashelp.vmacro;
    where name='G';
    put (name scope offset value)(=);
  run;
  %put ---------------------------------;
  %put;
  %put &when: Title;
  %put ---------------------------------;
  data _null_;
    set sashelp.vtitle;
    where number IN (1);
    put text=;
  run;
  %put ---------------------------------;
  %put;
  %put &when: Data MyData;
  %put ---------------------------------;
  data _null_;
    set mydata;
    put _ALL_;
  run;
  %put ---------------------------------;
  %put;
  %put;
  %put;
  %put;
%mend;

%macro test;
  %show(DURING);
%mend;


proc fcmp outlib=work.funcs.test;
  function test();
    G='TEST';
    rc=run_macro('test',g); /*run_macro calls %test, and creates macro var &g with value TEST*/
    return(rc);
    endsub;
run;

options cmplib=work.funcs obs=333 mergenoby=warn mprint;
%let g=GLOBAL;
Title 'GLOBAL';
data mydata;
  y='GLOBAL';
run;
%show(BEFORE)
data _null_;
  x=test();  /* test() invokes %test which calls %show to show the local environment, labeled DURING */
run;
%show(AFTER)

Log:

BEFORE: Options
---------------------------------
    SAS (r) Proprietary Software Release 9.3  TS1M0

 OBS=333           Number of the last observation to process MERGENOBY=WARN    
 Action for DATA step MERGE statement with no associated BY statement
  ---------------------------------

BEFORE: Macro variables G:
---------------------------------
name=G scope=GLOBAL offset=0 value=GLOBAL 
 ---------------------------------

BEFORE: Title
---------------------------------
text=GLOBAL
  ---------------------------------

BEFORE: Data MyData
---------------------------------
y=GLOBAL _ERROR_=0 _N_=1
---------------------------------

DURING: Options
---------------------------------
    SAS (r) Proprietary Software Release 9.3  TS1M0

 OBS=9223372036854775807                   Number of the last observation to process 
 MERGENOBY=WARN    Action for DATA step MERGE statement with no associated BY statement 
 ---------------------------------

DURING: Macro variables G:
---------------------------------
name=G scope=GLOBAL offset=0 value='TEST' 
name=G scope=GLOBAL offset=0 value=GLOBAL 
 ---------------------------------

DURING: Title
---------------------------------
text=The SAS System 
 ---------------------------------

DURING: Data MyData
---------------------------------
y=GLOBAL _ERROR_=0 _N_=1
---------------------------------

AFTER: Options
---------------------------------
    SAS (r) Proprietary Software Release 9.3  TS1M0

 OBS=333           Number of the last observation to process
 MERGENOBY=WARN    Action for DATA step MERGE statement with no associated BY statement

---------------------------------

AFTER: Macro variables G:
---------------------------------
name=G scope=GLOBAL offset=0 value=GLOBAL
---------------------------------

AFTER: Title
---------------------------------
text=GLOBAL
---------------------------------

AFTER: Data MyData
---------------------------------
y=GLOBAL _ERROR_=0 _N_=1
---------------------------------

Thanks,

--Q.

Re: When user-written functions call a macro that executes steps, what environment do they execute in?

Quentin — Thu, 05 Jul 2012 16:43:44 GMT

Hi All,
I've been playing around more, still trying to understand the environment in which code executes when you have a user-written function which calls run_macro() to execute a macro which generates data steps. I've also had some very helpful input from a little birdie, so wanted to share. (Any mistakes below are my own, I haven't separated what I was told from what I have deduced/guessed...)

When I heard that a data step could call a function which executed another data step, one of my first thoughts was "Well, where does that second data step run?". You have a global SAS session which has system options, a macro symbol table, titles, etc. As a sub-routine, does the code executed by run_macro excecute in that same environment? Is there inheritance? Could there be collissions?

So here is what I think I have figured out.

When a DATA step that invokes a user-defined function that calls run_macro is compiled, a local run_macro environment is initialized which is shared by all user functions executed within that step. This environment is local to the DATA step, and persists for the duration of the DATA step. Some attributes of this local environment are inherited from the global SAS session, and others are not. Some attributes of the local environment persist during the data step, and others are initalized each time a user function is executed. Some changes made in the local environment will collide with the global environment, and some will not.

Helpful explanation, right?

Test script below is basically:
1. Create global session environment: titles, options (OBS, MERGENOBY), macro vars, data set.
2. Data step which calls 2 user-written functions. The functions report on the environment in which they execute, change the environment, and report on the changed environment.
3. Report on global session environmment at the end

And here is what I see [9.3 (TS1M0), results differ in 9.2]:
1. TITLES. The run_macro environment does not inherit titles from the global environment. Titles are initialized to "The SAS System" when the run_macro environment is initalized. Titles persist within the run_macro evironment (so two functions called in the same step, or one function called on two iterations of the same step, share the same title space). Changes made to the titles in the run_macro environment do not collide with the global environment.

2. MERGENOBY. The run_macro environment inherits option MERGENOBY from the global environment. MERGENOBY setting persists within the run_macro environment. Changes made to MERGENOBY in the run_macro environment do not collide witht the global environment.

3. OBS. The run_macro environment does not inherit option OBS from the global environment. Obs is reset to OBS=MAX every time run_macro is called. So the OBS option setting does NOT persist within run_macro evironment. Changes made to the OBS in the run_macro environment do not collide with the global environment.

4. GLOBAL MACRO VARS. The run_macro environment inherits global macro vars from the global session environment (or perhaps shares the same symbol table?). If a macro variable exists in the global session environment, and the value is changed in the run_macro environment, the change will persist, and will collide with the global environment (MVAR2 below). If a macro variable is created in the run_macro evironment, it will not persist within the run_macro environment (see MVAR3 below, which does is created in the first function call, but does not exist in the initial environment for the second function call, or the global environment at the end). Interestingly, macro variables created by the run_macro call itself do not collide with the global environment, but instead lead to scenario in which the symbol table of run_macro environment has two macro variables with the same name in the same scope (MVAR below).

5. Data Sets. Looks like the run_macro environment uses the same WORK library as the global environment, so data sets are inherited, they persist, and will collide.

The above is my understanding of the environment created for user functions when they are called inside a DATA step (and I assume the same for PROC steps, but haven't tested). When a user function is called outside of a datastep, e.g. by %sysfunc(), the rules are different. It looks like a run_macro is environment is created, but more attributes seem to persist accross function calls. For example, titles persist. Birdie tells me that this should change in next 9.3 maintenance release. I think the run_macro environment created when run_macro is invoked by %Sysfunc() is independent of the environment created by a data step.

That is about all that I think I know. My test script is below. I do think this is neat stuf, especially as featured in Mike Rhoads' paper on the Macro Function Sandwich. I haven't been able to find much in the documentation or elsewhere on these scoping issues for run_macro(), and clearly this is still new functionality, and the rules are evolving. Hopefully SAS (or someone else) will be able to put out a better description than I have tried to cobble together, but wanted to start the ball rolling.

Warm Regards,
--Q.

%macro GetEnvironment
(when=
,out=Environment
);

data __tit (keep=var text rename=(text=Value));
set sashelp.vTitle;
var=catt("Title",number);
where type='T';
run;

data __opts(keep=optname setting rename=(optname=Var setting=Value));
set sashelp.vOption;
where optname IN ("MERGENOBY","OBS");
run;

data __mvars(keep=name value rename=(name=Var));
set sashelp.vmacro;
where scope ne "AUTOMATIC" and name=: "MVAR";
run;

data __mydata;
set mydata;
var="MyData";
run;

data __all;
length Var When $40 ;
set
      __tit
      __opts
      __mvars
      __mydata
;
When="&when";
run;

proc append base=&out data=__all;
run;

proc datasets library=work memtype=data;
delete __tit __opts __mvars __mydata __all;
run;
quit;

%mend GetEnvironment;

%macro test1;
%GetEnvironment(when=Function 1 Initial Environment)

%*Change Environment;
title1 "Function 1 Title";
options obs=222 mergenoby=error;
%let mvar2=Function1 MVAR2 revised by macro; %*scope not declared;
%let mvar3=Function1 MVAR3 created in macro; %*scope not declared;
data mydata;
Value="Function1 work.mydata";
run;

%GetEnvironment(when=Function 1 Changed Environment)
%mend;

%macro test2;
%GetEnvironment(when=Function 2 Initial Environment)

%*Change Environment;
title1 "Function 2 Title";
options obs=333 mergenoby=nowarn;
%let mvar2=Function2 MVAR2 revised by macro; %*scope not declared;
%let mvar3=Function2 MVAR3 created in macro; %*scope not declared;
data mydata;
Value="Function2 work.mydata";
run;

%GetEnvironment(when=Function 2 Changed Environment)
%mend;

proc fcmp outlib=work.funcs.test;
function func1();
    mvar='Func1 MVAR';
    rc=run_macro('test1',mvar); /*run_macro calls %test1, and creates macro var &mvar with value Func1 MVAR*/
    return(rc);
endsub;
function func2();
    mvar='Func2 MVAR';
    rc=run_macro('test2',mvar); /*run_macro calls %test2, and creates macro var &mvar with value Func2 MVAR*/
    return(rc);
endsub;
run;

options cmplib=work.funcs varlenchk=nowarn nocenter;

*Create initial Global envionment;
Title1 "Global Title";
options obs=111 mergenoby=warn;
%let mvar=Global Macro Var;
%let mvar2=Another Global Macro Var;
data mydata;
Value="Global work.mydata";
run;

%GetEnvironment(when=Global Session Beginning)

data _null_;
x=func1();
y=func2();
run;

%GetEnvironment(when=Global Session End)

proc print data=Environment noobs;
by When notsorted;
run;

=================== Output =====================

When=Global Session Beginning

Var Value

Title1       Global Title
MERGENOBY    WARN
OBS          111
MVAR         Global Macro Var
MVAR2        Another Global Macro Var
MyData       Global work.mydata

When=Function 1 Initial Environment

Var Value

Title1       The SAS System
MERGENOBY    WARN
OBS          9223372036854775807
MVAR         'Func1 MVAR'
MVAR         Global Macro Var
MVAR2        Another Global Macro Var
MyData       Global work.mydata

When=Function 1 Changed Environment

Var Value

Title1       Function 1 Title
MERGENOBY    ERROR
OBS          222
MVAR3        Function1 MVAR3 created in macro
MVAR         'Func1 MVAR'
MVAR         Global Macro Var
MVAR2        Function1 MVAR2 revised by macro
MyData       Function1 work.mydata

When=Function 2 Initial Environment

Var Value

Title1       Function 1 Title
MERGENOBY    ERROR
OBS          9223372036854775807
MVAR         'Func2 MVAR'
MVAR         Global Macro Var
MVAR2        Function1 MVAR2 revised by macro
MyData       Function1 work.mydata

When=Function 2 Changed Environment

Var Value

Title1       Function 2 Title
MERGENOBY    NOWARN
OBS          333
MVAR3        Function2 MVAR3 created in macro
MVAR         'Func2 MVAR'
MVAR         Global Macro Var
MVAR2        Function2 MVAR2 revised by macro
MyData       Function2 work.mydata

When=Global Session End

Var Value

Title1       Global Title
MERGENOBY    WARN
OBS          111
MVAR         Global Macro Var
MVAR2        Function2 MVAR2 revised by macro
MyData       Function2 work.mydata

Re: When user-written functions call a macro that executes steps, what environment do they execute in?

chang_y_chung_hotmail_com — Thu, 05 Jul 2012 18:19:45 GMT

It is not fair to compare SAS to "free" software, but still..."R has been lexically scoped and has supported function closures since day one." -- Darren Wilkinson