topic Re: DOSUBL scope of returned macro variables in SAS Programming

DOSUBL scope of returned macro variables

Quentin — Mon, 26 Jan 2015 18:04:12 GMT

Hi All,

I'm playing around with DOSUBL. Trying to understand how macro variables created (or modified) in the DOSUBL environment are returned to the calling environment. (tested in 9.3M2 and 9.4M0 on Win7 PC SAS).

Hit one oddity that looks like a bug to me.

Conditions:

In main SAS session, macro variable &x exists in global symbol table and local symbol table, with different values.
In dosubl code, value of &x is assigned a new value.

Result

After dosubl has executed, the value of &x in the local symbol table has been set to the value of &x from the global symbol table. I would expect the value of &x in the local symbol table to be set to the value of &x returned from the DOSUBL environment.

Here is sample code and log:

1    %macro TestDoSubl(x=);
2      %local rc;
3
4      %let rc=%sysfunc(dosubl(%nrstr(
5        %let x=dosubl;
6        %put in DOSUBL environment;
7        %put _user_;
8        %put; %put; %put;
9      )));
10
11     %put after DOSUBL;
12     %put _user_;
13     %put; %put; %put;
14
15   %mend TestDoSubl;
16
17   %let x=global;
in DOSUBL environment
GLOBAL X dosubl
GLOBAL RC
TESTDOSUBL X local     <--- The X in the local symbol table has value local, as expected
TESTDOSUBL RC
GLOBAL X global

18   %TestDoSubl(x=local)
after DOSUBL
TESTDOSUBL X global     <--- After DOSUBL completed, X in the local table has value global.  I would expect it to have value dosubl. 
TESTDOSUBL RC 0
GLOBAL X global

19
20   %put after macro;
after macro
21   %put _user_;
GLOBAL X global

Observations:

In the DOSUBL, environment, the local macro variables that exist in main session are imported as global macro variables (fair). Note that the %PUT statement in the DOSUBL environment shows two different global macro variables named X. (This can also be seen when FCMP/run_macro is used to run code on the side).

After the DOSUBL has completed, I would expect it to return the global macro variable &x=dosubl back to the local symbol table. But it looks to me like DOSUBL gets confused, and instead is returning the other global macro variable &x=global back to the local symbol table.

Thoughts whether this is a bug, or if I'm not understanding the expected results?

So far the only other oddity I noticed is that if in the main environment, &x exists in the global symbol table but not the local symbol table, DOSUBL will update the global macro variable (fair) and also create a local macro variable with the same value (odd, but not much of a concern).

Thanks

Re: DOSUBL scope of returned macro variables

Quentin — Mon, 26 Jan 2015 19:19:43 GMT

Ack, seems my idea of using %NRSTR() may be part of the problem? I'm not sure how it effects the timing. I did notice that order of the log looks odd (results of first %put statement before the macro invocation). Will keep playing.

Re: DOSUBL scope of returned macro variables

jakarman — Mon, 26 Jan 2015 20:24:44 GMT

The function dosubl is called from within a running macro. In my vision that is a perfect reason to see all new macro vars as defined local.

As you have used x as a amco-function parameter from that point it is local SAS(R) 9.4 Macro Language: Reference, Third Edition not touching the global one anymore.

Re: DOSUBL scope of returned macro variables

Quentin — Tue, 27 Jan 2015 13:27:35 GMT

Hi Jaap,

I agree, that new macro vars should be local. And I am happy that the end result here is that the global macro variable X is not touched.

The surprise to me (and suspect bug), is after DOSUBL has completed, instead of it writing the value it computed for X to the local symbol table, somehow the value from the global symbol table is written to the local symbol table. That's the surprise. I will add a comment to the log to in my post to try to make this more clear.

Re: DOSUBL scope of returned macro variables

jakarman — Tue, 27 Jan 2015 17:54:11 GMT

Quentin you coded the macro var x also at: %macro TestDoSubl(x=);

At that moment it is becoming local defined by x=....

Re: DOSUBL scope of returned macro variables

Quentin — Tue, 27 Jan 2015 20:49:54 GMT

Yes, there is a local macro variable x that I passed in as a parameter, and that is the correct behavior.

There is also a global macro variable x that I created with a %LET statement in open code, and that is the correct behavior.

The problem is that the value of the local macro variable x becomes over written by the value of the global macro variable x after the DOSUBL has completed.

Re: DOSUBL scope of returned macro variables

jakarman — Wed, 28 Jan 2015 06:28:49 GMT

I see the following:

%let x=global; (x as global defined)

Anything referring to x will use the global definition.

%TestDoSubl(x=); (call/start Will active level1 of macro-processing start dedicated variables scope)

Anything referring to x will use this local definition.

Macro TestDoSubl will proceed to execute.

> %let x=dosubl; (from macro) will process x as local as the macro TestDoSubl is active (level1)

%TestDoSubl(x=); (call/end Will end level1 of macro processing removing variables belonging to this scope )

Only the original global macro-vars still there.

You can build up a nesting sequence of local-vars (explicitly local defines) or having automatic inheritance of the local to a caller.

It is a nesting scope of varaibles, SAS(R) 9.4 Macro Language: Reference, Third Edition.

Not being explained well in older doc's or very difficult to get when you are trying to learn by example. Seen more programming languages you could deduce this.

The same variable-name is intended to have different contenst depending on its scope.

Re: DOSUBL scope of returned macro variables

Quentin — Wed, 28 Jan 2015 11:00:55 GMT

Thanks for not giving up on the discussion, Jaap. But I don't think your seeing my point.

Yes, I understand the scoping rules of the macro language, and they are as you described them. Here is a shorter example of some test code to show the surprising result.:

%macro testNoDosubl(x=);
  %put x=&x;
%mend testNoDosubl;

%macro testWithDosubl(x=);
  %let rc=%sysfunc(dosubl(%nrstr(%let x=3;)));
  %put x=&x;
%mend testWithDosubl;

%let x=1;
%testNoDosubl(x=2)
%testWithDosubl(x=2)

When %testNoDosubl runs, what is the expected result shown in the log? It is 2. Because the %put statement returns the value of x from the local symbol table, not the global symbol table. That is the usual macro scoping rule. No surprise.

When %testWithDoSubl runs, what is the expected result? I'm not sure. But I would hope 3 (if the %let statement in the DoSubl block updated the local symbol table) or 2 (if it did not). But in fact it returns 1. What can explain why %testWithDoSubl returns 1?

My explanation would be that DoSubl is returning the wrong value to the local symbol table. If you add %put _global_; to the DoSubl block after the %let statement, it will show that in the 'side-session' where DoSubl runs, it can see two global macro variables named x. It sees the global macro variable x in the main session (with value 1), and the global macro variable x in the side-session (with value 3). I think DoSubl is intended to return the value from the side-session to the local symbol, but accidentally returns the value from the main session to the local symbol table.

Do you see another explanation for why %testWithDoSubl returns 1?

Re: DOSUBL scope of returned macro variables

jakarman — Wed, 28 Jan 2015 22:05:46 GMT

I see your issue now, it is not very clear to see. There are some new options that are very helpful with debugging.

During the scope of the macro the local var x get the value of the global version. That must be a bug.

42 ;

43 options mprint mprintnest mlogic mlogicnest symbolgen;

45 %macro testNoDosubl(x=);

46 %put x=&x %symlocal(x);

47 %mend testNoDosubl;

49 %macro testWithDosubl(x=);

50 %local rc;

51 %put x=&x %symlocal(x);

52 %let rc=%sysfunc(dosubl(%nrstr(%let x=3;)));

53 %put x=&x %symlocal(x);

54 %put _user_;

55 %mend testWithDosubl;

57 %macro testNESTsubl(x=);

58 %local rc;

59 %put x=&x %symlocal(x);

60 %let rc=%sysfunc(dosubl(%nrstr(%testWithDosubl(x=5)

61 )));

62 %put x=&x %symlocal(x);

63 %put _user_;

64 %mend testNESTsubl;

67 %let x=1;

68 %put x=&x %symlocal(x);

SYMBOLGEN: Macro variable X resolves to 1

x=1 0

69 %put _user_;

GLOBAL GRAPHINIT

GLOBAL GRAPHTERM

GLOBAL X 1

GLOBAL _CLIENTAPP SAS Studio

GLOBAL _CLIENTAPPVERSION 3.1

GLOBAL _SASPROGRAMFILE folders myshortcuts ikke macroscope_001.sas

GLOBAL _SASWSTEMP_ folders myfolders .images f9d75edd 6a9c 4b8e b177 abbba13bbe38

GLOBAL _SASWS_ folders myfolders

70 %testNESTsubl(x=4)

MLOGIC(TESTNESTSUBL): Beginning execution.

MLOGIC(TESTNESTSUBL): Parameter X has value 4

MLOGIC(TESTNESTSUBL): %LOCAL RC

MLOGIC(TESTNESTSUBL): %PUT x=&x %symlocal(x)

SYMBOLGEN: Macro variable X resolves to 4

x=4 1

MLOGIC(TESTWITHDOSUBL): Beginning execution.

MLOGIC(TESTWITHDOSUBL): Parameter X has value 5

MLOGIC(TESTWITHDOSUBL): %LOCAL RC

MLOGIC(TESTWITHDOSUBL): %PUT x=&x %symlocal(x)

SYMBOLGEN: Macro variable X resolves to 5

x=5 1

MLOGIC(TESTWITHDOSUBL): %LET (variable name is RC)

MLOGIC(TESTWITHDOSUBL): %PUT x=&x %symlocal(x)

SYMBOLGEN: Macro variable X resolves to 1

x=1 1

MLOGIC(TESTWITHDOSUBL): %PUT _user_

TESTWITHDOSUBL GRAPHINIT

TESTWITHDOSUBL GRAPHTERM

TESTWITHDOSUBL RC 0

TESTWITHDOSUBL X 1

TESTWITHDOSUBL _CLIENTAPP SAS Studio

TESTWITHDOSUBL _CLIENTAPPVERSION 3.1

TESTWITHDOSUBL _SASPROGRAMFILE folders myshortcuts ikke macroscope_001.sas

TESTWITHDOSUBL _SASWSTEMP_ folders myfolders .images f9d75edd 6a9c 4b8e b177 abbba13bbe38

TESTWITHDOSUBL _SASWS_ folders myfolders

GLOBAL RC

GLOBAL X 4

TESTNESTSUBL RC

TESTNESTSUBL X 4

GLOBAL GRAPHINIT

GLOBAL GRAPHTERM

GLOBAL X 1

GLOBAL _CLIENTAPP SAS Studio

GLOBAL _CLIENTAPPVERSION 3.1

GLOBAL _SASPROGRAMFILE folders myshortcuts ikke macroscope_001.sas

GLOBAL _SASWSTEMP_ folders myfolders .images f9d75edd 6a9c 4b8e b177 abbba13bbe38

GLOBAL _SASWS_ folders myfolders

MLOGIC(TESTWITHDOSUBL): Ending execution.

MLOGIC(TESTNESTSUBL): %LET (variable name is RC)

MLOGIC(TESTNESTSUBL): %PUT x=&x %symlocal(x)

SYMBOLGEN: Macro variable X resolves to 1

x=1 1

MLOGIC(TESTNESTSUBL): %PUT _user_

TESTNESTSUBL RC 0

TESTNESTSUBL X 1

GLOBAL GRAPHINIT

GLOBAL GRAPHTERM

GLOBAL X 1

GLOBAL _CLIENTAPP SAS Studio

GLOBAL _CLIENTAPPVERSION 3.1

GLOBAL _SASPROGRAMFILE folders myshortcuts ikke macroscope_001.sas

GLOBAL _SASWSTEMP_ folders myfolders .images f9d75edd 6a9c 4b8e b177 abbba13bbe38

GLOBAL _SASWS_ folders myfolders

MLOGIC(TESTNESTSUBL): Ending execution.

72 %put x=&x %symlocal(x);

SYMBOLGEN: Macro variable X resolves to 1

x=1 0

73 %put _user_;

GLOBAL GRAPHINIT

GLOBAL GRAPHTERM

GLOBAL X 1

GLOBAL _CLIENTAPP SAS Studio

GLOBAL _CLIENTAPPVERSION 3.1

GLOBAL _SASPROGRAMFILE folders myshortcuts ikke macroscope_001.sas

GLOBAL _SASWSTEMP_ folders myfolders .images f9d75edd 6a9c 4b8e b177 abbba13bbe38

GLOBAL _SASWS_ folders myfolders

75 ;

Re: DOSUBL scope of returned macro variables

Quentin — Thu, 29 Jan 2015 15:09:51 GMT

Thanks again Jaap. Glad you agree. I think I'll try posting this to SAS-L as well, to see if I can prompt more discussion there. Then will send it in to tech support and/or Rick Langston directly (pretty sure DOSUBL is his creation). I really like his last example of using DOSUBL for function-style macros (https://support.sas.com/resources/papers/proceedings13/032-2013.pdf), so feel like it's worth time to test out some of the magic involved in moving things back and forth between the main session and the side-session.

Re: DOSUBL scope of returned macro variables

pchegoor — Sat, 26 Sep 2015 13:16:20 GMT

Has this Issue been fixed ? Does anyone know?

Re: DOSUBL scope of returned macro variables

Quentin — Sat, 26 Sep 2015 20:39:40 GMT

I'm not sure. After posting this I followed up with Rick Langston, and he agreed it was a bug and said he was working on a hot-fix. I'm not sure if it was going to be an official hot-fix or if it would perhaps be slipped into a maintenance release.

Perhaps folks who have a fully hot-fixed version of 9.4M3 could try the example I posted and let us know?

I just ran it in AWS SAS Studio (9.04.01M3P06242015), and got the same scope collision I reported above. I'm not sure how hot-fixed the AWS version is.

Re: DOSUBL scope of returned macro variables

pchegoor — Mon, 28 Sep 2015 18:17:37 GMT

@Quentin I contacted SAS Support on the Status of this Issue and they responded saying this is a difficult issue to fix via a HOTFIX in SAS 9.4M3 and therefore will be planned for a future release of SAS. No timetable has been set for this however.

Also i have heard from one of my Colleauges at Work that SAS may be coming out with Release 10 next year instead of 9.4M4 or 9.5. Not sure how much of this is True.Need to wait and See.

Re: DOSUBL scope of returned macro variables

Quentin — Tue, 29 Sep 2015 15:29:10 GMT

Thanks @pchegoor sorry to hear there is no timetable, but good to know it's still on their list of things to work on. I think there be magic in this "side-session" processing, but until I have a better understanding of how the main environment and side environment communicate with each and inherit from each other, I don't acticipate using it for production code. But certainly the promise/potential of DOSUBL is very exciting.

Good to hear the rumor about SAS 10. Hope for 10 will be more motivation to hit Vegas for SGF2016.

Re: DOSUBL scope of returned macro variables

pchegoor — Thu, 15 Dec 2016 03:59:04 GMT

@Quentin I am happy to Report that the DOSUBL Macro variable Scope Issue has been fixed in SAS 9.4M4. I have run the below Codes on SAS University Edition with 9.4M4.

%let a = 1;
%let b = 2;
%let c = 3;

%macro test2;
	%local a b c;
	%let a = 10;
	%let b = 20;
	%let c = 30;
	%put BEFORE DOSUBL: a is &a, b is &b, c is &c;
	%let rc = %sysfunc(dosubl(%str(data _null_;run; )));
	%put AFTER DOSUBL: a is &a, b is &b, c is &c;
%mend test2;

%test2

Result :

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

62 %let a = 1;

63 %let b = 2;

64 %let c = 3;

66 %macro test2;

67 %local a b c;

68 %let a = 10;

69 %let b = 20;

70 %let c = 30;

71 %put BEFORE DOSUBL: a is &a, b is &b, c is &c;

72 %let rc = %sysfunc(dosubl(%str(data _null_;run; )));

73 %put AFTER DOSUBL: a is &a, b is &b, c is &c;

74 %mend test2;

76 %test2

BEFORE DOSUBL: a is 10, b is 20, c is 30

NOTE: DATA statement used (Total process time):

real time 0.00 seconds

cpu time 0.01 seconds

AFTER DOSUBL: a is 10, b is 20, c is 30

78 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

%macro testNoDosubl(x=);
  %put x=&x;
%mend testNoDosubl;

%macro testWithDosubl(x=);
  %let rc=%sysfunc(dosubl(%nrstr(%let x=3; )));
  %put x=&x;
%mend testWithDosubl;

%let x=1;
%testNoDosubl(x=2)
%testWithDosubl(x=2)

Result :

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

62 %macro testNoDosubl(x=);

63 %put x=&x;

64 %mend testNoDosubl;

66 %macro testWithDosubl(x=);

67 %let rc=%sysfunc(dosubl(%nrstr(%let x=3; )));

68 %put x=&x;

69 %mend testWithDosubl;

71 %let x=1;

72 %testNoDosubl(x=2)

x=2

73 %testWithDosubl(x=2)

x=3

    %macro TestDoSubl(x=);
      %local rc;

      %let rc=%sysfunc(dosubl(%nrstr(
        %let x=dosubl;
        %put in DOSUBL environment;
        %put _user_;
        %put; %put; %put;
      )));

    %put after DOSUBL;
     %put _user_;
     %put; %put; %put;

   %mend TestDoSubl;

   %let x=global;
   
   %TestDoSubl(x=local)

Result :

  1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
61
62             %macro TestDoSubl(x=);
63               %local rc;
64
65               %let rc=%sysfunc(dosubl(%nrstr(
66                 %let x=dosubl;
67                 %put in DOSUBL environment;
68                 %put _user_;
69                 %put; %put; %put;
70               )));
71
72             %put after DOSUBL;
73              %put _user_;
74              %put; %put; %put;
75
76            %mend TestDoSubl;
77
78            %let x=global;

in DOSUBL environment
GLOBAL    X     dosubl
TESTDOSUBL   RC
TESTDOSUBL   X   local

GLOBAL X global

after DOSUBL
TESTDOSUBL RC 0
TESTDOSUBL X dosubl

GLOBAL X global

Re: DOSUBL scope of returned macro variables

Quentin — Thu, 15 Dec 2016 14:49:12 GMT

Thanks @pchegoor

That is very exciting news. Maybe enough for me to renew my personal AWS subscription so I can play with this a bit in university edition (probably still a couple months away from having 9.4M4 at work).

The management of the DOSUBL side-sesssion environment / inheritance (not only macro variable scope, but system option scope, etc... ) must be very complex. I'm glad they have fixed this macro variable collision. Hope to be playing with it soon.

There's such amazing potential in DOSUBL, I'm glad they're putting in the development effort for fixes.

Re: DOSUBL scope of returned macro variables

pchegoor — Thu, 15 Dec 2016 15:46:19 GMT

@Quentin On a slightly different Topic. I wonder why you need AWS for SAS University Edition. You can directly download it on your Machine instead of using it via the cloud on AWS and save money on the subscription. Also if you prefer a Cloud based version of Free SAS Software then you do have the SAS On Demand for Academics . This one has SAS/GRAPH included too plus additional SAS products. The Comparision of SAS On Demand for Academics and SAS University Edition is given here

Right now SAS On Demand for Academics this is still at 9.4M3 but i am hoping by the end of Jan 2017 or so it may be also be updated to 9.4M4 . May be @AmyP_sas can comment on when this may occur.

Re: DOSUBL scope of returned macro variables

fury88 — Fri, 16 Dec 2016 19:44:37 GMT

I'm having a problem which I think MAY be related to this bug but I'm not sure because I'm venturing into new SAS coding territory!

I'm using a combination of PROC STREAM, a macro template to spit out XML, and inside the template calling a macro that has a logic to set a global variable which will get used after the dosubl call.

/** Something similar to this **/

proc stream outfile=REQUEST
begin%due_date_template(...)
;;;;
run;

.
.
.

/** sub section of the due_date_template macro **/
%do i=1 %to &numAlerts;
.
.
%let rc= %sysfunc(dosubl('due_date_macro(var1,var2,var3...)'));
.
.
.
/** inside the due_date_macro, there is logic to set a global variable called 
    due_date
**/
%put due_date = &due_date;
%end;

/**
  1st time through the due_date gets set properly
  the remaining times in the iteration the due_date remains the initial value set
  by the dosubl call even though a new due_date is being calculated.
**/

Is this part of the bug in M3? I've tried everything to get that &due_date macro to update after the dosubl call but it always remains the first value set!

-Matt

Re: DOSUBL scope of returned macro variables

pchegoor — Sat, 17 Dec 2016 01:08:04 GMT

@fury88

Yes it looks like it is due to the DOSUBL bug in 9.4M3 .

I ran the below code both on 9.4M3 and 9.4M4 :

%macro due_date_macro(flag);
 
%Put  &=flag;
 
%Let due_date=&flag;
 
%Put  INNER : &=due_date;
 
%mend  due_date_macro;
 
 
%Let  Due_Date=;
%Let Var1=20120304;
%Let Var2=20120805;
%Let Var3=20150709;
 
 
%Macro  Test;
 
%do i=1 %to &numAlerts;
 
%let rc= %sysfunc(dosubl('%due_date_macro(&&Var&i)'));
 
%put OUTER : &=due_date;
%put **************************;
%end;
 
%Mend  Test;
 
 
%Let NumAlerts=3;
%Test;

Result from SAS 9.4M3:

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

56 %macro due_date_macro(flag);

58 %Put &=flag;

60 %Let due_date=&flag;

62 %Put INNER : &=due_date;

64 %mend due_date_macro;

67 %Let Due_Date=;

68 %Let Var1=20120304;

69 %Let Var2=20120805;

70 %Let Var3=20150709;

73 %Macro Test;

75 %do i=1 %to &numAlerts;

77 %let rc= %sysfunc(dosubl('%due_date_macro(&&Var&i)'));

79 %put OUTER : &=due_date;

80 %put **************************;

81 %end;

83 %Mend Test;

86 %Let NumAlerts=3;

FLAG=20120304

INNER : DUE_DATE=20120304

87 %Test;

OUTER : DUE_DATE=

**************************

FLAG=20120805

INNER : DUE_DATE=20120805

OUTER : DUE_DATE=

**************************

FLAG=20150709

INNER : DUE_DATE=20150709

OUTER : DUE_DATE=

**************************

Result from SAS 9.4M4 :

62 %macro due_date_macro(flag);
63
64 %Put &=flag;
65
66 %Let due_date=&flag;
67
68 %Put INNER : &=due_date;
69
70 %mend due_date_macro;
71
72
73 %Let Due_Date=;
74 %Let Var1=20120304;
75 %Let Var2=20120805;
76 %Let Var3=20150709;
77
78
79 %Macro Test;
80
81 %do i=1 %to &numAlerts;
82
83 %let rc= %sysfunc(dosubl('%due_date_macro(&&Var&i)'));
84
85 %put OUTER : &=due_date;
86 %put **************************;
87 %end;
88
89 %Mend Test;
90
91
92 %Let NumAlerts=3;
FLAG=20120304
INNER : DUE_DATE=20120304
93 %Test;
OUTER : DUE_DATE=20120304
**************************
FLAG=20120805
INNER : DUE_DATE=20120805
OUTER : DUE_DATE=20120805
**************************
FLAG=20150709
INNER : DUE_DATE=20150709
OUTER : DUE_DATE=20150709
**************************