Why is CALL SYMPUT so fickle?

[rsbmore]

Hello SAS community!

I created a macro to help me rename variables by groups. The only problem is sometimes it works and sometimes it doesn't. The main issue is the first CALL SYMPUT. The creation of the "cw" variable always works, but the CALL SYMPUT doesn't always pull in the value. Instead when I run "%put &c." I just get "&c" in the log. Because the macro is never created, I get the following error: 

ERROR: A character operand was found in the %EVAL function or %IF condition where a numeric operand is required. The condition was: &c.
ERROR: The %TO value of the %DO I loop is invalid.
ERROR: The macro RENAME will stop executing.

I have no clue what the root of the issue is. I'll tweak the code and it'll run fine. Then to test the new code I will close out SAS and reopen the program and run the same code that just worked. I always get the same error. Hopefully, this is a simple problem! The code below has run successfully and unsuccessfully with no changes. Thanks for all of your help!

%macro rename(oldvarlist,suffix);
    length prefix var newvar $ 32.;
    cw = countw("&oldvarlist",' ');

    put cw = ;
    call symputx('c',cw);
    %put &c.;
    %let k=1;
    %let old = %scan(&oldvarlist, &k);
        %do i = 1 %to &c.;
            l = anylower("&old");
            u = anyupper("&old");
            var = left(substr("&old",(l-1)));
            prefix = left(substr("&old",u,l-2));
            newvar = strip(catx("_",prefix,"&suffix",var));
            call symput("&suffix.&i.",newvar);
            %put &&&suffix.&i.;
            rename &old = &&&suffix.&i.;
          %let k = %eval(&k + 1);
          %let old = %scan(&oldvarlist, &k);
        %end;
drop cw l u var prefix newvar;
%mend;

options mprint symbolgen;

data _null_;  %rename(PAForms PALevelChange PAGrad PAMilestone PAFuf PABenefits, admn); run;

[PaigeMiller]

There is probably no way anyone can answer you question unless you tell us the value of &oldvarlist when you get this error.

[rsbmore]

Here is an example:

%rename(PAForms PALevelChange PAGrad PAMilestone PAFuf PABenefits, admn);

so &oldvarlist would be "PAForms PALevelChange PAGrad PAMilestone PAFuf PABenefits" and cw equals 6.

Thanks!

[Reeza]

I'm going to make a general guess. 

I think you likely have some scoping issues. Notice that you use CALL SYMPUTX but don't specify the variable is local or global and so reusing that same macro variable for multiple calls could be problematic if you use it elsewhere. Is this at all possible?

[rsbmore]

Thanks, reeza! I tried with and without specifying the macro variable as local and haven't found a difference. Should I try global? Since I will be running this macro multiple times with different values I wasn't sure that assigning "c" as global was the right choice.

[Tom]

I don't know what error you are having with that macro since it is only generating parts of a data step there is no way to run it as it stands.

But this little sequence shows a clear miss understanding of macro processing.

call symputx('c',cw);
put c =;
%put &c.;

It is quite a nice little simple example of a number of misconceptions

Let's see what the three statements are doing. First is setting the macro variable C to the value of data set variable CW.  The second is printing the value of different variable, the data set variable C, to the log. And the last is printing the value of the macro variable C to the log. But since this block of code will obviously run in the middle of a data step the value that it prints while SAS is processing the macro logic and building up the code of the data step will have nothing to do with the value that will eventually be put into C by the CALL SYMPUTX() statement in the first line.  The first two lines will execute once the data step starts running, which will be long after the %PUT statement has executed.

This shows a misunderstanding of the differences between a macro variable and variable reference.  It shows a misunderstanding of the order that SAS processes macro code and code in general.

When SAS processes code it first evaluates the macro triggers and generates the code that they resolve to. That code is then passed onto the normal SAS interpreter/compiler to be executed.  When SAS sees that it has the complete set of code needed for it to be run (a DATA step,  a PROC step or a statement (proc sql) or block of statements (proc glm, etc.)) then it runs it.  

If you can explain what you are trying to do (and provide an example, with input data, that users can run) then perhaps someone can help solve your original problem.

[rsbmore]

Correct, in my program the code is within a data step. Although, input data isn't needed to test this out since I'm only using text strings to rename variables.

Run the macro and then the code below. You will be able to test the function of the macro by reviewing the log (assuming you've turned on symbolgen and mprint), even though no concrete data has been used.

data _null_;  %rename(PAForms PALevelChange PAGrad, admn); run;

The goal is below. If one were to hard code "&c" in the do loop as the count of the number of variables in &oldvarlist, the code would run fine.

rename PAForms = PA_admn_Forms

rename PALevelChange = PA_admn_LevelChange

rename PAGrad = PA_admn_Grad

And the misplacement and incorrect assignment of the put statements shows my exhaustion in tweaking my code so much that I forgot to add a w after c in the first one!

Thanks!

[Quentin]

You've got a mix of data step language code and macro code.  As  others have pointed out, you're trying to use call symput in the data step to generate a macro variable, and then you are trying to resolve the macro variable in the same step.  That won't work, because macro variable references are resolved before the data step code executes (before data step code is even compiled).  It's a timing issue.  If it feels like it works "sometimes" it could be because you created a global macro variable, so after it runs once the second time it might run without error, because the macro variable was created the first time.  But it's still not working as you've designed it. 

One option would be to use pure macro code. It sounds like you want to give it a list of variables, and then return a rename list where a prefix has been added.  It looks like your logic says to insert the prefix two characters before the first lower case variable in a variable name.  You can do that with macro code.  Something like:

%macro rename(oldvarlist,prefix);
  %local i oldvar newvar ;
  %do i=1 %to %sysfunc(countw(&oldvarlist,%str( ))) ;
    %let oldvar=%scan(&oldvarlist,&i) ;
    %let FirstLower=%sysfunc(anylower(&oldvar)) ; %*position of first lowercase character ;  
                        
    %let NewVar=%substr(&oldVar,1,%eval(&FirstLower-2))_&prefix._%substr(&oldVar,%eval(&FirstLower-1)) ;

    %Put DEBUGGING: &=OldVar &=FirstLower &=NewVar;

    /*return code to rename one variable*/
    &oldvar=&newvar 

  %end ;
%mend rename ;

You could use it like:

%put %rename(PAForms PALevelChange PAGrad PAMilestone PAFuf PABenefits ABCDefg ABCDEfg ABCDEFg, admn) ;

data want ;
  set have(rename=(%rename(PAForms PALevelChange PAGrad PAMilestone PAFuf PABenefits, admn))) ;
run ;

The code will error if the the first lower case letter is in the first or second character, and probably other stuff, but this is one way to approach it.

[Tom]

---

@rsbmore wrote:

Correct, in my program the code is within a data step. Although, input data isn't needed to test this out since I'm only using text strings to rename variables.

Run the macro and then the code below. You will be able to test the function of the macro by reviewing the log (assuming you've turned on symbolgen and mprint), even though no concrete data has been used.

data _null_;  %rename(PAForms PALevelChange PAGrad, admn); run;

The goal is below. If one were to hard code "&c" in the do loop as the count of the number of variables in &oldvarlist, the code would run fine.

rename PAForms = PA_admn_Forms

rename PALevelChange = PA_admn_LevelChange

rename PAGrad = PA_admn_Grad

And the misplacement and incorrect assignment of the put statements shows my exhaustion in tweaking my code so much that I forgot to add a w after c in the first one!

Thanks!

---

So I think I can translate this into a problem description and then we can propose some solutions.  It looks like you want a way to take a list of variable names and generate RENAME statements.  The individual variable names are in the form PAxxxx and the new names are in the form PA_yyy_xxxx where YYY is a separately supplied list.

So one way to code a macro for that is make one that just generates the OLD=NEW values.  You could then use it in either a RENAME statement or with the RENAME= dataset option.

%macro rename
/* Generate OLD=NEW variable list */
(varlist  /* Space delimited list of variable names */
,insert   /* word to insert to form new names */
);
%local i var;
%do i=1 %to %sysfunc(countw(&varlist,%str( )));
  %let var=%scan(&varlist,&i,%str( ));
 &var=%substr(&var,1,2)_&insert._%substr(&var,3)
%end;
%mend rename ;

Here is an example usage.

84    options mprint;
85    data want ;
86      set have ;
87      rename %rename(PAForms PALevelChange PAGrad, admn);
MPRINT(RENAME):   PAForms=PA_admn_Forms PALevelChange=PA_admn_LevelChange PAGrad=PA_admn_Grad
88    run;

[Rick_SAS]

Maybe I am missing something, but macro variables that are set by CALL SYMPUT aren't copied into the global macro symbol table until the DATA step exits.

Can you guess what the following program does BEFORE you run it?

%let c = 1;

data a;
input cw @@;
call symputx('c',cw);
%put &=c;
do i = 1 to &c;
   y = i;
   output;
end;
datalines;
2 3 4
;

%put &=c;

proc print; run;

For this DATA step, the loop is constantly equal to 1, which is the value of &c BEFORE the DATA step runs. The value of &c does not change until after the DATA step exits, at which time it becomes 4.

[Quentin]

Nice example, @Rick_SAS.  If you don't mind me being picky, with SYMPUTX the value of the macro variable C changes while the data step is executing, each time SYMPUTX executes.  It's just that typically we don't reference the macro variable until after the step boundary, because if we referenced it before the step boundary, it would resolve before SYMPUTX has executed.

If you use the DATA step SYMGET function, it can look up the changing value of the macro variable C during each iteration of the DATA step loop.  I think I've used SYMGET less than 5 times over the past 20 years, but it's still a useful addition to your thought exercise:

%let c = 1;

data a;
  input cw @@;
  call symputx('c',cw);
  %put MACRO %nrstr(%PUT) statement executes before data step: &=c;
  c=symget('c') ;
  put "CALL SYMPUTX and SYMGET execute during data step " c= ;
datalines;
2 3 4
;

%put &=c;

[Rick_SAS]

@Quentin Yes, I was simply trying to make my program resemble the OP's. For more of my thoughts on SYMPUTX and SYMGET (albeit in the context of PROC IML, which is an interactive procedure), see my blog post from 2013:

https://blogs.sas.com/content/iml/2013/06/19/macros-and-loops.html

[Astounding]

I think @Rick_SAS has identified a key issue.   Also note, for the same reason, the %PUT statement will not function properly.

Here's an approach to modify your code appropriately.  I'm not sure I dotted all the i's and crossed all the t's, but these are a good set of tools to use:

%do k=1 %to %sysfunc(countw(&oldvarlist));

   %let old = %scan(&oldvarlist, &k);

See if this makes sense as to where to put it and how to use it.

[ballardw]

If you aren't adding any variables and only renaming them then you might look at creating a dataset with the old and new names. Then use a data _null_ to use that data and call execute to rename the variables in the source data set using proc datasets.

The following code assumes you have built a data set WORK.RENAME with variables named "name", holding the current name of a variable, and "newname" which is the name you want.

data _null_;
   set work.rename end=LastName;
   if _n_ = 1 then do;
      Call execute ("Proc datasets library=somelib nodetails nolist;");
      Call execute ("modify somedataset;");
      Call execute ("rename ")  ;
   end;
   Call execute(catx(' ',name,' = ',newname)) ;
   if LastName then do;
      Call execute (";") ;
      Call execute ("quit;");
   end;
run;

The example code would be used to rename those variables in the data set somelib.somedataset .

Placing the name and newname into a dataset may be much easier than all of the macro code. Plus if you keep that data set around you have documentation of the names.