How to find what macros are called by a specific program

[PaigeMiller]

I have written a very long SAS program that calls many different macros that I have written. Now, I want a way to find out the exact list of macros I am calling in this program. I suppose I could search the code for any time a % sign appears, but that seems like a very inefficient way of doing it, as it will find %LET and %IF and there are a lot more of those than actual macro calls. So what is a smarter way to do this?

[Reeza]

If the macros are not conditionally executed perhaps PROC SCAPROC?

[Astounding]

Messy, but you could try:

options mlogic mprintnest;

Then run the program, and parse the log to find the macro references.

Going by memory now, but you may not need MPRINTNEST.  If I recall properly, MLOGIC gives a message about "macroname" macro beginning execution whenever a macro begins to execute.  But you would still need to extract and dedup all such messages in the log.

[Quentin]

I saw a paper once (sorry, not finding it now), where they wanted to track usage of autocall macros shared by a team.  

Instead of parsing the logs, they programmed each of their macros so that every time it was executed, it would write a record to an audit trail dataset that would say "Macro X was invoked in Program Y on date-time T by Programmer P etc."   

Of course its overhead, but I thought it was an interesting approach for broad tracking of how macros are being used.  

But agree with @Astounding , for one-offs, I'd probably go with MPRINTNEST.

[Tom]

What I have done in the past is split the file into words and look for words that start with %.

I had to manually build a list of macro keywords and functions (%IF %SCAN etc.) to exclude.  That list has probable grown since the last time I used that program.

I also added a pre-process step to exclude any text between /* and */ tokens to prevent finding calls in comment blocks.

[Tom]

So I took a look at that old program from 20+ years ago and updated it to take advantage of new SAS functions like HASH objects.

data _null_;
  length type $1 name $32 word $256;
  length delim $256 ;
  retain delim;
  if _n_=1 then do;

* Treat anything that is not part of name or &, % or : as delimiters;
    delim=compress(collate(0,256),'_:%&','ad');

* Hash to store macro calls ;
    declare hash macros(ordered:'a');
    rc=macros.definekey('name');
    rc=macros.definedata('name');
    rc=macros.definedone();

* Hash to store symbol (macro variable) references ;
    declare hash symbols(ordered:'a');
    rc=symbols.definekey('name');
    rc=symbols.definedata('name');
    rc=symbols.definedone();

* Hash to store system symbols ;
    declare hash auto(dataset:"sashelp.vmacro(where=(scope='AUTOMATIC'))");
    rc=auto.definekey('name');
    rc=auto.definedata('name');
    rc=auto.definedone();

* Hash to store macro keywords and functions ;
    declare hash reserved();
    rc=reserved.definekey('name');
    rc=reserved.definedata('name');
    rc=reserved.definedone();

   do name='MACRO','MEND','DO','END','IF','THEN','ELSE','WHILE','UNTIL','GOTO'
          ,'TO','BY','BQUOTE','DISPLAY','EVAL','GLOBAL','INCLUDE','INDEX'
          ,'KEYDEF','LET','LOCAL','NRBQUOTE','NRQUOTE','NRSTR','PUT','QSCAN'
          ,'QSUBSTR','QSYSFUNC','QUOTE','QUPCASE','SCAN','STR','SUBSTR'
          ,'SUPERQ','SYSEXEC','SYSGET','SYSPROD','LENGTH','SYSEVALF'
          ,'SYSFUNC','TRIM','UNQUOTE','UPCASE','VERIFY','WINDOW','SYMDEL'
    ;
      rc=reserved.ref();
    end;
  end;
* Read file word-by-word ;
  infile &filename dlm=delim end=eof;
  input word @@;
  word=upcase(word);
* Split words with multiple & and % into individual names ;
  do i=1 to countw(word,'%:&');
    call scan(word,i,pos,len,'%:&');
    name=substrn(word,pos,len);
    type=substrn(word,pos-1,1);
    if 0=cmiss(name,type) then do;
* Macro calls start with %. Exclude macro labels that start with % and end with : ;
      if type='%' and reserved.find() and substrn(word,pos+len,1) ne ':' then rc=macros.ref();
* Symbol references start with &. ;
      if type='&' and auto.find() then rc=symbols.ref();
    end;
  end;
  if eof then do;
* Write the lists of macros and symbols to datasets ;
     rc=symbols.output(dataset: 'symbols');
     rc=macros.output(dataset: 'macros');
     stop;
  end;
run;

Try it out.  Let me know if there are missing macro keywords that need to be added to the list.  %RETURN perhaps?

Here is a crude program for removing /* ... */ comment blocks if you want to cut down on the number of false positives.

Spoiler

filename stripped temp;

data _null_;
  infile &filename truncover lrecl=500;
  file stripped;
  retain next '/*' ;
  input line $char500.;
*----------------------------------------------------------------------
Loop until then input line is empty
-----------------------------------------------------------------------;
 do while (line ne ' ');

   i = index(line,next);
   if i > 0 then do;
     if next='/*' then do;
       if i>1 then do;
         outline=substr(line,1,i-1);
         l = length(outline);
         put outline $varying500. l @;
         anyout=1;
       end;
       line = substr(line,i+2);
       next='*/';
     end;
     else do;
       line = substr(line,i+2);
       next='/*';
     end;
   end;
   else if next='/*' then do;
     l = length(line);
     put line $varying500. l @;
     anyout=1;
     line=' ';
   end;
   else do;
     line=' ';
   end;
 end;
 if anyout then put;
run;

[PaigeMiller]

@Tom thanks.

Since my Autoexec calls %SGANNO and %ANNOMAC plus some personal macros that I have created, there are always about 80–100 macros present (for example, to access on of the bazillion data bases here, instead of memorizing a LIBNAME statement to link to database ABC, I create and place in my autoexec this LIBNAME statement inside macro %ABC, so I can in the code use %ABC instead of the hard-to-remember LIBNAME statement). So every time I would add another macro to my Autoexec, the code you have provided would have to be updated.

Naturally, I could add those macro names into your code, but I'm still thinking that my BEFORE/AFTER modification of @Ksharp's is more general and future proof.

[Tom]

Depends on what you trying to discover and how invasive you want to be.   

Gathering the macros used by running the program will not work if the the program has conditional logic that only calls some macros based on the inputs.

Trying to scan the code will miss complex things like 

%&macro

It is not hard to scan the SASAUTOS fileref and gather the names of the programs there and exclude them if you want (or perhaps just to classify them instead?).

Things like ANNOMAC that embed multiple macro definitions into a single physical file will be a problem either way.  It you look a the code you will see calls to macros that have no corresponding macro definition file.   And if you look at the compiled macros you might seem macros that are compiled that might never be used by the program.

[ballardw]

I might be tempted t to get the names of currently defined macros probably from the Work.SASMACR catalog.

If you need this in the middle of a job get them before the job then after to get the "added" macros.

Of course if using stored compiled macros this wouldn't help directly but could use the information in the stored catalong as well to make target strings of your defined macros to search a code file instead of all % usage.

[Amir]

I too recall a similar paper as described by @Quentin, but could also not find it.

In a similar vein, if this is more than a one-off, then perhaps try changing the macro invocations to call a shell macro which is passed the macro name and any arguments, then the shell macro can first output the macro name to the log (including arguments, which can be useful) then invoke the given macro with the arguments. Then in future if extra information needs to be output to the log, e.g., invocation count, then just the shell macro needs to be changed, instead of all the other defined macros.

Kind regards,

Amir

[Ksharp · Accepted solution]

If all these marco were written by yourself, Check dictionary.catalogs.

%macro xx;
data x;
 set sashelp.class;
run;
%mend;



%xx


proc sql;
create table macro as
select * from dictionary.CATALOGS
 where libname='WORK' ;
quit;

[PaigeMiller]

Thank you to everyone who has answered.

I like the answer from @Ksharp the best, although macros such as %LEFT and %ANNOMAC and many other system macros show up. But I still have to visually weed out the system macros from the ones I have written, but this gets me pretty close, with not a lot of effort.

I thought of a further enhancement to @Ksharp 's code. I can run it before my very long program, create a data set named BEFORE, then run his code after I run the very long program, create a data set named AFTER, and then merge the two to see which macro names have been added. So far, that seems like the best option to me.