topic Re: Bug in Macro: Fail to output 1st run result. in SAS Programming

Bug in Macro: Fail to output 1st run result.

jiangmi — Mon, 08 Dec 2014 19:18:56 GMT

I am running a macro. I need to output the results into a fileref (lst_out).

The problem is: it will only output the 2nd run and after when you run 2 and more macros together.

I tried to debug it for a while now. I think I need some extra help.

The original macro is long, I downsized it to show the problem.

Sample data (bugdata.txt) was attached.

Thanks.

**************************************************;

/*1. Prepare the data.*/

data have ;

infile '/SASData/... .../BUG/bugdata.txt' ;

input score bad index;

run;

data have1 have2 have3;

set have;

if index=1 then output have1;

if index=2 then output have2;

if index=3 then output have3;

run;

/*2. the macro*/

%macro calculation (data = , score = , y = );

options nonumber nodate orientation = landscape linesize = 160 nocenter

formchar = "|----|+|---+=|-/\<>*" formdlim=' ' ls = 150;

data _tmp1 (keep = &y &score);

set &data;

run;

filename lst_out "/SASData/... .../BUG/&data..txt";

proc printto print = lst_out new; run;

*** Step_1 ***;

/*ods output kolsmir2stats = _ks;*/

proc npar1way wilcoxon edf data = _tmp1;

class &y; var &score; run;

proc printto; run;

*** Step_2 ***;

ods listing close;

/*ods output spearmancorr = _cor;*/

/*proc corr data = _tmp1 spearman;*/

/* var &y;*/

/* with &score;*/

/*run;*/

ods listing;

%mend calculation;

/*3. The problem.*/

/*No matter which order the three macro run, the first run will not have 'lst_out' output*/

%calculation(data = have2, score = score, y = bad);

%calculation(data = have1, score = score, y = bad);

%calculation(data = have3, score = score, y = bad);

Re: Bug in Macro: Fail to output 1st run result.

Astounding — Mon, 08 Dec 2014 20:11:04 GMT

The second run shouldn't be producing anything in the final file either. The problem lies with:

proc printto print = lst_out new; run;

"NEW" means that the file should be overwritten. You need to add NEW the first time, and remove it for the second and third runs.

Good luck.

Re: Bug in Macro: Fail to output 1st run result.

Patrick — Mon, 08 Dec 2014 20:28:34 GMT

The code you've posted works for me the very first time already (code as run posted below).

What you observe is normally caused by some wrong sequence in the code, so for example by trying to use a global macro variable before it has been created. If you then run the code for a second time in the same session the macro variable will exist and things work.

First thing I would do: Consult the SAS log after the first run for any Warning messages or even SAS Notes which could give you a hint what's missing.

%let path=c:\temp;

/*1. Prepare the data.*/

data have;

infile "&path\bugdata.txt";

input score bad index;

run;

data have1 have2 have3;

set have;

if index=1 then

output have1;

if index=2 then

output have2;

if index=3 then

output have3;

run;

;

/*2. the macro*/

%macro calculation (data = , score = , y = );

options nonumber nodate orientation = landscape linesize = 160 nocenter

formchar = "|----|+|---+=|-/\<>*" formdlim=' ' ls = 150;

data _tmp1 (keep = &y &score);

set &data;

run;

filename lst_out "&path\&data..txt";

proc printto print = lst_out new;

run;

*** Step_1 ***;

/*ods output kolsmir2stats = _ks;*/

proc npar1way wilcoxon edf data = _tmp1;

class &y;

var &score;

run;

proc printto print=print;

run;

%mend calculation;

/*3. The problem.*/

/*No matter which order the three macro run, the first run will not have 'lst_out' output*/

%calculation(data = have2, score = score, y = bad);

/*%calculation(data = have1, score = score, y = bad);*/

/*%calculation(data = have3, score = score, y = bad);*/

Re: Bug in Macro: Fail to output 1st run result.

art297 — Mon, 08 Dec 2014 22:10:16 GMT

The only way I got it to run was by commenting out the 2 ods listing lines, i.e.,

/*ods listing close;*/

/*ods listing;*/

didn't mention that, but had removed those two lines in his response.

Also, you don't need the semi-colons at the end of the lines where you call the macro. i.e.:

%calculation(data = have2, score = score, y = bad)

%calculation(data = have1, score = score, y = bad)

%calculation(data = have3, score = score, y = bad)

Re: Bug in Macro: Fail to output 1st run result.

jiangmi — Mon, 08 Dec 2014 23:13:17 GMT

The log did not trigger any warning or error message.

I am not getting the 1st run output as showed in the picture (have3 was in the 1st order and thus did not have the result).

I am run SAS EG on Server, may be something wrong beyond my control.

Re: Bug in Macro: Fail to output 1st run result.

Reeza — Tue, 09 Dec 2014 00:02:39 GMT

can you post the log?

My guess is that a macro variable isn't moving to the server

Re: Bug in Macro: Fail to output 1st run result.

jiangmi — Tue, 09 Dec 2014 05:04:08 GMT

I will post the log tomorrow.

However, it runs fine on my local SAS (see attached). Must be something wrong on the server end.

Re: Bug in Macro: Fail to output 1st run result.

jiangmi — Tue, 09 Dec 2014 14:01:04 GMT

(This is the log file copy)

The SAS System

1 ;*';*";*/;quit;run;

2 OPTIONS PAGENO=MIN;

3 %LET _CLIENTTASKLABEL='2014.12.08_GainTableBug';

4 %LET _CLIENTPROJECTPATH='';

5 %LET _CLIENTPROJECTNAME='';

6 %LET _SASPROGRAMFILE=;

8 ODS _ALL_ CLOSE;

9 OPTIONS DEV=ACTIVEX;

10 GOPTIONS XPIXELS=0 YPIXELS=0;

11 FILENAME EGSR TEMP;

12 ODS tagsets.sasreport12(ID=EGSR) FILE=EGSR STYLE=Analysis

12 ! STYLESHEET=(URL="file:///C:/Program%20Files/SAS/EnterpriseGuide/4.3/Styles/Analysis.css") NOGTITLE NOGFOOTNOTE GPATH=&sasworklocation

12 ! ENCODING=UTF8 options(rolap="on");

NOTE: Writing TAGSETS.SASREPORT12(EGSR) Body file: EGSR

14 GOPTIONS ACCESSIBLE;

15 /*2. the macro*/

16 %macro calculation (data = , score = , y = );

17 options nonumber nodate orientation = landscape linesize = 160 nocenter

18 formchar = "|----|+|---+=|-/\<>*" formdlim=' ' ls = 150;

20 data _tmp1 (keep = &y &score);

21 set &data;

22 run;

24 filename lst_out "/SASData/SASUSER/SAS_CREDRISK_SHARE/ART/Consumer/MK/BUG/&data..txt";

25 proc printto print = lst_out new; run;

27 *** Step_1 ***;

28 /*ods output kolsmir2stats = _ks;*/

29 proc npar1way wilcoxon edf data = _tmp1;

30 class &y; var &score; run;

31 proc printto; run;

33 *** Step_2 ***;

34 ods listing close;

35 /*ods output spearmancorr = _cor;*/

36 /*proc corr data = _tmp1 spearman;*/

37 /* var &y;*/

38 /* with &score;*/

39 /*run;*/

40 ods listing;

42 %mend calculation;

44 /*3. The problem.*/

45 /*No matter which order the three macro run, the first run will not have 'lst_out' output*/

46 %calculation(data = have3, score = score, y = bad);

NOTE: There were 50 observations read from the data set WORK.HAVE3.

NOTE: The data set WORK._TMP1 has 50 observations and 2 variables.

NOTE: DATA statement used (Total process time):

real time 0.01 seconds

cpu time 0.00 seconds

NOTE: PROCEDURE PRINTTO used (Total process time):

The SAS System

real time 0.00 seconds

cpu time 0.01 seconds

NOTE: There were 50 observations read from the data set WORK._TMP1.

NOTE: PROCEDURE NPAR1WAY used (Total process time):

real time 0.02 seconds

cpu time 0.02 seconds

NOTE: PROCEDURE PRINTTO used (Total process time):

real time 0.00 seconds

cpu time 0.00 seconds

47 %calculation(data = have1, score = score, y = bad);

NOTE: There were 50 observations read from the data set WORK.HAVE1.

NOTE: The data set WORK._TMP1 has 50 observations and 2 variables.

NOTE: DATA statement used (Total process time):

real time 0.00 seconds

cpu time 0.00 seconds

NOTE: PROCEDURE PRINTTO used (Total process time):

real time 0.00 seconds

cpu time 0.01 seconds

NOTE: There were 50 observations read from the data set WORK._TMP1.

NOTE: The PROCEDURE NPAR1WAY printed pages 1-2.

NOTE: PROCEDURE NPAR1WAY used (Total process time):

real time 0.02 seconds

cpu time 0.02 seconds

NOTE: PROCEDURE PRINTTO used (Total process time):

real time 0.00 seconds

cpu time 0.00 seconds

48 %calculation(data = have2, score = score, y = bad);

NOTE: There were 50 observations read from the data set WORK.HAVE2.

NOTE: The data set WORK._TMP1 has 50 observations and 2 variables.

NOTE: DATA statement used (Total process time):

real time 0.00 seconds

cpu time 0.01 seconds

NOTE: PROCEDURE PRINTTO used (Total process time):

real time 0.00 seconds

cpu time 0.00 seconds

The SAS System

NOTE: There were 50 observations read from the data set WORK._TMP1.

NOTE: The PROCEDURE NPAR1WAY printed pages 3-4.

NOTE: PROCEDURE NPAR1WAY used (Total process time):

real time 0.03 seconds

cpu time 0.03 seconds

NOTE: PROCEDURE PRINTTO used (Total process time):

real time 0.00 seconds

cpu time 0.00 seconds

51 GOPTIONS NOACCESSIBLE;

52 %LET _CLIENTTASKLABEL=;

53 %LET _CLIENTPROJECTPATH=;

54 %LET _CLIENTPROJECTNAME=;

55 %LET _SASPROGRAMFILE=;

57 ;*';*";*/;quit;run;

58 ODS _ALL_ CLOSE;

61 QUIT; RUN;

Re: Bug in Macro: Fail to output 1st run result.

art297 — Tue, 09 Dec 2014 14:38:46 GMT

You didn't respond to my suggestion yesterday, but now have included a third ODS statement at the end. Do you really need those three ODS statements?

Here is a note from the documentation regarding ODS _ALL_CLOSE (see: SAS(R) 9.2 Output Delivery System: User's Guide 😞

Note: Be sure to open one or more ODS destinations before you execute your next program so that you can view or print your output within the same SAS session.

Re: Bug in Macro: Fail to output 1st run result.

jiangmi — Tue, 09 Dec 2014 17:02:25 GMT

Hi, Arthur,

Thanks for look into this issue.

The third ODS statement (line 58, in the log file I assume you are referring to) was add into the log file by SAS at the end of the run.

All I have is the two lines (line 34 & 40). These two lines are needed in the macro run (for line 35-39). If I comment out these two lines (line 34&40), I won't have anything output to the fileref (see the attached picture):

Again, the macro runs fine on my local SAS. I think it is something wrong on the server end. It actually happened before with other codes (like gchart) as well.

Re: Bug in Macro: Fail to output 1st run result.

Reeza — Tue, 09 Dec 2014 17:07:28 GMT

What version of SAS are you using locally and what version on your server?

ANd what are you trying to do oversll? I get that your current code doesnt work, but it also appears inefficient and cumbersome.

Re: Bug in Macro: Fail to output 1st run result.

jiangmi — Tue, 09 Dec 2014 18:35:26 GMT

Hi, Reeza,

Thanks for looking into this issue.

SAS9.1 on my local computer.

SAS EG 4.3 on the server.

The macro I posted here is actually a part of a big macro to evaluate model performance. I deleted a majority part to show the problem point. The purpose of 'filename lst_out' and 'proc printto print = lst_out new;' (line 24, 25 in log file) is to output the result and grab a number (ks_score, " at Maximum = ") from the outputted document and put it into a macro variable (&ks_score) which will be used in the report later. If the result was not able to be outputted, then nothing can be grabbed, therefore no value can be assigned to the macro variable. Here is the original full macro:

%macro calculation (data = , score = , y = );

options nonumber nodate orientation = landscape linesize = 160 nocenter

formchar = "|----|+|---+=|-/\<>*" formdlim=' ' ls = 150;

*** DEFAULT GROUP NUMBER FOR REPORT ***;

%let grp = 10;

data _tmp1 (keep = &y &score);

set &data;

where &y in (1, 0) and &score ~= .;

run;

filename lst_out temp;

proc printto new print = lst_out;

run;

*** CONDUCT NON-PARAMETRIC TESTS ***;

ods output wilcoxonscores = _wx;

ods output kolsmir2stats = _ks;

proc npar1way wilcoxon edf data = _tmp1;

class &y;

var &score;

run;

proc printto;

run;

proc sort data = _wx;

by class;

run;

*** CALCULATE ROC AND GINI ***;

data _null_;

set _wx end = eof;

by class;

array a{2, 3} _temporary_;

if _n_ = 1 then do;

a[1, 1] = n;

a[1, 2] = sumofscores;

a[1, 3] = expectedsum;

end;

else do;

a[2, 1] = n;

end;

if eof then do;

auc = (a[1, 2] - a[1, 3]) / (a[1, 1] * a[2, 1]) + 0.5;

if auc <= 0.5 then auc = 1 - auc;

gini = 2 * (auc - 0.5);

call execute('%let auc = '||put(auc, 10.4)||';');

call execute('%let gini = '||put(gini, 10.4)||';');

end;

run;

*** CALCULATE KS ***;

data _null_;

set _ks;

if _n_ = 1 then do;

ks = nvalue2 * 100;

call execute('%let ks = '||put(ks, 10.4)||';');

end;

run;

*** CAPTURE SCORE POINT FOR MAX KS ***;

data _null_;

infile lst_out;

input @" at Maximum = " ks_score;

output;

call execute('%let ks_score = '||put(ks_score, 10.4)||';');

stop;

run;

proc summary data = _tmp1 nway;

class &y;

output out = _data_ (drop = _type_ _freq_)

mean(&score.) = mean var(&score.) = variance;

run;

*** CALCULATE DIVERGENCE ***;

data _null_;

set _last_ end = eof;

array a{2, 2} _temporary_;

if _n_ = 1 then do;

a[1, 1] = mean;

a[1, 2] = variance;

end;

else do;

a[2, 1] = mean;

a[2, 2] = variance;

end;

if eof then do;

divergence = (a[1, 1] - a[2, 1]) ** 2 / ((a[1, 2] + a[2, 2]) / 2);

call execute('%let divergence = '||put(divergence, 10.4)||';');

end;

run;

*** CAPTURE THE DIRECTION OF SCORE ***;

ods listing close;

ods output spearmancorr = _cor;

proc corr data = _tmp1 spearman;

var &y;

with &score;

run;

ods listing;

data _null_;

set _cor;

if &y >= 0 then do;

call symput('desc', 'descending');

end;

else do;

call symput('desc', ' ');

end;

run;

%put &desc;

proc rank data = _tmp1 out = _tmp2 groups = &grp ties = low;

var &score;

ranks rank;

run;

proc summary data = _last_ nway;

class rank;

output out = _data_ (drop = _type_ rename = (_freq_ = freq))

min(&score) = min_score max(&score) = max_score

sum(&y) = bads;

run;

proc sql noprint;

select sum(bads) into :bcnt from _last_;

select sum(freq) - sum(bads) into :gcnt from _last_;

quit;

proc sort data = _last_ (drop = rank);

by &desc min_score;

run;

data _data_;

set _last_;

by &desc min_score;

i + 1;

percent = i / 100;

good = freq - bads;

odds = good / bads;

hit_rate = bads / freq;

retain cum_bads cum_freq;

cum_bads + bads;

cum_freq + freq;

cum_hit_rate = cum_bads / cum_freq;

cat_rate = bads / &bcnt;

retain cum_cat_rate;

cum_cat_rate + cat_rate;

format symbol $4.;

if i = 1 then symbol = 'BAD';

else if i = &grp - 1 then symbol = 'V';

else if i = &grp then symbol = 'GOOD';

else symbol = '|';

run;

proc printto print = "%upcase(%trim(&score))_SEPARATION_REPORT.TXT" new;

run;

proc report data = _last_ spacing = 1 split = "/" headline nowd;

column("GOOD BAD SEPARATION REPORT FOR %upcase(%trim(&score)) IN DATA %upcase(%trim(&data))/

MAXIMUM KS = %trim(&ks) AT SCORE POINT %trim(&ks_score)/

( AUC STATISTICS = %trim(&auc), GINI COEFFICIENT = %trim(&gini), DIVERGENCE = %trim(&divergence) )/ /"

percent symbol min_score max_score good bads freq odds hit_rate cum_hit_rate cat_rate cum_cat_rate);

define percent / noprint order order = data;

define symbol / "" center width = 5 center;

define min_score / "MIN/SCORE" width = 10 format = 9.4 analysis min center;

define max_score / "MAX/SCORE" width = 10 format = 9.4 analysis max center;

define good / "GOOD/#" width = 10 format = comma9. analysis sum;

define bads / "BAD/#" width = 10 format = comma9. analysis sum;

define freq / "TOTAL/#" width = 10 format = comma9. analysis sum;

define odds / "ODDS" width = 10 format = 8.2 order;

define hit_rate / "BAD/RATE" width = 10 format = percent9.2 order center;

define cum_hit_rate / "CUMULATIVE/BAD RATE" width = 10 format = percent9.2 order;

define cat_rate / "BAD/PERCENT" width = 10 format = percent9.2 order center;

define cum_cat_rate / "CUMU. BAD/PERCENT" width = 10 format = percent9.2 order;

rbreak after / summarize dol skip;

run;

proc printto;

run;

%mend ;

Re: Bug in Macro: Fail to output 1st run result.

Reeza — Tue, 09 Dec 2014 19:44:44 GMT

There's inconsistent programming here, was it written by more than one person?

1. You use call symput in some places and call execute with %let in other, any reason for that?

2. For proc printto, sometimes you use file and other times direct path to txt. I recommend changing them all to a direct path to txt

Is this portion correct? It's not what you initially posted.

filename lst_out temp;

proc printto new print = lst_out;

run;

*** CONDUCT NON-PARAMETRIC TESTS ***;

ods output wilcoxonscores = _wx;

ods output kolsmir2stats = _ks;

proc npar1way wilcoxon edf data = _tmp1;

class &y;

var &score;

run;

proc printto;

run;

Re: Bug in Macro: Fail to output 1st run result.

jiangmi — Tue, 09 Dec 2014 21:45:13 GMT

Thanks, Reeza.

1. The macro was written by somebody else, I basically use it to do certain calculations. Therefore I don't know the particular reason for the choice between call symput and call execute.

2. In the original Macro, the fileref 'lst_out' was set up as temporary. I changed it to an accessible file on the computer so that I can see what is going on there. It turns out that nothing was outputted to 'lst_out' during the first macro run when I run the macro on the server. But when I run it on my local computer, everything works. That's why I think it is something wrong at the server end. I feel a little guilty for people who spend their time and took a look into this issue when the problem is probably not within the code. You you do have other suggestions, I'd like to give it a try. Thank you.

Re: Bug in Macro: Fail to output 1st run result.

Tom — Wed, 10 Dec 2014 01:12:15 GMT

You have to turn on the listing destination before PROC PRINTTO will have anything to print. That is why the first one never prints, because you are not printing any text output yet. So just put ODS LISTING: at the top of the program.

To make it less confusing perhaps you should switch from using using PROC PRINTTO to using the FILE= option on the ODS LISTING statement.

Re: Bug in Macro: Fail to output 1st run result.

Patrick — Wed, 10 Dec 2014 04:25:05 GMT

Yes, of course! Good catch, Tom.

Re: Bug in Macro: Fail to output 1st run result.

jiangmi — Wed, 10 Dec 2014 14:16:16 GMT

Hi, Tom,

I have to say, it does fix the problem.

Thank you.

Also thanks for all the people who look into this issue.

Joe