topic how to calculate integrated discriminatory improvement (IDI) using SAS MACRO in SAS Programming

how to calculate integrated discriminatory improvement (IDI) using SAS MACRO

miminiari_naomi — Fri, 13 Nov 2020 04:17:42 GMT

hi, I doing a research which compare basic model with new model. This new model is basic model+ a new biomarker. I have a dichotomous outcome. I want to calculate IDI and NRI to show the added predictive ability of this new marker on the basic model.

I found a paper by Kevin Kennedy A SAS macro to compute added predictive ability of a new marker in logistic regression. I try to used the syntax in the appendix of this paper but the result did not come out. here I post what the log said. it keep saying uninitialized.

what did I do wrong? or could you help me where I could download the macro so that I could just used it instead? I already email Mr Kennedy but still not receive any reply. it would be nice if someone could help me.tx

label relative_idi='Relative IDI';format pvalue_idi pval.;

_____

484

NOTE 484-185: Format PVAL was not found or could not be loaded.

141 run;

NOTE: Numeric values have been converted to character values at the places given by: (Line):(Column).

129:23 129:51

NOTE: Variable p_event_new is uninitialized.

NOTE: Variable p_event_old is uninitialized.

NOTE: Variable p_nonevent_new is uninitialized.

NOTE: Variable p_nonevent_old is uninitialized.

NOTE: Variable eventstderr is uninitialized.

NOTE: Variable noneventstderr is uninitialized.

NOTE: Missing values were generated as a result of performing an operation on missing values.

Each place is given by: (Number of times) at (Line):(Column).

1 at 122:17 1 at 125:12 1 at 125:29 1 at 125:33 1 at 125:49 1 at 127:11 1 at 127:20 1 at 127:25 1 at 128:10

1 at 128:19 1 at 128:24 1 at 131:7 1 at 131:14 1 at 131:16 1 at 131:33 1 at 131:37 1 at 131:53 1 at 133:16

1 at 133:17 1 at 133:26 1 at 134:25 1 at 136:83 1 at 137:82 1 at 138:22 1 at 139:23

NOTE: The data set WORK.FIN has 1 observations and 20 variables.

NOTE: DATA statement used (Total process time):

real time 0.08 seconds

cpu time 0.04 seconds

142

143

144

145 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

157

Re: how to calculate integrated discriminatory improvement (IDI) using SAS MACRO

Oligolas — Fri, 13 Nov 2020 09:55:02 GMT

Hi,

the paper code seems to work with the example provided.

You may have copy/paste issues, try the code I've reindented.

Re: how to calculate integrated discriminatory improvement (IDI) using SAS MACRO

ballardw — Fri, 13 Nov 2020 16:14:13 GMT

When SAS reports a variable is uninitialized that means the variable name appears in code but has no values.

Example.

16   data example;
17      x=3;
18      label y ='Some nonexistent values';
19   run;

NOTE: Variable y is uninitialized.
NOTE: The data set WORK.EXAMPLE has 1 observations and 1
      variables.
NOTE: DATA statement used (Total process time):
      real time           0.02 seconds
      cpu time            0.01 seconds

This sort of thing is moderately common when using someone else's code without understanding it. The author likely used different variables at some point than you do but the code still uses the authors variables. But you did not provide any values for those so you get the note.

Macro code can compound this by building variable names dynamically and so you can't just search for the offending text in the program code.

If you are running a macro then you want to set OPTIONS MPRINT; prior to executing the macro so the entire code as generated appears in the log and you can get an idea of what needs to change.

Re: how to calculate integrated discriminatory improvement (IDI) using SAS MACRO

miminiari_naomi — Tue, 17 Nov 2020 04:26:04 GMT

hi, thank for your replay. I try to use your code. but it seems that I cannot make the data nri1. the output folder for data create nri1 data but it's empty. the log said this.

121 DATA nri1;

122 group="Category-Free NRI";

123 p_up_event=&num_event_up_user/&num_event;

WARNING: Apparent symbolic reference NUM_EVENT not resolved.

124 p_down_event=&num_event_down_user/&num_event;

what did I do wrong?tx

Re: how to calculate integrated discriminatory improvement (IDI) using SAS MACRO

miminiari_naomi — Tue, 17 Nov 2020 04:26:53 GMT

should I run the code step by step?

Re: how to calculate integrated discriminatory improvement (IDI) using SAS MACRO

miminiari_naomi — Tue, 17 Nov 2020 04:28:10 GMT

tx for your reply. I don't have the macro. I have the syntax. when should I put the OPTIONS MPRINT?
tx

Re: how to calculate integrated discriminatory improvement (IDI) using SAS MACRO

miminiari_naomi — Tue, 17 Nov 2020 04:29:09 GMT

it seems like data nri1 is not calculate

Re: how to calculate integrated discriminatory improvement (IDI) using SAS MACRO

Oligolas — Tue, 17 Nov 2020 11:05:13 GMT

Hi,

As far as I can see, the macros are variable independent and should work with any kind of data, you only need to adapt the model in the call.

You have the macros in the paper and I've extracted and provided them in my previous post. Why aren't you following the examples?

It works with sashelp.cars:

*APPENDIX C: EXAMPLE;
DATA cars;
   set sashelp.cars;
   msrp_40k=(msrp>40000); /*indicator for car being over $40,000*/
   cnt+1;
RUN;

/*Test to see if Country of Origin adds to the prediction of
   msrp_40k with engine size, weight, and
   MPG_Highway in model
   Using NRI groups (<10%, 10-30%, >30%)*/
/*initial model*/
PROC LOGISTIC DATA=cars descending;
   model msrp_40k=enginesize weight mpg_highway;
   output out=m1 pred=p1;
RUN;

/*new model*/
PROC LOGISTIC DATA=cars descending;
   class origin;
   model msrp_40k=enginesize weight mpg_highway origin;
   output out=m2 pred=p2;
RUN;

PROC SQL;
   CREATE table cars2 as SELECT *
   FROM m1 as a left join m2 as b on a.cnt=b.cnt;
QUIT;

%add_predictive(DATA=cars2,y=msrp_40k,p_old=p1,p_new=p2, nripoints=.1 .3);

and with sashelp.class:

*APPENDIX C: EXAMPLE;
DATA class;
   set sashelp.class;
   height_60=(height>60); /*indicator for height over 60*/
   cnt+1;
RUN;

PROC LOGISTIC DATA=class descending;
   model height_60=weight weight age;
   output out=m1 pred=p1;
RUN;

/*new model*/
PROC LOGISTIC DATA=class descending;
   class sex;
   model height_60=weight age sex;
   output out=m2 pred=p2;
RUN;

PROC SQL;
   CREATE table class2 as SELECT *
   FROM m1 as a left join m2 as b on a.cnt=b.cnt;
QUIT;

%add_predictive(DATA=class2,y=height_60,p_old=p1,p_new=p2, nripoints=.1 .3);

If you still can not fix it you should provide the code you are running together with sample data.