topic How to obtain an average ROC curve using multiple imuputation in Statistical Procedures

How to obtain an average ROC curve using multiple imuputation

AVA_16 — Tue, 05 Mar 2019 09:00:09 GMT

Hello,

My aim is to test the accuracy of a new marker to diagnos diabetes (as shown in the logistic model below). I am using multiple imputation to impute missing values of three important variables in the logistic model, i.e., glucose at 60 min, body-mass index, and treatment. I am performing multiple imputation using sas 9.4 in Windows using FCS method.

*imputation;

proc mi data= OGTT_4 nimpute=20 out=OGTT_5 seed=123;

class treatment;

fcs plots=trace(mean std);

var T2D_1 AGE sex BMI PFASTGLUC PGLUC60 PGLUC120 treatment ;

fcs discrim(treatment /classeffects=include) nbiter =100 ;

run;

Explaination of variables: T2D_1, type 2 diabetes; BMI, body mass index;PFASTGLUC, fasting glucose; PGLUC60, glucose at 60 min; PGLUC120, glucose at 120 min

T2D_1 and treatment are dichotmous and rest of the variables are continuous.

BMI (7/4639), PFASTGLUC (8/4638), PGLUC60 (19/4627), PGLUC120 (3/4643), treatment (369/4277) have missing values

PFASTGLUC and PGLUC120 are auxillary variables in Proc MI

*analysis;

proc logistic data=OGTT_5 descending;

class treatment (ref="0");

model T2D_1 = PGLUC60 age sex treatment BMI

/ covb outroc=roc;

roc;

by _Imputation_;

ods output ParameterEstimates=lgsparms;

run;

*combining estimates;

proc mianalyze parms (classvar=CLASSVAL )=lgsparms;

class treatment;

MODELEFFECTS intercept PGLUC60 AGE sex treatment BMI;

RUN;

I do get combined estimates of the logistic model using proc mianalyze, however, I am wondering, how can I get an average ROC curve? Is there any way to obtain combined estimates of senstivity and specificity in outroc data? Any help is appreciated.

Re: How to obtain an average ROC curve using multiple imuputation

StatDave — Tue, 05 Mar 2019 16:05:20 GMT

Using the example titled "Reading Logistic Model Results from a PARMS= Data Set" in the PROC MIANALYZE doc, these statements produce an ROC analysis using the original, unimputed data. Note that the final parameter estimates from MIANALYZE are saved and then transposed into a form that can be used as an INEST= data set. The combination of INEST= and MAXITER=0 forces PROC LOGISTIC to use the final estimates from MIANALYZE in the ROC analysis.

proc mianalyze parms=lgsparms;
   modeleffects Intercept Length Width;
   ods output parameterestimates=pe;
run;
proc transpose data=pe out=tpe; 
   var estimate; id parm; 
run;
proc logistic data=fish2 inest=tpe;
   class Species;
   model Species= Length Width / maxiter=0;
   roc;
run;

If you want to instead do an ROC analysis for each imputation and then get the average ROC area, the following statements can be used. The initial DATA step replicates the final parameter estimates once for each of the 25 imputations so it can again be used as an INEST= data set when the BY statement is used.

data inest;
   pt=1;
   do _imputation_=1 to 25;
     set tpe point=pt; output;
   end;
   stop;
run;
ods exclude all;
proc logistic data=outfish2 inest=inest;
   by _imputation_;
   class Species;
   model Species= Length Width / maxiter=0;
   roc;
   ods output rocassociation=roc(where=(rocmodel="Model"));
run;
ods select all;
proc means mean; 
   var area; 
run;

Re: How to obtain an average ROC curve using multiple imuputation

AVA_16 — Wed, 06 Mar 2019 09:43:51 GMT

Thank you for your reply:

I used the first method you suggested:

*imputation;

proc mi data= OGTT_4 nimpute=25 out=OGTT_5 seed=123;

class treatment;

fcs plots=trace(mean std);

var AGE BMI PFASTGLUC PGLUC60 PGLUC120 treatment ;

fcs discrim(treatment /classeffects=include) nbiter =100 ;

run;

*analysis;

proc logistic data=OGTT_5 descending;

class treatment (ref="0");

model T2D_1 = PGLUC60 age sex treatment BMI

/ covb ;

by _Imputation_;

ods output ParameterEstimates=lgsparms

CovB=lgscovb;

run;

*combining estimates;

proc mianalyze parms =lgsparms;

MODELEFFECTS intercept PGLUC60 AGE sex treatment BMI;

ods output parameterestimates=pe;

RUN;

proc transpose data=pe out=tpe;

var estimate; id parm;

run;

proc logistic data=OGTT_4 inest=tpe;

class treatment ;

model T2D_1 (EVENT='1')

= PGLUC60 age sex treatment BMI / maxiter=0;

roc;

run;

However, it gives me a warning in the very last step where I run logistic regression model after proc transpose step :

NOTE: Variable treatment1 is not found in the INEST= dataset.

NOTE: PROC LOGISTIC is modeling the probability that T2D_1=1.

ERROR: Missing initial values for some of the parameters.

NOTE: The SAS System stopped processing this step because of errors.

NOTE: There were 4646 observations read from the data set WORK.OGTT_4.

Any suggestions are appreciated.

Re: How to obtain an average ROC curve using multiple imuputation

StatDave — Wed, 06 Mar 2019 13:51:19 GMT

The problem is because your treatment variable is a CLASS variable. If it is binary and numeric and has levels 0 and 1, then simply remove the CLASS statements from your PROC LOGISTIC steps.

Re: How to obtain an average ROC curve using multiple imuputation

AVA_16 — Wed, 06 Mar 2019 14:20:11 GMT

Thank You Dave. It worked. Your assistance is highly appreciated.