ROC Curve with Multiple Predictors

[gabybarber]

I am trying to create an ROC curve using maternal race and smoking group as predictors with depression as the outcome. Below is the syntax, we have used to do this:

ods graphics on;

PROC LOGISTIC DATA=d.pramsv1 plots=effect plots=ROC(id=sensit);

Class SMOKE2 MATRACE;

WHERE (ANSAMPLE=1);

MODEL PPDSTATUS(DESC) = SMOKE2 MATRACE;

roc 'Smoking with race' SMOKE2 MATRACE;

RUN;

ods graphics off;

Our output for the ROC curve has several points labelled with their corresponding sensitivity (see below):

 Is there a way to determine which cutoffs for the variables correspond to each of these labelled sensitivities?

Any help would be much appreciated! Thanks!

[FreelanceReinh · Accepted solution]

Hello @gabybarber,

Section "Receiver Operating Characteristic Curves" of the PROC LOGISTIC documentation uses probability cutpoints. In your example with only two categorical predictors the distinct predicted probabilities correspond to combinations of predictor variable levels (20, as it seems), so you may be interested in both: the probability cutpoint and the combination of SMOKE2 and MATRACE corresponding to each of the 20 sensitivity values.

Here's an example where this is accomplished. The key is the SCORE statement which I added to the PROC LOGISTIC step.

/* Create sample data for demonstration */

data have;
call streaminit(27182818);
do smoke2=0 to 3;
  do matrace=0 to 4;
    do _n_=1 to 100;
      ppdstatus=rand('bern',logistic( 0.2*(smoke2=1)
                                     +0.5*(smoke2=2)
                                     +1.0*(smoke2=3)
                                     +0.3*(matrace=1)
                                     +0.7*(matrace=2)
                                     +0.4*(matrace=3)
                                     +0.6*(matrace=4)
                                     -3));
      output;
    end;
  end;
end;
run;

/* Perform logistic regression */

ods graphics on;
proc logistic data=have plots=effect plots=roc(id=sensit);
class smoke2 matrace;
model ppdstatus(desc) = smoke2 matrace;
roc 'Smoking with race' smoke2 matrace;
score out=scores outroc=rocdata;
run;
ods graphics off;

/* Combine sensitivities, probability cutpoints and predictor values */

proc sort data=scores out=scores1 nodupkey;
by p_1 smoke2 matrace;
run;

proc sql;
create table want as
select r._sensit_, r._prob_, s.smoke2, s.matrace
from rocdata r left join scores1 s
on _prob_=p_1
order by _sensit_;
quit;

Result:

Obs    _SENSIT_     _PROB_    smoke2    matrace

  1     0.11330    0.21221       3         2
  2     0.19704    0.18347       3         4
  3     0.29064    0.16009       2         2
  4     0.38424    0.14311       3         3
  5     0.44335    0.13719       2         4
  6     0.49261    0.13566       3         1
  7     0.52709    0.12538       1         2
  8     0.60591    0.10681       1         4
  9     0.65517    0.10569       2         3
 10     0.69951    0.10555       3         0
 11     0.73399    0.09996       2         1
 12     0.75862    0.08163       1         3
 13     0.81773    0.07709       1         1
 14     0.86700    0.07707       2         0
 15     0.90148    0.06232       0         2
 16     0.92611    0.05909       1         0
 17     0.94089    0.05253       0         4
 18     0.95567    0.03958       0         3
 19     0.98522    0.03728       0         1
 20     1.00000    0.02829       0         0

The _SENSIT_ values, rounded to two decimals, are those found in the ROC plot (not shown here).

Please see if the code is suitable for your data. Changes might be necessary if the correspondence between probabilities and (SMOKE2, MATRACE) pairs is not 1:1 or perhaps if the join condition _prob_=p_1 fails because of rounding errors, etc.

[gabybarber]

Thank you so much! It worked perfectly!