Comparing two ROC curves

[iuri_leite]

Dear Colleagues,

I have two measures, MPR and Relato30, that I wish to compare against a gold standard: TFV_CAT. Given that I'm working with longitudinal data, I am employing PROC GEE for a dichotomous response.

Below is the code I am using:

proc gee data=dbs_jovens;
class copid;
model tfv_cat(event="1")=mpr/ dist=bin;
repeated subject=copid;
output out=model_mpr p=p_mpr;
run;

proc logistic data=model_mpr;
model tfv_cat(event="1")= / nofit outroc=roc_mpr;
roc pred=p_mpr;
run;

proc gee data=dbs_jovens;
class copid;
model tfv_cat(event="1")=relato30/ dist=bin;
repeated subject=copid;
output out=model_relato30 p=p_relato30;
run;

proc logistic data=model_relato30;
model tfv_cat(event="1")= / nofit outroc=roc_relato30;
roc pred=p_relato30;
run;

I would like to test the differences between the curves. Do I need to create a unique dataset that includes the ROC curves from both models (roc_mpr and roc_relato30)?

Thank you in advance.

Regards,
Iuri

[StatDave · Accepted solution]

Put your two predicted probability variables together in the same data set and then use PROC LOGISTIC with two ROC statements and an ROCCONTRAST statement. Below is an example using the data in the "Comparing ROC curves" example in the PROC LOGISTIC documentation. It will work the same way with your GEE models.

proc genmod data=roc;
model popind(event="1")=alb / d=b;
output out=out p=palb;
run;
proc genmod data=out;
model popind(event="1")=tp / d=b;
output out=out p=ptp;
run;
proc logistic data=out;
model popind(event="1")= / nofit;
roc 'alb' pred=palb;
roc 'tp' pred=ptp;
roccontrast; 
run;

[iuri_leite]

Dear @StatDave,

it worked fine.

Many thanks.

Regards,

Iuri

[sbxkoenk]

Create and compare ROC curves for any predictive model
By Rick Wicklin on The DO Loop November 14, 2018
https://blogs.sas.com/content/iml/2018/11/14/compare-roc-curves-sas.html

Usage Note 52973: Plot and compare ROC curves from a fitted model used to score validation or other data
https://support.sas.com/kb/52/973.html

Usage Note 45339: Plot and compare ROC curves from logistic models fit to independent samples
https://support.sas.com/kb/45/339.html

Koen