topic longitudinal ROC cutoff? in Statistical Procedures

longitudinal ROC cutoff?

MeredithG — Tue, 04 Jun 2013 19:57:23 GMT

Hi all,

I'm working on creating an ROC curve for repeated-measures data. My outcome is dichotomous (pregnant yes/no) and my single predictor is a continuous measure, total number of motile sperm (TNM) drawn between 1 and 8 times. The doctor does not give me a time variable. I've been playing around with %GLIMMROC and am able to get a curve and and an AUC; however, the doctor is asking me about a cutoff for TNM and that is where I am stumped.

Side note, I also tried using PROC GLIMMIX and put the p-hats into PROC LOGISTIC and I got wildly different results - a much higher AUC than I got with %GLIMMROC, which is a little confusing to me since the macro uses GLIMMIX. Any insight would be welcome.

Thank you!

-M

Re: longitudinal ROC cutoff?

1zmm — Wed, 05 Jun 2013 11:58:00 GMT

Whether or not the ROC curve is based on a single measurement or repeated measurements, the choice of a cutoff depends on the trade-off between sensitivity and specificity. Ask the doctor whether sensitivity is more important than specificity or vice-versa and how much more important one is than another. For example, the sensitivity at or above level A of TNM if the proportion of all true pregnancies that are detected; the specificity below level A of TNM is the proportion of those who are truly not pregnant who are declared as not pregnant.

Or, ask the doctor whether level A of TNM yields an acceptable positive predictive value of the test: What proportion of the tests positive for pregnancy are associated with actual pregnancies? Unlike sensitivity and specificity, predictive values depend on the prevalence of the outcome (=pregnancy) in the population being tested; if pregnancy is rare in this population, positive predictive values are lower so that many positive tests are detected among those who are NOT pregnant.

I can't answer your other questions about differences in estimates of the area-under-the-curve between PROC LOGISTIC and PROC GLIMMIX. In general, multiple tests showing consistent results should improve sensitivity and specificity. Since you have no time indexes on your tests, you probably should consider either compound-symmetry or unrestricted variance-covariance matrices that do not depend on time.

Re: longitudinal ROC cutoff?

MeredithG — Wed, 05 Jun 2013 12:19:24 GMT

Thank you for the reply. I understand how to choose it, I just can't figure out how to actually locate it. I don't see, in any of the resulting datasets from %GLIMMROC or on the graph, an indication of the value of the TNM at the sens/spec level I'm after.