Calculating AUC and graphing ROC for a vendor model

[shidehrafigh]

Hi

I have a fraud vendor model for which I have the score and I have identified the TP, TN, FP and FN. I have been asked to graph ROC and calculate AUC for this model in order to assess the performance of the model. However, I am trying to figure out when in this case which we don't have access to the actual model since it is proprietary information how is it possible to calculate AUC ?

The way I know to calculate AUC is to create ROC first and for that we need to have the model in order to identify the confusion matrix for different cut offs, but in this case when we cannot do that, is there anything we can do?

[Rick_SAS · Accepted solution]

You have all the information you need. The ROC curve plots the False Positive Rate (FP divided by number of obs) as X and the True Positive Rate as Y.  Be sure to sort the (FPR, TPR) pairs by the FPR. To find the UAC, you can use the trapezoid rule to find the exact area. See

"The area under an ROC curve", which refers to the article "The trapezoidal rule of integration."