topic Re: Out-of-time Logistic regression test in Statistical Procedures

Out-of-time Logistic regression test

JinboZhao — Mon, 14 Aug 2017 16:32:39 GMT

Hi,

I used SAS Enterprise Miner and generated a scorecard sample and the logistic regression respectively.I want to do the out-of-time test on a new testing dataset. I tried to use SCORE node, I got a list of predicted percentage of good/bad. But I did not get any statistical comparison, like ROC, KINI, KS. I want to get those statistical parameters to test the reliability of my scorecard model.

I was suggested that Somer's D is similar as KINI, which one I can easily get from SAS. However, based on my research, I only saw some articles online which gives information about how to generate a logistic regression and also the Somer'D on the same dataset, not on another test dataset.

Could anyone give me some clue?

Thank you.

Re: Out-of-time Logistic regression test

Reeza — Mon, 14 Aug 2017 16:24:57 GMT

It's not exactly clear what you're asking. Are you looking for how to score new data with your logistic regression model? If so, look at the SCORE node.

Re: Out-of-time Logistic regression test

JinboZhao — Mon, 14 Aug 2017 16:35:21 GMT

Hi Reeza,

I used the SCORE node, but it did not give me statistical parameters like ROC, KINI on the new dataset.

Re: Out-of-time Logistic regression test

StatDave — Tue, 15 Aug 2017 14:55:52 GMT

If you fit your model in PROC LOGISTIC, you can use the SCORE statement with the FITSTAT option to score a new data set using the fitted model and get the area under the ROC curve (AUC) along with several other statistics. See the "Details: Scoring data sets" section of the the LOGISTIC documentation. Note that the Gini statistic equals 2*AUC-1. Or, you can use the predicted probabilities for the new data saved from the OUT= option of the SCORE statement in the PRED= option of the ROC statement in a subsequent run of PROC LOGISTIC to get Somers' D (which is the Gini statistic) along with the gamma and tau statistics (as well as the AUC again) as shown in this note.

Re: Out-of-time Logistic regression test

JinboZhao — Tue, 15 Aug 2017 16:04:37 GMT

Thank you for your answer. My problem is that I got my logistic regression in SAS Enterprise Miner under the REGRESSION node. It generated SAS scoring code for regression, but it is not a model which I can use to imput and then test on other dataset.

Do you know how to use the Regression Scoring Code to generate the respectively SAS model dataset? Thank you.