Hi The issue I have with the GINI/AUC calculation is that it is based on (and dependent upon) the choices made for where the cutoffs are made. I have been able to reproduce the results from the Model Manager procedure by using the proc assess-procedure with these settings: proc assess data=PM.ASSESS nbins=10 ncuts=100; target Target_Bad / event="1" level=nominal; input Em_Eventprobability; /* Fit statistics cannot be generated because no posterior probability variables for non-event levels are detected. Assign posterior probability variables on the OPTIONS tab. */ ods output ROCInfo=WORK._roc_temp LIFTInfo=WORK._lift_temp; run; Now, for most models this probably works fine. However, as the GINI/AUS calculation is dependent upon 100 cutoff values and these are distributed evenly across the 0-1 range (at each percentage point) this means that the first cutoff is made at 1% (em_eventprobabilty<0.01). For a PD model, this means that the first point of inspection is made at risk grade 4 or 5 dependent on the rating scale being used. This also means that the model's discriminatory power in the lower risk grades are not evaluated and that a large portion of the portfolio ends up in the first cutoff. The workaround I will probably use is to recalibrate the em_eventprobability to a range more suited for the assessment-procedure, but it would have been nice if the procedure rather had been implemented using deciles or something that is more adaptive to low-defualt probability models. Thanks for the link to your custom-KPI post. I'll be sure to look into this to see if this also can be used as a workaround.
... View more
