Difficulty in Visualizing Smooth AUC Curve in SAS Viya

[Ken_Ken]

Dear SAS Community Members,

I am currently facing difficulties in visualizing a smooth AUC curve in SAS Viya.

After building the model and reviewing the AUC curve, I noticed that the curve appears rough and zig-zagged.

Following the advice of a colleague who suggested that the AUC curve and values may not be accurate, I typically copy and paste the code from Viya to EG to verify the exact AUC curve and values. However, in this case, the issue seems too significant.

I suspect there might be a setting in Viya that can produce a more accurate AUC curve and values. Is there any solution for this?

Thank you in advance for your help.

Sincerely,

[sbxkoenk]

Hello ,

• Is this a ROC (AUC) - curve you plot with a statistical graphics procedure (like SGPLOT)?
• Or is this a ROC (AUC) - curve that is in the results pane after running a statistical procedure (like LOGISTIC with a ROC statement)?
• Or is this a ROC (AUC) - curve that is displayed in Model Studio?
• Or is this a ROC (AUC) - curve that is displayed in SAS Visual Analytics (Visual Statistics)?

Can you share that?

You can of course augment the number of interpolation points (i.e. more thresholds) ... but if the output of the model is very "discrete" (like with a 5-leaves decision tree) ... that will not help.

And with zig-zagged ... you probably mean step-function-like (as the ROC curve should be monotonically increasing from bottom-left to top-right).

Cheers,

Koen

[Ken_Ken]

Thank you for reaching out, Koen. I appreciate your help.

• Is this an ROC (AUC) curve you plotted using a statistical graphics procedure (like SGPLOT)?

Sorry, I’m not very familiar with statistical graphics procedures. I believe I’m using the default one.

• Or is this an ROC (AUC) curve displayed in the results pane after running a statistical procedure (like LOGISTIC with a ROC statement)?

After running the LOGISTIC procedure, I went to “Compare the Model” (I’m not sure if this is the correct name, as we are using the Japanese version), and the ROC curve appeared quite rough. Here’re the actual ROC curve outputted in Viya and EG.

 

• Or is this an ROC (AUC) curve displayed in Model Studio?

Yes, we created the model in Model Studio, not in SAS Visual Analytics.

[sbxkoenk]

There might indeed be a problem here. (??)

To be sure , can you make your ROC curve again using this technique?

Create ROC curve for any predictive model - SAS Support Communities

for more info , see : Create and compare ROC curves for any predictive model - The DO Loop

Take (from your Model Studio project) the dataset that contains target and prediction.

Then do a PROC LOGISTIC with ROC statement. Your target should be the dependent variable and your prediction (predicted probability) should be the -- only -- independent variable (explanatory variable).

Then compare ...

BR, Koen