topic Re: Problem with PROC logistic ROC with Large Dataset in Statistical Procedures

Problem with PROC logistic ROC with Large Dataset

CannonballZ — Sun, 16 May 2021 03:54:08 GMT

I am attempting to run Proc Logistic and the ROCPLOT macro on a dataset with 10^6 data points. The ROC plot will not display and the threshold labels for each classifier will not display. Only the composite threshold label will display:

Criterion Symbol Cutpoint Label Value
Correct C 0.36905 . . . . 0.369 C 0.9277

Typically, the threshold labels for each classifier will be sequentially listed where the "...." is. Is this just a limitation in the software secondary to too many datapoints?

Please help.

Thank you for any assistance.

Re: Problem with PROC logistic ROC with Large Dataset

sbxkoenk — Sun, 16 May 2021 10:32:42 GMT

Hello,

I have never used the ROCPLOT macro but produced thousands of ROC-plots. 😉

In cases like this, we cannot do much without you showing us the LOG. What is the LOG saying??

What version of SAS are you using? You can find out by submitting :

%PUT &=sysvlong;

Can't you use the PROC LOGISTIC ROC Statement and PLOTS=ROC option for ROC comparisons?

I believe the ROCPLOT macro is quite old already. There must be better alternatives by now. Maybe it has some unique features though that I am overlooking (again, I don't know the macro).

Cheers,

Koen

Re: Problem with PROC logistic ROC with Large Dataset

CannonballZ — Sun, 16 May 2021 15:08:06 GMT

Thanks for responding.

Version is 9.4.

The Log shows:
NOTE: Writing HTML Body file: sashtml.htm
NOTE: PROC LOGISTIC is modeling the probability that Infarct='1'.
NOTE: Convergence criterion (GCONV=1E-8) satisfied.
ERROR: Java virtual machine exception. java.lang.OutOfMemoryError: GC
overhead limit exceeded.
ERROR: Java virtual machine exception. java.lang.OutOfMemoryError: GC
overhead limit exceeded.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 1048573 observations read from the data set WORK.BAYDATA.
WARNING: The data set WORK.ROC may be incomplete. When this step was
stopped there were 899988
observations and 7 variables.
WARNING: The data set WORK.AGG may be incomplete. When this step was
stopped there were 0
observations and 7 variables.
NOTE: PROCEDURE LOGISTIC used (Total process time):
real time 1:24.00
cpu time 39.86 seconds

However, it looks like this was addressed in another past post. (
http://support.sas.com/kb/55/533.html) I'm sorry if I wasted your time.
I'll try what the post says. Thank you again for leading me in the right
direction.

Regards,
Adam

Re: Problem with PROC logistic ROC with Large Dataset

sbxkoenk — Sun, 16 May 2021 15:55:36 GMT

Hello,

I do not think that post (Problem Note 55533) will help you.

You get the same ERROR message but you are not using SAS® Asset Performance Analytics neither PROC PRINCOMP (unless there's a PROC PRINCOMP in the ROCPLOT macro but I doubt it).

The message

<< ERROR: Java virtual machine exception. java.lang.OutOfMemoryError: GC overhead limit exceeded. >>

is simply telling you, according to me, there's too much to plot (too many points, too many labels, too many ...).

It's a message probably thrown by a Statistical Graphics procedure (SG_something_ like SGPLOT). So it's not caused by your PROC LOGISTIC.

Kind regards,

Koen

Re: Problem with PROC logistic ROC with Large Dataset

sbxkoenk — Sun, 16 May 2021 16:16:41 GMT

Hello,

Maybe this post will help you better.

Usage Note 31184: The error "java.lang.OutOfMemoryError: Java heap space" might occur when you use ODS Graphics with a large data set.

https://support.sas.com/kb/31/184.html

If you do not want to edit your your SAS 9 configuration file, you may still be able to display your ROCPLOT by modifying your ODS options for the plot-procedure, there's a whole lot of options affecting ODS memory usage for graphs (See documentation).

Good luck,

Koen

Re: Problem with PROC logistic ROC with Large Dataset

StatDave — Sun, 16 May 2021 18:44:19 GMT

It might be that the Java errors will go away by simply closing and restarting SAS and then running the macro before doing a lot of other things. It is the number of observations in the OUTROC= data set from PROC LOGISTIC and input with the INROC= macro option that will be an issue rather than the number of observations used to fit the model. If there are a lot of observations in the INROC= data set, then you might want to use one the thinning methods described in the macro documentation. It would help to see the macro call that you used, but the "..." in the Label might be due to you specifying a large number of labeling items in the ID= macro option. Using fewer might help.

Re: Problem with PROC logistic ROC with Large Dataset

CannonballZ — Mon, 17 May 2021 00:09:50 GMT

Thanks again, Koen. I modified the .cfg file as the link instructed and it worked. In SAS 9.4 on my workstation, the file is in the following directory:

SASHome/SASFoundation/9.4/nls/en