topic Re: Warning Message when using Proc Logistic with Offset Option in Statistical Procedures

Warning Message when using Proc Logistic with Offset Option

Lobbie — Thu, 27 Aug 2020 02:51:24 GMT

Hi,

I have a training set for a binary classification problem that I balanced 50:50 for Responder = 1 or 0 respectively. I have also referred to this Note (https://support.sas.com/kb/22/601.html) about including the Offset variable because my data's true proportion is about 10:90 instead of 50:50. The code is below,

ods graphics on;
proc logistic 
	Data = work.train_stdize 
	outmodel=work.mymodel 
	outest=work.mdl_betas 
	namelen=32;
	class &class_var. / param=ref;
	model responder(event='1') = &class_var. &num_var. / stb lackfit ctable pprob=0.5 offset=off;
	score data=work.train_stdize fitstat out=work.trainpred outroc=work.troc;
run;
ods graphics off;

I get the warning message,

NOTE: Convergence criterion (GCONV=1E-8) satisfied.
WARNING: The information matrix is singular and thus the convergence is questionable.

If I exclude the offset=off in the option and rerun the logistic regression code, I don't get this warning message?! I checked all my class and numeric variables and can confirm there are no collinearity.

Any ideas on how I can fix this or just ignore the warning message?

Thanks,

Lobbie

Re: Warning Message when using Proc Logistic with Offset Option

SteveDenham — Thu, 27 Aug 2020 11:50:24 GMT

The note you referenced makes it clear that the WEIGHT option is generally superior to the offset method. I believe that you already have all the ingredients to create a weight variable, since these are included in the calculation of the offset variable 'off' (r sub1 and p sub1). Have you tried that approach? If you get the same sort of error then the problem might be that there is some sort of collinearity or quasi-separation in the &groupvar and &numvar variables.

SteveDenham

Re: Warning Message when using Proc Logistic with Offset Option

Ksharp — Thu, 27 Aug 2020 11:56:02 GMT

Try using PEVENT= instead of OFFSET .

proc logistic data=sashelp.class;
model sex=weight height/pevent=0.2 ctable;
run;

Re: Warning Message when using Proc Logistic with Offset Option

Lobbie — Fri, 28 Aug 2020 00:41:11 GMT

@Ksharp , I tried using PEVENT= as you suggested. It did not change the values of the intercept and the coefficients. It also did not adjust the predicted probabilities based on prior. It did give me a classification table from 0% to 100% just as the documentation stated. However, it is not what I am after. Thanks.

Re: Warning Message when using Proc Logistic with Offset Option

Lobbie — Fri, 28 Aug 2020 01:27:40 GMT

Hi @SteveDenham,

I did try using weight and am still getting the warning message. While I can confirm there is no collinearity, it is possible there are some sort of quasi-separation issue in the Class variables like you suggested. Questions are,

Why removing the offset or weight option, the warning message goes away? I read somewhere if it is quasi-separation, the model is still ok because only maximum likelihood cannot be computed and we can ignore the warning. I think I am fine with this as long as the model has good AUC, low misclassification, and the differences between these metrics of the train and test sets are not too big e.g. < 5% diff.

I suppose with the use of offset or weight option in Proc Logistic, and to get the adjusted prediction for each observation, I will still have to use priorevent=&pi1 in the score statement? Example below (&pi1 is the prior event value),

ods graphics on;
proc logistic 
	Data = work.train_stdize 
	outmodel=work.mymodel 
	outest=work.mdl_betas 
	namelen=32;
	class &class_var. / param=ref;
	model responder(event='1') = &class_var. &num_var. / stb lackfit ctable pprob=0.5 offset=off;
	score data=work.train_stdize fitstat out=work.trainpred outroc=work.troc priorevent=&pi1;
	score data=work.validate_stdize fitstat out=work.validpred outroc=work.vroc priorevent=&pi1;
	score data=work.osd_stdize fitstat out=work.osdpred outroc=work.oroc priorevent=&pi1;
run;
ods graphics off;

Thanks,

Lobbie

Re: Warning Message when using Proc Logistic with Offset Option

SteveDenham — Fri, 28 Aug 2020 13:22:30 GMT

Since the error message still appears using the WEIGHT statement, this has moved beyond my experience. Get with the folks at Tech Support - they should be able to decipher what is going on, and how to get the correction for oversampling done correctly.

SteveDenham

Re: Warning Message when using Proc Logistic with Offset Option

Lobbie — Tue, 01 Sep 2020 00:07:33 GMT

Hi @SteveDenham,

I tried again with some modifications to the Class variables with Weight method, and it worked now. I also notice when I used the Offset method, the DF of one of the levels in a categorical variable is 0 or NULL, and this may be the cause of the warning message.

All good and thanks for your help,

Lobbie