topic Re: Mixed effect logistic regression model in Statistical Procedures

Mixed effect logistic regression model

Barkamih — Wed, 13 Nov 2019 14:00:10 GMT

HI all

I'm trying to build a mixed-effects logistic regression model by using one variable as a random effect

I did proc logistic regression but can not take the random effect variable!

What do you think the best Prc to use in this situation?

These are the codes that I tried

the third one takes a random effect variable but I did get any results.

So guys any suggestion to solve this issue?

best regards

Ibrahim

proc logistic data = have;
class year month;
model mpd = year month;
run; quit;
*//////////////////////;

proc GENMOD data = have;
class year month ;
model mpd = year month;
run; quit;
 */////////////////////;

proc GLIMMIX data = have;
class year month;
model MPD = YEAR month;
RANDOM HERD; 
RUN; QUIT;

Re: Mixed effect logistic regression model

Rick_SAS — Wed, 13 Nov 2019 14:05:25 GMT

> the third one takes a random effect variable but I did [not] get any results.

1. What happened? Error? The proc never finished?

2. How many distinct levels of the HERD variable do you have?

3. Is the response variable binary? If so, you need to use DIST=BINARY on the MODEL statement for PROC GENMOD and GLIMMIX

If the response variable is not binary, you might want to read the 2009 paper "All the Cows in Canada: Massive Mixed Modeling with the HPMIXED Procedure",

which discusses efficiency issues in large linear mixed models.

Re: Mixed effect logistic regression model

Barkamih — Wed, 13 Nov 2019 14:25:12 GMT

thanks million for your response

1- this is the log message

NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE GLIMMIX used (Total process time):
real time 0.63 seconds
cpu time 0.53 seconds

2- I have a large number of herds, more than 8000.

3= Is the response variable binary?

yes, which is like 0 or 1.

So, in this case how my code should look like?

best regards

Re: Mixed effect logistic regression model

Rick_SAS — Wed, 13 Nov 2019 14:34:35 GMT

The NOTE says there are errors, so please check the log for ERROR and WARNING messages. Please post the complete portion of the code from the last PROC GLIMMIX statement until the end of the log.

Re: Mixed effect logistic regression model

Barkamih — Wed, 13 Nov 2019 14:37:20 GMT

this is the all message in log

156 proc GLIMMIX data = herd;
157 class testyear TIME;
158 model MPD = TESTYEAR TIME;
159 RANDOM HERD;
160 RUN;

160! QUIT;

NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE GLIMMIX used (Total process time):
real time 0.64 seconds
cpu time 0.54 seconds

Re: Mixed effect logistic regression model

Rick_SAS — Wed, 13 Nov 2019 14:52:55 GMT

Without error messages, I cannot determine what might be going wrong. Can you run the following small example? Do you get results and a clean error-free log?

data herd;
call streaminit(1);
do herd = 1 to 10;
   do testyear = 2005, 2015;
      do Time = 1 to 6;
         eta = -1 + 0.1*herd + 0.5*Time - 2*(testyear=2015);
         mu = logistic(eta);
         mpd = rand("Bernoulli", mu);
         output;
      end;
   end;
end;

proc GLIMMIX data = herd;
   class testyear TIME;
   model MPD = TESTYEAR TIME / s dist=binary;
   RANDOM HERD;
RUN;

Re: Mixed effect logistic regression model

Barkamih — Wed, 13 Nov 2019 15:02:05 GMT

not yet

49 data herd;
50 call streaminit(1);
51 do herd = 1 to 10;
52 do testyear = 2005, 2015;
53 do Time = 1 to 6;
54 eta = -1 + 0.1*herd + 0.5*Time - 2*(testyear=2015);
55 mu = logistic(eta);
56 mpd = rand("Bernoulli", mu);
57 output;
58 end;
59 end;
60 end;
61

NOTE: The data set WORK.HERD has 120 observations and 6 variables.
NOTE: DATA statement used (Total process time):
real time 0.03 seconds
cpu time 0.03 seconds

NOTE: Writing HTML Body file: sashtml.htm
62 proc GLIMMIX data = herd;
63 class testyear TIME;
64 model MPD = TESTYEAR TIME / s dist=binary;
65 RANDOM HERD;
66 RUN;

ERROR: Permanent copy of file WORK.UTDATA.UTILITY was deleted.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE GLIMMIX used (Total process time):
real time 0.79 seconds
cpu time 0.28 seconds

Re: Mixed effect logistic regression model

Rick_SAS — Wed, 13 Nov 2019 15:21:54 GMT

There is something wrong with your version of SAS (or GLIMMIX). That program should run correctly in a fraction of a second.

The first thing you should do is exit SAS (and probably reboot, too). Maybe SAS was corrupted by a previous submission, such as your big GLMMIX job.

When you log back in and start SAS, run the simple program again. If it still fails, contact SAS technical support and provide them with details about your OS, SAS installation, etc. Also, send them the code you just submitted and the complete SAS log.

Re: Mixed effect logistic regression model

StatDave — Wed, 13 Nov 2019 19:51:31 GMT

Of those three procedures, the only one that can fit a model involving random effects is PROC GLIMMIX. If you are not getting the results you expect, show the results you did get, any messages appearing after GLIMMIX in the SAS Log, and state what specifically you are looking for.

Re: Mixed effect logistic regression model

Barkamih — Thu, 14 Nov 2019 11:43:50 GMT

Firstly I'm so sorry about the late response, it was late night here.

show the results you did get,

this is What I got.

The GLIMMIX Procedure

Model InformationData SetResponse VariableResponse DistributionLink FunctionVariance FunctionVariance MatrixEstimation TechniqueDegrees of Freedom Method

WORK.HERD

mpd

Binary

Logit

Default

Not blocked

Residual PL

Containment

Class Level InformationClass Levels ValuesTESTYEARTIME

11	2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014
12	1 2 3 4 5 6 7 8 9 10 11 12

this is the log message

558
559 proc GLIMMIX data = herd;
560 class testyear TIME;
561 model MPD = TESTYEAR TIME / s dist=binary;
562 RANDOM HERD;
563 RUN;

NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE GLIMMIX used (Total process time):
real time 1.00 seconds
cpu time 0.82 seconds

I'm looking to have a multivariable logistic regression estimation of odd ratios of different months that having mpd including herd as a random effect.

I got my results when I used Logistic Proc, but the problem is I can't include Herd as a RANDOM EFFECT.

I hope this information will help to figure this out

best regards to all

Ibrahim

Re: Mixed effect logistic regression model

Rick_SAS — Thu, 14 Nov 2019 14:04:12 GMT

I assume you quit SAS and rebooted, as I suggested?

My first suggestion is to make sure the program can run on smaller data. Use a WHERE clause to restrict the number of herds to 10 and rerun the program to make sure it runs correctly.

IF the program runs correctly on smaller data, then my guess is that this problem has too many levels of the HERD variable and might be running out of memory. I suggest you read the paper by Kiernan, Tao, and Gibbs (2012)
http://support.sas.com/resources/papers/proceedings12/332-2012.pdf
and experiment to see if any of their suggestions make a difference in solving your problem.

Re: Mixed effect logistic regression model

Barkamih — Thu, 14 Nov 2019 15:41:15 GMT

633
634 proc GLIMMIX data = few;
635 class testyear TIME;
636 model MPD = TESTYEAR TIME / s dist=binary;
637 RANDOM HERD;
638 RUN;

this error from small data that has 3 herds with 100 90 Obs.

anyway, thanks millions for every single letter you had written to this conversation.

I'll try to work and read more to find out more about this issue.

best wishes to all

Re: Mixed effect logistic regression model

Rick_SAS — Thu, 14 Nov 2019 15:50:12 GMT

You quit SAS and rebooted, as I suggested?

Re: Mixed effect logistic regression model

Barkamih — Thu, 14 Nov 2019 16:14:42 GMT

Yes

Re: Mixed effect logistic regression model

Rick_SAS — Thu, 14 Nov 2019 16:26:56 GMT

I don't think there is anything else we can do on this forum. If you cannot run GLIMMIX on the simulated 'herd' data set that I wrote, then there is something wrong with your SAS installation.

Please contact SAS Technical Support and provide them information about your operating system, version of SAS/STAT, and how you are submitting the program (SAS Studio, Enterprise Guide, SAS UE, etc). Send them the following code and the complete SAS log that results when you run it

data herd;
call streaminit(1);
do herd = 1 to 10;
   do testyear = 2005, 2015;
      do Time = 1 to 6;
         eta = -1 + 0.1*herd + 0.5*Time - 2*(testyear=2015);
         mu = logistic(eta);
         mpd = rand("Bernoulli", mu);
         output;
      end;
   end;
end;

proc GLIMMIX data = herd;
   class testyear TIME;
   model MPD = TESTYEAR TIME / s dist=binary;
   RANDOM HERD;
RUN;

Re: Mixed effect logistic regression model

Barkamih — Thu, 14 Nov 2019 17:34:25 GMT

thanks a lot man, I appreciate your help, support and recommendations.

I will let you know about whatever I might get about this issue

Best wishes to all

Regards

Ibrahim Barkamih

Re: Mixed effect logistic regression model

Barkamih — Tue, 19 Nov 2019 15:44:58 GMT

	
proc GLIMMIX data=herd;
class testyear (ref='2004') time (ref='11');
model mpd = testyear time /dist=binary link=logit oddsratio solution;
random herd;
run;

Hi all,

Finally, I got my needs which is the code I posted in this reply.

Also, I would love to thank everyone who helped me to find a solution to this issue.

Rick_SAS your codes also now working perfectly after some changes on my machine.

thanks million man, I appreciate your time and all the suggestions to me.

Best Wishes to all

Ibrahim Barkamih