topic Re: Hessian and gradient in proc mixed in Statistical Procedures

Hessian and gradient in proc mixed

mgx — Thu, 19 May 2022 09:19:47 GMT

I would like to obtain the hessian and the gradients of my fixed effects in proc mixed. In proc glimmix I can obtain it with the code below, but I can not find how to have the equivalent my_gradient and my_hessian dataset in proc mixed. Due to characteristics of my model, I cannot use proc glimmix/proc nlmixed to fit my model instead of proc mixed.

proc glimmix data=data GRADIENT HESSIAN SUBGRADIENT=my_gradient;
model response0(ref='1') = var /noint s dist=byobs(distvar) link=byobs(linkvar) covb solution;
random intercept time/type=un subject=surveyID;
ods output hessian=my_hessian ;
run;

Re: Hessian and gradient in proc mixed

SteveDenham — Thu, 19 May 2022 12:53:12 GMT

Well, I think you have a handle on it using GLIMMIX. Check out @Rick_SAS 's blog post on ways to obtain the Hessian https://blogs.sas.com/content/iml/2019/02/13/hessian-maximum-likelihood-covariance.html#:~:text=Some%20regression%20procedures%20support%20the%20COVB%20option%20%28%22covariance,the%20PROC%20NLMIXED%20statement%20to%20display%20the%20Hessian.

If you have a different endpoint (continuous with normal errors), I think you can use a STORE statement in PROC MIXED, and then use the SHOW statement in PROC PLM to get the Hessian. I don't see an equivalence for the gradient, though.

SteveDenham

Re: Hessian and gradient in proc mixed

mgx — Thu, 19 May 2022 12:57:44 GMT

Thank you for your response!

Re: Hessian and gradient in proc mixed

mgx — Fri, 20 May 2022 11:02:11 GMT

It is possible to implement the model with proc nlmixed, but reaching convergence is very time-consuming and maybe not possible. However, even without convergence SAS creates a dataset with the gradients. Are these values valid, even if the model did not reach convergence?

Re: Hessian and gradient in proc mixed

Rick_SAS — Fri, 20 May 2022 11:42:33 GMT

> Are these values valid, even if the model did not reach convergence?

Yes, they are "valid". The numbers are the gradient at the end of each iteration. Because PROC NLMIXED supports the BOUNDS statement, the final gradient might not be the zero vector. If the model does not converge, the final gradient might not be zero.

For unconstrained linear regression, there is no reason to report the gradient because it is the zero vector at the solution.

Re: Hessian and gradient in proc mixed

mgx — Mon, 15 Aug 2022 10:32:19 GMT

Related to this question: in proc mixed you cannot get the hessian for the fixed effects estimates, only the variance-covariance matrix (the store statement gives the error that the model does not contain the hessian). What is the difference between the hessian en the variance-covariance matrixof the fixed effects?

Re: Hessian and gradient in proc mixed

Rick_SAS — Mon, 15 Aug 2022 11:03:02 GMT

The Hessian is the inverse of the covariance matrix of the fixed effects. See

https://blogs.sas.com/content/iml/2019/02/13/hessian-maximum-likelihood-covariance.html