Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Home
- /
- Analytics
- /
- Stat Procs
- /
- Re: Evaluating the Hessian

Options

- RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Posted 10-30-2013 02:12 PM
(2688 views)

I have been working with Proc Nlin to derive a Hessian matrix of 2nd order derivatives. I use the OUTEST= statement to generate the statistical output and analyze it when the _TYPE_='DETERMIN' and keep the automatic variable labelled _RHS_ which contains the value for the determinant. I don't evaluate the matrix that this determinant is derived from.

My questions concerns the meaning of the Hessian: its magnitude, sign, statistical significance, etc. I can't find any literature on evaluating it...is it a standardized statistic like a t-statistic or a Z-score? Any suggestions for tests to analyze whether or not it's significantly different from zero?

Thank you in advance.

6 REPLIES 6

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Hi Mike,

I think I'm missing something here. Are you asking about the determinant of the Hessian matrix? That tells you whether the Hessian matrix is singular (=bad) or nonsingular (=good). For most of the procs that use Hessians, you can tune this value. The closer it is to zero, the more likely it is to be singular.

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Steve-

Thanks for contributing your point of view. You are correct although the literature I'm familiar with also identifies the determinant as the Hessian. And I have an extended use or interpretation for the Hessian: if the slope is the rate of change in Y given a 1 unit change in X, the Hessian is the acceleration (deceleration) in that rate of change. Gallant spoke of it as an index for nonlinearity.

I understand that the smaller the value is, the more likely it is to be singular or insignificant. I'm looking for a test that allows me to go beyond eyeballing to make a more rigorous, statistical decision.

Mike

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Mike--

There doesn't seem to be a lot of info on my bookshelf on the distributional assumptions associated with the Hessian. I would think that to get a test you would need some sort of estimate of the error associated with the determinant, and then you could apply something like Tschebycheff's theorem. The hard part is that there doesn't seem to even be an empirical measure of the variability, since it is the root of a polynomial. Maybe bootstrapping could give you something.

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

The Hessian is the matrix, not the determinant.

I agree with Steve that a nonparametric bootstrap is probably the best that you can do: resample many times from your data, solve the NLMIX problem that gives the determinant at the optimal value, and look at the distribution of the determinant.

However, we might be able to say more if we have more context. Often the Hessian appears when maximizing the negative log-likelihood function. At a nondegenerate optimal solution, the Hessian will be positive definite, and thus the determinant is positive 100% of the time. At a degenerate solution, the determinant is zero.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Dr. Wicklin:

I wanted to know, if please, more elaborate on your last sentence regarding the maximization and estimation. As you know inverse of hessian in the Newton iteration step does not exist in the case of degenerate solution. How can we then direct the optimization routine to proceed for a solution?

Thank you in advance.

Moshiur Rahman

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

You've asked the million-dollar question. There are various heuristic methods that attempt to restart the optimization, but many times the real problem is that your data do not fit the model. The textbook response is that you should choose a different model, and that would be my recommendation for you.

If you are interested in the theory, there have been many research papers written about what to do when the Hessian is singular. I recommend the 2004 paper by Jeff Gill and Gary King, "What to do when your Hessian is not invertible." Gary King has done some excellent work in several areas of statistics, so I respect his opinions on this topic.

**Available on demand!**

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.