7

I used nlm function in R to do the optimization. When I calculated the correlation between estimated parameters using the inverse of Hessian matrix, I got negative values on the diagonal. My questions are what could be the reason for that issue, and what could be possible solution?

pthao
  • 163
  • 1
  • 7
  • Why is this an issue at all? – whuber Aug 10 '14 at 14:16
  • it means that the variance is negative, which is problematic.@whuber – pthao Aug 10 '14 at 15:40
  • 1
    Sorry, somehow I read "off" the diagonal rather than "on" it. (Thank you for being gentle with what amounted to a very stupid comment!) I am reminded of a interesting dialog that took place a few years back about a remarkably similar problem: see whether the thread at http://stats.stackexchange.com/questions/7308/can-the-empirical-hessian-of-an-m-estimator-be-indefinite gives you any useful ideas. Don't focus on my answer: take a look at all the answers and comments to appreciate the issues that came up. – whuber Aug 10 '14 at 18:56
  • What were you optimizing? Was it a max or a min problem? e.g. were you maximizing the likelihood directly? Maximizing the log of a likelihood? Minimizing $-\log\cal{L}$? Minimizing $-2\log\cal{L}$? – Glen_b Aug 11 '14 at 00:49
  • 1
    I was minimizing the -2loglikelihood @Glen_b – pthao Aug 12 '14 at 01:35
  • 1
    Hmm. More details might help people guess what might be going on. – Glen_b Aug 12 '14 at 01:55
  • 1
    So basically, I had 8 parameters in total and my response variable is serological data.The likelihood is defined as a mixture of 2 normal densities. The things is, the same code if applied to another antibody, it was ok. I have 22 antibodies, and some of them gave me negative variances after inverse the Hessian matrix. – pthao Aug 12 '14 at 02:20
  • 1
    As suggested here, some tricks like the package numDeriv might help. – Randel Jun 14 '16 at 17:31
  • 1
    3.3 years later ... The OP should show the exact manner in which nlm was used. Among other things, routines for calculating gradient and Hessian can be optionally supplied as inputs to nlm, else numerical derivatives are used. We don't know whether gradient and Hessian were supplied, and if they were, that they are correct. Is the non-psd Hessian which we are all so enjoyably discussing the Hessian which was returned as an optional output by nlm? What solution status was returned by the optimizer? – Mark L. Stone Nov 24 '17 at 15:20

1 Answers1

1

The variance covariance matrix can be aproximated by the inverse of the negative Hessian H (matrix of second order partial derivatives). May be the reason is that you are approximating using the inverse $H$, when it should be the inverse of the negative H.

Carlos S Traynor
  • 119
  • 5