1

There is a sizeable body of literature on the issue of multiple maximizers in maximum likelihood estimation, such as

https://projecteuclid.org/journals/statistical-science/volume-15/issue-4/Eliminating-multiple-root-problems-in-estimation-with-comments-by-John/10.1214/ss/1009213001.full

I am wondering if anyone is aware of any datasets (and choice of likelihood function) that exemplify this behavior? One possibility is to to a linear regression using a Cauchy loss function rather than square the residuals, but it feels contrived. Gaussian mixture models are another example, but I am unaware of any well-studied datasets in which the issue of multiple solutions comes into play.

kjetil b halvorsen
  • 77,844
Tom Solberg
  • 129
  • a linear regression with Gaussian errors and a rank-deficient design matrix is a nice example. – John Madden Feb 18 '24 at 17:14
  • @JohnMadden yes, but it's not an example of the phenomenon in the paper, which is multiple isolated maxima – Thomas Lumley Feb 18 '24 at 21:39
  • 1
    Here is an example of a bimodal likelihood function https://stats.stackexchange.com/questions/468227/ it happens because two means are estimated with a boundary restriction $0 \leq \mu_2-\mu_1 \leq 2$ – Sextus Empiricus Feb 18 '24 at 21:43
  • I can imagine some non-convex problems might be problematic, like penalty terms that are more extreme than lasso. Non-negative regression is another case of boundary restrictions that can cause local maxima. – Sextus Empiricus Feb 18 '24 at 21:54
  • Factor analysis is another famously multimodal problem – Thomas Lumley Feb 18 '24 at 21:57
  • The case of the Cauchy loss function has been described in a simple form here: https://stats.stackexchange.com/questions/562572/ it is the non-convexity of the log-likelihood function for a single data point that can cause problems. (a cost function that is a sum of convex cost functions, will be convex, a cost function that is a sum of non-convex functions may not be non-convex and can have multiple local maxima) – Sextus Empiricus Feb 18 '24 at 22:21

0 Answers0