1

I'm refreshing my (basic) knowledge about maximum likelihood and stumbled over

img

which summarizes the concept quite well, I'd say. I only wonder when $ argmax L(\theta) $ is maximized, how it is really done. Looking at the example above, wouldn't I need something that measures and compares the residuals like in least squares? What exactly is $ \theta $ (representing) ?

kjetil b halvorsen
  • 77,844
Ben
  • 3,443
  • 1
    Maximum likelihood estimators are not really about residuals (while in some circumstances using the mean [least squares] or median [absolute average] can be seen as being so). As the name suggests, they are about finding a model parameter which makes the the observed data more likely to have been seen – Henry Jul 12 '22 at 09:00
  • sure, but how does the model compare its correctness? Isn't there a kind of distance/deviation (like a residual) measurement needed? – Ben Jul 12 '22 at 09:02
  • Maximum likelihood is not designed to minimise expected error in some sense. So the answer to "needed?" is no. But you can define your own measure of accuracy and often find this for particular examples of maximum likelihood estimators. – Henry Jul 12 '22 at 11:02
  • Ok, obviously I don't get the concept in general. How exactly is ML done, I mean, $ \theta $ has to be varied, right? What exactly is this parameter representing? – Ben Jul 12 '22 at 11:17
  • I give you three observations: zero, one, and two. I tell you that these are draws from some Gaussian distribution with a variance of one but an unknown mean. You speculate that the mean of the Gaussian is one. How would you calculate the likelihood? What about if you speculate that the mean of the Gaussian is two, three, or eleven? – Dave Jul 12 '22 at 11:57
  • I would put the Gaussian at one so I guess I am doing ML in my head.. and I further guess I used $ \theta $ for it but what does it do? In case of the Gaussian I'd need to calculate/estimate the mean as well as the std. Ah, I remember.. when I wanna' maximize something I derive the function by the parameter I wanna' maximize. Right? But where does $ \theta $ come into play? – Ben Jul 12 '22 at 13:44
  • 1
    Your $\theta$ is $\mu$, as in estimate the $\theta$ of a $N(\theta, 1)$ distribution when you have observed $0$, $1$, and $2$. – Dave Jul 12 '22 at 13:46
  • Just to really grasp this: $ \theta $ becomes $ \mu $ and $ \delta $ in this case? – Ben Jul 12 '22 at 13:47
  • What is $\delta?$ – Dave Jul 12 '22 at 13:48
  • 1
    Residuals are not needed, (but can be useful when you have them). Look through https://stats.stackexchange.com/questions/112451/maximum-likelihood-estimation-mle-in-layman-terms/112480#112480 – kjetil b halvorsen Jul 12 '22 at 13:54
  • @Dave In case of the Normal the standard deviation(?) – Ben Jul 12 '22 at 13:58
  • I defined the variance as $1$ by saying that we want to estimate the $\theta$ of a $N(\mu, 1)$ distribution. – Dave Jul 12 '22 at 13:59
  • so I don't need $ \delta $ but does this heavily change my question that $ \theta $ becomes the parameter I want to maximize? – Ben Jul 12 '22 at 14:01

0 Answers0