5

Josh Starmer says it in here.

I have been searching for a simple way to understand likelihood and it's Bayesian and Frequentist use.

Josh's way seems simple to me.

likelihood is the y-axis coordinate

Is he correct?

User1865345
  • 8,202
Kirsten
  • 763
  • 2
    I have not looked at the links, but note that a likelihood function is not necessarily a PDF. – Galen Oct 16 '23 at 21:51
  • 2
    It depends on what exactly you mean by "a pdf." The likelihood is a function of two sets of variables -- data and parameters -- and, by definition, gives a probability value (or probability density value) for the data when the parameter values are specified. Have you looked at our highly voted threads about likelihood? – whuber Oct 16 '23 at 22:06
  • Thank you. I have updated the question to use the same words as Josh. – Kirsten Oct 16 '23 at 22:29
  • 2
    I hope he didn’t use that exact phrasing, because I think a reasonable interpretation of “distribution curve” is the CDF. – Dave Oct 16 '23 at 22:35
  • Thank you I have pasted some pictures in from the you-tube. From the picture I understand he means the pdf – Kirsten Oct 16 '23 at 22:53
  • 3
    Thank you for the images. The answer to "is he correct" is no: those images confuse probability density with likelihood. Please use those italicized terms to search our site for further information. – whuber Oct 16 '23 at 23:58
  • 2
    Starmer has some decent material, but I’ve definitely watched some of his videos and thought, “Not exactly…” – Dave Oct 17 '23 at 00:07

1 Answers1

4

WHERE STARMER IS CORRECT

When you limit consideration to continuous distributions (perhaps absolutely continuous), then the claim is correct: the likelihood value relates to the y-axis value of the PDF. Note that, contrasting colloquial use, “likelihood” and “probability” differ in meaning as technical terms in statistics, as these continuous distributions have the probability of any one point equal to zero, while the y-axis values have no upper bound on how high they can go.

WHERE STARMER IS WRONG

Likelihood is a general enough idea that it is inadequate to limit consideration to the continuous case. In the discrete case, for instance, think about how high the “PDF” goes on the y-axis. There is a sense in which it goes up to infinity! Thus, the idea of considering likelihood to be the y-axis value breaks down.

It might be worth thinking about what a “PDF” would look like for a distribution that has half of the density uniform on $[0,1]$ and the other half on exactly $1/2$. It’s easy to visualize the first half as just being a straight line $y=1$ on $[0,1]$ and zero everywhere else, but how high does the “PDF” go at exactly $1/2?$

Dave
  • 62,186
  • I am understanding from this post that likelihood can be infinite ~https://stats.stackexchange.com/questions/140463/can-the-likelihood-take-values-outside-of-the-range-0-1 – Kirsten Oct 17 '23 at 21:08
  • @Kirsten What in that link makes you think that likelihood can be infinite? I did not read it in detail but did not see any such claim. – Dave Oct 17 '23 at 21:10
  • Sorry. It is in the answer here ~https://stats.stackexchange.com/questions/4220/can-a-probability-distribution-value-exceeding-1-be-ok – Kirsten Oct 17 '23 at 21:30
  • @Kirsten Whose answer? – Dave Oct 17 '23 at 21:37
  • The accepted one with the green tick – Kirsten Oct 17 '23 at 21:56
  • 3
    @Kirsten That is correct: because probability densities can become infinite, so can likelihoods. It's virtually certain, though, that any likelihood (for any reasonable model) evaluated on data will be finite. As an example, consider a Gamma$(\theta)$ model for a single observation $x.$ When $\theta \lt 1,$ the likelihood for $x=0$ is infinite. However, the chance that $x=0$ is nil. That's the distinction between probability density and probability that the video is attempting to illustrate. – whuber Oct 17 '23 at 22:26
  • @whuber so likelihood is probability density? according to josh but not you? – Kirsten Oct 18 '23 at 01:18
  • 2
    @Kirsten The two can coincide but do not have to. “These are sometimes the same” hardly seems like a legitimate mathematical definition. – Dave Oct 18 '23 at 01:22
  • 1
    @Kirsten There's also a philosophical diference in that the likelihood fixes a value and varies the parameters(s), while the PDF fixes the parameters and then varies the value. What this means, at least in the (absolutely) continuous case such as with a Gaussian distribution, is that the likelihood looks at the height of the PDF at, say, $x=0$ as the parameters change, such as seeing which parameter gives the highest value at $x=0$ (so $\mu=0$ for a Gaussian distribution). In contrast, the PDF would fix the parameter and then calculate how high the PDF is at $x=0$. – Dave Oct 23 '23 at 20:46
  • Thanks Dave. Do you have a visual way of explaining this? Josh explains that the distributions can be moved. ~https://www.youtube.com/watch?v=pYxNSUDSFH4 – Kirsten Oct 24 '23 at 21:22
  • Should that be a fresh question? – Kirsten Oct 24 '23 at 21:38
  • 1
    @Kirsten I thought he explained it quite well and even mentioned in the beginning of the video that the explanation applies to continuous distributions. What do you find to be missing from his animation? – Dave Oct 24 '23 at 21:40
  • Thanks Dave. I just wanted to confirm there wasn't something missing. I am glad you like it. – Kirsten Oct 29 '23 at 06:55