19

I got a log-likelihood value of -34.82, so I am not getting whether the answer which I have got is right or not.

Can the likelihood take values outside of the range $[0, 1]$?

  • 11
    This should not be marked as a duplicate. The likelihood is not the same conceptually as a density and the distinction is important especially for beginners. Only an experienced statistician/econometrician would see the equivalence between this question and the one linked as a supposed duplicate. @whuber – Hirek Mar 05 '15 at 20:16
  • 1
    @Hirek The likelihood is defined as a probability: that is so elementary, it is reasonable to suppose the connection would be apparent to anyone employing the word "likelihood" in any technical sense. Moreover, we have many threads discussing precisely this same question about log likelihood, so the issue is not whether to close this question as a duplicate, but rather what duplicate would be the most helpful. – whuber Mar 05 '15 at 20:53
  • Doesn't a negative log-likelihood corresponds to a positive likelihood less than one? – user20637 Jul 25 '17 at 15:57
  • 2
    @whuber How can that be? Likelihood can be greater than 1, but probability can't! – Stand with Gaza Mar 13 '18 at 16:32
  • 1
    @BjörnLindqvist For continuous distributions, likelihoods drop the infinitesimal probability elements. Thus, they are only defined up to positive multiples in the first place. For discrete distributions likelihoods are indeed probabilities and therefore must be less than $1$. – whuber Mar 13 '18 at 16:54

2 Answers2

25

Likelihood must be at least 0, and can be greater than 1.

Consider, for example, likelihood for three observations from a uniform on (0,0.1); when non-zero, the density is 10, so the product of the densities would be 1000.

Consequently log-likelihood may be negative, but it may also be positive.

[Indeed, according to some definitions the likelihood is only defined up to a multiplicative constant (e.g. see here), so even if the density were bounded by 1, the likelihood still wouldn't be.]

Clarifications as a result of comments/chat: For a continuous distribution, likelihood is defined in terms of density. Density must be at least $0$ and can exceed $1$; and as a result, likelihood can exceed $1$.

Glen_b
  • 282,281
  • what do you mean that it is defined up to a multiplicative constant? The wikipedia article doesn't explain it well either. – Homero Esmeraldo Sep 04 '19 at 00:11
  • And the current wikipedia article explains it better why the likelihood can be calculated with the probability density rather than the probability itself. https://en.wikipedia.org/wiki/Likelihood_function#Likelihoods_for_continuous_distributions – Homero Esmeraldo Sep 04 '19 at 00:24
  • 1
    In response to your first comment, Fisher defined likelihood in such a way that $\mathcal{L}(\theta;\underline{x})=c\cdot \prod_i f_{X;\theta}(x_i)$ for any $c>0$ is a likelihood function (as long as any likelihood comparisons were performed with the same $c$, naturally). – Glen_b Sep 04 '19 at 00:27
  • Are we talking about likelihood of data given parameters here? – Kirsten Oct 09 '22 at 01:38
  • 3
    @Kirsten Sure (as is indicated by the link in my answer) ... I'm responding to the direct question "Can the likelihood take values outside of the range [0,1]?" to which the short version of my answer is "it cannot be below 0 but it can exceed 1". If the downvote was yours, could you clarify the problem you perceive? – Glen_b Oct 09 '22 at 04:36
  • Thanks Glen. Yes it was me and I may be out of my depth. However in my course I am understanding that similar to probability, likelihood of the data given the parameters can not be greater than zero. I found https://khakieconomics.github.io/2018/07/14/What-is-a-likelihood-anyway.html seeming to support this. – Kirsten Oct 09 '22 at 09:42
  • 2
    For a continuous random variable, the probability of the data given the parameters is indeed $0$, but in that case the likelihood is not the probability. Rather for continuous variables it's defined in terms of density. Many posts on site discuss the fact that density and probability are not the same thing. The page you link certainly shows likelihoods greater than 0. – Glen_b Oct 09 '22 at 09:55
  • greater than zero yes. but not exceeding 1? – Kirsten Oct 09 '22 at 10:28
  • 1
    Certainly density can exceed 1, just as I state in my answer. Again, density is not probability: (i) Consider a uniform distribution on (0, 0.1). ... what's the height of the density at x=0.05? (ii) consider a normal, mean 0, s.d. 0.1. What's the height of the density at 0? – Glen_b Oct 09 '22 at 10:50
  • Let us continue this discussion in chat. – Kirsten Oct 09 '22 at 10:57
  • 1
    Please make an edit, @Glen_b, to your post: anything trivial, if you may, so that the downvote could be removed. For context, see the comment. – User1865345 Oct 10 '22 at 05:57
  • 1
    I've done as requested by including clarifications from the discussion. – Glen_b Oct 10 '22 at 15:40
  • 1
    Thanks @Glen_b; +1 from me too. – User1865345 Oct 10 '22 at 16:25
  • How should I ask how likelihood > 1 fits with the discussion at https://stats.stackexchange.com/questions/2641/what-is-the-difference-between-likelihood-and-probability/2646?noredirect=1#comment1095711_2646 Surely it should be mentioned there? – Kirsten Oct 13 '22 at 20:38
  • 1
    I don't think it needs to be mentioned there at all; I think it's covered by the connection to density and the difference between density and probability (addressed in many questions elsewhere on site). If you have a question to ask about answers to another question that can be covered by clarifying one of the answers, you can comment (asking a question you need clarification on); otherwise post a new question (with a link to the old one for context is of value). – Glen_b Oct 14 '22 at 16:42
4

The likelihood function is a product of density functions for independent samples. A density function can have non-negative values. The log-likelihood is the logarithm of a likelihood function. If your likelihood function $L\left(x\right)$ has values in $\left(0,1\right)$ for some $x$, then the log-likelihood function $\log L\left(x\right)$ will have values between $\left(-\infty,0\right)$. For $L\left(x\right)\in\left[1,\infty\right)$ the $\log L\left(x\right)\in\left[0,\infty\right)$. So $-34.82$ is a typical value for a log-likelihood function.

Marco Breitig
  • 336