Definition of likelihood ratio test

Question

Do I understand this correctly:

You fit a model with unknown parameters to a dataset. You choose the parameters so the likelihood of the dataset under the model is maximal. Let this be $L_{max, model1}$. You do the same with a more advanced model. Again you pick the coefficients of the model so that the likelihood of the dataset under the model is maximal. Let this be $L_{max, model2}$

To see which model is the best, you compare the two (maximum) likelihoods. Now what confuses me is the definition of the likelihood ratio test:

"The likelihood ratio is the ratio of the likelihoods of the two models... ".

Shouldn't that be "The likelihood ratio is the ratio of the maximum likelihoods of the two models..."?

Stéphane Laurent · Accepted Answer · 2014-01-20T17:18:34.620

5

You're right in the context of "submodel" testing, the likelihood ratio statistic is the ratio of the maximum likelihoods (not the maximum likelihood estimates: the maximal values of the likelihoods). Consider a statistical model with likelihood $l(\theta \mid y)$ where $y$ is the vector of observations generated from a distribution with parameter $\theta$ belonging to some space $\Theta$. Let $\Theta_0 \subset \Theta$ and imagine you are interested in testing $H_0\colon\{\theta \in \Theta_0\}$. The likelihood ratio statistic is $$lr(y) = \frac{\sup_{\theta \in \Theta}l(\theta \mid y)}{\sup_{\theta \in \Theta_0}l(\theta \mid y)}. $$

But when the test hypotheses are $H_0 \colon\{\theta =\theta_0\}$ vs $H_1 \colon\{\theta =\theta_1\}$, as in the classical Neymann-Pearson lemma, then the likelihood ratio statistic is the ratio of the likelihoods: $$lr(y) = \frac{l(\theta_1 \mid y)}{l(\theta_0 \mid y)}.$$

edited Jan 20 '14 at 17:18

answered Jan 20 '14 at 14:09

Stéphane Laurent

19,179

Is this the distinction between a generalized likelihood ratio test for a composite hypothesis (first case) vs. a likelihood ratio test for a simple hypothesis (second case) as in the answer from @Scortchi ? – Jon Jan 20 '14 at 14:13
@psychometriko I don't know and I'd like to know that too. – Stéphane Laurent Jan 20 '14 at 14:16
My mathematical stats isn't the strongest, but that's what I suspected when reading your (very elegantly worded, for a complicated topic) answer. Hopefully someone can come and shed light on this! – Jon Jan 20 '14 at 14:20
Thank you for your answer! Just for my understanding, can you please explain what "sup" means? – Kasper Jan 20 '14 at 15:24
1

@Kasper $\sup_{\theta \in \Theta}\ldots$ means supremum (maximum) when $\theta$ runs over $\Theta$. – Stéphane Laurent Jan 20 '14 at 15:41
@psychometriko: Yes, 'GLRT' when one is or both are complex, 'LRT' when both are simple. Or 'LRT' for all cases is fine. – Scortchi - Reinstate Monica Jan 20 '14 at 16:16
Not 'complex'; I meant 'composite' – Scortchi - Reinstate Monica Jan 20 '14 at 16:45

Scortchi - Reinstate Monica · Answer 2 · 2014-07-07T15:46:29.300

2

The distinction between the likelihood ratio for completely specified probability mass or density functions (simple hypotheses) & the likelihood ratio for incompletely specified ones (composite hypotheses) is sometimes expressed by calling the latter a generalized likelihood ratio. So your quote could be giving a precise definition of the likelihood ratio proper, or an vague definition (because it doesn't specify how particular parameter values are picked to calculate the likelihood for each model) of the generalized likelihood ratio.

edited Jul 07 '14 at 15:46

answered Jan 20 '14 at 13:34

Scortchi - Reinstate Monica

29,907

I am sorry but I don't get this... – Kasper Jan 20 '14 at 15:51
@Stéphane's first formula is what's sometimes called the generalized likelihood ratio. – Scortchi - Reinstate Monica Jan 20 '14 at 16:11

Definition of likelihood ratio test

2 Answers2

Linked