6

This is a soft question, and I hope I can keep it on-topic.

According to the Neyman–Pearson lemma, (generalized) likelihood-ratio tests have the highest possible statistical power, yet LRTs are quite uncommon in my undergraduate textbooks. I have learnt $z$-tests, $t$-tests, ANOVA, a whole class of normality tests, along with their multivariate variants, to name just a few. All of these tests were derived by careful construction, so that they can have some "common" distributions like $\chi^2$, $F$ and $T^2$.

I'm just wondering why do we have to take the pain of constructing a statistic which has a simple distribution, instead of simply using LRT for all kinds of model validation. Indeed, many, if not all, of the above mentioned tests can be shown to be equivalent to a LRT, but I was referring to LRTs directly constructed by log-likelihood functions. The point is, a handcrafted test statistic can never do better than a no-brainer LRT, so why bother?

Well, I probably shouldn't call LRT a no-brainer, but we have the EM algorithm! From my understanding, performing a LRT is as simple as finding the MLE of unspecified parameters with the magical EM algorithm, plugging them to the likelihood function, and calculating a LRT statistic which asymptotically has a $\chi^2$ distribution. Isn't this the ultimate solution to every model validation problem? One issue is that you won't know the exact distribution of LRT, but in the case of small sample size, simulating its distribution with a computer shouldn't be very hard.

My own answer to this question would be "Because STATS people traditionally admire mathematical elegance, which is kinda pointless given the computational power of modern computers". I mean no offense, but I believe Statistics won't be what it is now if computers were invented one century earlier: people would spend less time tinkering with their models, and focus more on attacking real-world problems. I could be wrong, of course, and these constructed statistics will have some nice properties which can make the corresponding test better in some way.

The tone in this post might sound like trolling/complaining, but as a confused STATS major, I do intend to ask it in a helpful manner. I have did a lot of math (probability theory, to be more exact), and now I'm asking about the justification.

nalzok
  • 1,787
  • 7
    This is not an objection, but I want to note my surprise at reading "if computers were invented one century earlier: people would spend less time tinkering with their models, and focus more on attacking real-world problems." The methods found in your textbook were developed a century ago precisely to attack real-world problems, building on theory developed since c. 1500 CE to understand problems of insurance, mortality, and more. You might be interested to read some of the history, especially about Galton, Pearson, and Fisher. Steven Stigler has some nice books on the subject. – whuber Oct 04 '19 at 17:36
  • @nalzok , are you sure LRT can be used for any model comparison? I'm under the impression its assumptions only hold for nested models. – curious_dan Oct 04 '19 at 17:52
  • 2
    @curious_dan There are some differences between "LRT" and "generalized LRT". Please see https://stats.stackexchange.com/a/82822/108877 – nalzok Oct 04 '19 at 17:58
  • @nalzok , thank you for differentiating--- Do you have a reference for a good proof of the properties of the generalized LRT? I've been meaning to learn about this – curious_dan Oct 04 '19 at 18:04
  • @curious_dan Unfortunately no, it's not even clear to me why the generalized LRT is a generalization, but the properties of the original LRT were proved in On the Problem of the Most Efficient Tests of Statistical Hypothesis, 1933, Neyman & Pearson. – nalzok Oct 04 '19 at 18:12
  • @nalzok , thanks again! I will look into this. It seems without a proof of the properties of the generalized LRT, we don't know the assumptions its making, and will have a hard time answering your question. If there are any such assumptions, there may be situations in which those assumptions don't hold, and we may be better off deriving model comparison statistics that fit our situation. – curious_dan Oct 04 '19 at 18:17
  • @curious_dan OK, I'll keep you posted when I find a good proof! – nalzok Oct 04 '19 at 18:24

1 Answers1

1

Your question is very broad, so this is more of a comment. One possible problem is that the (generalized) likelihood ratio test might be suboptimal in some cases, this paper points to some such examples.

You say Indeed, many, if not all, of the above mentioned tests can be shown to be equivalent to a LRT, but I was referring to LRTs directly constructed by log-likelihood functions. The point is, a handcrafted test statistic can never do better than a no-brainer LRT, so why bother? Well, even then, we would need the null hypothesis distribution of the directly constructed LRT, and maybe it is easier to find that for some transformation?

In no-standard cases the non-asymptotic null distribution an be quite complicated ...

kjetil b halvorsen
  • 77,844