0

Let's say we do a classical ANOVA F-test to reject or not reject the hypothesis:

H0: $µ_1 = µ_2 = ... = µ_k$

with $k$ classes, and $n$ observations per class

Are there ranges of $k$, $n$ for which the F-test is not applicable or sub-obptimal?

Example 1: $k=10$, $n=1$ impossible because only 1 observation per class!

Example 2: $k=100$, $n=2$ will probably give a poor test because we have only 2 observations per class

Example 3: $k=3$, $n=100$ seems reliable because we have many observations per class

Is there a rule of thumb when we apply this test, are there requirements such that $n$ should be at least $2 k$ or something like that, for the test to be useful?

Basj
  • 498
  • 1
  • 4
  • 16
  • 1
    It's hard to imagine any such rule of thumb could exist generally, without qualifications, partly because the examples are questionable. Why exactly would "only 2 observations per class" be problematic, given there are therefore 50 replicates? Why would 100 observations be "reliable" without checking how the observations within the classes are distributed? – whuber Jan 08 '24 at 16:49
  • @whuber In Example 2, we don't have 50 replicates, we have 100 classes, and, for each class, 2 replicates. – Basj Jan 08 '24 at 21:40
  • Sorry: so you have 100 replicates. – whuber Jan 08 '24 at 21:42
  • @whuber I'm not sure about the terminology here, but I would say 100 classes for which we don't know if µ1 = µ2 = ... = µ100, and for each class, only two replicates. Why do you say 100 replicates? – Basj Jan 08 '24 at 21:44
  • Because, under the standard homoscedasticity assumption, each pair of replicates (of which there are 100 pairs) gives you some information about the within-group variance. That's 100 independent pieces of information, which isn't bad for estimating a variance. Whether it's a "poor test" or not depends on how large the within-group variance is relative to the between-group variance: and that's the entire idea behind the F test. – whuber Jan 08 '24 at 21:46
  • @whuber So k=3 groups with n=5 samples per group would be ok to perform the test? As well as k=5 groups with only n=3 samples per group? There is no general restriction for $k$, $n$, to perform the test? – Basj Jan 08 '24 at 21:49
  • I am suggesting the opposite: there is no general rule based solely on group sizes affirming when the F test is reliable. It can even work with $n=k=2:$ but in such cases you are relying on your distributional assumptions, whereas with more data you have an opportunity to test those assumptions. – whuber Jan 08 '24 at 21:54
  • You seem to asl about the power of the F-test? Does the following help? https://stats.stackexchange.com/questions/80048/calculating-power-function-for-anova – kjetil b halvorsen Jan 23 '24 at 14:55

0 Answers0