0

There exist many "variance equality tests", like the F-test and its more robust cousins: Levene's, Bartlett's, the Brown-Forsythe.

Given two samples A and B with variances $\sigma_A^2$ and $\sigma_B^2$, one can use them to evaluate the null and alternate hypotheses:

  • H0) $\sigma_A^2 = \sigma_B^2$
  • H1) $\sigma_A^2 \neq \sigma_B^2$

Specifically, a sufficiently small $p$ allows us to reject the null hypothesis. But a large $p$ just fails to reject the null. A large $p$ does NOT say the variances are equal. The test is inconclusive.

I am looking for a test which can say the variances are equal. One which constructs the hypotheses with some small $\Delta$ as

  • H0)

    $\sigma_A^2 - \sigma_B^2 \le -\Delta$

    or

    $\sigma_A^2 - \sigma_B^2 \ge \Delta$

  • H1)

    $-\Delta \lt \sigma_A^2 - \sigma_B^2 \lt \Delta$

and allows rejection of the non-equivalence H0 with a sufficiently small $p$.

I think I am asking for something very similar to TOST or Campbell and Ladkens Sec 4, but for distribution variances instead of means. Those are conveniently implemented in R packages here and here, resp.

I figured my problem would be commonplace enough that I could find a similar, single R call. But no luck so far. Help is very appreciated.

kdbanman
  • 857
  • 1
  • 8
  • 15
  • 2
    Isn’t this exactly what TOST does? The typical examples are for means, sure, but nothing about TOST is specific about means. – Dave Feb 05 '24 at 01:28
  • 1
    With variance it would make more sense to try to test whether the ratio was close to 1 than the difference close to 0, but (given that) you'd still seek to construct two one-sided tests. – Glen_b Feb 05 '24 at 01:39
  • @Dave The more I look, yes I think you're right. – kdbanman Feb 05 '24 at 01:41
  • With @Glen_b's point taken, this tutorial lays out the process pretty much exactly. – kdbanman Feb 05 '24 at 01:41
  • 2
    I suspect that a reason for this not being commonplace is that if $\Delta$ is small relative to $\sigma^2_A$ and $\sigma^2_B$ then the sample size to get a non-zero probability of rejecting the both the two null hypotheses becomes so big that it is usually not useful. Meanwhile a large $\Delta$ is hardly persuasive of equality. Your link suggests that even aiming for a ratio neither below $\frac23$ nor above $\frac32$, to get a power of $0.9$ when there is precise equality you need a sample size of $532$ ($266$ on each distribution). It would be bigger with a more plausible interval. – Henry Feb 05 '24 at 01:49
  • @Henry Very interesting and useful point! (Luckily, I'm working in the samples sizes of thousands to tens of thousands.) – kdbanman Feb 05 '24 at 01:51
  • Speaking about TOST for mean equivalence for a second: Is the method from Campbell and Ladkens Sec 4 considered a TOST method?? I think so, because it just sets a TOST-style confidence interval around ANOVA effect size $\eta^2$, rather than around the mean directly... – kdbanman Feb 05 '24 at 01:52
  • 1
    One thing to keep in mind is that the usual test under an assumption of normality will still tend to have the same sensitivity to non-normality (in particular, sensitivity to the situation where kurtosis is different from that of the normal) as you have when testing equality of variance. – Glen_b Feb 05 '24 at 02:08
  • 1
    @Glen_b Of course! Hence my mentioning a few variance tests that are robust to non-normality. – kdbanman Feb 05 '24 at 04:21

0 Answers0