3

I am given the following R output for a TOST test and would like to confirm whether my understanding is correct: p1 and p[2] are both <0.05 and thus one can reject the null hypothesis that both values are different and instead conclude that they are equal?

enter image description here

Alexis
  • 29,850
  • 1
    Equivalent, not equal (and with all of the usual caveats about p-values and hypothesis testing). – Dave Aug 26 '21 at 12:05

1 Answers1

2

You can reject the negativist null hypothesis, and conclude that you found evidence that the means are equivalent (not equal), at the $\alpha$ level of significance, and for your specific equivalence threshold.

Two one-sided tests for equivalence provide evidence against a negativist null hypothesis ($\text{H}^{-}_0$) that the difference in quantities (which I will label $\theta$) is at least some threshold level of relevance/equivalence which I will label $\Delta$ here: $\text{H}^{-}_{0}\text{: }|\theta| \ge \Delta$,$^{\dagger}$ and the alternative hypothesis is $\text{H}^{-}_{\text{A}}\text{: }|\theta| < \Delta$. Put another way, the null is that either $\text{H}^{-}_{01}\text{: }\theta \ge \Delta$, or that $\text{H}^{-}_{02}\text{: }\theta \le -\Delta$, with the alternative being both $\text{H}^{-}_{01}\text{: }\theta \ge \Delta$ and $\text{H}^{-}_{02}\text{: }\theta \le -\Delta$… i.e. $-\Delta < \theta < \Delta$.

Notice that very last part: $\theta$ lying within an interval bounded by $-\Delta$, and $\Delta$ is the definition of equivalence. So when you reject $\text{H}^{-}_{01}$ and $\text{H}^{-}_{02}$ (i.e. the two one-sided hypothesis from two one-sided tests) you have evidence of equivalence. Finally, notice that $-\Delta < \theta < \Delta$ is not the same thing as $\theta = 0$.

Finally, the conclusion drawn from an equivalence test is not only situated with respect to the researcher's choice of $\alpha$/type 1 error rate, but is also situated with respect to the researcher's choice of $\Delta$: the smalest size difference of quantities that the researcher finds relevant.


$^{\dagger}$ Negativist null hypotheses may also be expressed using asymmetric equivalence regions. For example, when testing the equivalence of two proportions or two incidence rates, it makes sense for the lower equivalence boundary to be $\frac{1}{\Delta}$ instead of $-1\times \Delta$.

Alexis
  • 29,850