3

I assume this question has been beaten to death and thus I am just looking for a reference which goes through the details.

Assuming all populations we deal with have finite means and variances (even higher moments if that helps) and the two samples are independent.

The two-sample t-test requires sampling from a normal population. There is a classic t-test assuming the variances of the two populations being sampled are equal and another so called Welch t-test assuming the variances are unequal (with complicated df).

In the case we do not sample from normal populations but suppose we know that the variances are equal we can still show (using CLT and Slustky's theorem) that two sample t-test (for equal variance) is valid (at least as $n_1,n_2\to \infty$). In short, $$\dfrac{(\overline{X}_1-\overline{X}_2)-(\mu_1-\mu_2)}{S_p\sqrt{1/n_1+1/n_2}}= t_{n_1+n_2-2} \to \mathcal{N}(0,1)$$ where the arrow denotes converge in probability.

In the case we do not sample from normal populations but suppose we do know that the variances are unequal does the Welch t-test become valid for large sample sizes? In short, what is the distribution of $$ \dfrac{(\overline{X}_1-\overline{X}_2)-(\mu_1-\mu_2)}{\sqrt{s_1^2/n_1+s_2^2/n_2}}$$ for large $n_1,n_2$?

My thoughts: The answer would be straightforward if $\dfrac{(\overline{X}_1-\overline{X}_2)-(\mu_1-\mu_2)}{\sqrt{s_1^2/n_1+s_2^2/n_2}}$ was exactly $t$-distributed. But (correct me if I am wrong) it is approximately $t$-distributed under appropriate conditions (because there is a $\chi^2$ distribution approximation for the denominator in Student's theorem). So in simpler terms, does the approximation to $t$ distribution improve as $n_1,n_2$ increase? In that case it would also converge in probability to a normal distribution. Does this follow from a general version of Slutsky's theorem and CLT?

user2902293
  • 133
  • 1
    One thing to keep in mind with all this is you're only considering the type I error rate. Presumably one should also care about power (compared to other possible choices). – Glen_b May 09 '22 at 03:18
  • The one-sample t statistic is a ratio with the sample mean $\bar X$ in the numerator and the sample SD $S$ in the denominator. It is only for normal data that $\bar X$ and $S$ are independent. The dist'n of the t statistic does not have a Student t dist'n without indep of numerator and denominator. // Similarly, but with more details for a pooled t test. So the CLT doesn't guarantee a Student t dist'n for non-normal data. Nevertheless, t methods are robust against modent departures from normality of data. – BruceET May 09 '22 at 04:29
  • @BruceET that is interesting. I did not know that sample mean and sample standard deviation for independent only for normal data. Is this easy to see or could you please give a reference? – user2902293 May 09 '22 at 04:45
  • 1
    For your question: the same argument holds when the variances are unequal. The denominator is still a consistent estimator for the standard deviation of the numerator. Then CLT gives (mean zero) AN of the numerator, and then CLT gives that the denominator cancels the sd. For your comment, see https://en.wikipedia.org/wiki/Basu%27s_theorem, particularly the example section – Ben May 09 '22 at 05:11
  • Cochran's Theorem. See Wikipedia. // Also, maybe see answer here. – BruceET May 09 '22 at 08:41
  • Examples/counterexamples illustrated by simulation. – BruceET May 09 '22 at 08:56

1 Answers1

3

The same logic (application of CLT and Slutsky's theorem) will also show convergence of the Welch T-test in the same way as the equal variance T-test. In both cases the denominator is a consistent estimator of the standard deviation of the numerator, so Slutsky's theorem applies in either case.

One thing that is notable here is that the sample mean and sample variance are only independent in the case where the underlying distribution is a normal distribution (this is Cochran's theorem). If the underlying distribution is unskewed but non-normal then the sample mean and sample variance are uncorrelated but not independent, and if the underlying distribution is skewed then the sample mean and sample variance are correlated. Specifically, if we have and underlying distribution with finite skewness $\gamma$ and finite kurtosis $\kappa$ then the correlation between the sample mean and sample variance is (see O'Neill 2014, p. 284):

$$\mathbb{Corr}(\bar{X}_n, S_n^2) = \frac{\gamma}{\sqrt{\kappa - (n-3)/(n-1)}}.$$

(Note that even as $n \rightarrow \infty$ the correlation converges to the non-zero value $\gamma / \sqrt{\kappa-1}$, so the values are asymptotically correlated.)

Ben
  • 124,856