Normality assumption for t/z tests

Question

I have what is probably a silly question regarding normality assumptions for t/Z tests. As I understand, t/z tests require that sample data was obtained from populations following a normal distribution.

...So, what does this actually mean in practice?

Does it imply that the distributions of each variable we collect should be approximately normal as well? Or does it just mean that, even if they are not normal, it's fine as long as the population they were sampled from is arguably normal? The logic in the last sentence seems tenuous.

Additionally, what does this imply for regression analysis? I understand that regression analysis 'only' requires that the error terms are normally distributed. But, I am wondering if the above implies anything for t tests on the OLS parameters.

Answer 1

Let's start with the t test. The assumption can be clarified if we consider the definition of a $t$-random variable. Let $Z\sim \mbox{Normal}(0, 1)$ be a standard normal random variable, and $V\sim \chi^2_{\nu}$ be a chi-square random variable with degrees of freedom $\nu$.

The quantity $t=Z / \sqrt{V/\nu}$ is then Student-T distributed with degrees of freedom $\nu$. This is easy to verify with simulation

N = 1e4
nu = 10
Z = rnorm(N)
V = rchisq(N, df=nu)
t = Z/sqrt(V/nu)
hist(t, probability = T)
x = seq(-5, 5, 0.01)
y = dt(x, df=nu)
lines(x, y, col='red')

Ok, so we have discussed the definition of a $t$ statistic. The $t$ test says that if we have an iid sample $x_1, \dots, x_n$ then

$$ \hat{t} = \dfrac{\bar{x} - \mu}{s} $$

The wikipedia entry on the t distribution has a very nice walk through on why this quantity has $t$ distribution here which references the definition of a $t$ random variable as the ratio of a standard normal random variable and the square root of a chisquare random varibale divided by its degrees of freedom.

Now that we have this out of the way, let's get to your questions:

As I understand, t/z tests require that sample data was obtained from populations following a normal distribution.

Not necessarily. If the data come from a normal population, then $\bar{x}$ will also be normal due to theorems concerning the sum of normal random variables. So if the population is normal, then the numerator of the $t$ statistic will have normal sampling distribution and assuming the denominator is appropriately distributed then the $t$ statistic will have $t$ sampling distribution.

Does this mean we can not use the t-test for non-normal population data? It depends. Nothing is exactly normal (I'd go so far as to say normal distributions do not exist in nature, at least not to the standard of mathematics, but I digress) so the question is "how normal is normal" and that is really a question concerning the validity of the inference made therefrom.

By the central limit theorem, the asymptotic sampling distribution of the sample mean is normal, but despite sophomoric claims that this theorem "kicks in" at some pre-determined sample size, the convergence to a normal sampling distribution can be very very slow.

There is the additional implicit assumption about $V$ and $Z$ being independent. For the case where $x$ are from a normal distribution, the sample mean and sample variance are independent. However, this is not always the case, and can threaten the validity of the t test.

Answer 2

This post may be helpful: https://www.johndcook.com/blog/2018/05/11/two-sample-t-test/

It shows that when you sample from populations that are non-normal, but symmetric and with tails that aren't too heavy, the t-test should work fine.

Here's the final conclusion:

These two examples show that you can replace the normal distribution with a moderately heavy tailed symmetric distribution, but don’t overdo it. When the data some from a heavy tailed distribution, even one that is symmetric, the two-sample t-test may not have the operating characteristics you’d expect.