How to prove $(\hat{X}-\mu)/(\hat{S}/\sqrt{n})$ is student t with $n-1$ degrees of freedom if $X_i$ are iid $N(\mu, \sigma)$?

Question

It is commonly stated that if $X_i$ are iid $N(\mu, \sigma)$, then with $\hat{X}$ the sample mean, and $\hat{S}$ the sample error (sample standard deviation), then $\frac{ \hat{X}-\mu}{\hat{S}/\sqrt{n}}$ follows a t distribution with $n-1$ degrees of freedom.

e.g., wikipedia link states it, and another Wikipedia link states it as the natural way a t distribution arises, but nothing proves this fact.

How do we know/prove this?

The definition of a t distributed variable is $\frac{U}{\sqrt{W/(n-1)}}$ with $U$~$N(0,1)$, $W$~$\chi^2(n-1)$.

So we can write $$\frac{ \hat{X}-\mu}{\hat{S}/\sqrt{n}} = \frac{ \frac{\hat{X}-\mu}{\sigma/\sqrt{n}}}{ \frac{\hat{S}}{\sigma}} = \frac{U}{D}$$.

$U$ is $N(0,1)$ b/c $X_i$ are, meaning $\hat{X}\sim N(\mu, \sigma/\sqrt n)$.

We need to show that the bottom, $D = \sqrt{W/(n-1)}$ with $W = \frac{(n-1)S^2}{\sigma^2} \sim \chi^2(n-1)$. Looking at other post here, it seems the proof is not quite complete, but it gives a good start. Basically, the proof is: (1.) define auxiliary variable $V=\sum_{i=1}^n(\frac{X_i-\mu}{\sigma})^2$, clearly $\chi^2(n)$. (2.) Algebra shows that $V= W + Y$, where $W = \frac{(n-1)\hat{S}^2}{\sigma^2 }$ (quantity of interest) and $ Y = (\frac{n(\bar{X}-\mu)}{\sigma})^2$ which is $\chi^2(1)$. (3.) Conclude that this implies $W$ is $\chi^2(n-1)$.

So I can follow (1.) and (2.). I do not think (3.) follows directly from (2.) without justification. (e.g., if $W$ and $Y$ were independent, I think the relationship above shows $W$ is $\chi^2(n+1)$!

Overall, how do we prove (3.)? OR Please provide another proof of this whole thing. I just want to understand this rigorously.

Answer 1

For $\zeta\sim \mathcal N(0,1),~ \chi^2\sim\chi^2_{n},$ both being independent to each other, \begin{align}\textrm{Fisher's t} := ~ \frac{\zeta}{\sqrt{\frac{\chi^2}n}}\sim t_n.\tag I\end{align}

Now take $\zeta:= \frac{\bar x-\mu}{\sigma/\sqrt n}\sim \mathcal N(0, 1), \chi^2 :=\frac{\sum(x_i-\bar x) ^2}{\sigma^2}=\frac{ns^2}{\sigma^2}\sim \chi^2_{n-1},$ both being independent (to be proved)

\begin{align} \text{Student's t}&:= \frac{\sqrt n (\bar x-\mu)}{\sigma}\times \frac\sigma{\sqrt{\sum(x_i-\bar x) ^2/(n-1)}}\\&= \frac{\bar x-\mu}{S/\sqrt n} \sim t_{n-1}.\tag{II} \end{align}

---

Fisher's lemma: If $X_i\mapsto Y_i$ via an orthogonal linear transformation, where $X_i, ~i\in\{1, 2,\ldots, n\}$ are independent $\mathcal N(0, \sigma^2), $ then $Y_i\sim \mathcal N(0, \sigma^2)$ and independent.

The proof is easy. Let $\mathbf A$ be the matrix corresponding to the transformation i.e. $\bf Y = AX; $ since $\bf A$ is orthogonal, $\bf Y^\mathsf TY = X^\mathsf TX$ and $\bf A^\mathsf TA= I. $ Now one can evaluate the joint density of $Y_i$ using the relation stated; the jacobian would yield $1$ and the rest would follow.

---

Now, take $\bf A$ such that

$$a_{11} = a_{12} =\ldots= a_{1n}= \frac1{\sqrt n}; $$ this would imply $$y_1 = \sqrt n \bar x. $$

Now, as the transformation is orthogonal,

\begin{align} \sum y_i^2 &= \sum x_i^2\\ &= \sum (x_i-\bar x) ^2 +\underbrace{ n\bar x^2}_{=y_1^2}\\\implies \sum_{i= 2}^n y_i^2 &= \sum_{i=1}^n (x_i -\bar x) ^2; \end{align}

moreover

$$\sum_{i=1}^n (x_i -\mu) ^2= \sum_{i= 2}^n y_i^2 + n(\bar x - \mu)^2.$$

Therefore, following Fisher's lemma and the results found,

\begin{align}\mathrm dG(y_1, y_2, \ldots,,y_n) &= \frac1{\sigma^n(2\pi)^{n/2} }\exp\left [-\frac1{2\sigma^2}\left\{ \sum_{i= 2}^n y_i^2 + n(\bar x - \mu)^2 \right\}\right]|\mathcal J|\prod \mathrm dy_i\\ &= \left[\frac{1}{\sqrt{2\pi}(\sigma/\sqrt n) }\exp\left\{-\frac{ n}{2\sigma^2}(\bar x -\mu)^2\right\}~\mathrm d\bar x\right]\times \left[\frac{1}{\left(\sigma{\sqrt {2\pi}}\right)^{n-1} }\exp\left\{-\sum_{i=2}^n\frac{ y_i^2}{2\sigma^2}\right\}~\prod \mathrm dy_i\right]; \end{align}

proving the independence of $\bar X$ and $\sum_{i=2}^n Y_i^2 = ns^2.$

Now, since $Y_i \sim \mathcal N(0, \sigma^2), ~i\in\{2, 3,\ldots, n\}, $

$$ \sum_{i=2}^n \frac{Y_i^2}{\sigma^2} = \sum_{i=1}^n \left(\frac{X_i -\bar X}{\sigma}\right)^2\sim \chi^2_{n-1}.$$

---

Reference:

Fundamentals of Mathematical Statistics, S. C. Gupta, V. K. Kapoor, Sultan Chand & Sons, 2014.