$t$-test and likelihood ratio test for testing the regression coefficient

Question

I am studying hypothesis testing for the regression coefficient, it is given that

The hypotheses for testing the significance of any individual regression coefficient, such as $\beta_{j},$ are $$ H_{0}: \beta_{j}=0, \quad H_{1}: \beta_{j} \neq 0 $$ If $H_{0}: \beta_{j}=0$ is not rejected, then this indicates that the regressor $x_{j}$ can be deleted from the model. The test statistic for this hypothesis is $t_{0}=\frac{\hat{\beta}_{j}}{\sqrt{\hat{\sigma}^{2} C_{j j}}}=\frac{\hat{\beta}_{j}}{\operatorname{se}\left(\hat{\beta}_{j}\right)}$ where $C_{j j}$ is the diagonal element of $\left(\mathbf{X}^{\prime} \mathbf{X}\right)^{-1}$ corresponding to $\hat{\beta}_{j} .$ The null hypothesis $H_{0}: \beta_{j}=0$ is rejected if $\left|t_{0}\right|>t_{\alpha / 2, n-k-1}$.

1st question: It is just given that this $t$ is test statistics, but how to show/ prove that this hypothesis can be tested using the given $t$ - statistic.

Given the linear model $G: \mathbf{Y}=\mathbf{X} \beta+\varepsilon,$ where $\mathbf{X}$ is $n \times p$ of rank $p$ and $\varepsilon \sim N_{n}\left(0, \sigma^{2} I_{n}\right),$ we wish to test the hypothesis $H: \mathbf{A} \beta=c,$ where $\mathbf{A}$ is $q \times p$ of rank $q$.
The likelihood ratio test of $H$ is given by $$ \Lambda=\frac{L\left(\hat{\beta}_{H}, \hat{\sigma}_{H}^{2}\right)}{L\left(\hat{\beta}, \hat{\sigma}^{2}\right)}=\left(\frac{\hat{\sigma}^{2}}{\hat{\sigma}_{H}^{2}}\right)^{n / 2} $$ and we can define $F$ statistic based on this ratio as $F=\frac{n-p}{q}\left(\Lambda^{-2 / n}-1\right)$ has an $F_{q, n-p}$ distribution when $H$ is true. We then reject $H$ when $F$ is too large.

2nd question: Can we prove that the likelihood ratio test is equivalent to the $t$-test in 1st question.

Answer 1

For linear regression a t-test and F-test are the same: Difference between t-test and ANOVA in linear regression

This is also true for your situation. You can transform the $X$ and $\beta$ in your equation $Y=X\beta +\epsilon$ such that $\mathbf{A}\beta$ is one of your parameters (*). And you can shift the Y-variable (subtract $c\cdot \mathbf{A}\beta$ from it) such that the test becomes $\mathbf{A}\beta = 0$. And then you have the same situation as typical regression where a t-test and F-test are the same.

This is a bit intuive explanation. Maybe it still needs the algebra to show that it actually works.

---

(*) Example of that transformation: recently we had a question Confidence interval for the difference of two fitted values from a linear regression model The question was about the distribution/confidence interval of the difference in the expectation of two points $\hat{y}_1$ and $\hat{y}_2$ (which can be expressed as some linear sum of the $\beta$). By a transformation of the regressors, this can be made directly equivalent to the distribution of a parameter in the regression.

Answer 2

1. This can be shown using the following results of a linear model $$\widehat{\epsilon} = y - X\widehat{\beta} \text{ and } \widehat{\epsilon} \sim N(0, \sigma^2(I-H))$$ $$\Rightarrow \left\| \widehat{\epsilon} \right\|^2 \sim \sigma^2 \chi^2_{n-p}$$ where p in the number of coefficients ($\{\beta_0, \beta_1 ...\beta_{p-1} \}$) $$\text{The above result comes from Eigen decomposition of I-H projection matrix}$$ Using the least squares estimates via orthogonalization of the jth column of matrix X, it can be shown that. $$\widehat{\beta}_j \sim N(\widehat{\beta}_j, \sigma^2 C_{jj})$$ and under the null hypothesis we have, $$\widehat{\beta}_j \sim N(0, \sigma^2 C_{jj})$$ or $$\frac{\widehat{\beta}_j}{\sigma\sqrt{ C_{jj}}} \sim N(0, 1)$$ Since we don't know the value of $\sigma$, we use the following test statistic $$t_j = \frac{\widehat{\beta}_j}{\widehat{\sigma}\sqrt{ C_{jj}}}$$ where $$\widehat{\sigma} = \frac{\widehat{\epsilon}}{n-p}$$ Now we can rewrite $t_j$ as $$t_j = \dfrac{\frac{\widehat{\beta}_j}{\sigma\sqrt{ C_{jj}}}}{\sqrt{\frac{\sigma^{-2} \left\| \widehat{\epsilon} \right\|^2}{n-p}}}$$ $$\Rightarrow t_j \sim \dfrac{N(0, 1)}{\sqrt{\dfrac{\chi_{n-p}^2}{n-p}}}$$ This distribution is called the t-distribution against which you can test your t-statistic.

2. Since F = $t^2$ you can use them interchangeably here.