3

Recently I read Wald test for logistic regression this post, which states why do we use Wald test in logistic regression and no t-test instead. It says, that the main reason behind it, is that for example in OLS the variance of residuals is unknown, and it has to be estimated, so t-test has to be used. On the another hand, since logistic regression is estimated using maximum likelihood estimation, we know that $\hat \beta - \beta$ will be asymptotically normally distributed, and therefore the expression:

$$W = \frac{\hat \beta - \beta}{\hat{\textrm{se}}(\hat \beta)}$$

will have asymptotically standard normal distribution. Where of course:

$$\hat{\textrm{se}}(\hat \beta) = \sqrt{s^2(X^TX)^{-1}_{jj}}$$

and $s^2$ is a variance of residuals.

I understand, that in linear regression, you replace $s^2$ by its estimator $\hat \sigma^2$, and then statistics $W$ has t-student distribution instead of normal. However, when you fit exactly the variance, $W$ statistics will have asymptotically normal distribution. And here I couldn't find any information how exactly this variance in logistic regression can be found. Could you please explain to me how it can be dervied?

kjetil b halvorsen
  • 77,844
John
  • 532

1 Answers1

0

I have to agree that reading the post you linked and other sources I was confused as well. Digging around I was able to find this source which finally gives the full answer on how to calculate $SE(\hat{\beta_j})$. In summary:

$$\hat{SE}(\hat{\beta_j}) = \sqrt{(X^t\hat{W}X)_{jj}^{-1}}$$

Where $\hat{W}$ is a diagonal $nXn$ matrix:

$$\hat{W} = diag (\frac{e^{\Sigma_{j=0}^p\beta_jx_{1j}}}{(1+e^{\Sigma_{j=0}^p\beta_jx_{1j}})^2}, \dots, \frac{e^{\Sigma_{j=0}^p\beta_jx_{nj}}}{(1+e^{\Sigma_{j=0}^p\beta_jx_{nj}})^2})$$

From there you can go and stick it in the Wald statistic to get the t-value for the test of $H_0:\beta_j=0$

Iyar Lin
  • 171