0

Is there a mathematical proove that conditional expection of residual is 0?

i.e. E(Residual | X) = 0

pqrz
  • 163
  • Other related questions https://stats.stackexchange.com/questions/369658/expected-value-of-the-residuals https://stats.stackexchange.com/questions/469212/proof-that-the-mean-of-predicted-values-in-ols-regression-is-equal-to-the-mean-o – Sextus Empiricus Oct 08 '22 at 10:27
  • @SextusEmpiricus I think since these questions come frequently, we need to take one a canonical and mark the others duplicate. – User1865345 Oct 08 '22 at 10:47
  • @User1865345 the first link that I gave is the one that many others link to. I mentioned some others because they are not easily found and may give the OP some more insight. The duplicate answers do not show up directly in the canonical question and only via the 'linked questions'. – Sextus Empiricus Oct 08 '22 at 11:02
  • 2
    Whether this is true depends on what the model is. Could you clarify? – whuber Oct 08 '22 at 15:03
  • Hi Wuber, We are using OLS model. I was confused - Does the OLS always ensures, that E(Residual | X) = 0 ? – pqrz Oct 09 '22 at 13:24
  • even in case of data contains endogeneity ... and we run OLS on it ... the OLS still ensures it, isn't it? – pqrz Oct 09 '22 at 13:25

1 Answers1

1

What is residual $\mathbf e? $

Specifically $$\mathbf e = \mathbf y -\mathbf P\mathbf y, \tag 1$$

where $\mathbf P:= \mathbf X\left(\mathbf X^\mathsf T\mathbf X\right) ^{-1}\mathbf X^\mathsf T. $

For fixed $\mathbf X, $

\begin{align}\mathbb E[\mathbf e|\mathbf X] &= (\mathbf I-\mathbf P) \mathbb E[\mathbf y|\mathbf X]\\&= (\mathbf I-\mathbf P)\mathbf X\boldsymbol \beta\\ &= \left[\mathbf X-\mathbf X\left(\mathbf X^\mathsf T\mathbf X\right) ^{-1}\mathbf X^\mathsf T\mathbf X\right]\boldsymbol\beta\\ &=\mathbf 0.\tag 2\end{align}

What happens when $\bf X$ is random? Can the result be valid? Does it make any difference in the treatment above? Can any comment be made on unconditional expectation $\mathbb E[\mathbf e]? $

User1865345
  • 8,202
  • But will it still be true ....if model contains endogenity? – pqrz Oct 08 '22 at 11:32
  • I.e. E(Residual | X) = 0 will still hold true? – pqrz Oct 08 '22 at 11:33
  • The classical LRM assumes exogeneity of independent variables: that means $\varepsilon$ is uncorrelated with information based on $\mathbf x$ i.e. $\mathbb E[\boldsymbol\varepsilon |\mathbf X]= \mathbf 0.$ That is the basis. Without exogeneity, then, can you infer such result? Think. – User1865345 Oct 08 '22 at 11:40
  • What if the model contains endogenity and we still fit OLS on it .... will this result still hold? – pqrz Oct 08 '22 at 12:08
  • That would be a separate question that you can ask in a different post. – User1865345 Oct 08 '22 at 16:04