3

I fail to understand the need for the linearity assumption in the Gauss-Markov theorem and, even worse, I do not understand its meaning. We say that the model should be linear in the coefficients or parameters, implying that regressing $y$ on $x$ and $x^2$ is linear as the marginal effect would be $\beta_1+2\beta_2 x$, which is certainly a linear function of $x$.

However, if we would include a power $3$, the marginal effect would be $\beta_1+2 x\beta_2+3 \beta_3 x^2$, which is no longer linear. Thus this mean that it is wrong to include powers higher than 2 in a simple regression model?

TPArrow
  • 2,295
  • 13
  • 22
Daniel Pinto
  • 633
  • 6
  • 16
  • 1
    The marginal effects are not linear when the model is quadratic. You have confused this with the effect used in econometrics which is the first derivative, which is linear. – Frank Harrell Oct 08 '16 at 13:23

1 Answers1

6

Linearity in parameters is different than in the covariates. Covariates can be transformed to be linear.

I start by an example. Assume the model is $y=\beta^2 x+e$ that clearly not linear with respect to the parameter $\beta$. We assume all typical assumptions of the linear model hold. To use the classical methods I assume $\phi=\beta^2$ . then the estimation of $\phi$ is given by

$$ \hat\phi=(x'x)^{-1}x'y $$

and

$$ \hat\phi\sim N(\phi,(x'x)^{-1}var(e)) $$

But $\phi=\beta^2$ that is $\hat\beta= \pm \sqrt{\hat\phi}$. Putting it in the equation above leads to, $$ \hat\beta\sim N(\pm\beta,(x'x)^{-1}var(e)) $$ The last equation shows that the sampling distribution of the estimated parameter is not consistant in sign. Then the estimation is not BLUE!

TPArrow
  • 2,295
  • 13
  • 22
  • brilliant! I've been struggling with that for years! I gave you kudos here in case you're interested (and in case you're interested in seeing all the other nonsensical answers.....) https://www.reddit.com/r/CFA/comments/m2r41g/linear_regression_question/ – QED Mar 13 '21 at 15:43