2

The difference between Linear regression and Polynomial regression is that in the last we manipulate our original explanatory variables in a way to create polynomial dependency between Y and X. For the sake of simplicity if we consider one feature (explanatory variable) only: X, the Polynomial regression with degree 2 will look like this:

$Y$ = $\beta_0$ + $\beta_1\cdot X$ + $\beta_2\cdot X^2$

In sklearn we can do this by initializing and calling fit_transform method of PolynomialFeatures(2). Eventually if we want to train our model we should use LinearRegression of sklearn to do so.

However, one of the assumptions of the Linear Regression is that all features should be uncorrelated. $X$ and $X^2$, on the other hand are perfectly correlated. So, doesn't this create any issue with the training process?

kjetil b halvorsen
  • 77,844
Carlos Muradyan
  • 21
  • 3
    There is no such assumption about uncorrelated predictors in linear regression. The Gauss-Markov theorem requires uncorrelated errors. Also, $X$ and $X^2$ are not perfectly correlated. – Dave Aug 11 '20 at 18:13
  • 2
    My post at https://stats.stackexchange.com/a/384636/919 illustrates and explains the possibilities for multicollinearity when introducing a quadratic term in a regression. Additional insight can be had by searching our site for posts on orthogonal polynomials: the idea is that the very same model can be expressed in terms of two new variables $Z_1$ and $Z_2,$ derived as linear combinations of $X$ and $X^2,$ where $Z_1$ and $Z_2$ are centered and orthogonal: that is, there's no collinearity whatsoever. – whuber Aug 11 '20 at 18:21

0 Answers0