1

Non linear line

Straight line

  1. When we fit a linear reg model, do we get a straight line equation? So, it means there is no way we can get a curved line from it. Then why in some examples over internet we see curved line for linear regression examples?

  2. 1st image is quantile regression fit on 3 quantiles and 2nd image is again quantile regression model fit on 3 quantile but 1st one is non linear and second one is linear.

Please provide some explanation on both the points.

Thanks for the first comment, now its pretty clear: In linear regression when we have linearity in parameters and variables both then we get a straight line. ex- $Y =a + bx$ but when we have linearity in parameter but not in variable then we get a curved line.

$$Y = a+bx+cx^2 $$

Both are linear regression but model lines looks different. Generalized additive algo is linear in terms of parameter but not in variable, thats is why it is capable of fitting non linear curve with similar level of interpretability like linear regression.

User1865345
  • 8,202
Sourabh Sharma
  • 11
  • Hint: research the difference between linearity in parameters and linearity in variables. – user2974951 Jan 16 '23 at 07:23
  • Got it thanks a lot! – Sourabh Sharma Jan 16 '23 at 07:44

  • https://stats.stackexchange.com/questions/106025/regression-that-creates-x-logx-functions/

  • https://stats.stackexchange.com/questions/59782/linear-regression-explanations/

  • https://stats.stackexchange.com/questions/100653/how-to-do-ordinary-least-squares-ols-when-the-observations-are-not-linear

  • https://stats.stackexchange.com/questions/177015/clues-that-a-problem-is-well-suited-for-linear-regression/

  • https://stats.stackexchange.com/questions/111266/polynomial-regression-rules/ ... etc etc

    • – Glen_b Jan 16 '23 at 07:54
  • Does this answer your question? Why is polynomial regression considered a special case of multiple linear regression? // @Glen_b VTC with me? – Dave Jan 16 '23 at 07:55
  • @Dave I've deleted my answer, but could you please clarify why it was wrong? – Shawn Hemelstrand Jan 16 '23 at 08:30
  • see also https://stats.stackexchange.com/a/597445/56940 – utobi Jan 16 '23 at 08:32
  • @ShawnHemelstrand I’m pretty sure you know what constitutes a linear model, but inclusion of polynomial terms (among other nonlinear basis functions, such as splines) absolutely fit in a standard linear regression framework. Your answer made it sound like linear regression was all about straight lines, which is a misconception that I have seen lead to problems, such as one presentation I saw last year where the speaker made the remarkable claim that linear models could not handle curvature, necessitating a neural network approach. (I like neural nets, but, hello, Stone-Weierstrass…) – Dave Jan 16 '23 at 08:39
  • I see. That wasn't my intent but I will choose my wording better next time. Thanks for clarifying btw – Shawn Hemelstrand Jan 16 '23 at 08:59
  • @ Dave my main reason for not just VTC was I thought Shawn 's answer was pretty close to a really good answer and might have ended up being worth merging into another thread. – Glen_b Jan 16 '23 at 23:42