1

Suppose we have a multiple linear regression model with two predictors, $X_1$ and $X_2$: $$Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + \epsilon.$$

We can interpret $\beta_1$ as the expected increase in $Y$ with a unit increase in $X_1$ when $X_2$ is held constant. This is because $\beta_1$ is the partial derivative of the expected value of $Y$ with respect to $X_1$.

Further, suppose that we also compute the simple linear regression of $Y$ against $X_1$: $$Y = b_0 + b_1X_1 + \epsilon.$$

Then I've seen by some authors that: $b_1$ is the expected increase in $Y$ with a unit increase in $X_1$ without holding $X_2$ constant.

But I really don't see the last point because for me in the simple linear regression is like holding constant $X_2$ by giving it a zero value.

So why do they say the in the simple linear regression all other predictors not considered are not constant?

I would really appreciate if you can help me clarify this idea.

Richard Hardy
  • 67,272
user128422
  • 111
  • 1
    In a caual inference sense, when $X_2 $ is a confounder between $X_1 $ and $ Y $, the effects of $ X_1 $ to $ Y$ given $X_2 $ is different from effects of $ X_1 $ to $ Y$ without control/knowledge on $ X_2 $. The simple linear regression is describing the latter effects, where the information on $ X_1 $ will both has its original effects on $ Y$ and also the effects of $ X_1 $ to $X_2 $ to $ Y$. So that should be your case that $X_2 $ is not holding constant since the distribution on $X_2$ changes accordingly when $X_1 $ increase. – Bayesian Jun 17 '22 at 06:16
  • Thank you for your comment @Bayesian. I'm still trying to understand the idea, but the thing that bothers me is that algebraically speaking we can just plug in the value of $X_2 = 0$ in the multiple model, which converts it to a simple linear regression with just $X_1$ as a predictor and perform the regression. We held fixed $X_2$ at the value of zero. – user128422 Jun 17 '22 at 06:57

2 Answers2

1

For the most part, you should read my answer to: Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?, of which, this is nearly a duplicate.

To address your explicit question more directly, $X_2$ is not being held constant. What you have done is set $\beta_2 = 0$, not adjust the data to account for what they would be like if $X_2$ were $0$ for all data in the dataset.

Unless $X_1$ and $X_2$ were perfectly uncorrelated in your dataset, controlling for $X_2$ would amount to shifting the $X_1$ values to some degree. As a result, the estimated $\hat\beta_1$s between the two models would differ.

gung - Reinstate Monica
  • 145,122
0

Try estimating the two models on some dataset:

$$Y = \beta_0 + \beta_1X_1 + \epsilon$$

and

$$Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + \epsilon$$

What you would usually notice is that the $\beta_0$ and $\beta_1$ parameters differ between the two models. In the second you control for $X_2$, in the first one you don't, hence the different parameter estimates. In the first model, by the fact that we didn't include $X_2$, it was accounted for by noise term $\epsilon$. If in the second model you set $X_2=0$ you would consider only the scenario where it is exactly zero. So in fact, in simple linear regression, all the other predictors are accounted for by the random noise, rather than considered constant.

Tim
  • 138,066