A conflict in regression coefficients interpretations (R code provided)

Question

There are many great answers on CrossValidated (e.g., HERE and HERE) regarding why and how regression coefficients are partial coefficients controlling for/holding constant "other predictors".

The answers cited above all say controlling the "other predictors" does NOT mean holding/FIXING other predictors at a particular value for them, though.

But when I do a regression with $2$ centered predictors (time_c and parent_c), I clearly see that partial coef. of each is obtained by FIXING the other predictor at 0 (its mean as they are centered).

Question: So, can somebody help me clarify this conflict in my mind?

library(tidyverse)
library(interactions)
data <- read.csv('https://raw.githubusercontent.com/rnorouzian/e/master/math.csv')
data <- mutate(data, time_c = time_hw - mean(time_hw), parent_c = pare_inv - mean(pare_inv))
m4 <- lm(math ~ time_c*parent_c, data = data)
summary(m4) ## partial coef. of time_c is .97 (holding par_inv constant but 
                                               # NOT at a particular value)

                     Est.   S.E.   t val.      p


(Intercept)             25.89   0.65    40.04   0.00
time_c                   0.97   0.14     7.12   0.00
parent_c                 2.62   0.64     4.08   0.00
time_c:parent_c         -0.47   0.14    -3.48   0.00

Check the simple slope of "time_c" corresponds to the summary(m4) partial coef. for time_c
sim_slopes(m4, pred = time_c, modx = parent_c, modx.values = 0, john = F)
SIMPLE SLOPES ANALYSIS
Slope of time_c when parent_c = 0.00:
Est.   S.E.   t val.      p

0.97   0.14     7.12   0.00
BUT Simple slope of time_c in the summary(m4) table earlier in fact is possible
when parent_c is FIXED at 0 as demonstrated by simple slope outtput.

Answer 1

The issue here, is that you are fitting an interaction. The interpretation of the main effects changes in the presence of an interaction.

Without an interaction, each of the main effects has the meaning of: the association of a 1 unit change in that variable, with a change in the outcome, leaving the other variable unchanged

With an interaction, when interpreting the main effect, we have a problem with "leaving the other variable unchanged" because now we have 2 other variables - the other main effect and the interaction, and when we change either of the variables, the interaction also has to change - unless the other variable is at zero

So, in the presence of an interaction, each main effect is interpreted as: the association of a 1 unit change in that variable, with a change in the outcome, when the other variable that it is interacted with is zero, and that is exactly what you have observed with these data.

---

Edit to address the question in the comment:

: "Without an interaction, each of the main effects has the meaning of: the association of a 1 unit change in that variable, with a change in the outcome, leaving the other variable unchanged." What do you mean by "leaving the other variable unchanged"? Can you update your answer by simply adding a model with only main effects to clarify this? That is using the model: lm(math ~ time_c + parent_c, data = data)?

> lm(math ~ time_c + parent_c, data = data) %>% coef()
(Intercept)      time_c    parent_c 
 26.5306122   0.9163097   2.5709370 

This can be interpreted as:

• when time_c and parent_c are both at zero, then math will take the value 26.53. Since both variables are centred at zero, this will coincide with the mean of math.

• A 1-unit change in time_c, with parent_c unchanged, is associated with a 0.92 change in the value of math.

• A 1-unit change in parent_c, with time_c unchanged, is associated with a 2.57 change in the value of math.

This is the same as the familiar scenario you might encounter in basic algebra. Let:

$$ y = 10 + 2x_1 + 3x_2$$

So, if we choose $x_1 = 2$ and $x_2 = 3$ we have $y = 23$. If $x_1$ now increases by 1 to 3, with $x_2$ unchanged, then $y = 25$, an increase of 2; because a 1 unit change in $x_1$ results in change of 2 in $y$. In statistics, we prefer to say "is associated with" rather than "results" in, or similar language, in order to avoid causal implications.

We can also think of this in terms of calculus. With the same funtion of $y$ above, let $y=f(x_1,x_2)$ where $f(x_1,x_2)=10 + 2x_1 + 3x_2$

Consider the partial derivative of $f$ with respect to $x_1$:

$$\frac{\partial f}{\partial x_1}=\lim_{\Delta x_1\to 0} \frac{f(x_1+\Delta x_1,x_2)-f(x_1,x_2)}{\Delta x_1}$$

That is, consider a small change in $x_1$, while holding $x_2$ unchanged. This tells us how sensitive $f$ is to a change in $x_1$ and will correspond to the regression coefficient for $x_1$. In the above we have:

$$ \frac{\partial f}{\partial x_1} = 2, \frac{\partial f}{\partial x_2} = 3 $$

In other words, in a simple linear model with continuous predictors the coefficients are partial derivatives (slopes).

When we say "with xxx unchanged", it implies that the two variables are independent. When we have an interaction, the main effects are not independent of the interaction, and that leads to the interpretation of your model that I gave in my original answer above.

Answer 2

I had a look at the simpler case of $ Y = aX+b $

To minimise $ E[(Y-aX-b)^2] $ take the partial derivative wrt $ a $ and $ b $.

$ \implies E[(Y-aX-b)X] = 0 $

$ \implies E[XY] = aE[X^2] + bE[X] $

Also,

$ E[Y-aX-b] = 0 $

$ \implies E[Y] = aE[X] + b $

$ \implies $

$ \hat a = (E[XY] - E[X]E[Y])/(E[X^2]-E[X]^2) $

$ \hat b = E[Y] - aE[X] $

Do the same for $ E[(Y-aX)^2] $

$ \implies E[(Y-aX)X] = 0 $

$ \implies \hat a = E[XY]/E[X^2] $

Next, try

$ Y = aX + b_0 $ where $ b_0 $ is a fixed real number.

Consider $ E[(Y-aX-b_0)^2] $

Differentiate wrt $ a $.

$ \implies E[(Y-aX-b_0)X] = 0 $ $ \implies E[XY] = aE[X^2]+b_0E[X] $

$ \implies \hat a = (E[XY] -b_0E[X])/ E[X^2] $

Now notice that if $ E[X] = 0 $ then $ \hat a $ is the same value in all three cases.