1

I have been looking through material on difference-in-difference (DID) models, and I understand the interaction term in a regression model can estimate the DID effect. However, fundamentally for statistical models, the interaction term may only be interpreted if the main effects are statistically significant as well.

Do we also apply the same reasoning and only interpret a DID estimate if the main effects in the model are statistically significant?

Sheryl
  • 11

1 Answers1

2

That is simply untrue, its more a rule of thumb than anything. For example in R.

> x1 <- rnorm(100)
> x2 <- rnorm(100)
> y  <- x1 * x2 + rnorm(100)
> 
> 
> summary(lm(y ~ x1 + x2))

Call:
lm(formula = y ~ x1 + x2)


Residuals:
    Min      1Q  Median      3Q     Max 
-5.4396 -0.9537  0.0591  1.0530  3.1617


Coefficients:
            Estimate Std. Error t value Pr(>|t|)

(Intercept)  0.06657    0.13804   0.482   0.6307

x1           0.23257    0.13433   1.731   0.0866 .
x2          -0.21193    0.13720  -1.545   0.1257




Signif. codes:  0 ‘*’ 0.001 ‘’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


Residual standard error: 1.37 on 97 degrees of freedom
Multiple R-squared:  0.05351,   Adjusted R-squared:  0.034 
F-statistic: 2.742 on 2 and 97 DF,  p-value: 0.06943


> summary(lm(y ~ x1 + x2 + x1 * x2))


Call:
lm(formula = y ~ x1 + x2 + x1 * x2)


Residuals:
     Min       1Q   Median       3Q      Max 
-2.79339 -0.72311  0.01982  0.78272  2.39362


Coefficients:
            Estimate Std. Error t value Pr(>|t|)

(Intercept)  0.05560    0.10767   0.516    0.607

x1           0.05168    0.10720   0.482    0.631

x2           0.04472    0.11176   0.400    0.690

x1:x2        0.98052    0.12309   7.966 3.35e-12 ***




Signif. codes:  0 ‘*’ 0.001 ‘’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


Residual standard error: 1.069 on 96 degrees of freedom
Multiple R-squared:  0.4302,    Adjusted R-squared:  0.4124 
F-statistic: 24.16 on 3 and 96 DF,  p-value: 9.881e-12
Kozolovska
  • 1,355
  • Thanks for your prompt comment - I guess my concern was that while we can fit models, the question is whether the estimates are valid for use. – Sheryl Jan 14 '22 at 08:13
  • I fail to understand why you think it is not possible.. – Kozolovska Jan 14 '22 at 09:12
  • @Sheryl there can be confusion about "main effect" terminology when there are interactions. See this thread, for example. If by that you mean the coefficients of the individual predictors in a model with interactions, like the second model in this answer, such "main effects" aren't uniquely defined as they depend on the reference values/centering of the predictors with which they interact. See this thread for a worked-through example. – EdM Jan 14 '22 at 17:58
  • @Sheryl if by "main effects" you mean the marginal estimates with respect to individual predictors in a model that hasn't incorporated an interaction, like the first model in this answer, then you must recognize that those estimates ignore both the correlations among the predictors that are typically present in observational studies and the potentially important interactions among the predictors. So, either way that you are using the term "main effects," you need to evaluate interaction terms. – EdM Jan 14 '22 at 18:13