1

I have the following regression outputs from a model that includes both quadratic and cubic interaction terms. I calculated the simple slopes using the simle_slope from reghelper.

x1 <- rnorm(100)
x2 <- rnorm(100)
x3 <- rnorm(100)
x4 <- rnorm(100)

y   <- x1 + x2 + x2*2 + x1x2 + x3 + x4 + rnorm(100)
fit <- lm(y ~ x1 + x2 + I(x2^2) + I(x2^3) + x1:x2 + x1:I(x2^2) + 
    x1:I(x2^3) + x3 + x4)


summary(fit)


library(reghelper)
test2 <- simple_slopes(fit, level = list(x2 = c(0,1,2,3), confint = T))
test2$L95 <- test2$Test Estimate-1.96test2$Std. Error
test2$U95 <- test2$Test Estimate+1.96test2$Std. Error
test2


test2
      x1 x2 Test Estimate Std. Error t value df  Pr(>|t|)        L95      U95 Sig.
1 sstest  0        1.1733     0.1642  7.1437 92 2.085e-10  0.8514158 1.495274  ***
2 sstest  1        2.0379     0.1895 10.7562 92 < 2.2e-16  1.6665505 2.409244  ***
3 sstest  2        2.7164     0.8014  3.3896 92  0.001033  1.1456684 4.287227   **
4 sstest  3        3.2395     2.6804  1.2086 92  0.229911 -2.0140209 8.493070

I understand simple slope can tell us whether or not a conditional effect based on a specific value of moderator is statistically different from zero.

My question is, the confidence interval (L95, U95) of the simple slope of x1 when x2 = 0,1 is not overlapped. Can I say there is a statistically significant difference in the simple slopes of x1 when x2 = (0,1)? In other words, can I tell the simple slopes of x1 when x2 = 0,1 are significantly different from each other?

EdM
  • 92,183
  • 10
  • 92
  • 267
zjppdozen
  • 347

1 Answers1

1

Non-overlap of confidence intervals is too stringent for determining "statistically significant" differences between estimates. This page has a superb illustration.

If your interest is in differences between "simple-slope"* estimates, then evaluate those differences directly. For example, the emmeans package provides ways to examine all pairwise differences among a set of "simple slopes" evaluated as emtrends:

library(emmeans)
emtrends(fit,pairwise~x2,at=list(x2=c(0,1,2)),var="x1")
# $emtrends
#  x2 x1.trend    SE df lower.CL upper.CL
#   0     1.02 0.134 90    0.755     1.29
#   1     2.39 0.176 90    2.040     2.74
#   2     2.80 0.365 90    2.069     3.52
# 
# Confidence level used: 0.95 
# 
# $contrasts
#  contrast  estimate    SE df t.ratio p.value
#  x20 - x21   -1.371 0.160 90  -8.542  <.0001
#  x20 - x22   -1.776 0.417 90  -4.255  0.0002
#  x21 - x22   -0.405 0.381 90  -1.063  0.5393

The "simple slopes" at each value of x2 are indicated by x1.trend in the $emtrends output, along with standard errors and confidence levels. They differ somewhat from yours as you didn't specify a random seed, and I limited the range of x2 as there weren't any values as high as 3 in the data. (I used set.seed(101) before running your code.) Your example shows a much higher standard error for the slope when x2=2, which I also assume is because of the difference in random seeds.

The differences in slopes among the pairs of x2 values are in the $contrasts output. x20 means the situation in which x2=0, etc., so the first line for the contrast x20-x21 is the particular comparison you note in the question.

*I personally find "simple slopes" to be not so simple, so I put quotes around the phrase.

EdM
  • 92,183
  • 10
  • 92
  • 267