compare contrasts from different models with emmeans

Question

I'm trying to compare two contrasts from different models (each model from data from different individuals), and I'm wondering if that is possible using the estimate, SE and t.ratio information from each contrast.

Just as an example, let's pretend I have two independent models mod1 and mod2:

data1 <- mtcars[1:2]
data$cyl <- factor(data$cyl)
rownames(data) <- paste("id",1:nrow(data), sep="")
mod1 <- lm(mpg ~ cyl, data=data)
data2 <- mtcars[1:2]
data2$cyl <- sample(data$cyl)
names(data2)[names(data2) == 'cyl'] <- 'cyl2'
rownames(data2) <- paste("id",1:nrow(data2)+nrow(data2), sep="")
data2$mpg = runif(length(data$mpg))
names(data2)[names(data2) == 'mpg'] <- 'mpgrand'
mod2 <- lm(mpgrand ~ cyl2, data=data2)

Using emmeans to compute the contrasts:

em1 <- emmeans(mod1, pairwise~cyl)
em2 <- emmeans(mod2, pairwise~cyl2)
em1$contrasts
 contrast    estimate   SE df t.ratio p.value
 cyl4 - cyl6     6.92 1.56 29   4.441  0.0003
 cyl4 - cyl8    11.56 1.30 29   8.905  <.0001
 cyl6 - cyl8     4.64 1.49 29   3.112  0.0112
em2$contrasts
 contrast    estimate    SE df t.ratio p.value
 cyl24 - cyl26  -0.0371 0.129 29  -0.288  0.9554
 cyl24 - cyl28  -0.0233 0.107 29  -0.218  0.9743
 cyl26 - cyl28   0.0137 0.123 29   0.111  0.9932

Since I know the estimate, SE, df and t.ratio values of each contrast, can I now somehow compare em1's first contrast

contrast    estimate   SE df t.ratio p.value
 cyl4 - cyl6     6.92 1.56 29   4.441  0.0003

with em2's first contrast

contrast    estimate    SE df t.ratio p.value
 cyl24 - cyl26  -0.0371 0.129 29  -0.288  0.9554

in order to claim that the em1 contrast value is significantly greater than the em2 contrast value?

score 0 · Answer 1 · answered Mar 12 '24 at 01:43

I am not quite sure what your underlying question/hypothesis is but given from what you said, why not calculating the difference between mpg and mpgrand and then run a new model on mpg_diff for example?

data$mpg_diff <- data$mpgrand - data$mpg
mod3 <- lm(mpg_diff ~ cyl, data = data)
em3 <- emmeans(mod3, pairwise~cyl)
> em3
$emmeans
 cyl emmean    SE df lower.CL upper.CL
 4    -26.3 1.009 29    -28.4    -24.2
 6    -19.2 1.264 29    -21.8    -16.6
 8    -14.6 0.894 29    -16.4    -12.8
Confidence level used: 0.95
$contrasts
 contrast    estimate   SE df t.ratio p.value
 cyl4 - cyl6     -7.1 1.62 29  -4.392  0.0004
 cyl4 - cyl8    -11.7 1.35 29  -8.684  <.0001
 cyl6 - cyl8     -4.6 1.55 29  -2.972  0.0158
P value adjustment: tukey method for comparing a family of 3 estimates

I don't think it makes much sense to compare the contrasts of different models (i.e. fit with different dependent variables) and then run some sort of significance test on those contrasts.

Thanks for your answer Stefan! Sorry, I should have been clearer in my question. The two models actually come from data from different individuals, so calculating the difference does not make sense in this case. I have updated the question to reflect that point. — locus, Mar 12 '24 at 07:43

compare contrasts from different models with emmeans

1 Answers1