1

After reading about interactions contrasts in emmeans, I just wanted to make sure I understood it correctly. Say I have a model with a group*time interaction effect, and I set up emmeans as follows:

emm <- emmeans(lme, ~ Group * Session)

And then use

contrast(emm, interaction = TRUE, "pairwise", adjust="mvt")

It outputs something like

Group_pairwise        Session_pairwise   estimate    SE     df   t.ratio p.value
Group_A - Group_B     Session1 - Session2  x.xxx       x.xxxx xx.x x.xxx   0.001
Group_A - Group_B     Session1 - Session3 ...
...

Does the first line for example then say that the difference between Session 1 and 2 within Group_A is significantly different to the difference between Session 1 and 2 within Group_B?

So it is basically a contrast of contrasts?

Zucchini
  • 41
  • 1
    Yes, that is correct, and interaction contrasts are indeed contrasts of contrasts. But the code looks wrong. Should be contrast(emm, interaction = "pairwise") – Russ Lenth Aug 22 '22 at 00:30
  • Thanks. Just following up on this - how can I use eff_size to calculate cohens d and it's confidence interval for the interaction contrast? For instance, I'm aiming at a cohens d and CI for the abovementioned example. So basically the effect size for the group difference of the change scores (Session 1 - Session 2). – Zucchini Feb 15 '23 at 21:44
  • When I write << interactioncontrast <- contrast(emm, interaction = "pairwise") >> and then << eff_size(interactioncontrast, sigma = sigma(lme), edf = df.residual(lme)) >> I'm not getting the desired effect size for << (A-B) - (Ses.1-Ses.2) >> but instead something like << (A - B Ses.1 - Ses.2) - (A - B Ses.1 - Ses.3) >> with cohens d and CI... – Zucchini Feb 15 '23 at 21:44
  • (The reviewer specifically asked for cohens d and CI in this case ...) – Zucchini Feb 15 '23 at 21:46
  • 1
    I honestly don't know, as I have never seen people report Cohen d effect sizes based on interaction contrasts. I guess it's possible to scale the interaction contrasts by dividing by sigma. But do you have an example of a published report where Cohen d effect sizes are reported for interaction contrasts? – Russ Lenth Feb 16 '23 at 03:53
  • Hi, thanks for your reply. No, I don't unfortunately... I can see a little bit why the reviewer thinks this is useful though, as is basically answers the question "How different is the change within one group to the change within the other". Hm, dividing by sigma sounds logical... Would it be possible to get a CI and SE for cohens d somehow this way, too? – Zucchini Feb 17 '23 at 00:29
  • The alternative would be to calculate the changes based on the data itself and then calculate cohens d from there. Not sure if that would be regarded appropriate, as the other post-hoc analyses are done with emmeans (and therefore based on the model)... – Zucchini Feb 17 '23 at 00:41

1 Answers1

2

I guess if a Cohen's d makes sense, you can do something like

CON <- contrast(emm, interaction = TRUE, "pairwise", adjust="mvt")
eff_size(CON, sigma = ???, edf = ???, method = "identity")

(see the examples in the help page for eff_size) but you need to replace the ???s with reasonable values -- and be able to explain and defend them. You apparently have a mixed model, and I'm not sure Cohen's d is even defined because there are multiple sigma values involved. But I suppose that for edf you can use #groups - 1 for the coarsest grouping. For sigma, possibly you can do VarCorr(model) to see all the variance estimates; and then combine them. For instance, if there are three SDs sd1, sd2, sd3 you might use

sigma <- sqrt(sd1^2 + sd2^2 + sd3^2)

I am very leery of all of this because you can't find an example where this was actually ever done with interaction contrasts, and because in any but the simplest situations, the whole idea of effect size is flaky; people improvise something such as what I suggest, and then nobody really knows what you're talking about. You wind up with numbers, but they answer a question that can't be stated clearly.

Russ Lenth
  • 20,271
  • 1
    BTW, sometimes reviewers ask for things that can't be produced. You need to learn how to talk back to them and patiently explain why you can't comply. – Russ Lenth Feb 21 '23 at 16:34
  • Thank you very much for your extensive replies! In fact I have already partly followed your advise: As the query sounded somewhat reasonable, I have computed the effect sizes (I simply calculated cohens by dividing by sigma), but only present those in the supplement and do not interet/mention them any further; while explaining the reviewer why. The reviewer seems to be happy with this compromise :) – Zucchini Feb 22 '23 at 18:42