5

I am doing a mixed method model:

m1 <- lmer(DV ~ IV*Country + (1+IV:Country|Region), data = data)

I am using the lme.dscore to get the Cohen's d for the model. I can get the effect size for the main effects, and the different levels of the interaction when the interaction is a categorical variance - IV through different levels of the main effect.

The challenge is I need to get the effect size of for the interaction term (IV:Country), since we are looking at how these effects influence the model with a meta-analysis.

Lowpar
  • 73
  • I would like to provide an answer, but to provide a contextually specific response, I need some clarification: (1) ¿what are the two levels of your nested data? (I'm assuming regions nested in countries...like counties nested in states) and (2) assuming country is a level-2 variable and IV is a level-1 variable, ¿why is there not a random effect for IV? while there is a random effect for the interaction? Lastly, I am uncertain what you mean by "interaction is a categorical variance". – Gregg H Jun 07 '19 at 11:54
  • Levels of nested data - IV denotes a categorical variable whether participants were in condition a or b. Country is a categorical variable with different countries, region is a categorical variable more refined than country, for instance state in USA. (2) so you are saying I would need two additional random effects? We only did the interaction since it is a replication and only the interaction was previously significant. What I meant by "interaction is a categorical variance" is when the interaction has a categorical variable it becomes a problem. Thanks so much for your question. – Lowpar Jun 07 '19 at 13:58
  • related, possible duplicate: https://stats.stackexchange.com/questions/257985/how-can-i-derive-effect-sizes-in-lme4-and-describe-the-magnitude-of-fixed-effect – setholopolus Aug 02 '21 at 20:46

1 Answers1

4

First, a comment regarding your model: given the hierarchical nature of your design (Region nested within Country) you should not include Country in by-Region random slopes: 

m1 <- lmer(DV ~ IV*Country + (1+IV|Region), data = data)

Also note that, when random slopes for interactions are specified, random slopes for the associated main effects should also be included according to the principle of marginality.

Coming back to your question, to my understanding there is no agreement on how to calculate an effect size in LMM. Given the simplicity of your model, you may consider using an ANOVA model for which effect sizes are well defined.

If you really need LMM, you could make use of the coefficient of determination $R^2$ that has been proposed for LMM and that can be interpreted as variance explained by the model (Nakagawa & Schielzeth 2012). It is implemented in the MuMIn package. In your case you could roughly define the effect size of the interaction of interest as the difference between the $R^2$ of the model with the interaction minus the $R^2$ of the model without the interaction:

library(MuMIn)
m1 <- lmer(DV ~ IV*Country + (1+IV|Region), data = data)
m2 <- lmer(DV ~ IV+Country + (1+IV|Region), data = data)
es <- r.squaredGLMM(m1)[1] - r.squaredGLMM(m2)[1]

That would not necessarily be helpful in the context of a meta-analysis though, as you could not compare such an effect size with effect sizes obtained differently by other authors.

Ous
  • 488
  • 2
  • 6
  • Hi Ous, thanks for taking the time to write this, however it does not answer the specified question, which is how do I get Cohen's d – Lowpar Jun 10 '19 at 09:42
  • 1
    Cohen's d is the effect size of the difference between the means of two samples. It is not defined for interactions. Effect sizes of interactions are commonly obtained by $\eta^2$ ("eta-squared ") in the context of an ANOVA. Hence my answer. – Ous Jun 10 '19 at 13:05