3

The emmeans vignette shows that we need to declare edf for calculating effect size. I just have no idea where to get this?

https://cran.r-project.org/web/packages/emmeans/vignettes/comparisons.html

enter image description here

enter image description here

myfatson
  • 193

1 Answers1

2

It seems like you could do something like

EMM <- emmeans(model, "filters")
pairs(EMM)

... and look at how many df are reported for the pairwise comparisons. I'd use the smallest of those df

jay.sf
  • 725
Russ Lenth
  • 20,271
  • 1
    Okay thanks! I don't get it though, what's the logic behind choosing that? – myfatson Feb 13 '21 at 00:28
  • Read the documentation? It's supposed to quantify the uncertainty in the variance estimate being used. BTW, edf has no impact on the effect size estimate m, just on the cis – Russ Lenth Feb 13 '21 at 01:35
  • I was more asking why do we use the df from the fixed effect pairwise comparisons, I cant find an explanation. Between the anova for fixed effects and likelihood ratio for random effects, and whats printed from the summary, theres a lot of dfs floating around! – myfatson Feb 13 '21 at 13:40
  • Well, you're right. In a simple lm() model, things are a lot more straightforward. In a mixed model, there is a lot more ambiguity. But to me, the d.f. to use is the smallest issue. What's bigger is deciding what you are even talking about when you compute an effect size. There is a lot of discussion on this, and from where I sit, the question is nearly unanswerable. I don't really believe in effect sizes. I provided the function be cause I thought is was important to account for uncertainty in the SD estimate if you insist on computing an effect size. – Russ Lenth Feb 13 '21 at 13:53
  • BTW, the sigma you provide should probably NOT be the residual SD. It needs to be the SD of the population you are referencing with regard to the effect size. I am pretty much talking myself out of the answer I gave,actually. I'll update my answer when I get a chance. – Russ Lenth Feb 13 '21 at 13:58
  • Thanks for all the input Russ! I'm actually looking for effect size currently to inform a power analysis im trying to perform. I know mixed effects power analysis/sample size determination is fickle, but I've been asked to at least give a rough estimate of what kind of n we need. As for the sigma comment, I don't know the population SD, so what do? Do I use sample SD? – myfatson Feb 14 '21 at 14:15
  • Then good luck, I guess. I disagree with using standardized effect size as a target. I wrote about it long ago (e.g., 1981 in The American Statistician), if you care to look it up. – Russ Lenth Feb 14 '21 at 15:19
  • So is the final answer "don't use effect size for mixed effects linear models?" – myfatson Feb 14 '21 at 16:36
  • Maybe. But really I was saying don't use it for power and sample size – Russ Lenth Feb 14 '21 at 17:56