0

My question is pretty similar to this one, but I don't seem to get an F-statistic.

Hi, I know that we would usually report an ANOVA result as F(between groups, within groups DF) = f-statistic, p value but I'm wondering how to report an Anova used to compare multiple models ?

#Anova 1: mod1 x intercept-only model

> anova(modInterceptOnly, mod1)
refitting model(s) with ML (instead of REML)
Data: data
Models:
modInterceptOnly: SCORE ~ (1 | ID)
mod1: SCORE ~ X1_c * X2 + (1 | ID)
                 npar    AIC    BIC  logLik deviance  Chisq Df Pr(>Chisq)

modInterceptOnly    3 1096.5 1104.7 -545.24   1090.5

mod1                6 1074.9 1091.3 -531.46   1062.9 27.567  3  4.477e-06 ***


#Anova 2: mod1 x intercept-only model x mod2 (this model excludes X2, which was not significant in the model)

anova(modInterceptOnly, mod1, mod2)

refitting model(s) with ML (instead of REML)
Data: dfModels1
Models:
modInterceptOnly: SCORE ~ (1 | ID)
mod2: SCORE ~ X1_c + (1 | ID)
mod1: SCORE ~ X1_c * X2 + (1 | ID)
                   npar    AIC    BIC  logLik deviance  Chisq Df Pr(>Chisq)    
modInterceptOnly      3 1096.5 1104.7 -545.24   1090.5                         
mod2                  4 1071.8 1082.7 -531.91   1063.8 26.671  1  2.412e-07 ***
mod1                  6 1074.9 1091.3 -531.46   1062.9  0.896  2     0.6389    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  • I wanna say something like this: We've performed two Analysis of Variance (ANOVA) in order to analyse the model fit. Mod1 seems to be significant better than an intercept-only model (F( ) = , p < 0.05). However, when compared to a model without X2, this model doesn't seem to be a statistically better fit (F( ) = , p > 0.05). Basically, what should I put within the parenthesis? I'm a bit confused. Thanks in advance!

  • Edit:

All models are lmers. Data is pretty much like the one I've used in this post (not the same, but they're alike)

mod1 <- lmer(SCORE ~ X1_c * X2 + (1|ID)
mod2 <- lmer(SCORE ~ X1_c + (1|ID)
modInterceptOnly <- lmer(SCORE ~ (1|ID)

X1 = continuons pred
X2 = 2-level cat predictor


summary(mod1) do not indicate a significant beta for neither X2 not the interaction between X1 and X2, only for X1
Larissa Cury
  • 741
  • You need to explain how the fit objects modInterceptOnly, mod1 and mod2 were created, i.e., what sort of model fits they are, before it would be possible to explain what anova might mean between them. The objects do not appear to be standard anova or lm objects in R. – Gordon Smyth Sep 19 '22 at 22:17
  • @GordonSmyth , all right! I'll add an edit! – Larissa Cury Sep 19 '22 at 22:30
  • It is not usually possible to conduct an F-test between mixed models fitted by REML. So what you are after is impossible by standard means, unless you are using a specialist package with specialist methods designed for this purpose, which is not something you mention. – Gordon Smyth Sep 26 '22 at 04:36
  • hi, @GordonSmyth , usually the anova() function in R converts the models to ML before the test, at least I get the message saying it does so, right? – Larissa Cury Sep 26 '22 at 12:02
  • Yes, indeed it does refit with ML, which means that (i) you are no longer comparing the original model fits that you wanted to compare and (ii) you're not doing an F-test. This is my point: there is no way to do an F-test by this route and, even if there was, it wouldn't correspond to the comparison that you actually wanted to make. – Gordon Smyth Sep 28 '22 at 09:00

0 Answers0