Linear Mixed Model With Partially Crossed Random Effects

Question

I am trying to apply a linear mixed model on my data. Unfortunately all howto's and papers I found yet do not help me with my design:

I have 30 different texts (stimuli) which are rated on 5 items (on the text quality) by the participants. Every participant rates 6 texts. Text 1 and 2 are rated by ALL participants. The other 28 texts are divided into 7 groups (4 each). The participants are randomly assigned to one of these text-groups (including Text 1 and Text 2 and 4 specific others 1,2,3,4,5,6; 1,2,7,8,9,10 and so on).

I already have the "ideal" ratings as a property of each text for all 30 texts. Now I want to see, if the participants are able to rate them similarly or not.

I therefore want to apply different linear mixed models as follows but I am not sure, if I am allowed to. The participants are, I guess, nested in the texts, but all participants are rating text 1 and 2 so there is no clear cluster. I want to account for text-effects and person effects.

So far, my code for the different models (I want to try all of them and test with an anova which one fits best) is:

#Texts as a random effect

model1 <- lmer(participant_rating ~ perfect_rating + (1|text), data = data)

#plus persons as a random effect

model2 <- lmer(participant_rating ~ perfect_rating + (1|text) + (1|participant), data = data)

#plus slopes

model3 <- lmer(participant_rating ~ perfect_rating + (1 + perfect_rating|text) + (1 + perfect_rating|participant), data = data)

I read in forums that the lme4 package actually has no problem with a lot of different variations in nesting and random effects but I'm afraid I will have to explain why I am allowed to do this.

I am really thankful for any advice :-)

...and I am sorry for the long post..

Edit:

I didn’t know a factor can be nested and not nested (because of texts 1 and 2). And thank you for drawing it. In my head though, it makes more sense that participants (n=300) are nested in the 28 texts. So the 28 texts are the grouping factor? And are models 2 and 3 really still appropriate, I thought I have to specify nested effects: in your example:

#plus persons as a random effect model2 <- lmer(participant_rating ~ perfect_rating + (1|participant/text) + (1|participant), data = data) #plus slopes model3 <- lmer(participant_rating ~ perfect_rating + (1 + perfect_rating|participant/text) + (1 + perfect_rating|participant), data = data)

And in my example:

#plus persons as a random effect model2 <- lmer(participant_rating ~ perfect_rating + (1|text) + (1|text/participant), data = data) #plus slopes model3 <- lmer(participant_rating ~ perfect_rating + (1 + perfect_rating|text) + (1 + perfect_rating|text/participant), data = data)

Thanks for a last advice and I guess then I will be good :-)

Answer 1

As discussed in the comments, either of models 2 or 3 should be appropriate.

To help understand the structure I have drawn a diagram. Sorry that it's done by hand !

It's obviously not possible to show all 30 items, and I don't know how many participants you have, so I can't make it exactly as you describe, but hopefully it makes sense to see where they are crossed and where they are nested.

If you look at the items, items 1 and 2 have lots of lines pointing to them, so they "belong" to many participants. This means they are crossed. Whereas items 3 and onwards "belong" to only 1 participant, so they are nested.

Hope it helps !

Edit: I just noticed I should have drawn a line from participant A to one of the items between 5 and 30 but it doesn't reallty change anything. The main point is the items 1 and 2 are crossed with participants but items 3-30 are not.

Edit: To address the points raised in the Edit to the OP:

In my head though, it makes more sense that participants (n=300) are nested in the 28 texts.

I don't think so. As mentioned, for factor A to be nested within factor B, levels of factor A must uniquely "belong" to levels of factor B - that is, levels of A must belong to one and only one level of factor B. So, referring to my diagram, you can see that the participants all have multiple lines so those levels do NOT uniquely belong to one and only one item/text, whereas if you look at items 3-30, all of them have only 1 line, so they belong to one and only 1 participant. Thus, those items are nested within participants. Now, referring back to your original post:

Text 1 and 2 are rated by ALL participants.

Clearly this means that they are crossed.

The other 28 texts are divided into 7 groups (4 each). The participants are randomly assigned to one of these text-groups (including Text 1 and Text 2 and 4 specific others 1,2,3,4,5,6; 1,2,7,8,9,10 and so on

Now, ignore texts 1 and 2 from this since we have already handled those, so items 3,4,5 and 6 "belong" to participant A, and 7,8,9 and 10 "belong" to particpant B etc. So this satisfies the definition of nesting.

So if we had a fully nested design, you would write (1 | participant/text) as a grouping variable, which is the same as (1 | participant) + (1 | participant:text) (it is literally just shorthand for it. However, since this is partially crossed, and the items (texts) are coded uniquely, what I think you need is ``(1 | participant) + (1 | text)` - ie., model2 or model3 in your original post.