Compare levels between each other in mixed model

Question

I have done an experiment where subjects where asked to recognized different emotions presented to them in two different situations: while in motion and at rest.

To analyze the model I have constructed a mixed model with lmer:

mod <- lmer(percentage ~ situation + emotion + 
            (1|subject), data = df)

where percentage is the percentage of correctly recognized emotions. I have verified this model is indeed linear by verifying the usual assumptions (linearity, homoskedasticity, normality, cook's distances) and it seems fine.

Now, I'd like to compare the levels between each other, for instance to determine if there is significant difference of emotion recognition at rest or in movement or between the emotions.

What I have done so fare is to use glht in the following way:

summary(glht(mod, mcp(situation="Tukey")))
     Simultaneous Tests for General Linear Hypotheses

Multiple Comparisons of Means: Tukey Contrasts


Fit: lmer(formula = percentage ~ situation + 
 emotion + (1 | subject), 
    data = df)

Linear Hypotheses:
                     Estimate Std. Error z value Pr(>|z|)    
static - motion == 0  -13.765      2.297  -5.992 2.08e-09 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Adjusted p values reported -- single-step method)

and
> summary(glht(mod,mcp(emotion="Tukey")))

     Simultaneous Tests for General Linear Hypotheses

Multiple Comparisons of Means: Tukey Contrasts


Fit: lmer(formula = percentage ~ situation + emotion + (1 | subject), 
    data = df)

Linear Hypotheses:
                          Estimate Std. Error z value Pr(>|z|)    
disgust - anger == 0         7.812      3.979   1.963  0.36360    
fear - anger == 0          -27.232      3.979  -6.844  < 0.001 ***
happiness - anger == 0      43.750      3.979  10.995  < 0.001 ***
sadness - anger == 0        -7.366      3.979  -1.851  0.43272    
surprise - anger == 0       14.955      3.979   3.758  0.00236 ** 
fear - disgust == 0        -35.045      3.979  -8.807  < 0.001 ***
happiness - disgust == 0    35.938      3.979   9.031  < 0.001 ***
sadness - disgust == 0     -15.179      3.979  -3.814  0.00190 ** 
surprise - disgust == 0      7.143      3.979   1.795  0.46888    
happiness - fear == 0       70.982      3.979  17.838  < 0.001 ***
sadness - fear == 0         19.866      3.979   4.992  < 0.001 ***
surprise - fear == 0        42.188      3.979  10.602  < 0.001 ***
sadness - happiness == 0   -51.116      3.979 -12.846  < 0.001 ***
surprise - happiness == 0  -28.795      3.979  -7.236  < 0.001 ***
surprise - sadness == 0     22.321      3.979   5.610  < 0.001 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Adjusted p values reported -- single-step method)

However, is this a correct approach? Or is there a better way to analyze the data?

Answer 1

Before you do the modelling, it will help if you have a look at some graphical summary of your data. For example,

library(ggplot2)
ggplot(df, aes(emotion, percentage, col=situation)) + geom_point() 

Subjects seem to be good in general at detecting happiness but not fear regardless of the situation.

ggplot(df, aes(emotion, percentage, col=situation)) + geom_point() + facet_grid(subject ~ .)

From your uploaded data, I cannot fit interaction effect between emotion and situation as the interaction completely indexes the data. From looking at this partial data, it's hard to fit a more complex model and so your initial model seems fine (although I would think situation is not significant for your partial data and can be dropped). This does not answer certain questions you may want though. For example, you cannot answer how different situations affect the ability to recognise certain emotions. My suggestion is to have a look at some graphical plot first.