1

My dependent variable is reading time. My two predictors are categorical.

I conducted a lmer model with the default family and the performance package indicated that the response distribution fits better with an inverse gamma family. enter image description here

So I conducted a glmer model with the gamma inverse family: enter image description here

And I conducted a glmer model with the gamma identity family: enter image description here

The formula was always the same, but the in the plot of the gamma inverse model seems almost like the symmetrical mirror of the x and y axis of the gamma identity. Is this a usual thing?

UPDATE:

I did not transform my data before running the models, but the RT are the sum of three reaction times:

The origin of the data is the measurement of reaction time for button presses in seconds. Which button press would reveal to the participant a new word of the sentence. I am following previous experiments, and the dependent variable is obtained as the sum of the reaction time of the last three words of the sentences (these are the words after we introduced the manipulated word). The purpose is to see in which conditions participants are slower and in which conditions they are quicker to read after the manipulation. (I personally think a gam model with the reading times predicted by word position and predictors would be more interesting here to see if the line tendency is to increase or decrease reading times in each conditions, but I am only a student.)

Model1:
minv <- glmer(RT ~ 1 + pred1 + pred2 + pred1:pred2 + (1 | id) + (1 | stimulus_id) + (1 | order), data=df, family= Gamma(link="inverse"), control = glmerControl(optimizer = "bobyqa", calc.derivs = TRUE))

cat_plot(minv, pred = pred1, modx = pred2, geom = "line", line.thickness = 2, interval = FALSE)


Model2:
mid <- glmer(RT ~ 1 + pred1 + pred2 + pred1:pred2 + (1 | id) + (1 | stimulus_id) + (1 | order), data=df, family= Gamma(link="identity"), control = glmerControl(optimizer = "bobyqa", calc.derivs = TRUE))


cat_plot(mid, pred = pred1, modx = pred2, geom = "line", line.thickness = 2, interval = FALSE)

Histogram of the reading time in each condition: enter image description here

Olivia
  • 121
  • 2
    In a comment to your previous question, @Glen_b explained that the model(s) you are fitting are for the conditional response. In other words, don't plot a histogram (and why does the histogram have only 2 bins?) for all responses lumped together. It's more appropriate to make one plot for each of the four groups determined by your two binary predictors. – dipetkov Mar 23 '23 at 10:36
  • 1
    As for the inverse relationship, it's to be expected as even though the formula $X\beta$ (on the right side) is the same, the link function is different. With identity link $\operatorname{E}(Y) = X\beta$ while with the inverse link $1 / \operatorname{E}(Y) = X\beta$. In your case $X\beta$ is very simple because you have 4 groups determined by your 2 binary predictors. – dipetkov Mar 23 '23 at 10:50
  • To optimize the conditional response, I opted for the performance package, which identifies the best fit. As for the histogram, it was a default output from the performance function 'check_distribution(model)', and I did not specifically choose it. I also conducted separated fitdist() for the conditional responses like @Glen_b and you are kindly suggesting. The inverse gamma keeps showing up as the best fitting distibution :/ The additional plots are focused on the interaction term, and I utilized the interaction package for their creation. – Olivia Mar 23 '23 at 10:52
  • So in my interpretion of the results about the reading time I should take it as it is (e.g., reading times in B condition are larger in H than in L), or should I think of the results in an inverted way (e.g., reading times in B condition are larger in L than in H)? – Olivia Mar 23 '23 at 11:01
  • 1
    I think I understand the question better now. In your plots the y-axis is labeled "RT". But do you plot the response on the original scale (the reading time scale) or the linear scale (the $X\beta$ scale)? – dipetkov Mar 23 '23 at 11:05
  • I did not perform any data transformation. I assume that the y-axis represents the raw values of the reading time because they are close to the raw descriptive means, and I believe that the interactions package does not perform any data transformations when plotting models. – Olivia Mar 23 '23 at 12:22
  • 2
    You use the inverse link, so yes, you are doing a kind of transformation. I would suggest to follow @SextusEmpiricus advice and show your code. – dipetkov Mar 23 '23 at 12:30
  • 1
    What is this graph 'distribution of response'? Are those the observed reading times? Is that reading time like a variable with only values 1 or 2? – Sextus Empiricus Mar 23 '23 at 13:03
  • I removed that default histogram since it was inducing error. The reading time variable is continuous and it ranges from 0.432 to 3.615. – Olivia Mar 23 '23 at 16:30
  • 1
    There are three random components in your model. Even though the code looks okay, you may want to read this about the challenges of fitting a Gamma GLMM: https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#gamma-glmms – dipetkov Mar 24 '23 at 20:13
  • 1
    In case these might be of use to you. I wasn't able to reproduce the "trippiness" using the reaction time data from this blog post: An (in)complete guide to reaction time analysis. CV threads about analyzing RT data: 1, 2, 3. In short, I think it would be hard to figure out what's going on without the actual data. Start with a simpler model that you can understand better? – dipetkov Mar 24 '23 at 21:31

1 Answers1

0

Is this a usual thing?

If you are plotting different models then you are gonna get different results. And yes, this might even completely reverse the effects.

That's a danger of the practice that you are using here. You seem to be plotting many different models until you find something that looks ok. Ideally you have some previous ideas about what model should be neccesary, with some technical/theoretical motivations, rather than picking out whatever model seems to give the best outcome for your case.

Sextus Empiricus
  • 77,915
  • Reconsidering my answer, it is actually weird that the value of RT as a whole is larger/smaller. The lowest point in the one graph is larger than the highest point in the other graph. So probably your RT plots are incorrect and one of the graphs shows 1/RT instead of RT. But still you do get that the methods differ. If you convert the axis of one of those graphs into 1/RT then slopes of the lines will change. – Sextus Empiricus Mar 23 '23 at 11:04
  • If you give more details about the code that you use to produce these images then this might be verified. – Sextus Empiricus Mar 23 '23 at 11:05
  • I agree that pre-registration of the methods and analysis are critical. I indeed have some previous ideas about the model: previous researchers that used the same experimental task used a simple Gaussian distribution, and other researchers in my field advised the use of an inverted Gaussian with the identity link. However, none of those families fit my data well. I am looking for a model that fits my data, so I can trust the results. – Olivia Mar 23 '23 at 12:30
  • @Olivia "However, none of those families fit my data well. I am looking for a model that fits my data, so I can trust the results." sure, but it is a practice that needs to be done carefully. It would be cherry picking if you would continue an analysis on the same data (I am not sure whether this are your intentions, but it needs to be mentioned for anybody else that comes across this question and might pick up that this is a good way of analysis). – Sextus Empiricus Mar 23 '23 at 12:57
  • 1
    Another thing, instead of working with weird distributions,l and link functions, why not model directly reading speed instead of reading time? What is the source of error in your experiment, is it your stopwatch (which would mean that using RT as variable makes sense) or is it the sampling of people (which might possibly make speed = 1/RT a better variable). Also, could you plot the reading times as a histogram for the four different cases? – Sextus Empiricus Mar 23 '23 at 12:57