4

This post is related to my previous ones, but now I'm looking at each year separately (i.e, this is not a repeated measures design). My data set looks like this:

SUBJECT    PROFICIENCY (in Eng) SCORE LANGUAGE

PETER      100                   154    Spanish 
PETER      100                   132    English
MARY       95                    191    Spanish
MARY       95                    139    English

So each SUBJECT was tested twice: one test in English and one test in Spanish. Therefore, each SUBJECT has 2 exam scores. I want to see the impact of Language on the test scores. However, I already know that Proficiency in English is highly correlated to the test score in English:

correlation PROFICIENCY and the ENglish Exam Score

What I'm wondering is if I can fit a model in which I could use proficiency as a "moderator" (something like a partial correlation). My idea was to fit a model with a random intercept for Proficiency. The thing is, I only have one Proficiency score per student (I don't have a Proficiency score for Spanish). Hence, I have two entries for Language and one for Proficiency (in Eng).

Would this make sense:

mod1 <- lmer(ExamScore ~ LANGUAGE + (1 | SUBJECT) + (1|PROFICIENCY IN ENG), data = data)

I have seen similar questions elsewhere with a categorical moderator (here and others

Thanks in advance, any thoughts would be much appreciated :)

Larissa Cury
  • 741
  • Can you explain educational setting: Is English the native language? Or "first" foreign language while Spanish -- the second? I'm wondering whether a model like Spanish_score ~ English_score + English_proficiency might not be interesting / applicable. – dipetkov Aug 28 '22 at 12:17
  • @dipetkov , so English is the language that they're learning (ie, the second), Spanish is their native language (that's why I don't have a proficiency score for Spanish). The ideia is to see whether the different languages play a role on their scores on their exams. So I had a wide table and turned it into a long table in order to do ExamScore ~ Language, but then I noticed that proficiency in English is correlated with the ExamScore, so I guess that I'm supposed to control its effect on the exam – Larissa Cury Aug 28 '22 at 14:52
  • because subjects with a higher Proficiency will likely have a higher score in English (ps: thank you once more!) – Larissa Cury Aug 28 '22 at 14:53
  • The idea "see whether the different languages play a role in the scores on the exams" sounds a bit imprecise like this is an exploratory data analysis. Have you thought about how you will interpret the coefficients in the models you are considering and what they tell you about your study? – dipetkov Aug 28 '22 at 15:10
  • So, I'm using dummy coding for Language. Hence, Spanish is the reference level (0) and English is (1). I've ran a simple regression with lm() for an illustration, 
    (Intercept)  147.712      4.924  30.000  < 2e-16 ***
LANGUAGEL2   -35.076      7.089  -4.948 2.66e-06 ***```                                                          So my interpretation is "the score decreases by 35 points when the task is performed in Eng in relation to when its performed in Span" (b0 = 147, b1 = -35)
    – Larissa Cury Aug 28 '22 at 16:05
  • So that's saying that on average students score 35 on the English which they are learning as a second language than on Spanish which is their mother tongue. Makes sense but also not surprising that's negative. Again I think what you'd like to learn from the data gets lost a bit in the details of should something be a random or a fixed effect. – dipetkov Aug 28 '22 at 16:14
  • @dipetkov , yes, the result is expected. But I guess that I should control the fact that data comes from the same participants (1 | Subject) --- concerning the profiency correlation with the English score, I just thought that I need to account for that, shouldn't I? – Larissa Cury Aug 28 '22 at 16:20

1 Answers1

2

Proficiency is a continuous variable, ranging from 0 to 100. It's not clear how you propose to include it as a random effect. Unless you bin it into levels of proficiency but why would you want to do that? See the discussion on Why should binning be avoided at all costs?

Instead treat proficiency as a covariate, that is, include it in the mean structure of the model: ExamScore ~ Language + Proficiency. Optionally, you can "spline" it to allow for a smooth, flexible and nonlinear relationship between proficiency and test scores. However, before including Proficiency in the model at all, consider carefully what it measures: English proficiency (as a second language) before the test? after the test? throughout the school year? It might not make sense to have this variable in the model at all.

How do I enter a continuous variable as a random effect in a linear mixed effects model? explains that a random effect is conceptualized as a population of groups (students) from which we have drawn a random sample (students who have taken English and Spanish language tests).

dipetkov
  • 9,805
  • thank you! I like this ExamScore ~ Language + English_Proficiency idea but I have two data points for Language (ie, one score in Eng and one in Span) and only one score for Proficiency (therefore, each subject has its Eng proficiency duplicated in the table), that's why I was thinking about a random intercept (thinking that each subject will start from a different point considering their Eng scores) what do you think about it? – Larissa Cury Aug 28 '22 at 14:56
  • Think about what the formula ExamScore ~ Language + English_Proficiency + (1|Student) says for the expected score of student with proficiency 95 and for student with proficiency 100. – dipetkov Aug 28 '22 at 15:09
  • thank you for this link, it was very useful! I've ran the model you've asked me about Fixed effects: Estimate Std. Error df t value Pr(>|t|) (Intercept) 118.4777 7.6677 80.2972 15.452 < 2e-16 *** LANGUAGEL2 -38.2114 5.5403 53.7005 -6.897 6.21e-09 *** PROFICIENCY 0.4954 0.1068 65.8902 4.639 1.71e-05 *** I'm confused, though, as to what extend does it make sense to use Proeficiency as a predictor? For ex, b0 = Score for Language when Lang = Spanish, but I don't have a proficiency score for Span (it seems weird to compare a measure for Eng to Span) – Larissa Cury Aug 28 '22 at 16:25
  • It's up to you to decide what makes sense for the analysis or not. I'm just pointing out a technical difficulty with the proposed analysis. You have a dataset, what you don't seem to have is a clear idea of what you want to ask of your data. I suggest you think about this more. PS: Not all questions we might want to ask can be answered by the data. – dipetkov Aug 28 '22 at 16:32
  • Some things to consider: What is the proficiency supposed to measure? What about the test scores? How are they plausibly related? What's the purpose of building a model for these test scores? ... – dipetkov Aug 28 '22 at 16:39
  • thank you very much!! Basically, I just need to test if language affect the testscores, period. So, yes, a model with Score ~ Language + (1|Subject) seems just fine. My problem is that I found out this correlation with proficiency and then I started wondering that I should take that into consideration since subjects with a highier proficiency score will likely have a better performance than the ones who have lower scores and then I started wondering how I could "control" for this, that's the idea (as I'd do with a partial correlation, for ex) – Larissa Cury Aug 28 '22 at 18:24