what would be the best way to find out if a student's marks is increasing with the number of attempts he or she takes for a test?

Question

Please bear with me. I am quite a beginner in statistics. I want to analyse what would be the best way to find out if a student's marks is increasing with the number of attempts he or she takes for a test?

So as to answer that if more practice means better performance? for example for a student in the first attempt he or she scores less marks then as he or she keeps taking more and more attempts his or her performance is increasing/decreasing.

Say there is a data set of 40 students. For all the students we have a data set of number of attempts by each student. Now I have to analyse whether more number of attempts by a student means better performance than a student making lesser attempts

Do students stop doing the test once they have passed? If the latter, then the analysis has to be done carefully, taking into account this "censoring". It might then be that the average mark over all students drops as the good students stop taking the test. You write that you only know the number of attempts per student, but surely you also know the result of the exams? — jarauh, Feb 12 '19 at 08:01
There are some ambiguities in this question, or the direction is unclear. 1) The marks may (likely) increase without any learning effect present. Often the students that make more attempts are in a situation where students fail a test. The failing can be due to the level of the students being low, or due to some random variation for a student around his/her performance level. Now if students that redo tests are only a selection of students with a specific first result, those with low marks, then they will perform better without a change of their mean performance (regression to the mean)... — Sextus Empiricus, Feb 12 '19 at 08:18
.... so the question is whether you just wish to describe the observation, or whether you wish to analyse the causes of the observation and test underlying models.... — Sextus Empiricus, Feb 12 '19 at 08:19
Please clarify your question in light of @MartijnWeterings' comments, Desmond. It isn't clear if the posted answers are correct until we have a better sense of what your situation is. Can you post your dataset? — gung - Reinstate Monica, Feb 12 '19 at 16:02
I agree with @gung, because the data you describe do not permit this question to be answered, even assuming you have data about each student's performance. — whuber, Feb 12 '19 at 17:24
Please see I can not post the data set as its not my property. Also to clarify @Martijin s doubts. It is safe to assume a student making attempts more often is practicing then taking those attempts in lieu of this if the student is scoring well in the later attempts as compared to earlier than that can successfully imply that more often attempts means more practice means more marks. — Desmond, Feb 13 '19 at 10:42
@Desmond I did not have doubts, I have questions about the question. It is unclear what the situation is exactly and what the question is exactly. — Sextus Empiricus, Feb 13 '19 at 17:29

naive · Answer 1 · 2019-02-12T09:53:05.450

So as to answer that if more practice means better performance?

This question talks about the relationship between practice and performance. Regression analysis can be used to answer the posed question. But, regression is not the only tool to get an idea of the relationship. Most commonly used Pearson correlation can also be used to get an idea about how do these random variables move together. A potential question that might come into the OP's mind is:

So, why did I point out regression and not correlation?

It doesn't matter which variable you call "independent" and which one "dependent" when calculating correlation. But, it really makes a difference in interpretation from a regression analysis.

With regression you might have to think about the directional relationship that would be there among the variables and model the potential cause as independent variable and the effect as dependent variable, if you want meaningful interpretation about the relationship among the variables.

(stolen from this link): Correlation is almost always used when you measure both variables. It rarely is appropriate when one variable is something you experimentally manipulate. With linear regression, the X variable is usually something you experimentally manipulate (time, concentration...) and the Y variable is something you measure.

coming back to the question: If you have a positive correlation among practice and performance then one might also claim that better performance means better practice, but essentially the relationship that you want to examine incorporates the "potential cause" as practice and performance as effect. So, correlation is not going to help you with your analysis.

For purposes of doing regression have a look at this thread. For a basic understanding of regression have a look at this answer of mine in the very popular thread.

Now I have to analyse whether more number of attempts by a student means better performance than a student making lesser attempts

To answer this you can perform a paired t-test for two different experimental conditions, i.e. you can compare the scores of the students from the $1^{st}$ test to $k^{th}$ test as long as the assumption is satisfied.

The assumption is : The differences $w_i = x_i−y_i$ where $x_i$ and $y_i$ are the performance scores of the $i^{th}$ student in the $1^{st}$ test and the $k^{th}$ test, between the paired samples are independent draws from a normal distribution $N(µ, σ^2)$, where µ and σ are unknown.

I find it a bit confusing what you see as the difference between regression and correlation. They are only subtly different in how the model coefficients are expressed and not so much different in the model for the data (although regression allows more complex deterministic relationships, but still, in the case of least squares regression, you will just look for the relation $f(x) $ that correlates the highest with $y $). But you seem to indicate that regression is some sort of indicator (or proof) for a causal relationship. I find it dangerous to equate regression with causation model. — Sextus Empiricus, Feb 12 '19 at 08:05
Agree to your comment about difference between regression and correlation. I am not saying that regression is an indicator for a causal relationship. I was saying that if you think about a variable as a potential cause and another as an effect, then you can incorporate them in the regression model because regression does have a much stronger connotation that one is estimating an explicit directional relationship than does estimating the correlation between two variables. Please feel free to edit the parts that you think needs to be addressed. Thank you — naive, Feb 12 '19 at 09:01
@MartijnWeterings I guess, the phrase potential cause is creating confusion. Should I do away with it? — naive, Feb 12 '19 at 09:18
A regression model is only showing a relation between two variables. It does not reveal/proof/show the direction of causal effects any more better than a correlation does. To study/proof a causal relation one should carefully define the experiment as in a randomized control study. So, the confusing phrase is "Because correlation doesn't let you talk about the potentially causal relationship that might be among the variables. But, with regression ...." — Sextus Empiricus, Feb 12 '19 at 09:25
"...the relationship that you want to examine incorporates the "potential cause... So, correlation is not going to help" It is still unclear to me why you believe that linear regression is gonna help you more than correlation. With regression, it is just as fine to switch the x and y variable and it does not matter which one is the dependent variable or which one is the independent variable. Regression is just giving you a model for the mean of the variable y conditional on x. Regression does not care, per se, which variable you manipulated/controlled or which is the cause of the other. — Sextus Empiricus, Feb 12 '19 at 12:55

score 0 · Answer 2 · answered Feb 11 '19 at 10:17

0

For me, two simple first analysis will calculate correlation between marks and number of attempts. If it is positive and high, it might by true (but correlation does not imply causation)

A second, more sophisticated approach will be using regression. Take into account that because number of attemps is a discrete one, you need to take care of that (see Multiple regression with categorical and numeric predictors and https://stackoverflow.com/questions/22192934/linear-model-lm-with-dependent-variable-being-a-factor-categorical-variable)

Best!

answered Feb 11 '19 at 10:17

LocoGris

313

Thanks for the help. I understood the correlation bit but am still unclear on the regression – Desmond Feb 11 '19 at 10:20
Hi! As a first approach I will run a linear regression with score as the dependent variable and number of attempts as the explanatory variable. Nevertheless, you need to take into account that number of attempts is discrete (1,2,3...) but linear regression will treat it as continuous. So as a third step I will try to find a way to solve this issue. – LocoGris Feb 11 '19 at 11:15

what would be the best way to find out if a student's marks is increasing with the number of attempts he or she takes for a test?

2 Answers2