1

I am exploring the effect of social media on the performance of students of two different majors:

  • Majors: Math and English
  • The students engage in group study for a period of time.
  • After the group study, all students take a scored test.
  • During the group study, each student has access (or not) to social media (SM) on their smartphones.
  • N trials are repeated, where each trial has 6 independent experiments:
    • 10 English students, without SM
    • 10 Math students, without SM
    • 5 Math students + 5 English students, without SM
    • 10 English students, with SM
    • 10 Math students, with SM
    • 5 Math students + 5 English students, with SM

My questions are:

  • Would the independent variables be SM and Major and the dependent variable be the score?
  • What would be a good formulation of the hypotheses?
  • What would be the ideal significance test to compare the performances across majors, with and without SM.

Any help is greatly appreciated. Thank you.

rmas
  • 45

1 Answers1

1

First question:
If you are only interested in the effect of SM, then the independent variable is only SM. If, however, you are interested in the effect of the combinations of SM and Major, then you have those two as your independent variables. According to your data, the factor "Major" would consist of three alternatives ("levels"): "pure Maths", "pure English", and "balanced mixture of Maths and English". The dependent variable is indeed the score.
Furthermore, as @dipetkov points out, taking Major as an additional independent variable could help reduce the variance.

Second question:
You could formulate the problem as a NHST with the null hypothesis that SM has no effect on study performance. (NHSTs are still the most popular approach to testing hypotheses.)

Third question:
The "ideal" significance test depends on how you measure the performance. E.g., if your measurements are from a continuous scale and you can assume that the measurements are independent and normally distributed, you could use ANOVA. If you cannot presume normality, even not approximative, you have to choose tests that match your problem. There are some nonparametric tests, like the Kruskal-Wallis test, but that will not account for the two-way property, or the Mann-Whitney U test, but that is only for two levels. If that is not enough, you might want to check out the answers to this question.

frank
  • 10,797
  • Thank you Frank for the insights! I actually want to know the effect between majors. For example: which major performs better with or without SM. For the third question, we assume that the measurements are continuous and are not normally distributed. What would be the adequate test in this case? Thank you. – rmas Jul 14 '22 at 07:45
  • 1
    @rmas I updated the answer. – frank Jul 14 '22 at 09:12
  • 1
    I disagree about your advice on Q1. There is a reason we include covariates in a model and the reason is that these covariates oftentimes explain quite a bit of the variance in the outcome, thus reducing the residual variance, so that we can estimate the effects of interest more accurately. – dipetkov Jul 16 '22 at 13:08
  • 1
    @dipetkov I edited the answer accordingly. – frank Jul 17 '22 at 05:39