1

I am trying to determine the effectiveness of a learning intervention that we developed. We had two groups, a control and experimental group, where we applied the intervention to the experimental group. Each group took a pretest and posttest and we have now tabulated the scores.

Here's what we have so far about the scores:

  • the pretest and posttest scores are not normally distributed
  • we were left with unequal sample sizes for the control and experimental groups as some students unexpectedly dropped off from the activity

Based from what I have searched on the Internet so far, here's what I think we can do

  1. Check for the similarity of the pretest scores across the two groups using Mann-Whitney U Test
  2. Do the same for the posttest scores
  3. Determine significance of increase of the posttest scores from the pretest scores (gain score) using Wilcoxon sign test

I'd like to know

  1. If I got those information right, and
  2. If I want to know if the gain scores obtained in the experimental group is significantly better than the gain scores obtained by those in the control group, what would be an appropriate test for that?

EDIT:

  • As suggested, I am adding a histogram and QQ plot of the residuals gain score (posttest-pretest score) for the test of normality (which I now understand applies to the residuals and not the data itself). I based the generation of the QQ plot from a couple of articles and YouTube videos, I hope I get it right.

Histogram of the residuals

QQ plot of the residuals

MG B
  • 11
  • Does the following post help: https://stats.stackexchange.com/questions/3466/best-practice-when-analysing-pre-post-treatment-control-designs – kjetil b halvorsen Feb 07 '21 at 14:56
  • Thanks, I looked at ANCOVA and it says it assumes normality of the distribution of the data (did I get that right?), which unfortunately does not apply to my case. – MG B Feb 08 '21 at 01:09
  • Note first that the residuals from a fitted model should be approximately normal, not the data itself. Maybe you can show us some plots, fit a model, show us the residuals (as a qqplot is best). So maybe it can be done ... you could also contemplate using bootstrap to get correct confidence intervals even in nin-normal case. – kjetil b halvorsen Feb 08 '21 at 04:23
  • Oh thanks for that clarification. I'll edit my post to add the QQ plot of the gain scores – MG B Feb 09 '21 at 01:18
  • Thats good! The residuals (assuming you've done everything correct, I'm not sure that only youtube videos are a good way of learning stats ...) have a right tail, meaning that some improvements are too large (compared to a normal). You could try bootstrap ... can you also tell us which software you used, and te code or models you used, with output? – kjetil b halvorsen Feb 09 '21 at 12:35
  • 1
    I just used Google Sheets since I found articles and videos on how to do the QQ plot by hand, but I am studying R right now to verify what I did.

    I also just found a discussion on bootstrap, and it looks like I can use this to compare my control and treatment/experimental group. Thanks for your help. I'll try to update my post with my results once I'm done.

     – MG B Feb 10 '21 at 05:19

0 Answers0