5

I have data on 26 participants (13 from computing and remaining 13 from non computing) who have participated in my research. Each participant is treated with a lab module (Hands on Robotics Session). Now each participant will be evaluated using a rubric on scale of 1 to 4. This experiment has both pre and post test.

For my research i want to evaluate the following questions:

Research question 1

  • Null Hypothesis: students do not learn about computational thinking (programming basics and algorithmic thinking) with the help of robotics.
  • Alternate Hypothesis: Students learn about computational thinking (programming basics and algorithmic thinking) with the help of robotics.

To evaluate the above question, the categories i will be considering are Plan, Implementation and Knowledge gained on a scale of Excellent, Good, Fair and Poor.

I think i should use DEPENDENT T-TEST FOR PAIRED SAMPLES. Am i correct?

Research question 2:

  • Null Hypothesis: Participants background (computing and non computing) has no effect in learning about algorithmic thinking with help of robotics
  • Alternate Hypothesis:Participants background (computing and non computing) has an effect in learning about algorithmic thinking with help of robotics

I will also evaluate question 2 on a scale of Excellent, Good, Fair and Poor, but with respect to background.

My Statistical Question

Which T test should I use for each of my research questions?

Jeromy Anglim
  • 44,984
Dumb_Shock
  • 209

3 Answers3

5

Since you are interested in measuring an increase pre- and post-test, it seems to me that you should use a paired test.

An issue here is that your variables are likely non Gaussian since they are among 4 categories. If your sample size is big (very roughly larger than 50), then it is no big deal. Otherwise, I would use the Wilcoxon signed rank test which is a non parametric analog of the paired t-test.

gui11aume
  • 14,703
  • 2
    very roughly is right. What if $P( {\rm category \ 1} ) = 1 - 10^{-200}?$ ;) – Macro Jun 18 '12 at 23:40
  • 1
    Aha (+1), then I am f... But how likely is that? Close to $10^{-200}$ I'd say :-) – gui11aume Jun 18 '12 at 23:45
  • 1
    It's also worth noting that this would also require an assumption that the categories comprise an interval scale. It also seems (unless I misread) that, in question 1, the role of the null and alternative are reversed from the way they normally are in a paired $t$-test; the null appears to be that the module does help. Finally, question 2 appears to require a two independent samples test, where the paired differences are the outcome variables. – Macro Jun 18 '12 at 23:51
  • 1
    I do not subscribe to using the Wilcoxon signed rank test alternative. If you do a simulation study, you can still show the power of the t-test is good --comprable to the signed rank test-- in moderately skewed distributions. If the researchers are interested in testing for a geometric difference in responses, then a log transformation would be justified. – AdamO Jun 18 '12 at 23:54
  • @marco: Thanks for pointing out that my formation for null and alternate was wrong and thanks for your suggestion on the second question. – Dumb_Shock Jun 19 '12 at 00:59
  • A big +1 for gui11aume for that answer! – Michael R. Chernick Jun 19 '12 at 01:06
1

gui11aume beat me to it. He said exactly what I had in mind when I read the question. Since scaled measurements are not going to fit a normal error distribution very well a nonparametric paired test is the best way to go in my view and for that I would have recommended the Wilcoxon signed rank test. On the other hand if you were comparing scores by domains where many responses are summed, normality is not a bad assumption and the t test is fairly robust anyway. So in that situation a paired t test might be okay.

Michael R. Chernick
  • 42,857
0

I think you are having problems with the nature and elaboration of your alternative hipotesis. Your investigation/reserach hypotesis (and all the theoretical model associated with it) is your alternative hipothesis. Since the variable is clearly ordinal, i´d prefer a Wilcoxon rank test. But anyway i suggest to work out a little bit more your hipothesis.

AnastD
  • 310