1

I'm about to start up a project (as a high school teacher) that involves comparing the change in average between two different groups.

It's an experiment, and the setup is the following:

  1. Obtain a population of respondents,
  2. Have each respondent answer a questionnaire that scores them on an index,
  3. Divide the respondents randomly into two groups,
  4. Subject them to different information,
  5. Let them answer the questionnaire again and see if there is any difference in the change between the two groups.

I'm thinking of using the z-test to compare the averages, but I'm not sure exactly how. Any ideas or other suggestions are welcome!

Cheers

EDIT: The statistical approach needs to be as simple as possible, as it's for high school students. The z-test looks implementable in Excel, that's part of the reason why I like that for this study.

There are probably numerous issues with that approach - but think along the lines of the weightless ropes and frictionless surfaces often used in high school physics ;-)

trolle3000
  • 97
  • 3
  • 11
  • Who do you want your results to generalize to? Only the students who took the questionnaire or others that could have taken it? In the first case a z test is justified. In the second case Peters answer is right and either t-test will be fine (they are computationally equivalent and easy to do in excel or by hand). – russellpierce Mar 15 '13 at 15:18
  • Best practices for this design have already been discussed on this site, see http://stats.stackexchange.com/questions/3466/best-practice-when-analysing-pre-post-treatment-control-designs – Gala May 14 '13 at 15:31
  • But obviously the focus was on power, correctness and flexibility, not on doing something easy in Excel. I think that the simplest reasonable approach would be to compute difference scores (score on the questionnaire after the manipulation - score at the beginning) and compare the two groups using a regular independent sample t-test. – Gala May 14 '13 at 15:33

1 Answers1

1

There are several approaches e.g.

  1. ANOVA/Regression/ANCOVA (all the same, really) with the final score as the DV and group and initial score as IV

  2. Paired t-tests (this does not allow other covariates)

  3. ANOVA on change scores

However, if you are still designing this study, I suggest more time points for measurement; this allows more interesting hypotheses. If you take that suggestion, then for analysis I would look at multilevel models.

Peter Flom
  • 119,535
  • 36
  • 175
  • 383
  • I am still designing this study, but it needs to be as simple as possible: it's for high school students :-) Anything apart from looking up a test statistic in a table / somewhere else needs to be done in MS Excel or similar. – trolle3000 Feb 13 '13 at 13:37
  • 2
    OIC.... Well, then I'd go with the paired t-test or (equivalently) a one-sample t-test of the differences in scores. – Peter Flom Feb 13 '13 at 14:24