0

I am writing a research proposal measuring the effects of a treatment on a population with a pretest and posttest as the dependent variable. What is the best statistical analysis to use for this data?

mkt
  • 18,245
  • 11
  • 73
  • 172
Hannah Ballard
  • 1
  • That depends on two factors. 1) What do you want to know? 2) How are the populations distributed, e.g., are they normally distributed or not? – Carl Dec 05 '16 at 05:46

1 Answers1

7

The default test would be a paired-samples t-test (aka related-samples t-test or dependent-samples t-test). This test assumes that you have a normal distribution of difference scores, though, so if that assumption is not met you might consider a Wilcoxon signed rank test.

Just one observation. The most basic pretest/posttest design is confounded with time. That is, if you only have one group that you pretest, then administer treatment to, then posttest, you can't determine whether the change is due to treatment or time (or things related to time). For example - a university professor decides to implement a writing skills program in his class. He tests students at the start of term, and again at the end of term. If he sees a change, he can't know whether it's caused by a) his program b) other education the students are receiving at the same time, c) simple maturation of his students, or d) anything else time-related.

To avoid this problem, run a concurrent control group that doesn't receive treatment, with random assignment to the treatment/nontreatment variable. This should allow you to determine the effect of time and related variables - allowing you to determine the portion of pretest/posttest change that is due to the actual treatment. The analysis would then be a mixed 2-factor ANOVA (pre/post as a within subjects variable and experimental/control as a between subjects variable). (Or you could run an independent-samples t-test on the before/after differences.)

(For ethical or practical reasons it's not always possible to run such a control group - which is why the education literature and the psychotherapy literatures sometimes have simple pretest/posttest studies. But I am skeptical of interpretations of such studies.)

J Taylor
  • 381