2

Let's say that I want to know three training program effectiveness. Our independent variable is categorical variable (with three levels). Our response variable is after-train scores. Then I introduce a co-variate variable: before-training scores. I introduce a new response variable the score difference between after train and before train. (Y = after train score - before train scores)

Now I have two models in R

(1) one way anova, Y~training program (p-value 4.921e-11)

(2) ancova: after train ~ before train + train program (p-value 3.360e-11)

both output leads to same conclusion and very similar p-value.

Now, I'm confused which model to choose, what criteria should I look at to select a better model?

ElleryL
  • 699
  • 5
  • 12

1 Answers1

1

If there is correlation between treatment assignment and pretest score, the ANCOVA will be biased, and you should use the change-score ANOVA instead. Otherwise, the ANCOVA will have a bit more power. If you did a randomized experiment, asymptotically the estimates will be equal (assuming effective randomization). Otherwise, if people get to choose (or the researchers choose) which treatment to take based on some characteristic of the participant, then there may be a correlation between treatment assignment and pretest score.

I would recommend presenting the change score ANOVA, and perhaps mentioning that the same results were obtained from the ANCOVA.

Noah
  • 33,180
  • 3
  • 47
  • 105