2

I have the feeling that this is a very basic question, but I have never had any formal schooling in statistics and after extensive googling I feel I just can't come up with the right search terms.

Suppose we have the question: are men faster runners than women? We have a limited group of, say, 30 subjects - 20 male, 10 female. But we have plenty of time, so we let those 30 subjects run a 2k distance every day for two weeks: in the end, we have 14 measurements (time in sec over 2k) for each subject.

What, then, is the best way to deal with this data?

You could collect all male measurements (280) and all female measurements (140) and do a two-sample t-test. But this is clearly wrong, as it would give you a false boost of statistical power (consider the more extreme example of 3 subjects, two male and one female: maybe one male is fast by chance (pretty likely under the null-hypothesis), but this could still cause a highly significant result).

You could do a two-sample t-test using the average time of each subject. (So far, this seems to me the best solution.) Say this test comes back insignificant. After all, your sample size is pretty small. Doing more tests with the same subjects would then not improve your statistical power for this particular question, which may be fair. But now consider the following:

If you do the test for the individual days, you do find that males are significantly faster about half of the time. Even when you correct the significance level for multiple comparison using bonferroni (ie. alpha = 0.05/14), males are significantly faster on two out of fourteen days.

So, can we then reject our null-hypothesis? Is there a more formal way to deal with this? How does one report the result? - "males were faster on two out of fourteen days, so we conclude males are faster" doesn't seem very convincing.

(Google gave many links to repeated-measures ANOVA, but from what I understand this test is designed to compare different points in time; I am not interested in that. All I want to know if men are faster runners than women)

Scipio
  • 141
  • Repeated measures ANOVA is used to compare means in groups from whom you have drawn repeated measures. It does not compare different points in time unless you classify those time periods as different groups, which would the wrong thing to do in an analysis such as this. – AdamO Jul 08 '15 at 20:38
  • @AdamO, thank you for your response! Everything I find talks about changes in the measured variable over time: "Repeated-measures analysis can be used to assess changes over time in an outcome measured serially" (http://cslras.pbworks.com/f/park_korean_j_lab_med_29_1_correct_use_of_repeated_measures_analysis_of_variance.pdf). Could you maybe provide a link or example of a case similar to what I described? – Scipio Jul 08 '15 at 20:59
  • I think you have missed the forest for the trees. Quoted from your link, "Repeated measures are obtained when we measure the same variable repeatedly, for example, at different time points. Matched data can be obtained when we have sepa- rate groups but they have been matched in some way. For both cases, there is correlation among measurements within subject, or within matched pair, so standard ANOVA can- not be applied to such data.". – AdamO Jul 08 '15 at 21:21
  • Haha yes you may be right. I haven't really used standard ANOVA before either, but I think I have managed to do this for categorical values. Does it make any difference if you have numerical values? Eg, try to see if subject length also has an influence? – Scipio Jul 09 '15 at 08:57
  • Thanks for your help @AdamO, I posted a follow-up on a more complex scenario of multiple continuous and categorical predictors. – Scipio Jul 09 '15 at 22:39

1 Answers1

1

As pointed out by @AdamO, an repeated measure anova is indeed appropriate for the case I described in the question. See a matlab example here.

Scipio
  • 141