2

I have two groups, Controls and Patients, with 6 measurements for each such as speed, amplitude and so on. I'd like to examine whether these measures are different for the 2 groups. So firstly I'd calculate the Mann-Whitney U-test between controls and patients for speed, and then for separation and so on until I have run 6 comparisons.

I'm aware that I should correct this for multiple comparisons, but am unsure of the best way. I'm playing around in R with the pairwise.wilcox.test function but it only seems to work in cases where there's 1 group and 1 set of readings.

Firstly, just so I can search more easily in the future, what's the name of my experimental setup called? And what's the best way to adjust for multiple comparisons in R?

Harvey Motulsky
  • 20,456
Stuart Lacy
  • 708

2 Answers2

2

It seems to me that you have three choices:

  • Don't correct for multiple comparisons, and say so clearly when you present or publish your work. This is probably the most common strategy. Anyone viewing the data needs to take the multiple comparisons into account when interpreting.
  • Correct using statistical hypothesis testing approach. The Bonferroni method is used most often.
  • Correct using the False Discovery Rate (FDR) approach.

Of course there is a fourth choice: Find the comparison with the most impressive results, and publish that without mentioning the other outcomes. This is cheating of course, but is commonly done.

Harvey Motulsky
  • 20,456
  • Thanks for the useful ideas, I'll have a read about the FDR approach but most likely will go with printing the raw p-values! – Stuart Lacy Jun 10 '15 at 21:02
1

You simply have two multivariate samples: Multiple, different measurements on the same subjects.

You can use stepwise procedures like Bonferroni-Holm in your case. They work only on the p-values, so you can do this adjustment even "by hand" for your 6 comparisons.

There is no general answer about which multiple comparison procedure to use. They all have different characteristics, so you have to ask you questions in order to decide, some of them are:

  • How much do you know about the distributions? (in your case not much, as you use Wilcoxon tests...)
  • Do you also need simulataneous confidence intervals?
  • Do you know the dependency structure between the test statistics (in your case probably not, unless you know that there is a fixed relationship e.g. between speed and amplitude)?
  • Do you really want to adjust for the family wise error rate or is a correct false discovery rate sufficient?
Horst Grünbusch
  • 5,395