0

I have conducted three tests (math, Biology, Chinese) on 4 groups of students who are coming from Hong Kong, Pakistan, Philippines, and Singapore respectively.

I have the following dataset with the following columns: Subject: math, Biology, Chinese Country: Hong Kong, Pakistan, Philippines, Singapore Score: values ranging 1-100

I want to test for each subject whether there is a significant difference in scores between the Hong Kong students and the other three group of students from different countries. I conducted independent samples T-test but my question is do I need a Bonferroni correction here?

Here is some simulated data:

df <- tibble(
Subject = rep(c("math", "Chinese", "Biology"), 40),
Country = rep(c("Phili","Sing", "HK", 'Pak'), 30),
Score=  sample(x=1:100, size = 120, replace = TRUE))
Michael
  • 13
  • 3
  • Thanks for pointing it out, Henry. I have edited that. – Michael Feb 28 '22 at 11:48
  • There is little philosophical justification for multiplicity correction (https://stats.stackexchange.com/questions/120362/whats-wrong-with-bonferroni-adjustments), so my suggestion is that you should only use it for political reasons - e.g. if your study is going to be assessed by people who believe that multiplicity correction is a good idea. – fblundun Mar 10 '22 at 14:13
  • Some thought here https://stats.stackexchange.com/questions/468620/correction-for-multiple-testing-in-multiple-regression-analysis/468622#468622 will also apply. – Christian Hennig Sep 09 '23 at 11:56

1 Answers1

0

I just need a bit of clarification :

  • Are you trying to compare (and test) the difference between Hong Kong students with students from each different countries or Hong Kong students with the others students from different countries as a whole ?

And the important thing is what would be your question ? (I work in healthcare so I might not be suited to your topic but I'll try to be as clear as possible)

  • if you want to have a strong claim (like in a randomized controlled trial testing if treatment A is better than treatment B), with hypothesis testing (H0, H1) that would be the approach of Neyman & Pearson. Here you would need a correction for multiple tests
  • if it's only observationnal data and you want to see if your result are surprising if it were only the effect of randomness (the smaller the p value, the more surprising), that would be closer to the approach of Fisher, then I wouldn't really need to adjust for multiple tests. You just look at the p values (or confidence intervals)

And my last comment on this is you could use Bonferroni which is the simplest method but you could use others (Holm procedure, Fallback, Hierarchical procedure)

I hope it helps, I can develop if it's not clear

Loris
  • 11