1

I couldn't find my answer from previous similar posts, hence asking this silly question.

I have two groups of samples, say A and B, with 3 replicates of each.

I used t.test(paired=T) function in R to figure out statistically significant (0.05) data points among the two groups.

Now, my question is shall I also go for the correction of obtained p-values from t.test() function. Something like this

p.adjust(p, method = p.adjust.methods, n = length(p))

Irrespective of your answer "yes" or "no", please explain. I am bit lost.

kjetil b halvorsen
  • 77,844
Angelo
  • 4,485
  • So these adjustements come from the fact that repeated trials have a cumulative p-value that is returned. So if you do 10 trials, you would need to take a p.value 10 times smaller for each individual test (if you apply Bonferroni correction). Essentially you want to have a family wise error of say .05, then you would need to take the single error as being .05/sample size. – FisherDisinformation May 09 '16 at 14:49
  • So, if I understand correctly, I can simply use: t.test(x,y, paired=T, conf.level = 0.95), correct? – Angelo May 09 '16 at 14:58
  • Can you clarify your data set? Do you end up with 3 p-values, n p-values where n is the size of each sample, 3 * n, or something else? – mdewey May 09 '16 at 15:21
  • my bad., I made a typo: instead of p-value/sample size, it is p-value/number of hypotheses to test. – FisherDisinformation May 09 '16 at 15:22
  • 1
    I have 23 thousand genes and they have been measured across two conditions with 3 replicates in each condition. I get a pvalue for each gene across the two condition and my cut-off is 0.05 to decide its significance. – Angelo May 09 '16 at 15:26
  • See this answer and the linked questions and answers. You have not supplied enough information for anyone to give a sensible answer to your question. https://stats.stackexchange.com/questions/633673/which-family-wise-correction-is-needed-for-one-sided-t-tests-across-3-groups-wit/633679?noredirect=1#comment1184029_633679 – Michael Lew Dec 26 '23 at 01:27

1 Answers1

1

If you do a genome-wide association study, GWAS, with 23,000 genes, and all the null hypotheses are true, then you'd expect 0.05*23000=1,150 to be less than 0.05. So yes, when you are doing multiple comparisons with tens of thousands of comparisons, you definitely need to account for multiple comparisons to make your conclusions make sense. Plenty has been written about analyses of GWAS. Here is one review:

enter image description here

Harvey Motulsky
  • 20,456
  • 1
    The multiple of multiple comparisons has also been well documented in fMRI analysis, in a particularly comical manner with the IgNobel prize winning study finding brain activation in a dead salmon’s brain when multiple comparisons aren’t corrected for https://blogs.scientificamerican.com/scicurious-brain/ignobel-prize-in-neuroscience-the-dead-salmon-study/ – JElder Dec 26 '23 at 04:42