3

My question is rather simple: When should you use the Bonferroni correction over the FDR correction?

I've read about that the Bonferroni correction applies a more conservative approach in comparison with the FDR correction. Additional, the FDR should be more accurate with more pvalues (how much is more?).

I applied both p adjustment methods and can clearly see that the Bonferroni is more conservative (table and figure). But how would I know which method to use?

In my case I have 20 pvalues derived from 20 separate glm's (binomial):

     Original    FDR           Bonferroni 
X1   0.00516     0.0121      0.1033 
X2   0.00000     0.0000      0.0000 
X3   0.00128     0.0051      0.0255 
X4   0.01730     0.0316      0.3461 
X5   0.00545     0.0121      0.1091 
X6   0.75503     0.7550      1.0000 
X7   0.54320     0.6036      1.0000 
X8   0.31668     0.4222      1.0000 
X9   0.68161     0.7175      1.0000 
X10  0.01737     0.0316      0.3474 
X11  0.02543     0.0391      0.5086 
X12  0.02055     0.0343      0.4110 
X13  0.04737    0.06767      0.9474 
X14  0.00063     0.0042      0.0126 
X15  0.00109     0.0051      0.0217 
X16  0.37707     0.4713      1.0000 
X17  0.00046     0.0042      0.0092 
X18  0.00191     0.0064      0.0381 
X19  0.00402     0.0115      0.0805 
X20  0.47011     0.5531      1.0000

enter image description here

As you can see the Bonferroni is as expected conservative as it leaves most of the pvalues above 0.05. When looking at the correlations between the significant X variables and the response variable (for both adjustment methods) an ecological meaningful explanation can be formulated. However, just looking at the pvalues is there a way to say which method is more appropriate?

Mark
  • 63
  • 1
  • 1
  • 7

1 Answers1

3

Some procedures control the familywise type I error rate (e.g. Bonferroni, the uniformly more powerful Bonferroni-Holm adjustment, the more flexible general class of closed testing procedure, for specific situations like multiple comparisons against a common control Dunnett's test etc.) i.e. the probability of making at least one false rejection of a null hypothesis. In general, Bonferroni should probably be avoided in favor of e.g. Bonferroni-Holm (or something else that is closely tailored to the specific problem). False discovery rate (FDR) controlling procedures (e.g. Benjamini-Hochberg) instead control the proportion of wrongly rejected null hypotheses amongst those that are rejected (instead of amongst all).

Thus, the difference between the two types of approaches is about the goal and desired error control. E.g. in settings when there is a huge number of signals that need to be screened based on not very much data to determine whether something should be explored further, FDR control may be the most sensible. However, the results should then not be interpreted, as if a strict familywise type I error control had been applied. In contrast, for deciding what claims a drug company can do about a drug based on a clinical registration trial, strict familywise type I error rate is more usual.

Björn
  • 32,022