P-value Adjustments for Multiple Testing
Adjust p-values for multiple testing online. Bonferroni, Sidak, Holm for FWER. BH and BY for FDR.
When to use it
Use when a study runs many tests (pairwise comparisons after ANOVA, multiple outcomes, many genes / SNPs / regions) and you need to control either the family-wise error rate (FWER) or the false-discovery rate (FDR). Required by most journals when the number of tests exceeds a small handful.
When NOT to use it
Do not adjust if your hypotheses are pre-registered, planned, and few in number; the family is small enough that no correction is needed beyond reporting them all. Do not use FWER (Bonferroni) when the cost of missing real findings is high and false positives are tolerable; use FDR (BH) instead. Do not adjust descriptive statistics or exploratory associations you are not testing formally.
What you get
Each input p with its raw value alongside the adjusted p under the chosen method: Bonferroni (FWER, simplest), Sidak (FWER, slightly less conservative), Holm (FWER, step-down, uniformly more powerful than Bonferroni), Hochberg, Benjamini-Hochberg (BH, FDR), and Benjamini-Yekutieli (BY, FDR under arbitrary dependence). Adjusted p compared to the chosen alpha gives the rejection / non-rejection decision.
Worked example
Eight raw p-values for Bonferroni / Holm / BH FDR adjustment.
Test1: 0.001 Test2: 0.012 Test3: 0.025 Test4: 0.040 Test5: 0.062 Test6: 0.150 Test7: 0.230 Test8: 0.500
Source: StatVeda built-in example.
How to interpret the output
FWER methods control the probability of at least one false positive across the whole family; FDR methods control the expected proportion of false positives among rejections. For confirmatory hypotheses (clinical primary endpoints), use Holm. For high-throughput exploratory work (genomics, brain imaging), BH FDR is standard (Benjamini and Hochberg 1995). The same raw p can be 'significant' under FDR but not under Bonferroni.
Common pitfalls
- Mixing FWER and FDR in the same analysis without explanation.
- Adjusting too few or too many p-values (the 'family' should be the set of confirmatory tests).
- Reporting only adjusted p without the raw p (transparency requires both).
- Treating BH-adjusted p of 0.05 as identical in meaning to a raw p of 0.05.
Try it in StatVeda
The Post-hoc P-value Corrections (Bonferroni / Holm / BH FDR) engine runs entirely in the browser. No signup, no install, no data sent to a server. Paste your data, hit Run, copy the output.
Open Post-hoc P-value Corrections (Bonferroni / Holm / BH FDR)References
- Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65 to 70.
- Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society B, 57(1), 289 to 300.
- Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29(4), 1165 to 1188.