P-hacking, or repeating statistical analyses until nonsignificant...
Nicholas Fabiano, MD@NTFabiano
47 views
Feb 12, 2025
~1 min read
Advertisement
1
P-hacking, or repeating statistical analyses until nonsignificant results become significant, is prevalent in science.
🧵1/10
🧵1/10
2
These findings are from a perspective in @PLOSBiology which used text-mining to demonstrate that p-hacking is widespread in published literature. journals.plos.org/plosbiology/ar… 2/10
3
Current scientific practices create strong incentives to publish statistically significant (i.e., “positive”) results, and there is evidence that journals, especially prestigious ones with higher impact factors, disproportionately publish statistically significant results. 3/10
4
This creates incentives for researchers to selectively pursue and selectively attempt to publish statistically significant research findings. 4/10
5
P-hacking occurs when researchers try out several statistical analyses and/or data eligibility specifications and then selectively report those that produce significant results. 5/10
6
This leads to false positives which hinders scientific progress and can inspire investment in fruitless research programs, and even discredit entire fields. journals.sagepub.com/doi/10.1177/09… 6/10
7
Text-mining for p-values in all Open Access papers in PubMed found widespread p-hacking across scientific literature. 7/10
8
Eliminating p-hacking entirely is unlikely when career advancement is assessed by publication output, and publication decisions are affected by the p-value or other measures of statistical support for relationships. 8/10
9
P-hacking arises due to questionable research practices. Over 50% of researchers admitted to “failing to report all of a study’s dependent measures” and “deciding whether to collect more data after looking to see whether the results were significant” journals.sagepub.com/doi/10.1177/09… 9/10
10
Therefore it is clear that we must educate both researchers and journals on more rigorous research practices in order to minimize the extent of p-hacking in science. 10/10
11
I have previously done research in this area in the field of radiology @CanRadJournal journals.sagepub.com/doi/full/10.11…
