Paper
2011
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Joseph P. Simmons, Leif D. Nelson & Uri Simonsohn
Ordinary, undisclosed researcher choices - when to stop collecting data, which conditions to drop, which covariates to control for - let a false hypothesis reach statistical significance far more often than the nominal 5% rate, demonstrated by using them to 'prove' an impossible effect.
Read itBefore you start
FreeIntermediate8 pageslink checked 17 Sept 2026
Groundwork for
Works in the library that name this one as a prerequisite.
- Estimating the Reproducibility of Psychological ScienceOpen Science Collaboration, 2015Replicated 100 psychology studies; roughly a third produced significant effects, and effect sizes halved on average.
- The Preregistration RevolutionBrian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven & David T. Mellor, 2018Argues that registering hypotheses and analysis plans before seeing the data is the single change most able to separate confirmatory testing from exploratory data analysis, and that failing to distinguish the two is what let so many false positives into the published literature.
- Promoting an Open Research CultureBrian A. Nosek et al., 2015Proposes the TOP Guidelines - shared standards for citation, data and materials sharing, preregistration and replication - and argues journals and funders, not individual virtue, are the lever that actually changes research practice.
- The Extent and Consequences of P-Hacking in ScienceMegan L. Head, Luke Holman, Rob Lanfear, Andrew T. Kahn & Michael D. Jennions, 2015Text-mines the p-values reported across the biomedical and life sciences literature and finds a pattern consistent with widespread p-hacking, but argues the resulting bias is small relative to the true effects being measured, so it inflates rather than manufactures most published findings.
Filed under Research Methods & the Replication Crisis in Psychology.