The most important thing to understand before believing any result in this domain.
10 topics · 18 curated works
No prior grounding assumed.
The Earth Is Round (p < .05)
Jacob Cohen · 1994
Argues null-hypothesis significance testing does not tell researchers what they think it does - it cannot say how probable a hypothesis is - and that…
+2 more at this level
Assumes you know the vocabulary.
Estimating the Reproducibility of Psychological Science
Open Science Collaboration · 2015
Replicated 100 psychology studies; roughly a third produced significant effects, and effect sizes halved on average.
+8 more at this level
Primary sources and full treatments.
Statistical Methods for Research Workers
Ronald A. Fisher · 1925
Lays out the statistical tools - significance testing, analysis of variance, the design of experiments - that became the working method of…
+5 more at this level
12 of 18 works
Jacob Cohen
Argues null-hypothesis significance testing does not tell researchers what they think it does - it cannot say how probable a hypothesis is - and that psychology's near-total reliance on it has done the field lasting damage.
Notes on this copyChristopher D. Chambers
Announces a journal format in which the introduction and methods are peer reviewed and provisionally accepted before the results are known, arguing this removes the incentive to change a study's story to fit whatever result comes out.
Notes on this copyBrian A. Nosek et al.
Proposes the TOP Guidelines - shared standards for citation, data and materials sharing, preregistration and replication - and argues journals and funders, not individual virtue, are the lever that actually changes research practice.
link checked 17 Sept 2026Notes on this copyOpen Science Collaboration
Replicated 100 psychology studies; roughly a third produced significant effects, and effect sizes halved on average.
8 pageslink checked 17 Sept 2026Notes on this copyJoseph P. Simmons, Leif D. Nelson & Uri Simonsohn
Ordinary, undisclosed researcher choices - when to stop collecting data, which conditions to drop, which covariates to control for - let a false hypothesis reach statistical significance far more often than the nominal 5% rate, demonstrated by using them to 'prove' an impossible effect.
8 pageslink checked 17 Sept 2026Notes on this copyDaryl J. Bem
Reports nine experiments finding that ordinary participants' current behaviour was statistically influenced by random events that had not yet happened, using entirely standard psychological methods - a result now widely read not as evidence for precognition but as proof that those standard methods could manufacture significant findings for almost anything.
Notes on this copyJim McCambridge, Marijn de Bruin & John Witton
Reviews studies that directly tested whether participants change their behaviour simply because they know they are being observed, and finds the evidence for such demand effects outside the laboratory is far weaker and more inconsistent than psychology usually assumes when designing a study around it.
Eric-Jan Wagenmakers, Ruud Wetzels, Denny Borsboom, Han L. J. van der Maas & Rogier A. Kievit
Argues psychology has collapsed the distinction between generating a hypothesis from data and confirming it with new data, and that separating the two by preregistering every confirmatory analysis before collecting data is the fix.
Notes on this copyGeoff Cumming
Argues psychology should replace single-study null-hypothesis significance testing with estimation, confidence intervals and cumulative meta-analysis across studies, because a single p-value cannot say how big an effect is or how much to trust it.
link checked 17 Sept 2026Notes on this copyMegan L. Head, Luke Holman, Rob Lanfear, Andrew T. Kahn & Michael D. Jennions
Text-mines the p-values reported across the biomedical and life sciences literature and finds a pattern consistent with widespread p-hacking, but argues the resulting bias is small relative to the true effects being measured, so it inflates rather than manufactures most published findings.
Notes on this copyBrian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven & David T. Mellor
Argues that registering hypotheses and analysis plans before seeing the data is the single change most able to separate confirmatory testing from exploratory data analysis, and that failing to distinguish the two is what let so many false positives into the published literature.
link checked 17 Sept 2026Notes on this copyMostafa Salari Rad, Alison Jane Martingano & Jeremy Ginges
Updates the WEIRD-samples critique a decade on, finding the imbalance toward Western, educated, industrialised, rich and democratic participants has barely improved, and argues journals should require authors to state explicitly what population a finding is meant to generalise to.
Notes on this copyThis subject genuinely sits in more than one domain. These fields approach the same ground with different methods.