P-values and the replication crisis: a maths EPQ idea
A title to start from
Why do so many published results fail to replicate, and how much is the p-value to blame?
Why it works as an EPQ
Hypothesis testing is on the course; seeing how it is misused is a serious and current question.
Scope and difficulty
Ambitious. Ambitious. Use simulation to show the effects rather than surveying all of science.
The maths
Builds on these A Level topics: Hypothesis testing · Statistical distributions · Sampling and data.
You would learn:
- Type I and II errors and power
- Multiple testing
- Publication bias
One possible plan
- Simulate many small studies of a true null and of a small real effect.
- Show how selective reporting inflates false positives and effect sizes.
- Compare remedies (pre-registration, larger samples, corrections).
- Judge whether p-values are the problem or how they are used.
Pitfalls
- Implying that every published finding is wrong.
- Definitions of a p-value that are subtly wrong.
Where to start reading
- The Art of Statistics (David Spiegelhalter)
- Search for: American Statistical Association statement on p-values
Similar ideas
- Can people be random?Can people generate random sequences, and how can statistics tell?
- How accurate are weather forecasts?How accurate are UK weather forecasts, and what is the fairest way to measure accuracy?
- Is the lottery a fair game?Is a national lottery a fair game, and what is a ticket really worth?
- Benford's law in UK dataDoes Benford's law hold for UK data sets, and could it detect made-up numbers?
All statistics and data ideas · all 93 ideas
Your EPQ must be your own work. These pages coach: ideas, structure, checklists and planning. Submitting text, proofs, code or analysis written by someone else or by an AI tool as your own is malpractice. How to use help and AI honestly.