P-values and the replication crisis: a maths EPQ idea
A title to start from
Why do so many published results fail to replicate, and how much is the p-value to blame?
Why it works as an EPQ
Hypothesis testing is on the course; seeing how it is misused is a serious and current question.
Scope and difficulty
Ambitious. Ambitious. Use simulation to show the effects rather than surveying all of science.
The maths
Builds on these A Level topics: Hypothesis testing · Statistical distributions · Sampling and data.
You would learn:
- Type I and II errors and power
- Multiple testing
- Publication bias
One possible plan
- Simulate many small studies of a true null and of a small real effect.
- Show how selective reporting inflates false positives and effect sizes.
- Compare remedies (pre-registration, larger samples, corrections).
- Judge whether p-values are the problem or how they are used.
Pitfalls
- Implying that every published finding is wrong.
- Definitions of a p-value that are subtly wrong.
Where to start reading
- The Art of Statistics (David Spiegelhalter)
- Search for: American Statistical Association statement on p-values
Real data for this idea
- Primary source: OSF (Open Science Framework): Reproducibility Project: Psychology. The project's data files give original and replication effect sizes and p-values for 100 studies. Licence: Varies by project.
Frequently asked questions
Is this a good maths EPQ idea?
It can be. We rate it Ambitious: it suits students happy to read first-year university material. Use simulation to show the effects rather than surveying all of science.
What maths does this EPQ use?
It builds on these A Level topics: Hypothesis testing; Statistical distributions; Sampling and data. You would learn: Type I and II errors and power; Multiple testing; Publication bias.
Next step
Turn this idea into your own title, then check it in two minutes.
Check your titleSimilar ideas
- Can people be random?Can people generate random sequences, and how can statistics tell?
- How accurate are weather forecasts?How accurate are UK weather forecasts, and what is the fairest way to measure accuracy?
- Is the lottery a fair game?Is a national lottery a fair game, and what is a ticket really worth?
- Benford's law in UK dataDoes Benford's law hold for UK data sets, and could it detect made-up numbers?
All statistics and data ideas · all 93 ideas
Your EPQ must be your own work. How to use help and AI honestly.