A finding reported without its limits is not a simplified version of the research.
It is a different claim — usually a stronger one than the researchers made, and one they will be held responsible for.
The errors that cause most harm
- Reporting association as causation.
- Generalising beyond the population studied.
- Reporting a single study as settled knowledge.
- Omitting sample size when it is small.
- Reporting relative change without the absolute figures.
- Each of these turns a careful finding into an overstatement.
Questions to ask researchers
- What exactly did you find, in one sentence?
- Who does this apply to?
- What does it not show?
- How confident are you, and why?
- Does other research agree?
- Researchers generally answer these readily and are glad to be asked.
Context that changes the meaning
- Sample size and who was studied.
- Whether it is a single study or part of a body of evidence.
- Whether it has been peer reviewed.
- Funding source and any conflicts.
- How it fits with existing work.
Reporting numbers
- Give absolute figures alongside any relative change.
- Give the sample size.
- Indicate uncertainty.
- Do not present a range's endpoint as the finding.
- A relative change without an absolute base is close to meaningless.
Working with researchers
- Offer to check quotes and technical statements — most will respond quickly.
- Ask what they are worried might be misreported.
- Give a right of response before publishing anything critical.
- Name the study so readers can find it.
Reporting on research from smaller systems
- Coverage differences mean output comparisons understate them — say so.
- Do not treat unindexed work as absent.
- Avoid framing that generalises from individual cases to whole systems.
- Seek comment from researchers in the system concerned.
One thing worth remembering
Ask the researcher: what does this not show?
They will tell you plainly, it takes one question, and the answer prevents the great majority of misreporting — because most of it comes from omitting the limits rather than from getting the finding wrong.
Câu hỏi thường gặp
Which errors cause most harm?
Reporting association as causation, generalising beyond the population studied, treating a single study as settled, omitting small sample sizes, and reporting relative change without absolutes.
What should be asked of researchers?
What exactly did you find in one sentence, who does this apply to, what does it not show, how confident are you and why, and does other research agree.
What context changes the meaning?
Sample size and who was studied, whether it is a single study or part of a body of evidence, whether it was peer reviewed, funding source, and how it fits existing work.
How should numbers be reported?
With absolute figures alongside relative change, the sample size, an indication of uncertainty, and without presenting a range's endpoint as the finding.
Which single question prevents most misreporting?
What does this not show — researchers answer plainly, and most misreporting comes from omitting limits rather than getting the finding wrong.
