Most published analyses cannot be reproduced from what was shared.
Not usually because anything was withheld — but because the documentation needed to follow the steps was never written, and reconstructing it afterwards is impossible.
What reproducibility requires
- The data as used, or clear access instructions.
- The code that produced the results, not a cleaned approximation.
- The software environment with versions.
- Documentation of every processing and exclusion step.
- Enough that a stranger reaches your number without contacting you.
Where it fails
- Manual steps that were never recorded.
- Undocumented exclusions.
- Code that only runs on one machine.
- Software version differences changing results.
- Data cleaned in ways nobody wrote down.
- Every one of these is a documentation failure rather than a technical one.
Habits that make it routine
- Script everything — manual steps cannot be reproduced.
- Version control from the start.
- Keep raw data untouched and script all processing.
- Record the environment.
- Write the analysis plan before analysing.
- These cost little when adopted early and are hard to retrofit.
Testing your own
- Run the analysis from raw data on a clean machine.
- Have a colleague follow your documentation without asking you anything.
- Note every question they need to ask — each is a gap.
- Do this before submission, not after a challenge.
When data cannot be shared
- Reproducibility is not all or nothing.
- Share the code and documentation even when data is restricted — this alone has substantial value.
- Describe the data structure fully.
- Offer controlled access where possible.
- State clearly what cannot be shared and why.
Why it matters for smaller systems
- Reproducible work is harder to dismiss — this matters where assumptions about origin exist.
- It supports reuse by others with fewer resources.
- It builds local methodological capability.
- It is a low-cost way to demonstrate rigour.
One thing worth remembering
Run your analysis from raw data on a clean machine before submitting.
It reliably reveals the undocumented steps and local dependencies that make the work irreproducible — and finding them takes an afternoon, while a reader finding them costs your credibility.
Câu hỏi thường gặp
What does reproducibility require?
The data as used or access instructions, the code that produced the results, the software environment with versions, and documentation of every processing and exclusion step.
Where does reproducibility fail?
Unrecorded manual steps, undocumented exclusions, code that only runs on one machine, software version differences, and data cleaned in ways nobody wrote down.
Which habits make it routine?
Scripting everything since manual steps cannot be reproduced, version control from the start, keeping raw data untouched, recording the environment, and writing the analysis plan first.
What if data cannot be shared?
Reproducibility is not all or nothing — share code and documentation even when data is restricted, describe the data structure fully, and offer controlled access where possible.
What test should precede submission?
Running the analysis from raw data on a clean machine — it reliably reveals undocumented steps and local dependencies.
