Quality Assurance
Methodology documentation: write it so someone could disagree with it
Methodology that cannot be examined cannot be challenged — and data nobody can challenge cannot be trusted.
#counting rules
Quality Assurance
Data & Metrics
Counting rules: the rule is part of the number
Two analysts with the same data and two defensible rules produce different totals. Publishing one without the other publishes half a finding.
Quality Assurance
Uncertainty: say what you do not know rather than presenting a clean number
Academic data is uncertain in ways that do not show in the output. Presenting it cleanly is a choice, and usually the wrong one.
For Partners
Working with a registry: what each side needs from the other
Registries depend on institutions for accurate source data. Institutions depend on registries for figures they cannot produce alone.
Data & Metrics
Metrics: every number answers a narrower question than it appears to
Research metrics are not wrong. They answer narrower questions than the ones people use them to answer.
For Partners
Using academic data: what the figures can and cannot support
Academic data supports some claims well and others not at all. The difference is not always visible in the data itself.