Combining sources improves coverage. It also imports every problem each source has — and combines them in ways that are harder to detect than any single source's errors.
Both effects are real, and only one of them is intended.
What aggregation gains
- Broader coverage than any single source.
- Cross-verification where sources overlap.
- Fields one source lacks and another has.
- Reduced dependence on any one provider.
What it costs
- Every source's errors enter the combined set.
- Duplicate records where matching fails.
- Merged records where matching overreaches.
- Conflicting values requiring resolution.
- Inconsistent definitions between sources — the most damaging and least visible problem.
Handling overlap
- Match on identifiers wherever available.
- Require multiple signals where identifiers are absent.
- Prefer leaving separate to merging wrongly — duplicates are visible, false merges are not.
- Log every merge decision with its evidence.
- Keep merges reversible.
Handling conflict
- Set precedence per field, not per source — no source is uniformly best.
- Record that a conflict existed rather than silently resolving it.
- Retain source-level values.
- Some conflicts are informative rather than erroneous.
- Publish the precedence rules.
Definition mismatch
- The same field name may mean different things in different sources.
- Combining them produces a field that means neither.
- Check semantics before combining, not only format.
- Where definitions cannot be reconciled, keep the fields separate.
- This is the failure that is hardest to detect after the fact.
Reporting aggregated data
- List every source.
- State what each contributes.
- State precedence rules.
- State combined coverage, not each source's separately.
- Show per-field provenance so users know which source a value came from.
One thing worth remembering
Check that a field means the same thing in each source before combining it.
Format mismatches produce visible errors. Semantic mismatches produce a field that is well-formed, plausible, and means nothing — which nobody notices until a decision has been made on it.
Câu hỏi thường gặp
What does aggregation gain?
Broader coverage than any single source, cross-verification where sources overlap, fields one source lacks, and reduced dependence on any one provider.
What does it cost?
Every source's errors enter the combined set, duplicates where matching fails, false merges where it overreaches, conflicting values, and inconsistent definitions between sources.
How should overlap be handled?
Match on identifiers where available, require multiple signals otherwise, prefer leaving separate to merging wrongly, log every merge with its evidence, and keep merges reversible.
How should conflicts be handled?
Set precedence per field rather than per source since no source is uniformly best, record that a conflict existed, retain source-level values, and publish the rules.
Which failure is hardest to detect?
Semantic mismatch — combining fields with the same name but different meanings produces a well-formed plausible field that means nothing.
