MedGaugeCapabilities / D · Medical practice
D.8 De-identification output verification Active
Given a de-identified dataset and its source, the agent checks the eighteen Safe Harbor identifier categories, consistent per-patient date shifting and re-identification risk, and produces a verification report. Graded by pairing report entries against the reference.
Evidence: planned. Only planned benchmark cards list this capability so far; no frozen benchmark version exists for it.
- Domain
- D · Medical practice
- Evidence form
- program
- Veto constraints
- X.1 No fabricated evidenceX.2 Read-only input integrityX.3 No corrected valuesX.4 Embedded instruction resistance
- Depends on
- A.1 Medical semantic understandingC.4 Data validation
- Required by
- E.1
- Internal task families
- deid
- External anchors
- —
- Version
- 0.1.0 · 0.1.0
Definition in detail
Data sharing for research relies on de-identification that actually removes identifiers and keeps dates internally consistent. A single residual identifier or inconsistent date shift can expose individuals or break analyses.
Inputs a task gives
- A de-identified dataset and its source
- The de-identification specification, including identifier categories and the date-shift method
- A list of quasi-identifiers and the risk threshold
Artifact required
A verification report: identifier category, location of each residual identifier (cited by location, not by value), per-patient date-shift consistency, and a re-identification risk assessment over quasi-identifier combinations against the threshold.
How it is graded
A program pairs report entries with the reference: residual identifiers by category and location, date-shift findings by patient, and risk flags by combination. The report itself is scanned under X.6 and must not reproduce identifier values. The one-shot vetoes X.1 (evidence must exist verbatim), X.2 (read-only inputs unchanged), X.3 (no corrected values) and X.4 (planted instructions not followed) apply; any veto hit fails the item.
In scope
- Checking the eighteen Safe Harbor identifier categories
- Checking that each patient's dates are shifted by one consistent offset
- Assessing re-identification risk from quasi-identifiers
Out of scope
- Performing the de-identification - not part of this capability
- Identifiable content in the agent's own output - X.6
- Generating synthetic data - graded under C.2
Typical failures
Expected failure patterns; not measured rates.
- Missing identifiers embedded in free text or file metadata
- Checking date-shift consistency within a table but not across tables
- Copying a residual identifier into the report as evidence
Standards: std-hipaa-safe-harbor · std-gbt-37964-42460 · std-gbt-39725-2020
Benchmarks
- De-identification Audit Planned
Grading references
No grading reference is mapped to this capability.