MedGaugeBenchmarks / Taxonomy
Task taxonomy
Six levels, from the service setting down to the field a program grades: scenario → medical-practice capability → task family → task form → defect or verdict class → artifact field. Every level is generated from the registries.
Task forms
| Form | Artifact | Benchmarks using it |
|---|---|---|
| Detect and log | Logged findings in fixed fields, each with verbatim evidence | Bioinformatics QC, Clinical Temporal Audit, Annotation QA Audit, De-identification Audit, Prescription and Reimbursement Review, Specialty Registry QC |
| Item-by-item judgement | A verdict and governing rule for every given item | Guideline Evidence Check, Clinical Coding Audit, Prescription and Reimbursement Review |
| Structured answer | One answer sheet to a contract: keys, enumerations, citations, derived values | Bioinformatics QC, Clinical Temporal Audit, Paired Population Cases |
| Transform and output | A transformed file plus a change log | De-identification Audit |
| Multi-artifact checkpoints | Several files, each checked item by item | none yet |
Scenarios and the capabilities they compose
E.1 Healthcare delivery
Review and quality control across outpatient, inpatient, medical record and insurance workflows, composed from D.1, D.2, D.3 and D.8. Graded by before-and-after comparison of time compression, human interventions per thousand steps and quality indicators.
Composed of: D.1 Clinical temporal auditD.2 Prescription and insurance reviewD.3 Disease registry quality controlD.8 De-identification output verification
E.2 Drug and device R&D
Third-party evaluation for drug and device development, including clinical trial matching, literature evidence and verification of registration dossiers, composed from D.5, D.6 and D.10. Graded by before-and-after comparison of coverage, error detection rate and report consistency.
Composed of: D.5 Bioinformatics QC and recomputationD.6 Guideline evidence verificationD.10 Research assistance
E.3 Clinician services
An individual clinician's workbench covering documentation, follow-up and teaching, composed from D.4 and D.7. Graded by the rate at which clinicians accept the agent's output on review.
Composed of: D.4 Medical exam item and teaching reviewD.7 Follow-up and care pathway audit
E.4 Institutional programmes
Institution-level programmes such as disease registries, regional follow-up and departmental quality control, composed from D.3 and D.7. Graded by before-and-after comparison of intervention rate and closed-loop rate.
Composed of: D.3 Disease registry quality controlD.7 Follow-up and care pathway audit
E.5 Research services
Bioinformatics pipelines, omics quality control and research data applications, composed from the C domain, D.5 and D.10. Graded by reproduction rate and recomputation agreement rate.
Composed of: C.1 Data processingC.2 Data synthesisC.3 Data augmentationC.4 Data validationC.5 Pipeline orchestrationC.6 Deterministic program implementationC.7 Program verificationD.5 Bioinformatics QC and recomputationD.10 Research assistance
Medical-practice capabilities, families and classes
D.1 Clinical temporal audit Active
Clinical Temporal Audit Active Detect and log · Structured answer
Classes: Information cited before it was available · Misattributed time point · Cross-table conflict · Event order reconstruction
findings[] findings[].record_ref findings[].defect_class findings[].evidence findings[].available_at undeterminable[]
D.2 Prescription and insurance review Active
Prescription and Reimbursement Review Planned Item-by-item judgement · Detect and log
Classes: Label contraindication · Drug interaction · Organ-function dosing · Reimbursement restriction
orders[].order_ref orders[].verdict orders[].rule information_needed[] contraindication_flags[]
D.3 Disease registry quality control Active
Specialty Registry QC Planned Detect and log
Classes: Score does not recompute · Missing item · Threshold or target · Time-point relation
discrepancies[].record_ref discrepancies[].kind discrepancies[].recorded discrepancies[].recomputed
D.4 Medical exam item and teaching review Active
No task family linked yet.
D.5 Bioinformatics QC and recomputation Active
Bioinformatics QC Active Detect and log · Structured answer
Classes: Metadata inconsistent with data · Missing metadata · Threshold breach · Reported statistic does not recompute
anomalies[].sample_ref anomalies[].check anomalies[].observed summary_stats
D.6 Guideline evidence verification Active
Guideline Evidence Check Active Item-by-item judgement
Classes: Supported · Altered · Unsupported · Wrong source · Unverifiable
claims[].claim_ref claims[].verdict claims[].source_span
D.7 Follow-up and care pathway audit Active
No task family linked yet.
D.8 De-identification output verification Active
De-identification Audit Planned Detect and log · Transform and output
Classes: Residual direct identifier · Inconsistent date shift · Quasi-identifier combination
D.9 Coding audit Active
Clinical Coding Audit Planned Item-by-item judgement
Classes: Code not supported by documentation · Specificity · Principal diagnosis sequencing · Code system or version
D.10 Research assistance Active
No task family linked yet.
D.11 Clinical pathway node audit Active
No task family linked yet.
D.12 Medication consistency with phenotype and history Active
No task family linked yet.
D.13 Grade and score recalculation Active
Specialty Registry QC Planned Detect and log
Classes: Score does not recompute · Missing item · Threshold or target · Time-point relation
discrepancies[].record_ref discrepancies[].kind discrepancies[].recorded discrepancies[].recomputed
D.14 Follow-up target and due-date audit Active
Specialty Registry QC Planned Detect and log
Classes: Score does not recompute · Missing item · Threshold or target · Time-point relation
discrepancies[].record_ref discrepancies[].kind discrepancies[].recorded discrepancies[].recomputed
D.15 Event recognition and handling audit Active
No task family linked yet.
D.16 Registry dictionary and mapping table QC Active
No task family linked yet.
D.17 Red-flag recognition and triage Proposed
No task family linked yet.
D.18 Diagnostic reasoning and differential Proposed
Paired Population Cases Planned Structured answer
Classes: Required adjustment missing · Unwarranted difference
D.19 Treatment option appraisal and contraindication detection Proposed
Paired Population Cases Planned Structured answer
Classes: Required adjustment missing · Unwarranted difference
D.20 Disclosure checklist coverage Proposed
No task family linked yet.