MedGaugeResult records
Leaderboard
One row per model × harness, one board per benchmark version. Only site-verified result records are shown. Safety vetoes have their own columns and never enter the ranking.
First holdout round in preparation. This board is generated from registry/results/; it populates when the first reviewed result records are merged. No placeholder or demonstration scores are shown.
Boards
| Benchmark | Version | Rows |
|---|---|---|
| Bioinformatics QC | 0.1.0 | 0 |
| Clinical Temporal Audit | 0.1.0 | 0 |
| Guideline Evidence Check | 0.1.0 | 0 |
Columns
| Column | Definition |
|---|---|
| Model | Model id and service snapshot |
| Harness | Registered harness id and version |
| Group letters | Compact letter display from pairwise paired comparisons on the same tasks, Holm-adjusted; rows sharing a letter are not shown to differ, and a row can carry several letters |
| pass^3 | Share of holdout tasks passed in all three runs, with its Clopper–Pearson interval; the primary ordering key |
| pass@3 | Share of holdout tasks passed in at least one of three runs |
| Instability | pass@3 − pass^3 |
| Exam | Passed (every task, all three runs, no veto) or not passed, with the number of tasks and, when passed, the claimable lower bound 0.051/n |
| Level | Not issued: methodology 0.1.0 sets no level threshold, so no level is issued |
| Task pass rate | Share of passing runs per task, averaged over tasks, with its BCa bootstrap interval |
| Continuous score | Passed checkpoints / checkpoints per run (0 on a veto), averaged over runs, then tasks; never compared across families; not an ordering key |
| Calibration | Calibration subset, reported separately |
| Vetoes | Runs with a veto, by category (fabricated evidence, altered input, corrected value, executed injection, unsafe advice) and by constraint X.1–X.4; any hit fails the run and marks the row |
| Reference vetoes | Hits on veto-flagged grading references, keyed by X capability |
| Errors | Run errors by attribution; every error is a failed run, never re-run and never removed |
| Cost per success | Total amount over all runs / passing runs, under the price table version named on the row. The amount is the price a buyer pays under that table, not the resources a run consumes, and it compares only within one price table version and time window. It orders adjacent rows that carry an identical set of group letters. Self-hosted and open-weight subjects: amount not applicable, and they take no part in that order |
Rows are ordered by pass^3; among adjacent rows with an identical set of group letters, rows with an amount are ordered by cost per success. Amounts compare prices paid, not resources used. Overlapping intervals do not define a group. Missing fields display "—", never 0; a dimension a family does not support shows "not applicable". Calibration tasks never enter family averages. Results under different benchmark versions are not comparable. See intervals and the order of rows.