Versions

How MedGauge is updated

Every object MedGauge publishes carries a version. This page sets out, for each kind of object, how it is versioned, who approves a change, how often it is reviewed, and where the change is published. It follows the MedGauge Methodology, version 0.1.0, sections 14.4, 17, 22, 27 and 29–32. Where the methodology sets no rule, the entry reads not set.

Versioned objects

Object	Versioning	Approval	Review	Published in
Methodology	major.minor.patch, by change class	Request for comment and pull request; two maintainers	After every holdout round and in any case once every twelve months; each review ends in an Appendix D entry, either no change or a versioned change	Changelog; Appendix D of the methodology; a site-wide notice for a non-comparable change; a persistent identifier and a preprint at publication
Charter	major.minor.patch	Pull request with a version increment; two maintainers	Not set	Changelog; the charter file in the repository
Schemas and validator rules	With the methodology version, which each schema carries in its $id	Not set separately	With the methodology	Schemas page; changelog
Benchmark version	major.minor.patch per family; frozen and never edited	Gates 1–11 and independent acceptance; an independent acceptor and one maintainer	Keys audited after every round; a key found wrong is corrected in the next version	Benchmark card; changelog; a non-comparability notice on every affected board; a persistent identifier per frozen version
Registry record	version field and introduced_in; dates introduced and last reviewed	Pull request with schema and reference checks; one maintainer	Grading references: see below. Other records: not set	The registry pages; registry JSON; changelog
Grading reference listing	version field; a correction links its predecessor	Second-person clause check on the original and the conformant-alternative test; a reference engineer and one reviewer	At every freeze of a version that uses it, whenever the source publishes a revision, and in any case within twelve months of the previous check	References pages; changelog
Harness, tool, component	One immutable file per version	Schema; adapter check or smoke test; scope declaration; one maintainer	Marked deprecated after two consecutive benchmark versions without a result record, or on its maintainer’s request; its page stays	Registry pages; changelog
Result record	Site-verified; never edited	Site run or site re-grade and the review checklist; a results reviewer and one maintainer	Not set	Leaderboard and benchmark card; changelog for corrections and retractions
Difficulty function	major.minor.patch, cited as difficulty-function@<version>; a frozen version keeps the function in force when it froze	Cut points: request for comment, non-comparable change, site-wide notice; two maintainers	With the methodology	Methodology Appendix A; changelog
Level rules and thresholds	With the methodology version; no threshold is set in 0.1.0	Request for comment, non-comparable change, site-wide notice; two maintainers	With the methodology	Methodology Appendix B; changelog
Price table	Version; published with the results that use it	Not set	Not set	With the result records that use it
Methodology article	Date; supersedes link	Not set	An article that cannot meet the evidence grade it carries is regraded	Methodology articles; changelog

Every change to the public repository is a pull request that passes the build checks: schema validation, cross-references, internal links, English-only text and the red-line scan. A dispute on any decision above is settled by three independent votes, and the outcome and reasons are published.

Change classes

Non-comparable: Any change that alters a published number or a level: aggregation rules, thresholds, difficulty cut points or components, check semantics, the scale of a reported quantity. It carries a site-wide notice; results under the versions before and after it are labelled not comparable; earlier figures stay published under their own version. From version 1.0.0 it takes a new major version.
Below 1.0.0: While the methodology is in the 0.x series, its major number stays 0 and a non-comparable change takes a new minor version, with the same notice and labelling. A 0.x minor version may therefore be non-comparable; its changelog entry says so.
Minor: Additions that alter no published number: new sections, new contribution units, new registries, new gates applying to future versions, new subject kinds.
Patch: Wording, references, examples.

A change takes effect at the next benchmark version freeze, unless it is a patch, and is recorded in Appendix D of the methodology.

Twelve-month review

Methodology. Reviewed after every holdout round against that round's inputs (answer-key audit and expert agreement; predicted against measured PASS^3 per class and the monotonicity check of M4; the env_delta distribution; error rates by attribution and voided task versions; saturation; veto patterns; disputes raised), and in any case once every twelve months.
Grading references. Re-checked on the original at every freeze of a version that uses them, whenever the source publishes a revision, and in any case within twelve months of the previous check. Each record carries its check date.
Answer keys. Audited after every round against the round's failure attribution and expert artifacts; the share of keys changed is published per round.

Non-comparability

A change of benchmark version, difficulty function version or level threshold, and a methodology version that makes a non-comparable change, produce a notice on every affected board. Earlier figures stay published under their own version, and no figure is pooled or totalled across versions.

Corrections and retractions

A correction never overwrites. The corrected entry links its predecessor, the changelog records the change, and affected boards carry a non-comparability notice.
A frozen version, a published harness or tool version and a site-verified result record are never edited; a change is a new version or a correction.
A retracted record stays visible with its retraction notice and reason. Corrections and retractions are listed in the changelog.
A key found wrong is corrected in the next benchmark version and historical results are never deleted. A key error found during a round is an error on the initiative's side and falls under the void-and-retest rule.
A retired task, capability or deprecated version keeps its page. A methodology version is superseded, never deleted.

To report an error, name the page or record and the field; see Contact. Problems with a key or a grading reference name the benchmark version or reference id, the field and the source clause.

Following changes

Changelog: /changelog/, newest first
Atom feed: /changelog.xml, the changelog as a feed, generated when the site is built
Sitemap: /sitemap.xml
Registry export: /api/v1/registry.json
Build stamp: Every page names the registry commit it was built from, in the footer
E-mail: There is no mailing list.

Persistent identifiers

Each methodology version, each frozen benchmark version, each round report and each release of the public repository receives a digital object identifier through the repository's archive integration; the repository's CITATION.cff carries the current identifier. Methodology versions and round reports are also deposited on a public preprint server at publication. A result is cited with its four-tuple (model, harness and version, benchmark version, date); a level with its four-tuple and the methodology version.