MedGaugeCapabilities / C · Data and program
C.6 Deterministic program implementation Active
Given a rule specification, the agent writes a deterministic program (grader, validator or calculator) that implements it. Graded on a hidden test set; the program must not share an implementation with the reference solution.
Evidence: none. No benchmark card lists this capability yet. No level is issued: methodology 0.1.0 sets no level thresholds.
- Domain
- C · Data and program
- Evidence form
- program
- Veto constraints
- —
- Depends on
- —
- Required by
- B.1C.7D.13E.5
- Internal task families
- —
- External anchors
- MedAgentGym, MedCalc-Bench
- Version
- 0.1.0 · 0.1.0
Definition in detail
Graders, validators and clinical calculators must be deterministic and must implement the written rule, not an approximation of it. Independent implementations are how a reference solution is checked.
Inputs a task gives
- A written rule specification for a grader, validator or calculator
- An interface definition
- A small set of public examples
Artifact required
Program source implementing the rule behind the declared interface, with no network access and no randomness.
How it is graded
A program runs the implementation on a hidden test set and on repeated runs to confirm identical output. The implementation must not share code with the reference solution; this is checked by provenance and similarity checks.
In scope
- Implementing every clause of the rule, including boundaries
- Specified behaviour for missing or out-of-range inputs
- Deterministic output
Out of scope
- Auditing an existing program for defects - graded under C.7
- General tools with their own tests - graded under B.1
- Applying a scoring rule to records in an audit - graded under D.13
Typical failures
Expected failure patterns; not measured rates.
- Implementing the public examples rather than the rule
- Inclusive versus exclusive errors at thresholds
- Floating-point rounding that changes a category at the boundary
- Undefined behaviour when an input item is missing
Benchmarks
No benchmark card lists this capability.
Grading references
No grading reference is mapped to this capability.