PROJECT 03 · PERFORMANCE REPAIR
Codex Benchmark Guardian
Updated: July 2026
Built an evidence-backed performance repair system that turns benchmark regressions into bounded Codex repair tasks, verifies proposed fixes against protected benchmark evidence, and leaves final merge approval to a human.
Problem
Performance regressions are difficult to repair safely when benchmark direction, thresholds, repair scope, and verification evidence are implicit or mutable.
Technical Highlights
Python · FastAPI · Next.js · TypeScript · Streamlit · GitHub Actions
- Deterministic regression triage
- Protected repair verification
- Human approval before merge
System Design
- Compares baseline and current benchmarks across higher-is-worse and lower-is-worse metrics.
- Applies configurable thresholds, severity classification, and release-readiness scoring.
- Produces deterministic triage and an immutable Repair Contract.
- Creates a Conditional Codex Goal for repair-required or verification-only paths.
- Re-runs a protected benchmark harness and evaluator before reporting readiness.
Architecture
A Python CLI, FastAPI service, Next.js dashboard, and Streamlit interface share a protected evaluator while keeping repair execution separate from trusted evidence.
Initial Regression
- Metric:
pr_gate_generation_latency_ms - Harmful change: +135.21%
- Protected threshold: 25%
- Critical severity · Needs Review · 70/100
Protected Verification
- Verified result: zero material regressions
- Final readiness: Ready
- Final score: 100/100
Results and Insights
- Explicit metric direction and thresholds make classification deterministic.
- Immutable repair contracts keep generated work bounded and auditable.
- Protected verification separates repair claims from trusted evidence.
- Readiness scoring communicates risk without replacing human approval.
Takeaway
Performance repair automation is trustworthy only when goals are bounded, evidence is protected, and final approval remains human-controlled.