The morning-after view of an autonomous calibration loop — drop eval/history.jsonl, or the whole calib-loop/ folder (config, observed CSVs, run.out) for hydrograph overlays.
Drag the calib-loop folder here — or any mix of history.jsonl, config.json, observed *.csv, and a SWMM5 .out file
Read the climb. Each dot is one experiment; the blue step line is the ratchet — the best score so far. Green committed, gray valid-but-not-better, red guard-tripped. Click any dot.
Score trajectory — mean NSE over calibration gauge-events
Watch every event. The ratchet requires the mean to rise without any single event regressing — these lines are that rule, drawn.
Per-event NSE across committed models
Best model — per event
Audit the experiments. Every run the loop made, kept or reverted, with its note — the lab notebook the agent wrote.
Experiment log
Check the guardrails. Every rejected "improvement", grouped by which anti-gaming guard caught it.
Guard activity
Incidents
Overlay the hydrographs. Observed points vs the simulated line from the .out file, per event window. Metrics recomputed live with the same NSE/PBIAS math as evaluate.py.
Observed vs simulated
What this dashboard is looking at
An autonomous calibration loop — propose one parameter change → run SWMM5 → score → keep or revert — writes one JSON line per experiment to eval/history.jsonl. The referee (evaluate.py) only advances the ratchet when the mean Nash–Sutcliffe efficiency rises and no calibration event regresses beyond tolerance, and it rejects runs that trip a guard: continuity error above the cap, flatlined output, volume bleed (|PBIAS| pushed past the cap and worse than the best model), or parameters outside physical bounds. The pattern is the recursive-self-improvement recipe applied to hydraulic modeling: a fast objective evaluator, a monotone ratchet, guards against metric gaming, and held-out storms the optimizer never sees.
Reading the numbers: NSE 1.0 is a perfect match, above 0.75 very good, above 0.5 acceptable, below 0 worse than using the observed mean. Positive PBIAS means the model under-predicts volume. The honest test is the holdout storm — if calibration NSE climbed while holdout collapsed, the loop overfit; add events and rerun.
Companion app to the SWMM5 Calibration Ratchet starter kit. More SWMM5 first-principles material at swmm5.org.