Five test files and five evals scripts merged since this branch's base still import the
moved names via gateway.platforms.base. They work (base.py imports the names for its own
use) but the PR's invariant is that in-tree code imports from the defining module.
evals/postmortem/ turns the one-off audit behind tracking issue #103563 into
something anyone with a Hermes state.db copy (and optionally rotated
agent.log*) can run on their own fan-out:
forensics/ common.py discovers the run tree (root = most descendants,
compression-rollover children excluded so cost buckets stay
disjoint), fits pricing from estimated_cost_usd, and five lanes
recompute the OBSERVED figures: tokens (buckets, depth/duration
shares, context reconstruction, excess-cache-write proxy, cap
replay), logcalls (per-call cache behaviour from agent.log with
coverage printed first; strict and loose plateau definitions
reported separately), delegation (timeouts, orphaned children,
polling hours, batch-join withheld child-hours, truncated
summaries), tools (hardline blocks, foreground refusals,
whole-file rewrites), goal_loop (nudges, parked barrier), rework
(public-surface drop at PR open + post-open commit inventory).
Every figure is labeled OBSERVED or MODELED.
live_ab/ the per-PR A/Bs (real code paths, fake providers, temp
HERMES_HOME), paths from argv.
review_probes/ the independent /review's probes, credited and adapted; each
reproduced a round-1 defect and the fixed head must pass it.
run.py runs the offline probes against one or two checkouts and prints
PASS/FAIL side by side (--live adds the ones that spend cents).
tests/ synthetic-DB smoke test for the lanes and runner.
On the run's DB the lanes reproduce the tracking issue's population exactly
(1,394 sessions, 93,284 calls, $19,302.59; cache_write $11,159.76) and on
main vs an integration checkout of the 13 PRs the runner shows every probe
FAIL -> PASS (two guard-only probes pass on both, noted in run.py).
The trajectories are deliberately not shipped: the DB holds 51,956 home
paths, 5,341 e-mails, private IPs, chat ids and real-shaped credentials in
tool output. The lane reports and recomputed JSON are in a secret gist
linked from #103563.
kshitijk4poor