Address the hermes-sweeper review of #61233: the bounded retry budget
is now a clean-exit-specific streak, not a share of the unified
consecutive_failures counter.
- detect_crashed_workers stamps a protocol_violation marker into the
violation run's metadata (via the event payload _end_run copies);
_protocol_violation_streak derives the streak from run history:
consecutive most-recent violation runs, rate_limited runs neutral
(mirroring their unified-counter treatment), any other closed run
resets it. Mixed failure kinds can neither consume nor extend the
budget.
- Below-budget violations no longer call _record_task_failure at all:
the task returns to ready with last_failure_error stamped directly
(including the corrective retry guidance wording adopted from #61817,
which build_worker_context surfaces to the retry worker) and the
unified counter is untouched, keeping the two budgets independent.
- At the bound the trip funnels through _record_task_failure with a new
keyword-only force_trip=True: the reaper has already resolved the
per-task max_retries override against the violation streak itself, so
the threshold comparison is skipped rather than double-applied.
max_retries keeps its documented top precedence in both directions:
max_retries=1 blocks on the first violation, max_retries=5 blocks on
the fifth consecutive one, unset uses the default bound of 3.
- Replace the first-violation-blocks regression test with five tests:
first occurrence retries (ready + guidance stamped + no gave_up +
unified counter untouched); streak trips exactly at the bound with
protocol_violations/protocol_violation_limit in the gave_up payload
and the auto-blocked side channel set; a prior nonzero crash does not
consume the violation budget; a non-violation failure between
violations resets the streak; max_retries precedence both directions.
All five fail against the previously reviewed diff and pass with this
follow-up. The test harness resolves hermes_cli.kanban_db fresh and
uses that single module object for the exit registry, liveness patch,
and reaper — earlier suite tests reload the module, and the old
mixed-object harness made _classify_worker_exit return unknown (the
reason the old test failed in full-suite runs on main).
Kanban suite: zero introduced failures vs upstream/main tip (62 vs 63
pre-existing environmental failures — the one no longer failing is the
old violation test this replaces; 662 passed vs 657).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
slow4cyl
452861fdc1