Files
hermes-agent/tests
Alexander Russell ae900cba9c fix(gateway): redeliver a flood-refused final reply once the penalty passes, without a false duplicate marker
The Telegram adapter fails a flood-controlled final send closed as
flood_control:<seconds> so the send coroutine never sleeps a long penalty
(#91969), on the understanding that the delivery ledger owns the wait. The
ledger did not: sweep_failed_for_runtime only replayed send_path_degraded
rows, so a flood-refused row sat in 'failed' until the next restart, whose
sweep_recoverable then redelivered it hours late prefixed with "Recovered
reply, the gateway restarted during delivery, so this may be a duplicate".
Observed: a reply refused at 08:58 UTC (both the MarkdownV2 send and the
plain fallback got flood_control:185) arrived at 12:15 UTC after a restart,
labelled as a possible duplicate although the platform had never accepted it.

Ledger (gateway/delivery_ledger.py):
- flood_control rows are runtime-retryable, but only once their own deadline
  has passed: the refusal's updated_at plus the platform's wait
  (flood_not_before). Neither an early timer nor a reconnect sweep spends a
  redelivery attempt inside the penalty window.
- A flood refusal of a reply that fits in one Telegram message (4096 UTF-16
  units) proves non-delivery, so that redelivery carries no duplicate marker.
  A chunked reply may have had its first chunk accepted before the refusal
  and keeps the marker.
- Claiming a flood row clears the stale refusal (last_error NULL, state
  'attempting'), so a resend interrupted before mark_delivered is seen as
  uncertain by the next boot and gets the marker.
- At boot, a dead owner's flood row that is not yet due is adopted (owner
  re-stamped, no attempt spent) instead of being resent early.
- pending_flood_retries() lists this process's waiting flood rows per adapter
  identity with the earliest deadline.

Runner (gateway/run_startup.py):
- _schedule_flood_redelivery arms one timer per adapter identity that runs
  the existing runtime sweep after the wait (capped at 15 minutes per sleep;
  the row's deadline, not the timer, decides eligibility, so a capped timer
  wakes early, sends nothing, and re-arms for the remainder). The slot stays
  occupied until the timer ends and only the running timer may arm its
  successor into it, so a refusal during the sweep can never strand the row.
- _arm_flood_timers_for_waiting_rows runs after every redelivery pass (boot
  and runtime), covering adopted rows, rows skipped as not yet due, and rows
  refused again.

Adapter (gateway/platforms/base.py): _finalize_delivery_obligation arms the
timer on a flood_control failure, best-effort inside the existing try.

Tests: 35 in tests/gateway/test_delivery_ledger_flood_retry.py, with a
controllable clock shared by ledger and runner. Each guard was mutation
tested. Existing ledger, reconnect and redelivery suites (131 tests) pass.
2026-09-07 04:46:20 -07:00
..