Files
hermes-agent/tests
webtecnica 83fe362e80 fix(gateway): prevent reconnect watcher wedge after network-loss fatal error (#70344)
Three-part fix for the gateway going silently deaf after a retryable
fatal adapter error (e.g. httpx.ConnectError on Telegram):

1. **Detach-on-timeout in _connect_adapter_with_timeout** — Replaced
   plain asyncio.wait_for with the task-detach pattern used by
   _await_adapter_cleanup_with_timeout. asyncio.wait_for cancels the
   overdue task but then waits for it to exit, so a connect() that
   catches CancelledError can block recovery forever. The detach
   pattern releases the runner at the deadline via
   consume_detached_task_result.

2. **Ensure reconnect watcher always runs after escalation** — Added
   _ensure_reconnect_watcher_running(), called after queueing a
   retryable fatal error. If the reconnect watcher task has died
   (exhausted restart budget, terminal exception), it is respawned
   so queued platforms are never permanently stranded.

3. **Faulthandler at gateway startup** — Enabled faulthandler +
   SIGUSR2 dump to a rotating file under HERMES_HOME/logs/ for
   post-mortem diagnosis of future event-loop freezes.

Tests added for _ensure_reconnect_watcher_running (alive, dead,
not-started, not-running), fatal-error integration (retryable calls
ensure, non-retryable does not), and _connect_adapter_with_timeout
(timeout raises, success returns).
2026-07-24 16:03:10 -07:00
..