Files
hermes-agent/gateway/run_inbound.py
T
Ben Barclay 866332bfb5 fix(relay): authorize send_message targets and surface egress declines (P5) (#99220)
* fix(relay): authorize send_message targets and surface egress declines

P5 of the relay egress-authorization workstream. The relay path
authenticated the SENDER but never authorized the DESTINATION, and the
gateway compounded it from both ends.

(a) send_message could silently name an arbitrary relay target. Its
`target` parameter is free-form ('platform:chat_id'), so a model could
name ANY chat id and the gateway would emit an outbound frame for it.
gateway/relay/egress.py adds an attestation floor: a relay-routed
destination must have a provenance this gateway can show -- the
operator's home channel, the channel directory, or its own gateway
session origins. Anything else is refused HERE, with a visible tool
error naming the target, before a frame is written. Non-relay platforms
and platforms served by a live native adapter in this process are
untouched (same precedence resolve_delivery_transport applies).

(b) Connector declines were swallowed into apparent successes. The
connector's egress floor answers an unauthorized destination with a
DEFINITE failure whose text is deliberately uniform (F-005). Several
relay lanes degrade a *transport drop* by design and were degrading an
*authorization refusal* the same way:

  - _send_media returned None, sending the caller into
    BasePlatformAdapter's text fallback -- a DIFFERENT op re-addressed at
    the very chat the connector had just refused.
  - _send_prompt returned None, so exec-approval / slash-confirm /
    clarify reported "relay prompt op unavailable" (a wrong reason) and
    ran their numbered-text fallbacks into the refused chat.
  - task_card_stop discarded the error entirely.
  - typing / delete / react / thread ops degraded silently at debug.

is_egress_decline() classifies THAT a decline happened (never why --
the uniform text is not parsed for reasons) and requires a definite,
non-ambiguous failure, so a lost-ack retry is still a transport
outcome. Lanes with an error-carrying contract now report the decline
verbatim; cosmetic bool/None lanes still degrade but log it at WARNING.

Advisory progress drops that legitimately degrade are unchanged: the
task_card send lane, the draft ambiguous/except branches, and every
transport-exception path keep their existing fail-open behaviour.

Tests: 21 mutations of the production source, all KILLED.

* fix(relay): authorize the RESOLVED target; declines must not fall back

Review round 1 (independently confirmed by a second reviewer) found three
blockers. Two are fixed here; the third (B-2, Telegram @username) is a policy
decision left open deliberately.

B-1 — THE FIX CAUSED THE OUTAGE IT PREVENTED (tools/send_message_tool.py)

The P5(a) guard ran ABOVE Slack user->DM resolution, so it authorized the
internal pseudo-id `_parse_target_ref` emits (`user_name:ben`, `user:U...`).
Provenances only ever hold RESOLVED conversation ids, so a fully attested DM
was compared as a handle against a set of `D...` ids and refused:

  base  slack:@ben  SENT        head(before)  slack:@ben  REFUSED

Every Slack DM by handle was broken. Moved the guard below resolution; it now
authorizes the destination that is actually sent to, and the refusal names the
resolved id. Position is load-bearing, so it is commented as such and pinned:
reverting the move turns exactly the four new cases red.

B-3 — A DECLINE IS NOT A LANE FAILURE (gateway/run.py)

`_approval_send_outcome` had only sent/failed/ambiguous, so a connector
decline collapsed into `failed` — which is the cue to run the plain-text
fallback into the chat the connector had just refused. The adapter fix in the
previous commit improved the error STRING while user-visible behaviour stayed
identical to base; the commit message overstated it. Fixed properly:

  - new `declined` verdict, recognised via the shared `is_egress_decline`
    contract (not string sniffing at the call site)
  - exec-approval returns without the text fallback
  - slash-confirm suppresses the text reply AND clears the registration, so a
    card that never rendered cannot capture the user's next message

`send_clarify` was already correct (returns early inside the adapter).

MUTATIONS (production source; both directions)
  classifier never returns 'declined'        -> KILLED (4 cases)
  ALL failures classified as 'declined'      -> KILLED (2 cases)
  guard moved back above Slack resolution    -> KILLED (4 cases)
  decline CODE changed (review M05)          -> KILLED
  marker match made case-sensitive (M10)     -> KILLED

M05 was a tautology: the test asserted the imported constant against itself,
so changing the constant could not fail it. The wire contract is now pinned as
a literal, because the connector stamps that exact string and a one-sided
change is a silent cross-repo break.

REGRESSION CHECK: the 12 failures + 1 collection error in this test selection
are PRE-EXISTING cross-test contamination — the identical set fails at
7cf86188ac. Verified by diffing the failing sets: no new failures, 363 -> 374
passed.

NOT FIXED (deliberate): B-2, Telegram `@username`. The Bot API resolves handles
at send time, so there is no id to compare and no canonicalization exists yet.
That is a policy decision, not a code move.

* fix(relay): fail CLOSED on guard faults; classify the structured decline

Third independent review. Two more blockers, both reproduced before fixing.

1. THE GUARD ITSELF FAILED OPEN (tools/send_message_tool.py:158)

`_authorize_relay_target` wrapped BOTH the import and the call in one
`except Exception: return None` — and None means AUTHORIZED at every call site.
So any runtime bug inside the guard silently switched the entire P5(a) boundary
off. Reproduced: with the guard raising, an unattested target sent.

The docstring already stated the correct intent ("must not fail closed on its
own IMPORT error") and the code did something broader. The two failures are not
the same: a missing gateway package means there is no relay egress to
authorize; a fault inside the guard means authorization did not happen. The
import is tolerated, the call is not — a guard that cannot answer refuses.

2. THE STRUCTURED DECLINE WAS THROWN AWAY (gateway/run.py)

The adapter preserves the connector's dict in `SendResult.raw_response`. My
previous commit rebuilt a dict from the error STRING, which loses two
contracts:

  * a decline carrying `code: egress_declined` and NO text renders as
    "relay egress declined" — no marker colon — so it classified as `failed`,
    which is exactly the cue to run the fallback into the refused chat;
  * `ambiguous: True` (lost ack) was flattened into a DEFINITE failure,
    re-sending a card that may already be on the user's screen. That is the
    duplicate-card bug the ambiguous verdict exists to prevent, reintroduced
    by the fix meant to harden the same path.

Both call sites now classify `raw_response` when present, ambiguity first, and
fall back to the wire sentence only for connectors that send no structured
response.

I had fixed the text-marker path and tested only the text-marker path. Worth
naming: the review's probe was a shape my tests never produced.

MUTATIONS (production source)
  guard fault returns None (fail open again)   -> KILLED
  classifier ignores raw_response              -> KILLED (3 cases)
  ambiguous treated as a definite failure      -> KILLED (2 cases)

40 focused tests pass. Regression check vs be321faf27: identical 13-item
failing set (pre-existing cross-test contamination), no new failures.

STILL OPEN: B-2 / finding 3, Telegram `@username`. The reviewer is right that
this is a REGRESSION of an existing contract (#53573 added Bot API username
support), not merely an unspecified input, since relay provenance stores the
numeric chat id. Fixing it means resolving the handle before authorization, or
explicitly revoking the contract. That is a policy decision, not a code move,
and it is Ben's call.

* test(relay): pin M21 and M25, the survivors whose comments called them load-bearing

Round-2 review reported six unpinned survivors from round 1. Two guard real
behaviour and are now covered; the other four are cosmetic-lane warnings and
fail-open branches I am leaving documented rather than pretending to close.

M25 — thread-qualified session ids. `_session_ids` adds BOTH "chat:thread" and
the bare chat, because the connector authorizes the CHAT. Without the split a
gateway whose session origin is `-100999:77` cannot send to `-100999`, the chat
it is demonstrably already talking in. KILLED.

M21 — the generic `relay` plane must union every fronted platform, since a
relay session is filed under its LOGICAL platform. KILLED.

MY FIRST M21 TEST WAS THE DEFECT IT WAS TESTING FOR. I patched `_relay_fronted`
— the very function the mutation empties — so emptying it changed nothing the
test could see, and the mutation SURVIVED against a green test. Rewritten to
drive the real `relay_fronted_platforms()` through its env source
(`GATEWAY_RELAY_PLATFORMS`), which is how production learns it.

That is the same "the test verifies my stand-in" failure I have spent this
workstream removing from the connector harnesses, reproduced here in three
lines of Python. The tell was identical: a mutation that survives a test
written specifically to kill it.

334 tests pass.

NOT PINNED, deliberately: M03 (success-guard on a malformed dict), M24
(empty-target allowance — the one fail-open branch, reachable only when the
bare-platform path already resolved a home channel), M35/M36 (decline WARNINGs
on cosmetic lanes). All four are observability or defence-in-depth rather than
authorization, and the review agrees they are non-blocking.

* fix(relay): defer Telegram @username authorization to the connector (B-2)

Closes the last blocker. Two reviewers independently called this a REGRESSION
of the public-channel username support added in #53573, not an unspecified
input, and they were right: provenance stores RESOLVED numeric chat ids, so
comparing `@channel` against them could only ever refuse.

WHY THE GATEWAY CANNOT ANSWER IT. The guard fires only when there is no live
native adapter — i.e. relay-fronted deployments — and on exactly those the
CONNECTOR holds the bot token, not this process. There is no local way to turn
a handle into the numeric id. Refusing here is not "fail closed", it is "fail
always".

WHY DEFERRING IS SAFE. The destination is still authorized one layer out: the
connector's Telegram egress floor (gg#238, merged 743a7c2) classifies and
refuses unauthorized destinations after ITS resolution — the layer that closed
the reported vulnerability in the first place. Handles go from two guards to
one, the authoritative one, not to zero.

The carve-out is deliberately narrow and its EDGES are pinned, because the
failure mode of an exemption is silent widening:

  telegram `@handle`        -> deferred            (the regression case)
  telegram numeric id       -> still guarded
  matrix `@user:server`     -> still guarded       (telegram-only)
  bare name, no `@`         -> still guarded
  attested handle           -> normal path, attestation still consulted

MUTATIONS
  carve-out widened to all platforms   -> KILLED
  carve-out widened to every target    -> KILLED
  carve-out removed (regression back)  -> KILLED
  carve-out checked BEFORE attestation -> KILLED

THE ORDERING MUTANT SURVIVED MY FIRST TEST. Both orderings return None, so
asserting the verdict could not tell them apart — the test asserted the claim
instead of the mechanism. Rewritten to observe that attestation is actually
consulted. Same defect class as the M21 test earlier in this branch: a
mutation surviving a test written specifically to kill it means the test is
measuring the wrong thing.

341 tests pass.

FOLLOW-UP (option 2, Ben's call, deliberately NOT done here): resolve the
handle before authorizing so BOTH layers apply. That needs a resolution
round-trip through the connector — new wire surface — so it belongs in its own
phase rather than bolted onto this one. Recorded in the code comment at the
carve-out, not just here.

* fix(relay): close two fail-open boundaries; test the code-only decline for real

Both blockers from review, each REPRODUCED before fixing.

1. STRUCTURED DECLINE HAD NO GUARD. Deleting `raw_response=result` from both
   `_send_prompt` return branches left all 34 tests green — a surviving,
   non-equivalent security mutant. The `code` field is the documented
   PREFERRED signal precisely because a connector may send no prose, and a
   caller rebuilding `{"success": False, "error": ...}` cannot see it.

   Cause: every existing case declines with marker TEXT. The evidence for the
   code-only path was a hand-built SimpleNamespace in a different file — a
   stand-in for the adapter, so it verified my fixture instead of production.

   Fixed with a CodeOnlyDecliningConnector driving the real
   `send_exec_approval` -> `_send_prompt`, feeding the REAL SendResult to the
   REAL `_approval_send_outcome`, plus the same shape on the media lane.
       drop raw_response  SURVIVED (34 passed) -> KILLED

2. TWO FAIL-OPEN BOUNDARIES, both "absence" and "fault" sharing a return.

   `_relay_fronted` swallowed EVERY exception and returned an empty set, which
   `relay_routed_platform` reads as "not relay-routed" — skipping the guard.
   Probe, with a positive control in the same run:
       positive_control_denied   = True
       discovery_fault_denied    = False   <- unattested target AUTHORIZED

   `_authorize_relay_target` caught every exception during IMPORT as "no
   gateway package". A module that exists and fails to initialize is a fault,
   not an absence, and returning None there means authorized.

   Now: ImportError alone is absence; anything else raises RelayRouteUnknown
   and `authorize_relay_target` converts it to a REFUSAL STRING (not a raised
   exception — every caller treats the return value as the verdict, so raising
   would trade a fail-open for a crash).

   Kept the converse under test so "fail closed" does not silently become
   "refuse everything in CLI/cron", which is the outage the broad except
   existed to prevent.
       discovery fault -> empty set        KILLED
       RelayRouteUnknown -> authorized     KILLED
       import fault -> authorized          KILLED

397 passed (was 392, +5 new cases), zero failures.

* fix(relay): close all seven review-round-3 blockers

Every finding reproduced before fixing; every fix mutation-checked after.

CONTENT LEAKS (the decline was laundered into a different op, same chat)

#1 A declined DRAFT SEAL replayed as a plain send. On stream-is-the-message
   platforms the turn-final becomes draft(final=True); `_seal_open_draft`
   dropped the structured body, so `_absorb_into_open_draft` read a REFUSAL as
   a lane failure and fell through. Probe, Slack descriptor:
       before: draft(partial) -> draft(final,SECRET) -> send(SECRET)
       after:  draft(partial) -> draft(final,SECRET)
   My first probe of this used a discord descriptor and showed no seal at all —
   the leak is real, my probe was wrong (streams only arm for Slack).

#6 Task-card PROGRESS had the same defect one lane over: a bare failed
   SendResult reads as "card lane unavailable", and TurnRunner then sends the
   task text to the same chat. Both card methods now carry raw_response and
   the caller suppresses the fallback on a decline.

AUTHORIZATION BYPASSES

#2 `except ImportError` was NOT the fix I claimed last round. ImportError also
   covers a broken dependency inside an INSTALLED gateway; review probed
   `ImportError.name = "gateway.relay.dependency"` and got an authorized
   verdict. Now only a name identifying the gateway relay module itself is
   absence. An ImportError with NO name stays absence — refusing on a fault we
   cannot attribute would trade an unidentifiable bug for a real CLI/cron
   outage, and an existing test caught exactly that when I first got it wrong.

#3 `relay_routed_platform` lowercases the requested platform; `_relay_fronted`
   returned configured names verbatim. A platform configured as "Discord"
   missed the membership test, looked native, and skipped the guard:
       'discord' => refused    'Discord' => ALLOWED    'DISCORD' => ALLOWED
   An attestation bypass on a string comparison.

UNDELIVERABLE PROMPTS THAT HUNG

#4 `_clarify_send_disposition` handled `failed` and `ambiguous` but not
   `declined`, so a REFUSED clarify card fell through to wait_for_response and
   blocked until clarify_timeout — indefinitely when configured non-positive.
   A decline is more definitive than a failure, not less.

#5 The exec-approval decline branch returned quietly, which suppressed the text
   fallback (right) but left the CENTRAL approval entry pending (wrong) — the
   dangerous command stayed blocked until the approval timeout. My comment
   claimed the registration was torn down; only RelayAdapter's private map was.
   It now raises `_ExecApprovalDeclined`, which propagates to
   `_await_gateway_decision`'s existing notify-failure path (drops the entry,
   unblocks the tool). A dedicated type, re-raised past the local
   `except Exception` that would otherwise have restored the leak.

#7 THE GAP THAT LET ALL OF THIS SHIP. Both caller-level suppressions were
   unfalsifiable: deleting either branch left 36/38 tests green. The suites
   drove `_approval_send_outcome` and `RelayAdapter` but never the real
   TurnRunner / busy-session callers, so nothing observed whether a text send
   FOLLOWED a decline — which is the whole property.
   tests/gateway/test_decline_fallback_suppression.py drives both real callers
   and records every send. Each decline case is paired with an ordinary-FAILURE
   control, because without one a caller that never falls back would also pass.

MUTATIONS (all on production source, anchors count-checked, restored after)

  #1  seal decline -> plain send                KILLED
  #1b seal drops raw_response                   KILLED
  #2  nested ImportError -> authorized          KILLED
  #3  fronted set not normalized                KILLED
  #4  clarify declined branch removed           KILLED
  #5  approval decline returns not raises       KILLED
  #6  task_card drops raw_response              KILLED
  #7  slash-confirm suppression removed         KILLED

#7's two were the reviewer's SURVIVORS (36/38 passing); both now die.

425 passed, zero failures.

* fix(relay): close the three round-4 blockers

Round 4 confirmed six of seven round-3 fixes and found three more. Each
reproduced before fixing, each mutation-checked after.

1. A NAMELESS ImportError still authorized. Last round I admitted it as
   "absence" to protect the CLI/cron path. That reasoning was WRONG and the
   interpreter says so:

       import gateway.relay.nope  -> ModuleNotFoundError, name="gateway.relay.nope"
       import totally_absent_pkg  -> ModuleNotFoundError, name="totally_absent_pkg"

   Genuine absence is ALWAYS ModuleNotFoundError with `.name` set, so the
   CLI/cron path never produces a bare ImportError and nothing legitimate was
   being protected. A plain or nameless ImportError comes from an import hook
   or a module that failed while initializing — an unattributable FAULT.
   Now: absence is ModuleNotFoundError naming gateway / gateway.relay /
   gateway.relay.egress; everything else refuses. Two existing tests raised a
   bare ImportError to simulate absence and were corrected to the real shape.

2. SESSION ATTESTATION INVENTED IDS. `_session_ids` split every id on the first
   colon to recover "chat" from "chat:thread". Matrix ids contain a colon
   natively, so `!room:server.org` attested a bare `!room` — the guard
   vouching for a destination on its own fabrication. The split now applies
   only to platforms whose ids genuinely carry a `:thread` suffix (allow-list;
   unknown platforms are treated as un-splittable, which can only refuse more).
   Kept a Slack control: dropping the split entirely would refuse legitimate
   thread replies, which is the outage the split exists to prevent.

3. THE TASK-CARD FIX WAS UNFALSIFIABLE — my own round-3 mistake, and the same
   one round 3 caught me making. I added the production branch AND a test, but
   the test stopped at RelayAdapter: it proved `raw_response` is carried and
   never called `TurnRunner._task_card_publish`, which owns the property.
   Deleting the real branch left 30 tests green. Now driven through the real
   caller, with an ordinary-failure control.

   The lesson generalises: proving the DATA reaches the boundary is not proving
   the CALLER acts on it. Every one of these decline fixes has two halves and
   the second half is where the security lives.

Also closed the round-4 non-blocking finding: `gateway/relay/egress.py` has its
OWN import boundary, and the existing test intercepted the earlier import in
tools/send_message_tool.py, so it was never exercised. Mutating that classifier
to treat every ImportError as absence now dies.

MUTATIONS (production source, anchors count-checked, restored after)

  R4-1 nameless ImportError -> authorized        KILLED
  R4-2 session split unconditional               KILLED
  R4-3 task-card caller branch removed           KILLED  (was SURVIVED)
  egress classifier: any ImportError = absence   KILLED

Also probed and found NOT a leak: a refused OPENING draft frame disarms the
stream and the turn-final goes out via `send`. That send is itself guarded and
the connector refuses it too, so no content is delivered — unlike the seal case
(round 3, #1) where the seal was the only check on that path.

452 passed, zero failures.

* fix(relay): recover the thread parent from thread_id, not a colon split

Round 4 blocker 2 was closed with an allow-list of platforms whose ids have no
native colon. Reviewing my own fix while round 5 ran, the allow-list is the
wrong mechanism: it NARROWS a guess instead of removing it, and it still gets
Matrix wrong the moment a Matrix session is thread-qualified
(`!room:server.org:$thr` -> split yields `!room`).

The structured field was there all along. `_session_entry_id` composes the id
as f"{chat_id}:{thread_id}" and the entry still carries `thread_id`
separately, so the parent is knowable EXACTLY: strip the known suffix, or add
nothing. No platform list, no guessing, correct for ids that contain colons.

Mutations:
  back to splitting on the first colon        KILLED
  thread parent never recovered (over-refuse) KILLED

Both directions matter: the first invents attestations, the second refuses
legitimate thread replies.

One existing test (M25) asserted the right PROPERTY with a fixture that omitted
`thread_id` — a shape real entries never have. Fixture corrected, assertions
untouched.

453 passed.

* fix(relay): close the four round-5 blockers

Each reproduced before fixing, each mutation-checked after.

R5-1 A DISABLED NATIVE ADAPTER BYPASSED AUTHORIZATION. `_has_live_native_adapter`
     treated any entry in the adapter map as native; `resolve_delivery_transport`
     ignores a native adapter whose config is disabled and routes over Relay.
     Two independent routing classifiers, disagreeing:
         guard says native: True   delivery routes relay: True
     So the guard skipped authorization for a send that went over the relay.
     The guard now applies the router's enabled-state rule; probed both
     configurations and they agree.

R5-2 THREAD IDS WERE NEVER AUTHORIZED. The parser splits chat_id and thread_id;
     only chat_id reached the guard. On Discord the thread IS the destination —
     `POST /channels/{thread_id}/messages` — so an attested parent channel
     authorized an arbitrary caller-supplied thread. `authorize_relay_target`
     now takes thread_id and requires its own attestation (bare id or the
     `chat:thread` form a session origin produces); both call sites forward it.

R5-3 A DECLINED **INITIAL** DRAFT WAS RETRIED AS A PLAIN SEND. Round 3 fixed the
     declined SEAL; the declined OPEN was a different path. `send_draft`
     returned a bare failure, so the stream consumer read "draft transport
     unusable", disabled drafts and fell through to `_first_send`. Measured
     through the real adapter and real StreamTransportMixin:
         before: ops ['draft', 'send']      after: ops ['draft']
     send_draft now carries raw_response; a decline is terminal for the run and
     the guard sits in `_first_send`, where every fallback path converges.

R5-4 MY ROUND-4 TASK-CARD FIX SUPPRESSED EXACTLY ONE UPDATE. It set
     `native_failed`, which the entry gate already uses for an ordinary broken
     lane, so the next progress event skipped the decline branch and went
     straight to the text fallback:
         after first publish: []      after second: ['send']
     Terminal declines are now a separate `egress_declined` state checked at the
     entry gate. A refusal does not expire after one tick.

MUTATIONS

  R5-1  disabled native counts as native        KILLED
  R5-2  thread_id not authorized                KILLED
  R5-2b tool does not forward thread_id         KILLED (was SURVIVED)
  R5-3  initial-draft decline not terminal      KILLED
  R5-3b _first_send guard removed               KILLED
  R5-4  declined state not persistent           KILLED

R5-2b is the same gap that produced findings 3 and 4 of the last two rounds, a
third time: every test called `authorize_relay_target` directly, so dropping the
argument from the TOOL WRAPPER changed nothing. Testing the callee never proves
the caller uses it — now pinned explicitly.

Each fix ships with an ordinary-failure control, because every one of these
makes the guard refuse MORE, and over-refusal is now the larger risk.

474 passed, zero failures.

* refactor(relay): declare the terminal-decline state where it lives

Both terminal-decline flags were set dynamically. They worked (neither class is
frozen or slotted) but an undeclared attribute hides the state from anyone
reading the class, and this one is security-relevant.

  _TaskCardState.egress_declined  — declared dataclass field
  StreamConsumer._egress_declined — initialised in __init__

Lifetime verified while checking whether a refusal can leak ACROSS turns and
mute a healthy destination: it cannot. _TaskCardState is constructed per
progress-drain (run_turn_runner.py:420) and the consumer's flags per run
(stream_consumer.py:163), so both are fresh each turn.

Also verified the guard's blast radius after adding thread authorization: the
ONLY callers of authorize_relay_target are the two model-facing send_message
call sites. Gateway-internal sends — notably the handoff path, which creates a
thread and immediately posts to it with no session provenance yet — go through
transport.adapter directly and are unaffected. That was the most plausible
over-refusal, and it does not reach this guard.

461 passed.

* fix(relay): close the four round-6 blockers — the edit lane

R6-1 MY OWN R5-1 FIX REINTRODUCED THE BYPASS IT CLOSED. I wrote
     `except Exception: return True` around the config lookup, so a config read
     fault declared the platform native while the ROUTER, reading the real
     config, sends over the relay:
         guard_has_live_native True   guard_verdict None   router relay
     Routing we cannot determine is UNKNOWN. It now raises RelayRouteUnknown,
     which the outer handler must re-raise rather than flatten to False, and
     `authorize_relay_target` turns into a refusal. This is the second time a
     convenience `except` in this function created a bypass; there is now no
     permissive return left in it.

R6-2/3/4 THE NINTH LANE: `edit`. ONE dropped field, THREE leaks.
     `RelayAdapter.edit_message` discarded the connector response, and three
     independent callers read a bare edit failure as "editing is unavailable"
     and re-send the content as a NEW message to the same chat:

       stream edit fallback   ['edit', 'edit', 'send']  the unseen tail
       queued reconciliation  ['edit', 'send']          the WHOLE response
       task-card fallback     ['edit', 'send']          the task text again

     Fixed at the source (edit_message carries raw_response) plus each caller:
     `_on_edit_failure` — the single funnel for stream edit failures — makes a
     decline terminal for the run, `_send_fallback_final` refuses to deliver a
     continuation after one, the queued reconciler returns instead of sending,
     and the task-card fallback sets the same terminal state R5-4 introduced.

     R5-4 fixed the native task-card op and I did not check its sibling
     fallback path. The pattern across rounds 3-6 is consistent: the fix goes
     where the decline is OBSERVED, and the leak lives wherever someone else
     later decides to retry.

MUTATIONS

  R6-1  config fault -> assume native            KILLED
  R6-1b RelayRouteUnknown swallowed as False     KILLED
  R6-2  edit drops raw_response                  KILLED
  R6-2b edit-failure decline not terminal        KILLED
  R6-3  queued reconcile falls back on decline   KILLED
  R6-4  task-card fallback edit decline          KILLED

Each with an ordinary-failure control: a genuinely un-editable message must
still be delivered, and a broken card lane must still reach the user.

481 passed, zero failures.

* fix(relay): add a terminal-decline latch at the adapter choke point

THE STRUCTURAL FIX, not a twelfth local check.

Rounds 3-6 of review found ONE defect in eleven lanes: the connector refuses an
op, and some caller downstream reads that as 'this lane is unavailable' and
retries the same content through a DIFFERENT op against the SAME chat. Media,
prompt, draft-open, draft-seal, native task card, task-card fallback edit,
slash-confirm, exec-approval, clarify, stream edit, queued reconciliation.

Each was closed by adding a check at one more call site. That approach cannot
converge: gateway/ has ~60 outbound call sites, every one of them a place a
future change can reintroduce this, and four consecutive review rounds each
found another. The reviewer's own count of lanes is the argument against the
per-site design.

Every relay frame from every one of those callers passes through
_transport.send_outbound. One latch there covers them all: once the connector
refuses a chat, this adapter stops emitting CONTENT frames for that chat.

Proven to subsume the local checks: with the stream-edit per-site check
DISABLED, the leak probe still reports blocked=true — the frame never reaches
the wire. The local checks stay as defence in depth and for their better error
messages, but they are no longer the only thing standing between a decline and
a re-addressed send.

Scope is deliberately narrow, and each limit is mutation-pinned:
  per CHAT       - a refusal must not mute other conversations
  CONTENT ops    - typing/delete carry nothing; latching them would leave a
                   stuck typing indicator for no security gain
  self-healing   - cleared when the connector accepts that chat again, so a
                   transient policy change does not need a restart

Mutations:
  latch never set                KILLED
  latch never consulted          KILLED
  latch is global, not per-chat  KILLED
  latch never clears             KILLED

485 passed.

* fix(relay): one route source; the latch already covered round 7's lanes

Round 7 reviewed 573e41e294 — one commit BEFORE the terminal-decline latch —
and independently reached the same conclusion I had: 'The per-call-site
approach is structurally wrong. Use one turn-scoped choke point.' That is the
latch in 6dbc004594.

Its four 'still broken' lanes (tool-progress edit, progress-overflow edit,
long-running heartbeat edit, stale streamed-final reconciliation) all share the
shape edit_message->declined->adapter.send(same chat, same content), and NONE
has a local check. Probed all four against the latch:

  tool_progress      ops ['edit']  blocked
  progress_overflow  ops ['edit']  blocked
  heartbeat          ops ['edit']  blocked
  stale_final        ops ['edit']  blocked

That is the argument for the choke point, measured: lanes nobody patched are
safe anyway. Pinned by a parametrized test named for those four lanes.

R7-1 IS A REAL BYPASS THE LATCH DOES NOT COVER, and it is fixed here. The guard
rebuilt routing from GATEWAY_RELAY_PLATFORMS while resolve_delivery_transport
asks the CONNECTED adapter (fronts_platform, from the handshake identity set).
Different snapshots: with env discovery stale or momentarily empty, the guard
said 'native' and the router sent over the relay, skipping authorization.

  before: guard_relay_routed False / delivery relay
  after:  guard_relay_routed True  / delivery relay / unattested target refused

The guard now asks the live adapter first and falls back to config only when
there is no runner (CLI/cron) — pinned in both directions.

R7-5 (non-blocking, and a fair hit): my stream-fallback test asserted
_egress_declined and never drove _send_fallback_final, so removing that early
return SURVIVED. The test now calls the real fallback and asserts the wire is
untouched; the mutation dies.

Mutations:
  R7-1 guard ignores the live adapter    KILLED (was SURVIVED)
  R7-5 fallback early return removed     KILLED (was SURVIVED)
  latch not consulted                    KILLED

491 passed.

* fix(relay): close three holes found by attacking my own latch

Round 8's brief told the reviewer to attack the latch. I did the same in
parallel and found three real holes in it before the review returned.

1. send_for_platform BYPASSED THE LATCH ENTIRELY. It builds and posts its frame
   directly rather than through _outbound — and it is the delivery resolver's
   OWN entry point, so it is the single most important caller.
       before: ops ['edit', 'send']   after: ops ['edit']
   gateway/AGENTS.md states the rule I had just broken: 'Seal-interception
   exists at BOTH egress doors (send() and send_for_platform()); a new egress
   door needs the same two checks.' The latch is a third such check and I had
   wired it to one door.

2. A COSMETIC SUCCESS CLEARED THE LATCH. Clearing on ANY success meant a
   typing indicator — routinely allowed for a chat whose content is refused —
   re-opened the door for the very next send:
       ops ['edit', 'typing', 'send']
   Only a CONTENT op the connector accepted may clear it now.

3. A THREAD INSIDE A REFUSED CHAT WAS NOT COVERED. A thread lives inside its
   parent, so the same content reached the same conversation one level down:
       ops ['edit', 'send']
   The latch key now strips the thread suffix.

Also normalised int/str chat ids (callers pass both; a type mismatch would
silently unlatch).

MUTATIONS
  send_for_platform not latched          KILLED
  cosmetic success clears the latch      KILLED
  thread suffix not stripped             KILLED
  draft-seal retry not latched           SURVIVED — EQUIVALENT, proven:
       is unreachable while latched (a declined edit before the seal
      produces ZERO seal frames, measured). Kept as defence in depth because it
      posts directly, and documented at the site rather than covered by a
      test that could not fail.

One self-inflicted bug on the way: a blanket replace put 1Password CLI brings 1Password to your terminal.

Turn on the 1Password app integration and sign in to get started. Run
'op signin --help' to learn more.

For more help, read our documentation:
https://www.1password.dev/cli

1Password CLI is built using open-source software. View our credits and
licenses:
https://downloads.1password.com/op/credits/stable/credits.html

Usage:  op [command] [flags]

Management Commands:
  account         Manage your locally configured 1Password accounts
  connect         Manage Connect server instances and tokens in your 1Password account
  document        Perform CRUD operations on Document items in your vaults
  events-api      Manage Events API integrations in your 1Password account
  group           Manage the groups in your 1Password account
  item            Perform CRUD operations on the 1Password items in your vaults
  plugin          Manage the shell plugins you use to authenticate third-party CLIs
  service-account Manage service accounts
  user            Manage users within this 1Password account
  vault           Manage permissions and perform CRUD operations on your 1Password vaults

Commands:
  completion      Generate shell completion information
  inject          Inject secrets into a config file
  read            Read a secret reference
  run             Pass secrets as environment variables to a process
  signin          Sign in to a 1Password account
  signout         Sign out of a 1Password account
  update          Check for and download updates.
  whoami          Get information about a signed-in account

Global Flags:
      --account account    Select the account to execute the command by account shorthand, sign-in address, account ID, or user ID. For a list
                           of available accounts, run 'op account list'. Can be set as the OP_ACCOUNT environment variable.
      --cache              Store and use cached information. Caching is enabled by default on UNIX-like systems. Caching is not available on
                           Windows. Options: true, false. Can also be set with the OP_CACHE environment variable. (default true)
      --config directory   Use this configuration directory.
      --debug              Enable debug mode. Can also be enabled by setting the OP_DEBUG environment variable to true.
      --encoding type      Use this character encoding type. Default: UTF-8. Supported: SHIFT_JIS, gbk.
      --format string      Use this output format. Can be 'human-readable' or 'json'. Can be set as the OP_FORMAT environment variable.
                           (default "human-readable")
  -h, --help               Get help for op.
      --iso-timestamps     Format timestamps according to ISO 8601 / RFC 3339. Can be set as the OP_ISO_TIMESTAMPS environment variable.
      --no-color           Print output without color.
      --session token      Authenticate with this session token. 1Password CLI outputs session tokens for successful 'op signin' commands when
                           1Password app integration is not enabled.
  -v, --version            version for op

Run 'op [command] --help' for more information on the command. into
send_for_platform, which has no such variable. Two existing unfurl tests caught
it — NameError at adapter.py:1407.

504 passed.

* fix(relay): Telegram handle exemption + a turn boundary for the latch

Round 8 blockers. Two of its four were already closed by 93750e351a (it
reviewed the commit before it); these two are real and both are mine.

B1 — THE TELEGRAM @HANDLE EXEMPTION COVERED A NATIVE SEND.

_is_unresolved_handle exempts telegram @handles from attestation because
"the connector resolves and authorizes it". That justification is FALSE
whenever the gateway holds its own token: _send_to_platform calls
_send_telegram(pconfig.token, ...) directly and no connector is involved.
So an unattested @handle went out under the gateway's own credential
while the numeric control was correctly refused.

The exemption now requires that no native credential exists. A probe
fault WITHDRAWS the exemption (falls back to the ordinary attestation
check) rather than granting it.

Shipped with the converse control: relay-only config still exempts
@handles, and numeric targets stay guarded in both modes.

B4 — THE LATCH HAD NO BOUNDARY, SO IT WAS AN OUTAGE MECHANISM.

My own regression, and worse than reported. Removing "clear on cosmetic
success" (correctly) removed the ONLY way the latch could ever clear: a
content op can never reach the connector to succeed, because the latch
blocks it locally first. A refusal at 09:00 muted that chat forever.

A new inbound message for a chat is the generation marker — the natural
teardown point. Suppression still holds for the whole turn.

    same_turn_blocked: true     next_turn_delivered: true

MUTATIONS (all killed)
  handle exemption ignores native credential
  native-credential fault GRANTS the exemption
  no turn boundary (latch never clears)
  teardown clears ALL chats not just this one
  teardown ignores the chat

The last two SURVIVED first: I tested _clear_declined_for_turn directly
and never proved _on_inbound calls it — the caller-level gap that has now
produced four blockers on this branch. Added a test driving the real
inbound entry point.

One self-inflicted bug, caught by my own fault test: the probe imported
load_config, which does not exist (it is load_gateway_config), so it
always threw and returned the fault default. The test that pinned fault
behaviour is what exposed it.

510 passed.

* fix(relay): correct latch identity and boundary; one config snapshot

Round 9, four blockers, all reproduced.

B1+B4 — THE TEARDOWN WAS AT THE WRONG PLACE, twice over.

It sat on the adapter's raw _on_inbound, which runs BEFORE profile
routing, the ignored-channel guard, plugin hooks and user authorization.
An unauthorized or dropped event could therefore clear a refusal
belonging to an active turn, and stale content then went out as a
different op. The same placement missed Discord interaction passthrough,
which builds its own MessageEvent and calls handle_message directly, so
slash commands and modal submits stayed muted after an earlier decline.

Both are one mistake: I picked a lane instead of a boundary. Teardown now
runs immediately after _hm_admit_event, the single admission gate every
entry path shares.

  dropped event  -> latch survives, stale send blocked
  admitted event -> latch clears

B2 — THE LATCH KEY SPLIT ON ':', WHICH IS A MISTAKE I ALREADY FIXED ONCE.

_latch_key did str(chat_id).split(":", 1)[0], so !room:tenant-a and
!room:tenant-b both keyed !room: a decline in one Matrix room muted
another, and inbound from one cleared the other's refusal. egress.py
::_session_ids stopped doing exactly this in round 4 and I reintroduced
it three rounds later.

Parent identity is never recoverable from identifier TEXT. Thread
coverage is now structural: _thread_parent looks the relationship up in
the recorded auto-thread map.

B3 — AUTHORIZATION AND DISPATCH USED DIFFERENT CONFIG SNAPSHOTS.

_handle_send retains one pconfig; the guard independently reloaded
config. Across a transition the authorization snapshot could see a
connector-only setup (exemption granted) while dispatch still held the
native token and sent the unattested @handle itself. The guard now takes
native_token from the SAME snapshot dispatch will use. A caller that
omits it does not silently look like "no token".

NB-1/2/3 also closed: real-object snapshot tests, an exception shield
that faces a real exception, and send_follow_up no longer discards the
connector's verdict (that discard is exactly how the edit lane laundered
declines).

MUTATIONS (all killed)
  latch key splits on colon again
  thread parent lookup disabled
  dispatch token ignored by guard
  tool drops the snapshot token
  admission teardown removed
  teardown moved BEFORE admission
  exception shield removed
  follow_up drops raw_response

"admission teardown removed" SURVIVED first: I had tested the helper, not
_handle_message. Added a test driving production _handle_message with
admission stubbed both ways. Fifth caller-level gap on this branch.

One self-inflicted bug caught before commit: I passed pconfig.token in
_handle_react, which has no pconfig — a NameError on every reaction.

516 passed.

* docs(relay): pin the latch's thread coverage limit as a deliberate trade

_thread_parent only sees connector auto-threads, and that map is capped at
256 entries, so a user-created or evicted thread does not inherit its
parent's latch. Documented at the site and asserted by a test, because the
alternative - deriving parents from identifier text - is exactly what muted
unrelated Matrix rooms in round 9.

The primary control is unaffected: authorize_relay_target takes thread_id as
part of the destination and attests it on every send (6 thread tests).

* refactor(relay): one SendResult decline classifier for all 8 gateway lanes

The extraction found a DEFECT, not just repetition.

Eight gateway lanes each hand-rolled the unwrapping of a decline from a
SendResult, and they did not agree. Six checked only raw_response. Two
also checked the error text. A connector that answers with the uniform
decline SENTENCE and no structured code - the documented contract for
older connectors, per _approval_send_outcome - was therefore classified
as an ordinary failure by those six lanes, so each treated a refusal as
"editing unavailable" and retried through another op.

Measured:

    text-only decline    six-site check False    two-site check True
    structured decline   six-site check True     two-site check True

No content leaked, because the adapter latch classifies the transport
dict directly and catches both shapes (verified: text-only decline still
latches C1 and keeps SECRET off the wire). The cost was wrong verdicts
and futile retries, not disclosure.

declined_send(result) in gateway/relay/egress.py now owns this. It checks
raw_response when structured, else the error text, and preserves the
ambiguous exclusion - an ambiguous result is a transport outcome, so it
must never read as a refusal.

run.py keeps its own shape deliberately: that lane has three verdicts
(ambiguous / declined / failed), so it checks ambiguous first and then
delegates the boolean.

MUTATIONS (all killed)
  helper drops the text-only branch
  helper drops the structured branch
  ambiguous no longer excluded
  draft lane decline check removed
  edit-failure lane decline check removed
  prompt verdict lane check removed
  slash-confirm lane check removed
  draft lane goes terminal on ANY failure   (over-refusal direction)

"draft lane decline check removed" SURVIVED first: _send_draft_frame had
no test driving an unsuccessful send_draft at all. Added one, with an
ordinary-failure control so the fix cannot silently become "one flaky
frame mutes the chat". A non-unique anchor also masked the edit-failure
lane on the first pass - the trap my own skill warns about.

This closes the duplication that caused four of nine rounds of blockers:
a new lane now calls one classifier instead of copying three lines.

519 passed.

* fix(relay): latch identity, new-turn boundary, seal arming, ambiguity

Round 10, four blockers, each reproduced before fixing. Two are my own
regressions from the previous two rounds.

B1 - ADMISSION IS NOT A NEW-TURN BOUNDARY.

Round 9 moved teardown to just after _hm_admit_event. That is only an
ADMISSION gate: an authorized message can be steered into a running
session, answer a pending prompt, run a busy slash command, or be refused
by the pause/drain gates - all without starting a turn. Each of those
cleared the ACTIVE turn's refusal, and a later fallback from that turn
reached the wire (probe: latch emptied, wire ops ['edit', 'send']).

Teardown now runs after _claim_active_session_slot, the first point the
runner OWNS a new turn. The new test drives production _handle_message
through all four non-turn lanes plus the real new-turn path.

B2 - LATCH IDENTITY OMITTED THE LOGICAL PLATFORM.

One relay adapter fronts several platforms, so native ids collide. A
Discord refusal for chat 42 was cleared by clear_egress_latch("telegram",
"42") - the method took a platform and ignored it - and the Discord
fallback then reached the connector. Keyed by normalized platform plus
exact chat id; thread-parent expansion keeps the platform component.

B3 - THE DIRECT DRAFT-SEAL PATH DID NOT ARM THE LATCH.

_seal_open_draft posts through _attempt directly rather than _outbound,
so a definite decline logged and returned but never latched. The
immediate plain-send fallback was suppressed by the caller's own check;
later same-turn sends were not (wire ['draft', 'draft', 'send'], the
third frame carrying refused content).

B4 - MY OWN REFACTOR MADE AMBIGUOUS RESULTS TERMINAL.

send_draft's ambiguous projection discarded raw_response, so
declined_send fell through to the error-text branch - and an ambiguous
result whose text carries the decline marker ("... egress declined: ack
lost") read as a DEFINITE refusal and terminated the run. Ambiguous means
the frame may well have been delivered: a transport outcome, never an
authorization one.

Fixed on both layers: the projection carries the body (and the seal's
ambiguous return is now explicit too), and declined_send's text-only
branch - which cannot see the ambiguous flag - treats ack-lost text as
transport ambiguity. Audited every SendResult projection in adapter.py
for the same shape.

MUTATIONS (all killed)
  latch key drops the platform
  clear_egress_latch ignores platform
  draft seal does not arm the latch
  ambiguous projection drops raw body
  declined_send infers decline from ack-lost text
  teardown back at admission

523 passed.

* refactor(relay): split the terminal-decline latch out of the guard PR

The latch moves to feat/p5-egress-decline-latch (pushed at 3cf45736d7,
which retains the full history) for redesign. This PR keeps the
authorization guard and the per-site decline checks.

WHY. Across eleven review rounds the two halves behaved very differently.
The guard is a PURE FUNCTION of the destination - its blockers were all
"you asked the wrong question" (case sensitivity, nested ImportError,
missing thread_id, config snapshot skew), each a one-line correction that
then stayed fixed. Rounds 7-10 found nothing new in it.

The latch is MUTABLE STATE WITH A LIFETIME living on RelayAdapter - an
object registered once per process that holds the WebSocket and has no
concept of a turn. Nine of its blockers reduce to three questions the
adapter cannot answer: when does it end, who arms it, what is it keyed
on. Every answer so far has been a proxy (a successful op, an inbound
message, an admitted event, a claimed session slot) and every proxy was
wrong in a lane found later.

The per-site checks hold identical information on `st` - a PER-TURN
object - and have produced zero blockers, because the state dies with the
turn and nobody has to decide when it ends.

The no-relaunder property does NOT depend on the latch. Measured on the
real consumer path with the latch absent: a declined draft frame sets
_egress_declined and puts nothing on the wire.

Removal verified structurally rather than by eye: an AST diff of every
symbol between HEAD and this tree reports only latch symbols gone,
nothing added. That check caught two over-deletions my strip made -
_on_inbound (consumed by a "next def" boundary) and _SEEN_INBOUND_MAX
(a class constant inside the removed span). Both restored; 19 failures
went to 0.

ALSO: RESTORED A TEST I WRONGLY REPORTED AS PASSING.

test_tool_guard_forwards_thread_id never made it into the repo - `git log
-S` finds it in no commit - though round 5 recorded its mutant as killed.
Dropping thread_id from the guard call therefore survived the entire
tests/tools suite (146 passed). Written properly this time, driving the
real _handle_send far enough to reach the guard. It now KILLS that
mutant.

MUTATIONS on this tree
  guard fault authorizes instead of refusing      KILLED
  thread_id dropped from the guard call           KILLED  (was SURVIVED)
  handle exemption ignores native credential      KILLED
  draft lane decline check removed                KILLED
  prompt verdict lane check removed               KILLED
  slash-confirm lane check removed                KILLED

503 passed.

* test(relay): close the phantom-coverage gaps the guard audit found

The thread_id test that was reported as killing a round-5 mutant turned
out never to have been committed. That is a reason to distrust the other
claimed kills, so I re-ran every guard mutation against the COMMITTED
tree instead of trusting the earlier reports.

Result: 9 of 11 killed, and the two "SKIPPED" ones had non-unique
anchors hiding SIX separate sites. Mutating those individually found
three real survivors.

CASE NORMALISATION (round 3, finding 3) WAS HALF-COVERED.

test_relay_fronted_matching_is_case_insensitive varies the CONFIGURED
name but always requests lowercase "discord", so it pins _relay_fronted's
normalisation and nothing else. The REQUESTED name's `.lower()` was
covered by nothing at all. Probe with it removed:

    relay_routed("Discord") -> False
    authorize("Discord", unattested) -> AUTHORIZED

which is exactly the bypass round 3 reported, alive again and untested.

Two further sites were untested in the OVER-REFUSAL direction: the
attested store is keyed lowercase, so a mixed-case request missed its own
attested set and refused legitimate traffic. attested_relay_targets' own
normalisation was invisible to every existing test because they all
monkeypatch that function away; it is now asserted against the real
function with only its leaf sources stubbed.

Three tests added. All six case sites now die when mutated.

I also re-did the three fail-closed RelayRouteUnknown mutations properly.
The first pass swapped whole lines and produced IndentationErrors, so
"KILLED" there proved nothing but a syntax error. Neutralising each raise
at correct indentation: all three genuinely KILLED.

FINAL AUDIT ON THIS TREE — 17 mutations, zero survivors
  guard: thread_id dropped at the call site
  guard: react path unguarded
  guard: handle exemption ignores native credential
  guard: 3x fail-closed raise neutralised
  guard: 6x case-normalisation site
  classifier: ambiguous treated as a decline
  classifier: text-only decline branch removed
  lane: draft / stream-edit / prompt / slash-confirm checks removed

511 passed.

* test(relay): make the stream-edit test fail for the right reason

Review of 45835a282d raised one blocking issue and three non-blocking
ones. All four are addressed; none was a production defect.

BLOCKING — the stream-edit test failed on the double, not on a leak.

test_declined_stream_edit_does_not_send_the_unseen_tail implemented only
the GUARDED path in its consumer double. Removing either guard therefore
raised AttributeError inside the fake before any send could be observed:

  guard 1 removed -> AttributeError: no attribute '_is_flood_error'
  guard 2 removed -> AttributeError: no attribute '_clean_for_display'

Red, but for the wrong reason — the test could not have caught the leak
it is named for. My own docstring claimed it drove the fallback and
checked the wire; it did neither.

The double now implements everything the UNGUARDED path reaches
(_is_flood_error, _flood_strikes, _current_edit_interval, _last_edit_time,
_notify_new_message, _try_strip_cursor, _clean_for_display,
_fallback_prefix, _metadata_for_send). Both mutations now fail on real
assertions:

  guard 1 removed -> assert consumer._egress_declined is True
  guard 2 removed -> AssertionError: the unseen tail reached the wire:
                     ['send']

NON-BLOCKING 1 — a docstring claimed more than the test exercises.

test_requested_platform_name_is_also_normalised described a mixed-case
send_message(target="Discord:999") bypass. That entry point cannot reach
it: _resolve_tool_target lowercases the platform at
tools/send_message_tool.py:47 before the guard runs. The test still pins
a real contract — the helpers must not assume a lowercased argument, for
the gateway lanes and any future non-normalising caller — so the claim is
narrowed to that rather than the test removed.

NON-BLOCKING 2 — the module docstring said "every lane drives the REAL
RelayAdapter". The stream tests drive mixin doubles by design, because
the behaviour under test belongs to the adapter's CALLER. Docstring now
distinguishes the two kinds.

NON-BLOCKING 3 — latch-deletion residue in gateway/relay/adapter.py:418:

      return None
      return latched if surface_declines else None

The second line was unreachable and referenced a name deleted with the
latch. Removed, along with the 20-line comment block describing the latch
as "the structural fix" — that mechanism now lives on
feat/p5-egress-decline-latch, not here.

The reviewer independently confirmed the large deletion: an AST census
between 3cf45736d7 and f57a2298fa reports only latch symbols removed and
nothing added.

511 passed.

* docs(relay): correct three claims that outran the code

Review of 41ce3cc765 found no new production defect but three overstated
claims, one of them in my own commit message.

1. THE LATCH COMMENTARY WAS STILL THERE. My previous commit message said
   it removed "the 20-line comment block describing the latch as the
   structural fix". It removed only the unreachable statement. Twenty
   lines at adapter.py:361-380 still described a per-chat latch, a choke
   point and its scope rules - none of which exist on this branch. In a
   refusal-sensitive module that reads as coverage this branch does not
   have. Now removed for real.

   This is the same defect class as the tests: a claim that outran what
   the code does. I made it while fixing that class.

2. THE STREAM-TEST DOCSTRING OVERSTATED BOTH MUTANTS. It said the
   mutation "now fails on the assertion that a send reached the wire" -
   true of one guard, not both. Verified separately:

     remove the _on_edit_failure check  -> dies on _egress_declined,
                                           never reaches the fallback
     remove the fallback early return   -> dies on the wire: ['send']

   Both are valid behavioural failures, which is what the blocker asked
   for; they are different observables and the docstring now says so.

3. Duplicate `from types import SimpleNamespace` from an earlier scripted
   insert; imports reordered.

112 tests pass in the four focused files.

* fix(relay): close two authorization defects found in review

Both were reproduced before fixing and both mutants are pinned.

1. A LIVE relay adapter whose fronts_platform() raised degraded into the
   config fallback. `_live_relay_fronted` returned None for every failure,
   and None means "no live adapter, use the config snapshot" — so a faulting
   adapter plus an empty/stale snapshot made the guard conclude "not
   relay-routed" and authorize an unattested destination, while
   resolve_delivery_transport asks that same adapter and still routes over
   the relay. Measured: relay_routed=False, verdict None for chat 999.

   Absence and fault now have separate return values: None only when there
   is no runner or no relay adapter; a live adapter that cannot answer
   raises RelayRouteUnknown. This is the third instance of this bug class in
   this file, and the first two were also mine.

2. An attested chat whose id equalled the requested THREAD id vouched for
   that thread. The `thread in attested` arm proved nothing about parentage.
   Measured: attested {"-100A", "7"} authorized (-100A, thread 7).

   Only the bound `parent:thread` form is accepted now. Nothing legitimate
   needed the bare arm — _session_entry_id records a threaded origin as
   f"{chat_id}:{thread_id}", and a thread addressed as its own channel
   arrives as chat_id and passes the parent check.

The existing test blessed the bare form via parametrize, so it PINNED the
defect. Corrected, plus negative controls for the sibling-chat and
other-parent cases and a positive control proving genuine absence still
takes the config path (otherwise fix 1 would break native-only deploys).

Merged origin/main (was 22 behind). 428 passed via scripts/run_tests.sh;
full 10-row mutation ledger re-killed on the merged tree, none dying on an
exception rather than an assertion.

* fix(relay): only a missing adapter is absence; everything else is a fault

Reviewer BLOCKER, reproduced before fixing. Two more paths where a PRESENT
relay adapter still degraded into the config snapshot:

1. `fronts_platform` may be a property or descriptor, so the ATTRIBUTE
   LOOKUP can raise — and the lookup sat inside the absence handler. Probed
   with a raising property plus an empty snapshot: live=None, routed=False,
   verdict=None, i.e. an unattested target authorized. The previous test made
   an already-retrieved METHOD raise, so it could not reach this.

2. A present adapter with no usable `fronts_platform` returned None for the
   same reason. An adapter that cannot say what it fronts is broken, not
   absent, so it now raises too.

Also found by my own spot-check while the review ran: the nested imports of
`gateway.config` / `gateway.run` inside the live probe shared the broad
handler, so a broken installation degraded to the snapshot as well. Probed
with a healthy-adapter positive control in the same run — healthy refused
the unattested target, faulted authorized it. `_relay_fronted` one function
below already drew this exact distinction for its own import.

The boundary is now: `relay is None` is the ONLY absence. Everything about a
present adapter — attribute access, callability, the call itself, and the
imports needed to reach it — is a fault and raises RelayRouteUnknown.

This is the fourth variant of absence-vs-fault in this file and all four
were mine. The lesson is in the code as a comment rather than in a commit
message nobody re-reads.

Four controls keep genuine absence benign: no runner, no relay adapter in
the runner, a real ModuleNotFoundError naming the gateway package, and the
configured-attested-target-still-sends case.

434 passed via scripts/run_tests.sh; 9-row mutation ledger re-killed
including both new guards, none dying on an exception.

* fix(relay): invert the live probe to fail closed by default

Reviewer BLOCKER round 2, reproduced: reading the adapter registry can also
raise. A runner whose `adapters.get()` raised gave relay_present=True,
live=None, routed=False, verdict=None — unattested discord:999 authorized.

That was the FIFTH boundary in one function with the same defect: the call,
the attribute lookup, a non-callable attribute, the nested imports, and now
the registry lookup. Each round I patched the reported boundary and the
defect moved one statement up. The cause was the shape, not the statements:
the function asked "did something go wrong?" and answered None, and None
MEANS "no live adapter, use the config snapshot" — so every statement was a
new chance to fail open, and every new statement would have been too.

Inverted rather than patched a sixth time. Each `return None` now sits
behind an explicit narrow check that cannot itself be the fault (no runner,
no adapters, no relay key, gateway package genuinely absent), and one outer
handler turns anything else into RelayRouteUnknown. A statement added inside
this function is now fail-CLOSED by default.

Verified all six fault shapes raise (call, attribute, missing method,
registry .get, .adapters property, runner ref) and all five absence shapes
stay benign, plus a liveness control where the config snapshot disagrees
with a healthy adapter and the adapter still wins.

Four new tests, including the two absence controls that keep native-only and
CLI deployments working. 438 passed via scripts/run_tests.sh. Mutation
ledger: 8 killed. One survivor recorded as a proven equivalent mutant —
widening `if not registry` to `or {}` is behaviourally identical because
`{}.get()` returns None, i.e. the same absence; it is a readability guard.
2026-09-08 17:51:35 +10:00

2057 lines
110 KiB
Python

"""Inbound message pipeline (_handle_message, text/media preparation, durable-turn markers, plugin injection) for GatewayRunner.
Split out of ``gateway/run.py``; bound onto ``GatewayRunner`` via the MRO.
``gateway.run`` internals are imported lazily inside method bodies (import cycle),
so ``patch("gateway.run.X")`` keeps intercepting them at call time.
"""
from __future__ import annotations
import logging
from typing import TYPE_CHECKING
import asyncio
import concurrent.futures
import dataclasses
import json
import os
import re
import time
from contextlib import suppress
from gateway.config import Platform
from gateway.platforms.base import EphemeralReply
from gateway.platforms.event import MessageEvent, MessageType
from gateway.run_common import _UNSET
from gateway.session import (
SessionSource, is_shared_multi_user_session, neutralize_untrusted_inline_text
)
from gateway.turn_lease import TurnLeaseTimeoutError
from typing import Any, Dict, List, Optional, Tuple
if TYPE_CHECKING: # string annotations only; never imported at runtime (cycle)
from gateway.run import GatewayRunner # noqa: F401
from gateway.run_turn_runner import TurnRunner # noqa: F401
# Log-record parity with the origin module.
logger = logging.getLogger("gateway.run")
class GatewayInboundMixin:
"""Inbound message pipeline (_handle_message, text/media preparation, durable-turn markers, plugin injection) for GatewayRunner."""
def _hm_pre_gateway_dispatch_hook(
self, event: "MessageEvent", source: SessionSource
) -> Optional["MessageEvent"]:
"""Run the ``pre_gateway_dispatch`` plugin hook; None = drop, else the (maybe rewritten) event.
Results: ``{"action": "skip"}`` → drop; ``{"action": "rewrite", "text"}`` → replace ``event.text``;
``allow``/None → normal dispatch. Runs BEFORE auth so plugins can handle unauthorized senders."""
try:
from hermes_cli.lifecycle import invoke_hook as _invoke_hook
_hook_results = _invoke_hook(
"pre_gateway_dispatch", event=event, gateway=self,
# getattr: bare-runner tests build GatewayRunner via object.__new__ without __init__.
session_store=getattr(self, "session_store", None),
)
except Exception as _hook_exc:
logger.warning("pre_gateway_dispatch invocation failed: %s", _hook_exc)
_hook_results = []
for _result in _hook_results:
if not isinstance(_result, dict):
continue
_action = _result.get("action")
if _action == "skip":
logger.info(
"pre_gateway_dispatch skip: reason=%s platform=%s chat=%s",
_result.get("reason"), source.platform.value if source.platform else "unknown",
source.chat_id or "unknown",
)
return None
if _action == "rewrite":
_new_text = _result.get("text")
if isinstance(_new_text, str):
event = dataclasses.replace(event, text=_new_text)
break
if _action == "allow":
break
return event
async def _hm_offer_pairing_code(self, source: SessionSource) -> None:
"""DM an unauthorized sender a pairing code (rate-limited; groups never reach here)."""
platform_name = source.platform.value if source.platform else "unknown"
pairing_store = self._pairing_store_for(source)
if pairing_store is None:
logger.error("Cannot offer pairing code on %s: no pairing store", platform_name)
return
# Rate-limit ALL pairing responses (code or rejection) so a burst of DMs doesn't spam.
if pairing_store._is_rate_limited(platform_name, source.user_id):
return
code = pairing_store.generate_code(platform_name, source.user_id, source.user_name or "")
adapter = self._adapter_for_source(source)
if code:
store_profile = getattr(pairing_store, "profile", None)
profile_arg = (
f"-p {store_profile} "
if isinstance(store_profile, str) and store_profile and store_profile != "default"
else ""
)
reply = (
f"Hi~ I don't recognize you yet!\n\n"
f"Here's your pairing code: `{code}`\n\n"
f"Ask the bot owner to run:\n"
f"`hermes {profile_arg}pairing approve "
f"{platform_name} {code}`"
)
else:
reply = "Too many pairing requests right now~ Please try again later!"
if adapter:
await adapter.send(source.chat_id, reply)
if not code:
# Record rate limit so subsequent messages are silently ignored
pairing_store._record_rate_limit(platform_name, source.user_id)
async def _hm_admit_event(
self, event: "MessageEvent"
) -> Optional[Tuple["MessageEvent", SessionSource, bool]]:
"""Ingress gates for ``_handle_message``; None when dropped, else ``(event, source, is_internal)``
(the ``pre_gateway_dispatch`` hook may have rewritten ``event``)."""
from gateway.run import _is_slack_ignored_channel
source = event.source
# getattr(self, ...) throughout: bare test runners build GatewayRunner via object.__new__.
_config = getattr(self, "config", None)
# 🔴 Cross-session leak guard: this per-message task was create_task()'d with a copy of the
# spawning context, which may carry ANOTHER message's HERMES_SESSION_* ContextVars; until
# _set_session_env binds ours a subprocess would read the foreign identity. Reset to _UNSET.
try:
from gateway.session_context import reset_session_vars
reset_session_vars()
except Exception:
logger.debug("reset_session_vars failed at handler entry", exc_info=True)
# Most adapters resolve profile routes in build_source(); internal/voice paths construct
# SessionSource directly, so resolve those here as the shared fail-closed ingress gate.
# Strict boolean marker: require the literal True so duck-typed test/internal sources with
# dynamic attributes are not mistaken for a rejection.
if (
getattr(_config, "multiplex_profiles", False)
and not getattr(source, "profile", None)
and getattr(source, "profile_route_rejected", False) is not True
):
from gateway.profile_routing import ProfileRouteRejected
try:
source.profile = self._profile_name_for_source(source)
except ProfileRouteRejected:
source.profile_route_rejected = True
if getattr(source, "profile_route_rejected", False) is True:
logger.warning(
"Dropping inbound message because its explicit profile route "
"targets an unserved profile"
)
return None
is_internal = bool(getattr(event, "internal", False)) # e.g. background-process notifications
# Ignored-channel guard runs FIRST — before startup-restore queueing, plugin hooks, auth,
# and session setup — so an ignored channel can never reach pairing/auth/session state.
_chat_id = getattr(source, "chat_id", None)
if (
# See #51899.
not is_internal
and getattr(source, "platform", None) == Platform.SLACK
and _is_slack_ignored_channel(_config, _chat_id)
):
logger.info("Dropping Slack message from configured ignored channel %s", _chat_id)
return None
if (
getattr(self, "_startup_restore_in_progress", False)
and not is_internal
and not getattr(event, "_hermes_startup_restore_replay", False)
):
self._queue_startup_restore_event(event)
return None
if is_internal:
return event, source, True
# scale-to-zero: only real user-originated inbound stamps the last-inbound clock;
# counting internal/system events would keep a genuinely idle gateway awake.
self._scale_to_zero_note_real_inbound()
event = self._hm_pre_gateway_dispatch_hook(event, source)
if event is None:
return None
source = event.source
if not self._is_user_authorized_for_source(source):
if source.user_id is None:
# No user identity (Telegram service messages, channel forwards, anonymous admin
# posts, sender_chat): can't be paired but may be authorized via a chat allowlist.
logger.debug("Ignoring message with no user_id from %s", source.platform.value)
return None
logger.warning("Unauthorized user: %s (%s) on %s", source.user_id, source.user_name, source.platform.value)
# In DMs: offer pairing code. In groups: silently ignore.
if (
source.chat_type == "dm"
and self._get_unauthorized_dm_behavior(source.platform, profile=source.profile) == "pair"
):
await self._hm_offer_pairing_code(source)
return None
return event, source, False
def _hm_estop_turn_allowed(self, event: "MessageEvent", source: SessionSource) -> bool:
"""Whether a turn may bypass the global emergency stop: pause blocks NEW agent turns, never
running work or control traffic — recognized slash commands (incl. /pause off, the in-band
resume) and replies owned by in-flight work (pending update prompt, running session,
pending slash-confirm, dangerous-command approval) all pass through."""
with suppress(Exception):
_estop_cmd = event.get_command()
if _estop_cmd:
from hermes_cli.commands import resolve_command as _resolve_estop_cmd
if _resolve_estop_cmd(_estop_cmd) is not None:
return True
with suppress(Exception):
_estop_key = self._session_key_for_source(source)
_estop_state = self._peek_session_state(_estop_key)
if _estop_state is not None and _estop_state.persistent.update_prompt_pending:
return True
# A running session covers steering plus pending clarify / tool approvals it holds.
if self._is_session_running(_estop_key):
return True
from tools import slash_confirm as _estop_confirm_mod
if _estop_confirm_mod.get_pending(_estop_key):
return True
from tools.approval import has_blocking_approval as _estop_has_approval
if _estop_has_approval(_estop_key):
return True
return False
def _hm_estop_gate(
self, event: "MessageEvent", source: SessionSource, is_internal: bool
) -> Optional[str]:
"""Global emergency-stop (`hermes pause`) notice when this turn must be blocked, else None.
Placed after auth so unauthorized senders can't probe pause state."""
if is_internal:
return None
try:
from agent.estop import paused_reply as _estop_paused_reply
except ImportError:
return None
_paused_notice = _estop_paused_reply()
if _paused_notice is None or self._hm_estop_turn_allowed(event, source):
return None
logger.info(
"Gateway turn paused by global emergency stop (platform=%s chat=%s)",
getattr(getattr(source, "platform", None), "value", "unknown"),
getattr(source, "chat_id", None) or "unknown",
)
return _paused_notice
@staticmethod
def _hm_write_update_response(response_text: str) -> Optional[str]:
"""Atomically hand *response_text* to the detached update process; returns the OSError str."""
from gateway.run import _hermes_home
response_path = _hermes_home / ".update_response"
try:
tmp = response_path.with_suffix(".tmp")
tmp.write_text(response_text, encoding="utf-8")
tmp.replace(response_path)
(_hermes_home / ".update_prompt.json").unlink(missing_ok=True)
except OSError as e:
return str(e)
return None
def _hm_update_prompt_reply(self, event: "MessageEvent", _quick_key: str) -> Optional[str]:
"""Consume a reply to a pending ``/update`` prompt (routed to the detached update process via
``.update_response``); None when nothing was consumed. Recognized slash commands must bypass
this or /new, /help etc. get silently consumed as update answers."""
_up_state = self._peek_session_state(_quick_key)
if _up_state is None or not _up_state.persistent.update_prompt_pending:
return None
# Accept /approve and /deny as shorthand for yes/no
cmd = event.get_command()
_recognized_cmd = None
if cmd in {"approve", "yes"}:
response_text = "y"
elif cmd in {"deny", "no"}:
response_text = "n"
else:
if cmd:
with suppress(Exception):
from hermes_cli.commands import resolve_command as _resolve_update_cmd
_cmd_def = _resolve_update_cmd(cmd)
_recognized_cmd = _cmd_def.name if _cmd_def else None
response_text = "" if _recognized_cmd else (event.text or "").strip()
if response_text:
err = self._hm_write_update_response(response_text)
if err is not None:
logger.warning("Failed to write update response: %s", err)
return f"✗ Failed to send response to update process: {err}"
_up_state.persistent.update_prompt_pending = False
label = response_text if len(response_text) <= 20 else response_text[:20] + "…"
return f"✓ Sent `{label}` to the update process."
# Recognized slash command during a pending update prompt: write a blank response so the
# detached update's ``_gateway_prompt`` returns the prompt's default (typically a safe
# "n" / skip) and exits instead of blocking on stdin until the watcher timeout.
if _recognized_cmd:
err = self._hm_write_update_response("")
if err is None:
logger.info(
"Recognized /%s during pending update prompt for %s; "
"cancelled prompt with default and dispatching command",
_recognized_cmd, _quick_key,
)
else:
logger.warning("Failed to write cancel response for pending update prompt: %s", err)
_up_state.persistent.update_prompt_pending = False
return None
async def _hm_clarify_reply(
self, event: "MessageEvent", source: SessionSource, _quick_key: str
) -> Optional[str]:
"""Intercept a reply to a pending clarify prompt; None when the message falls through.
Free text answers open-ended/"Other" prompts; "2" answers a multi-choice one. Resolved/retained
replies return "" so adapters don't double-post — the agent produces the next user-facing message."""
try:
from tools import clarify_gateway as _clarify_mod
_pending_clarify = _clarify_mod.get_pending_for_session(_quick_key, include_choice_prompts=True)
except Exception:
return None
if _pending_clarify is None:
return None
_clarify_has_audio = bool(self._pending_event_audio_paths(event))
_raw_clarify_reply = await self._prepare_clarify_reply_text(event)
def _retain(why: str) -> str:
logger.info(
"Gateway retained pending clarify after %s (session=%s, id=%s)",
why, _quick_key, _pending_clarify.clarify_id,
)
return ""
if _clarify_has_audio and not _raw_clarify_reply:
return _retain("voice transcription produced no usable text")
# Slash commands: the user wanted a command, not to answer the clarify. Leave it pending so
# they can retry; on timeout the agent unblocks with an empty response.
if not _raw_clarify_reply or _raw_clarify_reply.startswith("/"):
return None
_text_outcome = _clarify_mod.attempt_text_response_for_session(_quick_key, _raw_clarify_reply)
if _text_outcome == _clarify_mod.TEXT_RESOLVED:
logger.info(
"Gateway intercepted clarify text response (session=%s, id=%s)",
_quick_key, _pending_clarify.clarify_id,
)
# The clarify callback pauses the platform typing/status indicator while waiting so
# Slack users can type; the active agent resumes now, so re-enable its indicator.
_clarify_adapter = self._adapter_for_source(source)
if _clarify_adapter:
try:
_clarify_adapter.resume_typing_for_chat(source.chat_id)
except Exception:
logger.debug("Failed to resume typing after clarify response", exc_info=True)
return ""
if _text_outcome == _clarify_mod.TEXT_REJECTED_SELECTION:
# Selection-shaped but invalid (out-of-range number, bad comma-list): keep the clarify
# armed for retry — don't cancel, don't treat as an unrelated follow-up.
return _retain("invalid selection attempt")
if _text_outcome == _clarify_mod.TEXT_REJECTED_PROSE:
# Native-choice prompts reject unmatched prose so it continues through normal busy
# routing. Release this clarify first: redirect() degrades to steer() while tools
# execute, and that steer cannot drain until the clarify tool returns.
_clarify_mod.resolve_gateway_clarify(_pending_clarify.clarify_id, "")
return None
# Reply → choice for a pending slash-confirm prompt; the command spelling wins over the
# bang/slash-stripped free-text spelling.
_SLASH_CONFIRM_CMD_CHOICES = {
"approve": "once", "yes": "once", "ok": "once", "confirm": "once",
"always": "always", "remember": "always",
"cancel": "cancel", "no": "cancel", "deny": "cancel", "nevermind": "cancel",
}
_SLASH_CONFIRM_TEXT_CHOICES = {
"approve": "once", "approve once": "once", "once": "once",
"always": "always", "always approve": "always",
"cancel": "cancel", "nevermind": "cancel", "no": "cancel",
}
async def _hm_slash_confirm_reply(self, event: "MessageEvent", _quick_key: str) -> Optional[str]:
"""Resolve a reply (/approve, /always, /cancel + aliases) to a pending slash-confirm prompt;
None when it falls through — a stale pending confirm does NOT block other commands. A pending
dangerous-command approval takes precedence: /approve there unblocks the waiting tool thread."""
from tools import slash_confirm as _slash_confirm_mod
_pending_confirm = _slash_confirm_mod.get_pending(_quick_key)
if not _pending_confirm:
return None
with suppress(Exception):
from tools.approval import has_blocking_approval
if has_blocking_approval(_quick_key):
return None
# Accept bang-prefixed replies (`!always`, `!cancel`) verbatim: Slack/Matrix show the `!`
# prefix (typed `/` is blocked in Slack threads) and adapters only rewrite
# `!<known-command>` — confirm keywords aren't commands, so the `!` survives to here.
_norm_reply = (event.text or "").strip().lstrip("!/").lower()
_confirm_choice = (
self._SLASH_CONFIRM_CMD_CHOICES.get(event.get_command())
or self._SLASH_CONFIRM_TEXT_CHOICES.get(_norm_reply)
)
if _confirm_choice is not None:
_resolved = await _slash_confirm_mod.resolve(
_quick_key, _pending_confirm.get("confirm_id"), _confirm_choice,
)
return _resolved or ""
# Stale pending + unrelated command: the user moved on, so drop the pending state rather
# than let the confirm block normal usage indefinitely.
_slash_confirm_mod.clear_if_stale(_quick_key)
return None
def _hm_evict_idle_stale_agent(self, _quick_key: str) -> None:
"""Evict a leaked lock from a hung/crashed handler: only when the agent has been *idle* past
the threshold (active tasks can run for hours), or has no activity tracker and an extreme
wall-clock age. The pending sentinel is never evicted (no get_activity_summary() → idle
reads inf and would race the async setup path)."""
from gateway.run import _AGENT_PENDING_SENTINEL, _float_env
_raw_stale_timeout = _float_env("HERMES_AGENT_TIMEOUT", 1800)
_quick_state = self._peek_session_state(_quick_key)
_stale_ts = _quick_state.turn.started_ts if _quick_state else 0
if _quick_state is None or _quick_state.turn.agent is None or not _stale_ts:
return
_stale_age = time.time() - _stale_ts
_stale_agent = _quick_state.turn.agent
_stale_idle = float("inf") # assume idle if we can't check
_stale_detail = ""
_activity_summary_valid = False
if _stale_agent and hasattr(_stale_agent, "get_activity_summary"):
with suppress(Exception):
_sa = _stale_agent.get_activity_summary()
from gateway.session_stall import resolve_session_idle_seconds_from_activity
_sa_d = _sa if isinstance(_sa, dict) else {}
_resolved_idle = resolve_session_idle_seconds_from_activity(
_sa if isinstance(_sa, dict) else None, now=time.time(),
)
if _resolved_idle is not None:
_stale_idle = _resolved_idle
_activity_summary_valid = True
_stale_detail = (
f" | last_activity={_sa_d.get('last_activity_desc', 'unknown')} "
f"({_stale_idle:.0f}s ago) "
f"| iteration={_sa_d.get('api_call_count', 0)}/{_sa_d.get('max_iterations', 0)}"
)
# A valid activity clock is authoritative: total age alone never makes an actively
# progressing turn stale. The emergency wall TTL is only a fallback when the agent cannot
# report usable activity.
_wall_ttl = max(_raw_stale_timeout * 10, 7200) if _raw_stale_timeout > 0 else float("inf")
_should_evict = _stale_agent is not _AGENT_PENDING_SENTINEL and (
(_activity_summary_valid and _raw_stale_timeout > 0 and _stale_idle >= _raw_stale_timeout)
or (not _activity_summary_valid and _stale_age > _wall_ttl)
)
if _should_evict:
logger.warning(
"Evicting stale _running_agents entry for %s "
"(age: %.0fs, idle: %.0fs, timeout: %.0fs)%s",
_quick_key, _stale_age, _stale_idle, _raw_stale_timeout, _stale_detail,
)
self._hm_evict_running_agent(_quick_key, "stale_running_agent_eviction")
def _hm_evict_reaped_agent(self, _quick_key: str) -> None:
"""Evict the in-memory turn slot of a session whose durable row was ended while the gateway
lived (``ws_orphan_reap`` / ``agent_close``): otherwise the fast-path queues every next
message into the dead runtime. The cold path re-attaches via ``get_or_create_session``."""
try:
# #99106: durable-reaped guard. This is the live-gateway variant of #54878 and the #632
# detached/ 405 suppressions in production. Evict the stale slot so the next message falls
# through to the cold path and re-attaches or creates a fresh session; /status then correctly
# shows 代理运行中: 否 before the heal and a live turn after.
_reap_store = getattr(self, "session_store", None)
# Public, lock-held accessors: peek_session_id returns a non-str on stubbed stores in
# bare test runners — the isinstance() / ``is True`` gates keep this inert unless a
# real SessionStore answers.
_reap_peek = getattr(_reap_store, "peek_session_id", None)
_is_ended = getattr(_reap_store, "_is_session_ended_in_db", None)
_reap_sid = _reap_peek(_quick_key) if callable(_reap_peek) else None
if isinstance(_reap_sid, str) and _reap_sid and callable(_is_ended) and _is_ended(_reap_sid) is True:
logger.warning(
"Evicting stale _running_agents entry for %s — "
"durable session %s is ended (reaped) in state.db; "
"healing routing on next message (#99106)", _quick_key, _reap_sid,
)
self._hm_evict_running_agent(_quick_key, "reaped_session_eviction")
except Exception:
logger.debug("reaped-session staleness check failed", exc_info=True)
def _hm_evict_running_agent(self, _quick_key: str, reason: str) -> None:
self._invalidate_session_run_generation(_quick_key, reason=reason)
self._release_running_agent_state(_quick_key)
def _hm_merge_pending_for_source(
self, source: SessionSource, _quick_key: str, event: "MessageEvent", *, merge_text: bool = False
) -> None:
"""Merge *event* into the source adapter's pending slot (no-op without an adapter)."""
from gateway.platforms.base import merge_pending_message_event
adapter = self._adapter_for_source(source)
if adapter:
merge_pending_message_event(adapter._pending_messages, _quick_key, event, merge_text=merge_text)
async def _hm_busy_slash_or_photo(
self, event: "MessageEvent", source: SessionSource, _quick_key: str
) -> Tuple[bool, Optional[str]]:
"""Slash-command / photo-burst handling on the busy fast-path → ``(handled, result)``. Each
command's mid-run behavior is declared on its CommandDef (busy_policy / busy_handler)."""
from hermes_cli.commands import resolve_command as _resolve_cmd_inner
_evt_cmd = event.get_command()
_cmd_def_inner = _resolve_cmd_inner(_evt_cmd) if _evt_cmd else None
if _cmd_def_inner:
# /status and /context are intentionally pre-gate so users always see session state.
if _cmd_def_inner.name == "status":
return True, await self._handle_status_command(event)
if _cmd_def_inner.name == "context":
return True, await self._handle_context_command(event)
# Slash access control mirrors the cold-path gate so non-admins can't bypass gating
# just because an agent is busy. /help and /whoami are the always-allowed floor.
_denied = self._check_slash_access(source, _cmd_def_inner.name)
if _denied is not None:
return True, _denied
# Any recognized slash command dispatches per its declared busy_policy (dispatch /
# interrupt_then_dispatch / reject). Unrecognized commands and plain text fall through.
return True, await self._dispatch_busy_slash_command(event, _cmd_def_inner, _quick_key, source)
# Telegram photo bursts arrive as near-simultaneous updates — never interrupt for a
# photo-only follow-up; adapter-level batching absorbs them.
if event.message_type == MessageType.PHOTO:
logger.debug("PRIORITY photo follow-up for session %s — queueing without interrupt", _quick_key)
self._hm_merge_pending_for_source(source, _quick_key, event)
return True, None
return False, None
def _hm_busy_telegram_grace_queue(
self, event: "MessageEvent", source: SessionSource, _quick_key: str, effective_busy_input_mode: str
) -> bool:
"""Queue a Telegram text follow-up that lands within the post-start grace window."""
_grace = float(os.getenv("HERMES_TELEGRAM_FOLLOWUP_GRACE_SECONDS", "3.0"))
_grace_state = self._peek_session_state(_quick_key)
_started_at = _grace_state.turn.started_ts if _grace_state else 0
if not (
source.platform == Platform.TELEGRAM and event.message_type == MessageType.TEXT
and _grace > 0 and _started_at and (time.time() - _started_at) <= _grace
):
return False
logger.debug(
"Telegram follow-up arrived %.2fs after run start for %s — queueing without interrupt",
time.time() - _started_at, _quick_key,
)
if effective_busy_input_mode != "queue":
self._hm_merge_pending_for_source(source, _quick_key, event, merge_text=True)
else:
adapter = self._adapter_for_source(source)
if adapter:
self._enqueue_fifo(_quick_key, event, adapter)
return True
@staticmethod
def _hm_text_only(event: "MessageEvent") -> bool:
return event.message_type == MessageType.TEXT and not event.media_urls and not event.media_types
def _hm_busy_steer(self, event: "MessageEvent", running_agent: Any, _quick_key: str) -> None:
"""Steer mode: inject text mid-run via ``agent.steer()``, else fall back to queue semantics."""
steer_text = (event.text or "").strip()
steered = False
if self._hm_text_only(event) and steer_text and hasattr(running_agent, "steer"):
try:
steered = bool(running_agent.steer(self._steer_text_with_origin(steer_text, event)))
except Exception as exc:
logger.warning("PRIORITY steer failed for session %s: %s", _quick_key, exc)
if steered:
logger.debug("PRIORITY steer for session %s", _quick_key)
return
logger.debug("PRIORITY steer-fallback-to-queue for session %s", _quick_key)
self._queue_or_replace_pending_event(_quick_key, event)
async def _hm_busy_interrupt(
self, event: "MessageEvent", source: SessionSource, running_agent: Any, _quick_key: str
) -> None:
"""Interrupt path: redirect text-only corrections when supported, else ``agent.interrupt()``."""
from gateway.run import _build_media_placeholder
# Text-only corrections redirect the live turn (preserving displayed context) when the
# runtime supports it; media/voice and older runtimes use the interrupt path below.
_can_redirect = getattr(running_agent, "_supports_active_turn_redirect", False) is True
if self._hm_text_only(event) and _can_redirect and hasattr(running_agent, "redirect"):
try:
if running_agent.redirect(
self._steer_text_with_origin((event.text or "").strip(), event)
):
logger.debug("PRIORITY redirect for session %s", _quick_key)
return
except Exception as exc:
logger.warning("PRIORITY redirect failed for session %s: %s", _quick_key, exc)
logger.debug("PRIORITY interrupt for session %s", _quick_key)
_interrupt_text = event.text
if self._pending_event_audio_paths(event):
_interrupt_text, _ = await self._transcribe_and_echo_pending_voice(
event, self._adapter_for_source(source), source, event.text or "",
log_context="Voice-priority-interrupt",
)
elif not _interrupt_text and getattr(event, "media_urls", None):
_interrupt_text = _build_media_placeholder(event)
# Delivered via adapter._pending_messages (read by _run_agent); never also buffered on self
# — that copy was never consumed and grew unbounded.
running_agent.interrupt(_interrupt_text)
async def _hm_handle_running_session_message(
self, event: "MessageEvent", source: SessionSource, _quick_key: str
) -> Optional[str]:
"""Fast-path while this session's agent is running: interrupt by default (minimal latency);
busy_input_mode queue/steer, subagent and compression protection demote to queue."""
from gateway.run import _AGENT_PENDING_SENTINEL
_handled, _result = await self._hm_busy_slash_or_photo(event, source, _quick_key)
if _handled:
return _result
effective_busy_input_mode = self._effective_busy_input_mode(source)
if self._hm_busy_telegram_grace_queue(event, source, _quick_key, effective_busy_input_mode):
return None
_ra_state = self._peek_session_state(_quick_key)
running_agent = _ra_state.turn.agent if _ra_state else None
if running_agent is _AGENT_PENDING_SENTINEL: # agent still being set up
if event.get_command() == "stop": # force-clean the sentinel so the session is unlocked
self._release_running_agent_state(_quick_key)
logger.info("HARD STOP (pending) for session %s — sentinel cleared", _quick_key)
return EphemeralReply("⚡ Force-stopped. The agent was still starting — session unlocked.")
self._hm_merge_pending_for_source(source, _quick_key, event, merge_text=True) # picked up after start
return None
if self._draining:
queue_during_drain = self._queue_during_drain_enabled(effective_busy_input_mode)
if queue_during_drain:
self._queue_or_replace_pending_event(_quick_key, event)
return (
f"⏳ Gateway {self._status_action_gerund()} — queued for the next turn after it comes back."
if queue_during_drain
else f"⏳ Gateway is {self._status_action_gerund()} and is not accepting another turn right now."
)
if effective_busy_input_mode == "queue":
logger.debug("PRIORITY queue follow-up for session %s", _quick_key)
self._queue_or_replace_pending_event(_quick_key, event)
return None
if effective_busy_input_mode == "steer":
self._hm_busy_steer(event, running_agent, _quick_key)
return None
# Subagent protection: an interrupt cascades through ``_active_children`` and aborts
# in-flight delegate_task work (/stop reached its handler above — still an escape hatch).
# Compression protection: an interrupt would start a new turn on the pre-rotation parent
# while compression rotates the id away, forking orphaned siblings.
if self._agent_has_active_subagents(running_agent):
_demote = "because the running agent has active subagents (#30170)"
elif await self._session_has_compression_in_flight(_quick_key):
_demote = "because context compression is in flight (#56391)"
else:
await self._hm_busy_interrupt(event, source, running_agent, _quick_key)
return None
logger.info("PRIORITY interrupt demoted to queue for session %s %s", _quick_key, _demote)
self._queue_or_replace_pending_event(_quick_key, event)
return None
def _hm_quick_commands(self) -> dict:
"""User-defined ``quick_commands`` mapping from config (empty dict when unset/malformed)."""
cfg = self.config
qc = (cfg.get("quick_commands") if isinstance(cfg, dict) else getattr(cfg, "quick_commands", None)) or {}
return qc if isinstance(qc, dict) else {}
@staticmethod
def _hm_expand_alias_quick_command(event: "MessageEvent", qcmd: dict) -> Optional[str]:
"""Rewrite ``event.text`` to an alias quick command's target; returns the new command name."""
target = (qcmd.get("target") or "").strip()
if not target:
return None
target = target if target.startswith("/") else f"/{target}"
event.text = f"{target} {event.get_command_args().strip()}".strip()
target_command = target.lstrip("/")
return target_command.split()[0] if target_command else target_command
async def _hm_command_hooks(
self, event: "MessageEvent", source: SessionSource, _quick_key: str, command: str, canonical: str
) -> Tuple[bool, Optional[str], Optional[str]]:
"""Fire ``pre_command`` (observer) and ``command:<canonical>`` (interceptor) hooks →
``(handled, result, new_command)`` (``new_command`` set when a handler rewrote the command).
The running-agent path deliberately does NOT fire these — a slow or hostile plugin must not
interfere with the operator's escape hatches for a live agent."""
raw_args = event.get_command_args().strip()
platform = source.platform.value if source.platform else ""
try:
from hermes_cli.plugins import fire_pre_command_hook
fire_pre_command_hook(
surface="gateway", command=str(canonical), alias_used=str(command),
args_raw=raw_args, session_key=_quick_key, platform=platform,
)
except Exception as _pre_cmd_err:
logger.debug("pre_command hook dispatch failed (non-fatal): %s", _pre_cmd_err)
# Handlers may return ``{"decision": "deny" | "handled" | "rewrite", ...}`` to intercept
# dispatch; handlers returning nothing behave as plain observers.
hook_ctx = {
"platform": platform, "user_id": source.user_id, "command": canonical,
"raw_command": command, "args": raw_args, "raw_args": raw_args,
}
try:
hook_results = await self.hooks.emit_collect(f"command:{canonical}", hook_ctx)
except Exception as _hook_err:
logger.debug("command:%s hook dispatch failed (non-fatal): %s", canonical, _hook_err)
hook_results = []
for hook_result in hook_results:
if not isinstance(hook_result, dict):
continue
decision = str(hook_result.get("decision", "")).strip().lower()
message = hook_result.get("message")
message = message if isinstance(message, str) and message else None
if decision == "deny":
return True, message or f"Command `/{command}` was blocked by a hook.", None
if decision == "handled":
return True, message, None
if decision == "rewrite":
new_command = str(hook_result.get("command_name", "")).strip().lstrip("/")
if new_command:
event.text = f"/{new_command} {str(hook_result.get('raw_args', '')).strip()}".strip()
return False, None, event.get_command()
return False, None, None
async def _hm_resolve_command(
self, event: "MessageEvent", source: SessionSource, _quick_key: str
) -> Tuple[bool, Optional[str], Optional[str], Optional[str]]:
"""Resolve the slash command (aliases, access gate, hooks) → ``(handled, result, command,
canonical)``; when ``handled`` the caller returns ``result`` as-is (may be None)."""
from hermes_cli.commands import is_gateway_known_command, resolve_command as _resolve_cmd
def _canon(cmd):
# Aliases resolve to the canonical name so dispatch and hook names don't depend on them.
_def = _resolve_cmd(cmd) if cmd else None
return _def, (_def.name if _def else cmd)
command = event.get_command()
_cmd_def, canonical = _canon(command)
# Expand alias quick commands before built-in dispatch so targets like /model openai/gpt-5.5
# --provider openrouter reach the /model handler. Built-ins keep precedence: aliases only
# need early handling when the typed command is not already known.
if command and _cmd_def is None:
qcmd = self._hm_quick_commands().get(command)
if qcmd is not None and qcmd.get("type") == "alias":
new_command = self._hm_expand_alias_quick_command(event, qcmd)
if new_command is not None:
command = new_command
_cmd_def, canonical = _canon(command)
if not (command and canonical and is_gateway_known_command(canonical)):
return False, None, command, canonical
# Per-platform slash access control: only active when the operator set ``allow_admin_from``
# for the source's scope; then non-admins get ``user_allowed_commands`` plus the
# /help, /whoami floor. Plain chat is never gated.
_denied = self._check_slash_access(source, canonical)
if _denied is not None:
return True, _denied, command, canonical
_handled, _result, new_command = await self._hm_command_hooks(
event, source, _quick_key, command, canonical
)
if _handled:
return True, _result, command, canonical
if new_command is not None:
command = new_command
_cmd_def, canonical = _canon(command)
return False, None, command, canonical
async def _hm_confirm_destructive(self, event, command: str, detail: str, handler) -> Tuple[bool, Optional[str]]:
async def _execute():
return await handler(event)
return True, await self._maybe_confirm_destructive_slash(
event=event, command=command, title=f"/{command}", detail=detail, execute=_execute,
)
async def _hm_cmd_new(self, event, source, _quick_key):
if await asyncio.to_thread(self._is_telegram_topic_root_lobby, source):
return True, self._telegram_topic_root_new_message()
return await self._hm_confirm_destructive(
event, "new", "This starts a fresh session and discards the current conversation history.",
self._handle_reset_command,
)
async def _hm_cmd_start(self, event, source, _quick_key):
logger.info("Ignoring /start platform ping for session %s", _quick_key)
return True, ""
async def _hm_cmd_egress(self, event, source, _quick_key):
from hermes_cli.proxy_cli import format_status_text
return True, format_status_text()
async def _hm_rewrite_turn_to_prompt(self, event, source, name: str, ack: str, build) -> Tuple[bool, Optional[str]]:
"""Ack, then rewrite the turn to ``build()`` and fall through to the agent (keeps role
alternation; works on any backend). A failing builder replies with a retry hint."""
await self._send_command_ack(source, ack, name)
try:
event.text = build()
except Exception:
return True, f"Could not start /{name} — please try again."
return False, None
# /learn and /plan: ack, rewrite the turn to a builder prompt, fall through to the agent.
async def _hm_cmd_learn(self, event, source, _quick_key):
from agent.learn_prompt import build_learn_prompt
req = event.get_command_args().strip()
_ack = f"Learning a skill from {'what you described' if req else 'this conversation'}…"
return await self._hm_rewrite_turn_to_prompt(event, source, "learn", _ack, lambda: build_learn_prompt(req))
async def _hm_cmd_plan(self, event, source, _quick_key):
from agent.plan_prompt import build_plan_prompt
task = event.get_command_args().strip()
_ack = f"Planning: {task[:80]}{'…' if len(task) > 80 else ''}" if task else "Planning from this conversation's context…"
return await self._hm_rewrite_turn_to_prompt(event, source, "plan", _ack, lambda: build_plan_prompt(task))
async def _hm_cmd_init(self, event, source, _quick_key):
# /init builds the prompt first: the ack wording depends on whether AGENTS.md exists.
from hermes_cli.init_command import build_init_prompt_for_cwd
try:
_init_prompt = build_init_prompt_for_cwd(extra=event.get_command_args().strip())
except Exception:
return True, "Could not start /init — please try again."
_ack = (
"Updating AGENTS.md from a project scan…"
if "UPDATE the existing AGENTS.md" in _init_prompt
else "Generating AGENTS.md from a project scan…"
)
await self._send_command_ack(source, _ack, "init")
event.text = _init_prompt
return False, None
async def _hm_cmd_blueprint(self, event, source, _quick_key):
_blueprint_result = await self._handle_blueprint_command(event)
_text = getattr(_blueprint_result, "text", "") or ""
_blueprint_seed = getattr(_blueprint_result, "agent_seed", None)
if not _blueprint_seed:
return True, _text or None
# Blueprint matched — rewrite the turn to the seed and fall through so the agent collects
# each slot value conversationally, then calls the cronjob tool (the /steer pattern).
if _text:
await self._send_command_ack(source, _text, "blueprint")
try:
event.text = _blueprint_seed
except Exception:
return True, _text or None
return False, None
async def _hm_cmd_undo(self, event, source, _quick_key):
_undo_n = 1
_undo_raw = event.get_command_args().strip()
if _undo_raw:
with suppress(ValueError, IndexError):
_undo_n = max(1, int(_undo_raw.split()[0]))
_undo_detail = (
"This removes the last user/assistant exchange from history."
if _undo_n == 1
else f"This removes the last {_undo_n} user turns from history."
)
return await self._hm_confirm_destructive(event, "undo", _undo_detail, self._handle_undo_command)
# /queue and /steer on the idle path: no agent is running, so strip the prefix and send the
# payload as a regular user turn; an empty payload surfaces the usage hint.
async def _hm_cmd_queue(self, event, source, _quick_key):
return self._hm_send_payload_as_turn(event, "Usage: /queue <prompt>")
async def _hm_cmd_steer(self, event, source, _quick_key):
return self._hm_send_payload_as_turn(
event, "Usage: /steer <prompt> (no agent is running; sending as a normal message)"
)
@staticmethod
def _hm_send_payload_as_turn(event, usage: str) -> Tuple[bool, Optional[str]]:
payload = event.get_command_args().strip()
if not payload:
return True, usage
with suppress(Exception):
event.text = payload
return False, None
async def _hm_cmd_moa(self, event, source, _quick_key):
# /moa is one-shot sugar only: run a single prompt through the default MoA preset, then
# restore the prior model. To *switch* to a MoA preset for the session, pick it from the
# model picker (MoA presets surface as a virtual "Mixture of Agents" provider).
from hermes_cli.moa_config import moa_usage, normalize_moa_config
from hermes_cli.config import load_config
moa_payload = event.get_command_args().strip()
if not moa_payload:
return True, moa_usage()
try:
cfg = load_config()
moa_cfg = normalize_moa_config(cfg.get("moa") if isinstance(cfg, dict) else {})
except Exception:
moa_cfg = normalize_moa_config({})
try:
event.text = moa_payload
_moa_state = self._session_state(_quick_key)
event._moa_restore_override = _moa_state.conversation.model_override
_moa_state.conversation.model_override = {
"provider": "moa", "model": moa_cfg["default_preset"], "base_url": "moa://local",
"api_key": "moa-virtual-provider", "api_mode": "chat_completions",
}
self._evict_cached_agent(_quick_key)
event._moa_disable_after_turn = True
except Exception:
return True, "Failed to prepare MoA turn."
return False, None
# Idle-path built-ins with bespoke flow (confirmations, prompt rewrites, one-shot MoA), each
# handled by ``_hm_cmd_<name>`` → ``(handled, result)``; ``(False, None)`` falls through to the agent.
_HM_CANONICAL_COMMANDS = frozenset({
"new", "start", "egress", "learn", "plan", "init", "blueprint", "undo", "queue", "steer", "moa",
})
async def _hm_dispatch_canonical_command(
self, event: "MessageEvent", source: SessionSource, _quick_key: str,
canonical: Optional[str],
) -> Tuple[bool, Optional[str]]:
"""Dispatch built-in idle-path commands → ``(handled, result)``; prompt-rewriting commands
mutate ``event.text`` and return ``(False, None)`` to fall through to the agent."""
plain_handler = (
self._gateway_plain_command_handlers().get(canonical)
or self._gateway_idle_command_handlers().get(canonical)
)
if plain_handler is not None:
return True, await plain_handler(event)
if canonical in self._HM_CANONICAL_COMMANDS:
return await getattr(self, f"_hm_cmd_{canonical}")(event, source, _quick_key)
return False, None
async def _hm_run_exec_quick_command(self, command: str, exec_cmd: str) -> str:
"""Run a ``type: exec`` quick command in the gateway process (30 s cap, sanitized env — the
gateway process has every API key in os.environ; output is redacted too)."""
try:
from tools.environments.local import build_subprocess_env
proc = await asyncio.create_subprocess_shell(
exec_cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE,
env=build_subprocess_env(),
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=30)
output = (stdout or stderr).decode().strip()
if output:
from agent.redact import redact_sensitive_text
output = redact_sensitive_text(output)
return output or "Command returned no output."
except asyncio.TimeoutError:
return "Quick command timed out (30s)."
except Exception as e:
return f"Quick command error: {e}"
async def _hm_dispatch_quick_and_plugin_commands(
self, event: "MessageEvent", source: SessionSource, command: Optional[str]
) -> Tuple[bool, Optional[str], Optional[str]]:
"""Drain gate, user-defined quick commands (exec/alias) and plugin slash commands →
``(handled, result, command)``; an alias quick command rewrites ``command``."""
if self._draining:
return True, f"⏳ Gateway is {self._status_action_gerund()} and is not accepting new work right now.", command
# User-defined quick commands (bypass agent loop, no LLM call)
qcmd = self._hm_quick_commands().get(command) if command else None
if qcmd is not None:
# Quick commands are slash capabilities too — and type:exec ones run a shell command in
# the gateway process. They are never in the registry, so the early gate never fires for
# them; apply the same admin/user policy to the raw typed name here.
# The early gate above only fires for registry-known commands, so quick commands (never in the
# registry) would otherwise reach this dispatch sink unchecked. (#44727)
_denied = self._check_slash_access(source, command)
if _denied is not None:
return True, _denied, command
qtype = qcmd.get("type")
if qtype == "exec":
exec_cmd = qcmd.get("command", "")
if not exec_cmd:
return True, f"Quick command '/{command}' has no command defined.", command
return True, await self._hm_run_exec_quick_command(command, exec_cmd), command
if qtype != "alias":
return True, f"Quick command '/{command}' has unsupported type (supported: 'exec', 'alias').", command
new_command = self._hm_expand_alias_quick_command(event, qcmd)
if new_command is None:
return True, f"Quick command '/{command}' has no target defined.", command
command = new_command # Fall through to normal command dispatch below
# Plugin-registered slash commands. Underscores normalize to hyphens so Telegram's
# underscored autocomplete form matches plugin commands registered with hyphens.
if command:
try:
from hermes_cli.plugins import get_plugin_command_handler
plugin_handler = get_plugin_command_handler(command.replace("_", "-"))
if plugin_handler:
result = plugin_handler(event.get_command_args().strip())
if asyncio.iscoroutine(result):
result = await result
return True, str(result) if result else None, command
except Exception as e:
logger.warning("Plugin command dispatch failed: %s", e)
return False, None, command
def _hm_bundle_slash_rewrite(
self, event: "MessageEvent", source: SessionSource, _quick_key: str, command: str
) -> bool:
"""Rewrite ``/<bundle>`` to the bundle invocation message; True when handled.
Skill bundles take precedence over individual skill commands (mirrors CLI dispatch)."""
try:
from agent.skill_bundles import (
build_bundle_invocation_message, resolve_bundle_command_key
)
bundle_key = resolve_bundle_command_key(command)
if bundle_key is None:
return False
# Pass the platform explicitly: bundle skill loading bypasses get_skill_commands()'
# scan-time disabled filter, and one gateway process serves several platforms, so
# env-var platform resolution can't be trusted here.
# Mirrors the stacked-skill gate (#58888).
bundle_result = build_bundle_invocation_message(
bundle_key, event.get_command_args().strip(), task_id=_quick_key,
platform=source.platform.value if source.platform else None,
)
if not bundle_result:
return False
event.text, _loaded, missing = bundle_result
if missing:
logger.info("Bundle %s skipped missing skills: %s", bundle_key, ", ".join(missing))
return True # Fall through to normal message processing with bundle content
except Exception as exc:
logger.warning("Bundle dispatch failed: %s", exc)
return False
@staticmethod
def _hm_unknown_slash_reply(command: str, source: SessionSource) -> Optional[str]:
"""Reply for a /command that is not built-in/plugin/skill; None when it is known."""
from gateway.run import _check_unavailable_skill
from hermes_cli.commands import GATEWAY_KNOWN_COMMANDS
# Known-but-disabled or uninstalled skill → actionable guidance.
_unavail_msg = _check_unavailable_skill(command)
if _unavail_msg:
return _unavail_msg
# Genuinely unrecognized: warn instead of forwarding to the LLM as free text (it invents
# tool calls). Normalize to hyphenated form first: the quick-command block may have set an
# alias target, so the resolved def can be stale.
if command.replace("_", "-") in GATEWAY_KNOWN_COMMANDS:
return None
logger.warning(
"Unrecognized slash command /%s from %s — replying with unknown-command notice",
command, source.platform.value if source.platform else "?",
)
return (
f"Unknown command `/{command}`. "
f"Type /commands to see what's available, "
f"or resend without the leading slash to send "
f"as a regular message."
)
def _hm_skill_slash_rewrite(
self, event: "MessageEvent", source: SessionSource, _quick_key: str, command: Optional[str]
) -> Optional[str]:
"""Rewrite ``/<bundle>`` / ``/<skill>`` invocations into the skill prompt on ``event.text``;
returns a reply string when the command is disabled/unknown/failed, else None.
resolve_skill_command_key() handles the Telegram underscore/hyphen round-trip (/claude_code)."""
if not command or self._hm_bundle_slash_rewrite(event, source, _quick_key, command):
return None
try:
from agent.skill_commands import (
get_skill_commands, build_skill_invocation_message, resolve_skill_command_key
)
skill_cmds = get_skill_commands()
cmd_key = resolve_skill_command_key(command)
if cmd_key is None:
return self._hm_unknown_slash_reply(command, source)
_plat = source.platform.value if source.platform else None
user_instruction = event.get_command_args().strip()
# Stacked slash-skill invocations: `/skill-a /skill-b do XYZ` loads every leading skill
# (up to 5), not just the first. Mirrors CLI.
try:
from agent.skill_commands import (
build_stacked_skill_invocation_message as _build_stacked,
split_stacked_skill_commands,
)
extra_keys, stacked_instruction = split_stacked_skill_commands(user_instruction)
except Exception:
_build_stacked = None
extra_keys, stacked_instruction = [], user_instruction
_skill_name = skill_cmds[cmd_key].get("name", "")
if _plat and (_skill_name or extra_keys):
# Per-platform disabled check: get_skill_commands() only applies the *global*
# disabled list at scan time (process-global cache across platforms), and
# split_stacked_skill_commands() only checks each extra token is a KNOWN skill.
from agent.skill_utils import get_disabled_skill_names as _get_plat_disabled
_plat_disabled = _get_plat_disabled(platform=_plat)
if _skill_name and _skill_name in _plat_disabled:
return (
f"The **{_skill_name}** skill is disabled for {_plat}.\n"
f"Enable it with: `hermes skills config`"
)
_disabled_extra = [
skill_cmds.get(k, {}).get("name", "")
for k in extra_keys
if skill_cmds.get(k, {}).get("name", "") in _plat_disabled
]
if _disabled_extra:
return (
f"The **{', '.join(_disabled_extra)}** skill(s) in this "
f"stacked invocation are disabled for {_plat}.\n"
f"Enable them with: `hermes skills config`"
)
if extra_keys and _build_stacked is not None:
stacked_result = _build_stacked(
[cmd_key, *extra_keys], stacked_instruction, task_id=_quick_key,
)
if not stacked_result:
return f"Failed to load stacked skills for /{command}."
event.text, _loaded, _missing = stacked_result
else:
msg = build_skill_invocation_message(cmd_key, user_instruction, task_id=_quick_key)
if msg:
event.text = msg
# Fall through to normal message processing with skill content
except Exception as e:
logger.debug("Skill command check failed (non-fatal): %s", e)
return None
async def _hm_pending_reply_intercepts(
self, event: "MessageEvent", source: SessionSource, _quick_key: str
) -> Optional[str]:
"""Replies owned by in-flight work: pending /update prompt, clarify, slash-confirm.
Only events that may control the gateway (``allow_gateway_control``) can answer them."""
if not event.allow_gateway_control:
return None
_reply = self._hm_update_prompt_reply(event, _quick_key)
if _reply is None:
_reply = await self._hm_clarify_reply(event, source, _quick_key)
if _reply is None:
_reply = await self._hm_slash_confirm_reply(event, _quick_key)
return _reply
async def _hm_dispatch_idle_commands(
self, event: "MessageEvent", source: SessionSource, _quick_key: str
) -> Tuple[bool, Optional[str]]:
"""Idle path: resolve + dispatch slash commands; rewriting commands fall through to the agent."""
_handled, _result, command, canonical = await self._hm_resolve_command(event, source, _quick_key)
if not _handled:
_handled, _result = await self._hm_dispatch_canonical_command(event, source, _quick_key, canonical)
if not _handled:
_handled, _result, command = await self._hm_dispatch_quick_and_plugin_commands(event, source, command)
if not _handled:
_result = self._hm_skill_slash_rewrite(event, source, _quick_key, command)
_handled = _result is not None
return _handled, _result
def _hm_rescue_orphaned_fifo(
self, event: "MessageEvent", source: SessionSource, is_internal: bool, _quick_key: str
) -> Tuple["MessageEvent", SessionSource, bool]:
"""FIFO orphan rescue: a session that went idle with a populated overflow (post-turn drain
never promoted, e.g. a compression-demoted follow-up) silently orphaned those events. The
oldest orphan runs as THIS turn and the incoming event is parked behind the chain. Skipped
for control commands and internal events."""
try:
# ── FIFO orphan rescue (#99882) ──────────────────────────────── If this session went idle with
# a populated overflow (queued during a busy window whose post-turn drain never promoted — e.g.
# a compression-demoted follow-up after the compression window ended through an exit that
# skipped the promotion site), those events were silently orphaned. We are starting the next
# turn for this session NOW: re-stage the orphans in FIFO order and enqueue the incoming event
# behind them, so arrival order (#28503) holds: oldest orphan runs as this turn, the rest drain
# in order, the new message last.
_orphan_adapter = self._adapter_for_source(source)
if _orphan_adapter is None or getattr(event, "internal", False) or event.get_command():
return event, source, is_internal
_rescued = self._rescue_orphaned_overflow(_quick_key, _orphan_adapter)
if _rescued is None:
return event, source, is_internal
# Into the slot when the chain was a single orphan (post-turn drain picks it up),
# otherwise into overflow behind the already-staged next orphan.
self._enqueue_fifo(_quick_key, event, _orphan_adapter)
# Same session key by construction; carry the orphan's own source so reply anchors /
# thread metadata point at the message actually being answered.
_rescued_source = getattr(_rescued, "source", None)
source = _rescued_source if _rescued_source is not None else source
return _rescued, source, bool(getattr(_rescued, "internal", False))
except Exception:
logger.debug("FIFO orphan rescue pre-claim failed for %s", _quick_key, exc_info=True)
return event, source, is_internal
async def _handle_message(self, event: MessageEvent) -> Optional[str]:
"""Handle an incoming message from any platform: auth → command check → running-agent
interrupt → get/create session → build context → run agent → return response."""
from gateway.run import _AGENT_PENDING_SENTINEL
_admitted = await self._hm_admit_event(event)
if _admitted is None:
return None
event, source, is_internal = _admitted
# TERMINAL-DECLINE LATCH TEARDOWN. Deliberately placed AFTER admission,
# not on the adapter's raw inbound: profile routing, the ignored-channel
# guard, plugin hooks and user authorization all reject events above,
# and a rejected event must not be able to clear a refusal belonging to
# an active turn. This is also the single entry point every lane shares
# — Discord interaction passthrough builds its own MessageEvent and
# calls handle_message directly, so a teardown on the relay's inbound
# handler left those turns muted.
_paused_notice = self._hm_estop_gate(event, source, is_internal)
if _paused_notice is not None:
return _paused_notice
_quick_key = self._session_key_for_source(source)
_reply = await self._hm_pending_reply_intercepts(event, source, _quick_key)
if _reply is not None:
return _reply
# Evict a leaked/reaped ``_running_agents`` slot before the busy-session fast-path.
self._hm_evict_idle_stale_agent(_quick_key)
if self._is_session_running(_quick_key):
self._hm_evict_reaped_agent(_quick_key)
if self._is_session_running(_quick_key):
return await self._hm_handle_running_session_message(event, source, _quick_key)
_handled, _result = await self._hm_dispatch_idle_commands(event, source, _quick_key)
if _handled:
return _result
# Pending exec approvals go through /approve and /deny only — no bare-text matching, or a
# conversational "yes" would execute a dangerous command.
if not is_internal:
if await asyncio.to_thread(self._is_telegram_topic_root_lobby, source):
# Debounced so a user who forgets about topic mode doesn't get ten reminders.
if self._should_send_telegram_lobby_reminder(source):
return self._telegram_topic_root_lobby_message()
return None
# External-drain new-turn gate: when NAS engaged an external drain (.drain_request.json,
# seen by _drain_control_watcher), refuse to START new turns so the in-flight set can
# only fall to zero. Reversible.
if self._external_drain_active:
logger.info("Refusing new turn for session %s — external drain active.", _quick_key)
return (
"⏳ This agent is draining for a maintenance action and isn't "
"accepting new turns right now. It'll be back in a moment — "
"please resend shortly."
)
# Claim this session before any await: many awaits sit between here and _run_agent
# registering the real AIAgent; without this sentinel a second message during any of them
# passes the "already running" guard and spins up a duplicate agent for the same session.
_active_session_lease, _limit_message = self._claim_active_session_slot(_quick_key, source)
if _limit_message is not None:
logger.info("Rejecting new active session %s: max_concurrent_sessions reached", _quick_key)
return _limit_message
event, source, is_internal = self._hm_rescue_orphaned_fifo(event, source, is_internal, _quick_key)
_claim_state = self._session_state(_quick_key)
if _active_session_lease is not None:
_claim_state.turn.lease = _active_session_lease
_claim_state.turn.agent = _AGENT_PENDING_SENTINEL
_claim_state.turn.started_ts = time.time()
self._persist_active_agents()
_run_generation = self._begin_session_run_generation(_quick_key)
try:
try:
_agent_result = await self._handle_message_with_agent(event, source, _quick_key, _run_generation)
except TurnLeaseTimeoutError as exc:
# A rejected message, not a completed turn: return before the /goal judge so it
# cannot consume the resend notice and enqueue a synthetic continuation loop.
logger.error(
"Rejecting turn for routing key %s on session %s after "
"turn-lease timeout; transcript load was not started and "
"the user must resend",
_quick_key, exc.session_id,
)
return (
"⏳ Another turn is still running on this session. To "
"protect the transcript, this message was not processed. "
"Wait for the active turn to finish, then resend it."
)
try:
await self._run_post_turn_hooks(
agent_result=_agent_result, source=source, is_internal=is_internal, event=event,
)
except Exception as _goal_exc:
logger.debug("post-turn hook failed: %s", _goal_exc)
return _agent_result
finally:
# MoA one-shot restore must run on EVERY exit path (success, exception, interrupt):
# the restore data lives on the per-turn event and would leak permanently otherwise.
self._restore_moa_one_shot(event, _quick_key)
self._restore_pending_one_turn_model_override(_quick_key)
# SIGKILL/OOM skips finally, leaving the durable marker for the next unclean startup's
# recovery pass.
await self._clear_durable_active_turn(event)
# Unconditional, idempotent release without a run_generation guard: evicts the zombie
# left when session_reset bumps the generation mid-flight (gen-N's guarded release in
# _run_agent returns False; a sentinel-only check would lock forever).
self._release_running_agent_state(_quick_key)
# Turn lease is keyed by (routing key, run generation) so this unwind can only free
# the lease its own turn acquired, never a newer turn's.
# Unconditional release covers every exit path. _release_running_agent_state is idempotent
# (pop-on-absent is harmless) and, called without a run_generation guard, always clears the slot
# regardless of which generation it holds. This evicts the zombie left when session_reset bumps
# the generation (N -> N+1) mid-flight: gen-N's guarded release inside _run_agent returns False,
# and the old sentinel-only check here missed the leftover real agent — locking the session out
# forever (#28686).
self._release_turn_lease(_quick_key, _run_generation)
def _restore_moa_one_shot(self, event: "MessageEvent", quick_key: str) -> None:
"""Revert a ``/moa <prompt>`` one-shot model override after its turn (called from the
message-handling ``finally``). ``_moa_restore_override`` holds the prior per-session
override (``None`` = clear the MoA override outright)."""
if not getattr(event, "_moa_disable_after_turn", False):
return
with suppress(Exception):
self._session_state(quick_key).conversation.model_override = getattr(event, "_moa_restore_override", None)
self._evict_cached_agent(quick_key)
def _restore_pending_one_turn_model_override(self, session_key: str) -> None:
"""Restore a per-session model override after ``/model --once`` runs."""
if not session_key:
return
try:
_otr_state = self._peek_session_state(session_key)
snapshot = _otr_state.conversation.one_turn_restore if _otr_state else None
if _otr_state is not None:
_otr_state.conversation.one_turn_restore = None
if snapshot:
self._restore_session_model_override(session_key, snapshot)
except Exception:
logger.debug("Failed to restore one-turn model override", exc_info=True)
def _prefix_inbound_sender_context(self, event: MessageEvent, source: SessionSource, message_text: str) -> str:
"""Attribute the sender in shared multi-user sessions and prepend history-backfill channel context."""
_is_shared_multi_user = is_shared_multi_user_session(
source, group_sessions_per_user=getattr(self.config, "group_sessions_per_user", True),
thread_sessions_per_user=getattr(self.config, "thread_sessions_per_user", False),
)
if _is_shared_multi_user and source.user_name:
# Display names are attacker-influenceable: neutralize newlines/control chars or a
# hostile name masquerades as a fake markdown section (mirrors build_session_context_prompt).
_safe_user_name = neutralize_untrusted_inline_text(source.user_name)
# Slack: expose the CURRENT speaker's verifiable `<@U...>` id so "mention me again" has a
# trusted target (display names are ambiguous). user_id comes from the envelope, not user-editable.
# See #17916.
if source.platform == Platform.SLACK and source.user_id:
_safe_user_name = f"{_safe_user_name} | Slack user <@{source.user_id}>"
message_text = f"[{_safe_user_name}] {message_text}"
# After the sender-prefix so the prefix applies only to the trigger message, not the backfill.
if getattr(event, "channel_context", None):
message_text = f"{event.channel_context}\n\n[New message]\n{message_text}"
return message_text
@staticmethod
def _classify_inbound_media(
event: MessageEvent, pending_stt_prepared: bool
) -> Tuple[list, list, list, list]:
"""Split ``event.media_urls`` into (image, STT-voice, audio-file, video) paths. Per-attachment
MIME wins over the message-level type (a document sent alongside an image must not be routed
as an image). MessageType.AUDIO / mixed DOCUMENT audio is a file attachment, never STT."""
from gateway.run import _event_media_is_audio, _event_media_is_image, _event_media_is_stt_input
image_paths, audio_paths, audio_file_paths, video_paths = [], [], [], []
for i, path in enumerate(event.media_urls or []):
mtype = event.media_types[i] if i < len(event.media_types) else ""
if _event_media_is_image(event, i):
image_paths.append(path)
if _event_media_is_audio(event, i):
if event.message_type in {MessageType.AUDIO, MessageType.DOCUMENT}:
audio_file_paths.append(path)
elif not pending_stt_prepared and _event_media_is_stt_input(event, i):
audio_paths.append(path)
if mtype.startswith("video/") or (not mtype and event.message_type == MessageType.VIDEO):
video_paths.append(path)
return image_paths, audio_paths, audio_file_paths, video_paths
async def _enrich_inbound_images(
self, source: SessionSource, session_key: str, message_text: str, image_paths: list[str]
) -> str:
"""Route images natively (attach pixels at run_conversation) or pre-analyze them into text."""
# See agent/image_routing.py. Offloaded to a thread: the decision does blocking network I/O
# (models.dev fetch on cache miss, Ollama /api/show probe) that would stall the event loop.
_img_mode = await asyncio.to_thread(
self._decide_image_input_mode, source=source, session_key=session_key,
)
if _img_mode == "native":
self._session_state(session_key).persistent.native_image_paths = list(image_paths)
logger.info(
"Image routing: native (model supports vision). %d image(s) will be attached inline.",
len(image_paths),
)
return message_text
logger.info(
"Image routing: text (mode=%s). Pre-analyzing %d image(s) via vision_analyze.",
_img_mode, len(image_paths),
)
# Vision enrichment runs before AIAgent.run_conversation(), so bind this session's resolved
# runtime explicitly rather than consulting process-global compatibility mirrors.
vision_runtime = None
try:
turn_model, runtime_kwargs = self._resolve_session_agent_runtime(
source=source, session_key=session_key,
)
vision_runtime = {**(runtime_kwargs or {}), "model": turn_model}
except Exception:
logger.debug("vision enrichment: session runtime resolution failed", exc_info=True)
from agent.auxiliary_client import scoped_runtime_main
with scoped_runtime_main(vision_runtime):
return await self._enrich_message_with_vision(message_text, image_paths)
async def _echo_stt_transcripts(
self, adapter, source: SessionSource, transcripts: List[str], *, metadata=None, log_context: str = "Transcript"
) -> None:
"""Send each transcript back as ``🎙️ "…"`` (best-effort; failures are logged, never raised)."""
for tx in transcripts:
try:
await adapter.send(source.chat_id, f'🎙️ "{tx}"', metadata=metadata)
except Exception as echo_exc:
logger.debug("%s echo failed (non-fatal): %s", log_context, echo_exc)
async def _enrich_inbound_voice(
self, event: MessageEvent, source: SessionSource, message_text: str, audio_paths: list[str]
) -> str:
message_text, _successful_transcripts = await self._enrich_message_with_transcription(
message_text, audio_paths,
)
# Echo each successful transcript back immediately when configured so users can verify STT
# quality in real time. On transcription failure do NOT send a hardcoded notice: that
# bypassed the LLM and produced two replies; enrichment leaves one neutral marker instead.
if _successful_transcripts and self._should_echo_stt_transcripts():
_echo_adapter = self._adapter_for_source(source)
if _echo_adapter:
_echo_meta = self._thread_metadata_for_source(source, self._reply_anchor_for_event(event))
await self._echo_stt_transcripts(_echo_adapter, source, _successful_transcripts, metadata=_echo_meta)
return message_text
@staticmethod
def _inbound_attachment_display_name(path: str) -> Tuple[str, str]:
"""``(display_name, agent_visible_path)``: cache filename is ``<id>_<id>_<original>``; the
path is translated to the in-container mount under a Docker backend."""
from tools.credential_files import to_agent_visible_cache_path
basename = os.path.basename(path)
parts = basename.split("_", 2)
return re.sub(r'[^\w.\- ]', '_', parts[2] if len(parts) >= 3 else basename), to_agent_visible_cache_path(path)
@classmethod
def _prepend_inbound_media_file_notes(cls, message_text: str, audio_file_paths: list[str], video_paths: list[str]) -> str:
"""Prepend a path-pointing note per audio-file / video attachment (content is not inlined)."""
for kind, noun, verb, tool, paths in (
("an audio file attachment", "audio", "transcribe or process", "a transcription or media tool", audio_file_paths),
("a video attachment", "video", "inspect or process", "a video analysis or media tool", video_paths),
):
for _path in paths:
_display, _agent_path = cls._inbound_attachment_display_name(_path)
message_text = (
f"[The user sent {kind}: '{_display}'. "
f"It is saved at: {_agent_path}. "
f"Its content is not inlined here. If the user's request involves "
f"what the {noun} contains, {verb} it yourself — for "
f"example by passing the path to {tool} — "
f"instead of asking the user to describe it. Only ask what to do "
f"with it if their intent is genuinely unclear.]"
f"\n\n{message_text}"
)
return message_text
@classmethod
def _prepend_inbound_document_notes(cls, event: MessageEvent, message_text: str) -> str:
"""Prepend a context note per non-media attachment (anything not routed as image/audio/video)."""
from gateway.run import (
_build_document_context_note, _event_media_is_audio, _event_media_is_image,
_event_media_is_video,
)
if not event.media_urls:
return message_text
import mimetypes as _mimetypes
_TEXT_EXTENSIONS = {".txt", ".md", ".csv", ".log", ".json", ".xml", ".yaml", ".yml", ".toml", ".ini", ".cfg"}
inline_flags = getattr(event, "media_text_inlined", None) or []
for i, path in enumerate(event.media_urls):
# A document mixed into a PHOTO/VOICE message (message-level type != DOCUMENT) still
# reaches the agent; only genuine non-media files get a note.
if any(f(event, i) for f in (_event_media_is_image, _event_media_is_audio, _event_media_is_video)):
continue
mtype = event.media_types[i] if i < len(event.media_types) else ""
if mtype in {"", "application/octet-stream"}:
_is_text = os.path.splitext(path)[1].lower() in _TEXT_EXTENSIONS
mtype = "text/plain" if _is_text else (_mimetypes.guess_type(path)[0] or "application/octet-stream")
# Every accepted file gets a note — a non-text/non-application MIME (font/*, model/*)
# must still tell the agent the file exists.
display_name, agent_path = cls._inbound_attachment_display_name(path)
inline_flag = inline_flags[i] if i < len(inline_flags) else None
context_note = _build_document_context_note(
display_name, agent_path, mtype, content_inlined=inline_flag is not False,
)
message_text = f"{context_note}\n\n{message_text}"
return message_text
@staticmethod
def _prepend_inbound_reply_context(event: MessageEvent, source: SessionSource, message_text: str) -> str:
"""Prepend the Discord triggering-message id and the reply-to pointer."""
# Discord: the triggering message id goes on the per-turn user message, never the cached
# system prompt — it changes every turn and would bust the agent-cache signature.
if (
source is not None
and getattr(source, "platform", None) == Platform.DISCORD
and getattr(event, "message_id", None)
):
from gateway.session import _discord_tools_loaded as _disc_tools_loaded
if _disc_tools_loaded():
message_text = (
f"[Triggering message id: `{event.message_id}` — use as "
f"`message_id` for reply/react/pin via the discord tools.]\n\n"
f"{message_text}"
)
if getattr(event, "reply_to_text", None) and event.reply_to_message_id:
# Always inject the reply-to pointer even when the quoted text is already in history:
# it's disambiguation (*which* prior message), not deduplication.
# Adapters resolve the original message (or the user's native partial quote).
# A preview here silently loses later list items and code; keep that context intact.
reply_text = event.reply_to_text
_who = " your previous message" if getattr(event, "reply_to_is_own_message", False) else ""
message_text = f'[Replying to{_who}: "{reply_text}"]\n\n{message_text}'
return message_text
async def _inbound_model_context_length(self, source: SessionSource, session_key: str) -> int:
"""Context length of the model this turn runs on. A global ``model.context_length`` pin
belongs to the configured model, not a /model or channel override; custom-provider limits win."""
from gateway.run import _load_gateway_config
from agent.model_metadata import get_model_context_length_async
_msg_config_ctx = None
_msg_cfg = None
_msg_model_cfg = {}
_msg_custom_providers = []
with suppress(Exception):
_msg_cfg = _load_gateway_config()
_msg_model_cfg = _msg_cfg.get("model", {})
if isinstance(_msg_model_cfg, dict):
_msg_raw_ctx = _msg_model_cfg.get("context_length")
if _msg_raw_ctx is not None:
_msg_config_ctx = int(_msg_raw_ctx)
try:
from hermes_cli.config import get_compatible_custom_providers
_msg_custom_providers = get_compatible_custom_providers(_msg_cfg)
except Exception:
_msg_custom_providers = _msg_cfg.get("custom_providers") or []
# GatewayRunner has no self._model/self._base_url; resolve the session's actual runtime.
_msg_model, _msg_runtime = self._resolve_session_agent_runtime(
source=source, session_key=session_key, user_config=_msg_cfg,
)
_msg_base_url = _msg_runtime.get("base_url") or ""
if isinstance(_msg_model_cfg, dict):
_msg_configured_model = _msg_model_cfg.get("default") or _msg_model_cfg.get("model")
else:
_msg_configured_model = _msg_model_cfg # (no dict → no pin was read; ctx is already None)
if _msg_model != _msg_configured_model:
_msg_config_ctx = None
if _msg_config_ctx is not None:
try:
from hermes_cli.route_identity import should_clear_context_pin_async
if await should_clear_context_pin_async(
None, None, # model match already checked above
_msg_model_cfg.get("base_url"), _msg_base_url,
_msg_model_cfg.get("provider"), _msg_runtime.get("provider"),
):
_msg_config_ctx = None
except Exception:
_msg_config_ctx = None
if _msg_custom_providers and _msg_base_url:
with suppress(Exception):
from hermes_cli.config import get_custom_provider_context_length
_msg_config_ctx = get_custom_provider_context_length(
model=_msg_model, base_url=_msg_base_url, custom_providers=_msg_custom_providers,
) or _msg_config_ctx
return await get_model_context_length_async(
_msg_model, base_url=_msg_base_url, api_key=_msg_runtime.get("api_key") or "",
config_context_length=_msg_config_ctx, provider=_msg_runtime.get("provider") or "",
custom_providers=_msg_custom_providers,
)
async def _expand_inbound_context_references(
self, source: SessionSource, session_key: str, message_text: str
) -> Optional[str]:
"""Expand ``@`` context references; returns None when the injection was refused (user notified)."""
try:
from agent.context_references import preprocess_context_references_async
try:
from tools.terminal_scope import terminal_env as _ts_env
except ImportError:
_ts_env = os.environ.get
_msg_cwd = _ts_env("TERMINAL_CWD", os.path.expanduser("~"))
_msg_ctx_len = await self._inbound_model_context_length(source, session_key)
_ctx_result = await preprocess_context_references_async(
message_text, cwd=_msg_cwd, context_length=_msg_ctx_len, allowed_root=_msg_cwd
)
if _ctx_result.blocked:
_adapter = self._adapter_for_source(source)
if _adapter:
await _adapter.send(
source.chat_id,
"\n".join(_ctx_result.warnings) or "Context injection refused.",
)
return None
if _ctx_result.expanded:
message_text = _ctx_result.message
except Exception as exc:
logger.warning("@ context reference expansion failed: %s", exc)
logger.debug("@ context reference expansion failure detail", exc_info=True)
return message_text
async def _prepare_inbound_message_text(
self, *, event: MessageEvent, source: SessionSource, history: List[Dict[str, Any]],
session_key: Optional[str] = None,
) -> Optional[str]:
"""Prepare inbound event text for the agent. Shared by the normal inbound and queued
follow-up paths so attribution, image enrichment, STT, document notes, reply context and
@ references behave the same. Side effect: buffers per-session native image paths when the
model supports native vision; the caller consumes that buffer at ``run_conversation``."""
_pending_stt_prepared = hasattr(event, "_gateway_pending_stt_text")
message_text = (event._gateway_pending_stt_text if _pending_stt_prepared else event.text) or ""
# Prefer the caller's resolved session key so this write key matches the consume key at the
# run_conversation site; derive it here only for tests and legacy standalone callers.
session_key = session_key or self._session_key_for_source(source)
# Reset only this session's per-call buffer; other sessions may be concurrently preparing.
self._consume_pending_native_image_paths(session_key)
message_text = self._prefix_inbound_sender_context(event, source, message_text)
image_paths, audio_paths, audio_file_paths, video_paths = self._classify_inbound_media(event, _pending_stt_prepared)
if image_paths:
message_text = await self._enrich_inbound_images(source, session_key, message_text, image_paths)
if audio_paths:
message_text = await self._enrich_inbound_voice(event, source, message_text, audio_paths)
message_text = self._prepend_inbound_media_file_notes(message_text, audio_file_paths, video_paths)
message_text = self._prepend_inbound_document_notes(event, message_text)
if "@" in message_text:
message_text = await self._expand_inbound_context_references(source, session_key, message_text)
if message_text is None:
return None
# After expansion: the quoted reply is someone else's text and stays literal — an
# ``@file:`` inside it must never read a local file on the replier's behalf.
return self._prepend_inbound_reply_context(event, source, message_text)
async def _prepare_profile_scoped_inbound_message_text(
self, *, event: MessageEvent, source: SessionSource, history: List[Dict[str, Any]],
session_key: Optional[str] = None,
) -> Optional[str]:
"""Run inbound preprocessing under the routed profile when multiplexed."""
from gateway.run import _async_profile_runtime_scope
kwargs = dict(event=event, source=source, history=history, session_key=session_key)
if getattr(getattr(self, "config", None), "multiplex_profiles", False):
async with _async_profile_runtime_scope(self._resolve_profile_home_for_source(source)):
return await self._prepare_inbound_message_text(**kwargs)
return await self._prepare_inbound_message_text(**kwargs)
async def _prepare_clarify_reply_text(self, event) -> str:
"""Return raw text or successful voice transcripts for a clarify reply."""
if not self._pending_event_audio_paths(event):
return (event.text or "").strip()
_, successful_transcripts = await self._transcribe_pending_audio_event_once(event, "")
return "\n\n".join(t.strip() for t in successful_transcripts if t.strip())
def _consume_pending_native_image_paths(self, session_key: str) -> List[str]:
state = self._peek_session_state(session_key)
paths = list(state.persistent.native_image_paths or []) if state is not None else []
if paths:
state.persistent.native_image_paths = []
return paths
async def _mark_durable_active_turn(self, event: "MessageEvent", session_key: str) -> bool:
"""Persist the exact resolved routing key for this running turn."""
try:
token = await self.async_session_store.mark_turn_active(session_key)
except Exception as exc:
logger.warning("Could not persist active-turn marker for %s: %s", session_key, exc)
return False
if not token:
return False
# Private event attributes are process-local ownership state: keep the token out of public
# metadata, transcripts, and platform payloads.
event._gateway_active_turn_session_key = session_key
event._gateway_active_turn_token = token
return True
async def _clear_durable_active_turn(self, event: "MessageEvent") -> bool:
"""Best-effort CAS clear of the marker owned by *event* (3 attempts; never blocks agent/lease
release — a stale marker is bounded by the agent timeout and clean-start discard)."""
session_key = getattr(event, "_gateway_active_turn_session_key", None)
token = getattr(event, "_gateway_active_turn_token", None)
try:
if not session_key or not token:
return False
last_error: Optional[Exception] = None
for attempt in range(1, 4):
try:
return bool(await self.async_session_store.clear_turn_active(session_key, token))
except Exception as exc:
last_error = exc
if attempt < 3:
logger.debug(
"Retrying active-turn marker cleanup for %s (%d/3): %s",
session_key, attempt, exc,
)
logger.warning(
"Could not clear active-turn marker for %s after 3 attempts: %s", session_key, last_error,
)
return False
finally:
for attr in ("_gateway_active_turn_session_key", "_gateway_active_turn_token"):
with suppress(AttributeError):
delattr(event, attr)
def _install_plugin_message_injector(self) -> None:
"""Publish this live gateway's plugin message scheduler."""
from hermes_cli.plugins import get_plugin_manager
get_plugin_manager().set_gateway_message_injector(
self, self._schedule_plugin_message_injection
)
def _clear_plugin_message_injector(self) -> None:
"""Remove this runner's scheduler without clobbering a newer owner."""
from hermes_cli.plugins import get_plugin_manager
get_plugin_manager().clear_gateway_message_injector(self)
def _schedule_plugin_message_injection(
self, *, session_key: str, content: str, plugin_id: str
) -> bool:
"""Schedule a plugin-triggered turn on the live gateway loop (thread-safe)."""
from gateway.run import safe_schedule_threadsafe
loop = getattr(self, "_gateway_loop", None)
if not getattr(self, "_running", False) or loop is None or loop.is_closed():
return False
coro = self._dispatch_plugin_message_injection(
session_key=session_key, content=content, plugin_id=plugin_id,
)
try:
current_loop = asyncio.get_running_loop()
except RuntimeError:
current_loop = None
if current_loop is loop:
try:
future = loop.create_task(coro)
except Exception:
coro.close()
logger.warning("Plugin message injection scheduling failed", exc_info=True)
return False
self._background_tasks.add(future)
future.add_done_callback(self._background_tasks.discard)
else:
future = safe_schedule_threadsafe(
coro, loop, logger=logger, log_message="Plugin message injection scheduling failed",
log_level=logging.WARNING,
)
if future is None:
return False
def _log_result(completed) -> None:
try:
if completed.result():
return
what, exc = "was not routed", None
except (asyncio.CancelledError, concurrent.futures.CancelledError):
return
except Exception as err:
what, exc = "failed", err
logger.warning(
"Plugin message injection %s: plugin=%s session=%s", what, plugin_id, session_key, exc_info=exc,
)
future.add_done_callback(_log_result)
return True
async def _dispatch_plugin_message_injection(
self, *, session_key: str, content: str, plugin_id: str
) -> bool:
"""Route a plugin-triggered turn through the session's live adapter."""
def _accepting() -> bool:
return getattr(self, "_running", False) and not getattr(self, "_draining", False)
if not _accepting():
return False
entry = await self.async_session_store.lookup_by_session_key(session_key)
if entry is None or entry.origin is None or not _accepting():
return False
source = dataclasses.replace(entry.origin)
try:
authorized = self._is_user_authorized(source, allow_adapter_delegation=False)
except Exception:
logger.warning(
"Plugin message injection authorization check failed: plugin=%s session=%s",
plugin_id, session_key, exc_info=True,
)
return False
if not authorized:
logger.warning(
"Plugin message injection denied by current gateway authorization: "
"plugin=%s session=%s", plugin_id, session_key,
)
return False
adapter = self._adapter_for_source(source)
if adapter is None:
return False
await adapter.handle_message(MessageEvent(
text=content, message_type=MessageType.TEXT, source=source, internal=True,
allow_gateway_control=False,
metadata={
"hermes_plugin_id": plugin_id, "hermes_plugin_injection": True,
"gateway_session_key": session_key, "gateway_session_id": entry.session_id,
"gateway_session_strict": True,
},
))
logger.info(
"Plugin message injection dispatched: plugin=%s session=%s session_id=%s",
plugin_id, session_key, entry.session_id,
)
return True
def _decide_image_input_mode(
self, *, source: Optional[SessionSource] = None, session_key: Optional[str] = None,
user_config: Optional[dict] = None, provider: Optional[str] = None,
model: Optional[str] = None,
) -> str:
"""Resolve image-input routing (``"native"`` / ``"text"``) for the effective model this turn
(see agent/image_routing.py). Sessions can carry /model overrides and this runs before AIAgent
sets the auxiliary_client runtime globals, so resolve the per-session runtime bundle the
upcoming turn will use, not just the persisted default."""
try:
from agent.image_routing import decide_image_input_mode
from agent.auxiliary_client import _read_main_model, _read_main_provider
from hermes_cli.config import load_config
cfg = user_config if isinstance(user_config, dict) else load_config()
resolved_provider = (provider or "").strip()
resolved_model = (model or "").strip()
resolved_requested_provider = ""
if (not resolved_provider or not resolved_model) and (source is not None or session_key):
try:
turn_model, runtime_kwargs = self._resolve_session_agent_runtime(
source=source, session_key=session_key, user_config=cfg,
)
rk = runtime_kwargs if isinstance(runtime_kwargs, dict) else {}
if not resolved_model and isinstance(turn_model, str):
resolved_model = turn_model.strip()
if not resolved_provider and isinstance(rk.get("provider"), str):
resolved_provider = rk["provider"].strip()
if isinstance(rk.get("requested_provider"), str):
resolved_requested_provider = rk["requested_provider"].strip()
except Exception as exc:
logger.debug(
"image_routing: session runtime resolution failed, falling back to config — %s",
exc,
)
return decide_image_input_mode(
resolved_provider or _read_main_provider(), resolved_model or _read_main_model(),
cfg, requested_provider=resolved_requested_provider,
)
except Exception as exc:
logger.debug("image_routing: decision failed, falling back to text — %s", exc)
return "text"
async def _enrich_message_with_vision(self, user_text: str, image_paths: List[str]) -> str:
"""Auto-analyze user-attached images with the vision tool and prepend the descriptions.
Description *and* local cache path are injected so the model understands the image without
a tool call and can re-examine it with vision_analyze."""
from tools.vision_tools import vision_analyze_tool
from agent.memory_manager import sanitize_context
analysis_prompt = (
"Concisely describe this image in 2-4 sentences "
"(~200 Chinese characters or ~150 English words). "
"Cover the main subject, key visible text/data/code, and overall context. "
"If it is a chart, diagram, or scientific figure, include the important "
"labels, legend, and key values. Skip decorative details."
)
enriched_parts = []
for path in image_paths:
try:
logger.debug("Auto-analyzing user image: %s", path)
result = json.loads(await vision_analyze_tool(image_url=path, user_prompt=analysis_prompt))
if result.get("success"):
description = sanitize_context(result.get("analysis", ""))
note = (
f"[The user sent an image~ Here's what I can see:\n{description}]\n"
f"[If you need a closer look, use vision_analyze with "
f"image_url: {path} ~]"
)
else:
note = (
"[The user sent an image but I couldn't quite see it "
"this time (>_<) You can try looking at it yourself "
f"with vision_analyze using image_url: {path}]"
)
except Exception as e:
logger.error("Vision auto-analysis error: %s", e)
note = (
f"[The user sent an image but something went wrong when I "
f"tried to look at it~ You can try examining it yourself "
f"with vision_analyze using image_url: {path}]"
)
enriched_parts.append(note)
if not enriched_parts:
return user_text
prefix = "\n\n".join(enriched_parts)
return f"{prefix}\n\n{user_text}" if user_text else prefix
_EMPTY_TEXT_PLACEHOLDER = "(The user sent a message with no text content)"
@classmethod
def _prepend_media_prefix(cls, prefix: str, user_text: str) -> str:
"""``prefix`` + the user's text; the Discord empty-content placeholder is dropped as redundant."""
if user_text and user_text.strip() != cls._EMPTY_TEXT_PLACEHOLDER:
return f"{prefix}\n\n{user_text}"
return prefix
@staticmethod
def _untranscribed_audio_note(path: str) -> str:
"""One minimal neutral marker for every STT failure. Never mention "no STT provider" or setup
steps — persisted in history they make the model keep volunteering STT-setup advice."""
from tools.credential_files import to_agent_visible_cache_path
agent_path = to_agent_visible_cache_path(os.path.abspath(path))
return f"[voice message could not be transcribed automatically; the audio is available at: {agent_path}]"
async def _transcribe_one_clip(self, path: str, transcribe_audio, transcribe_audio_local_fallback) -> Tuple[Optional[str], str]:
"""``(transcript_or_None, note)`` for one clip via configured STT with local fallback."""
result = await asyncio.to_thread(transcribe_audio, path, None, "gateway")
if not result.get("success"):
fallback = await asyncio.to_thread(transcribe_audio_local_fallback, path)
if fallback.get("success"):
logger.info("Configured STT failed for %s; recovered with local STT", path)
result = fallback
if not result["success"]:
logger.info("Voice transcription failed for %s: %s", path, result.get("error", "unknown error"))
return None, self._untranscribed_audio_note(path)
transcript = result["transcript"]
# STT may return success=True with an empty/whitespace transcript (silence, cut-off);
# empty quotes make the agent reply to nothing and can loop, so emit a sentinel note.
# See #41603.
if not (transcript or "").strip():
return None, (
"[The user sent a voice message but it came through "
"empty or inaudible — speech-to-text returned no "
"words. Do not guess at the content; ask the user "
"to resend or type it out.]"
)
# Plain quoted line: a "The user sent a voice message..." wrapper read as a meta-instruction
# and made the LLM comment on voice mode instead.
return transcript, f'"{transcript}"'
async def _enrich_message_with_transcription(
self, user_text: str, audio_paths: List[str]
) -> tuple[str, List[str]]:
"""Transcribe voice clips with the configured STT provider and prepend the transcripts →
``(enriched_text, successful_transcripts)``; the transcripts (input order; empty if every clip
failed or STT is disabled) let callers echo them back before the agent loop."""
from gateway.run import _probe_audio_duration
audio_paths = list(dict.fromkeys(audio_paths))
if not getattr(self.config, "stt_enabled", True):
notes = []
for path in audio_paths:
abs_path = os.path.abspath(path)
duration_str = await _probe_audio_duration(abs_path)
suffix = f" (duration: {duration_str})" if duration_str else ""
notes.append(f"[The user sent a voice message: {abs_path}{suffix}]")
return (self._prepend_media_prefix("\n\n".join(notes), user_text) if notes else user_text), []
try:
from tools.transcription_tools import (
transcribe_audio, transcribe_audio_local_fallback
)
except ModuleNotFoundError as e:
logger.error("Transcription module unavailable: %s", e)
return self._prepend_media_prefix("[voice message could not be transcribed]", user_text), []
enriched_parts = []
successful_transcripts: List[str] = []
for path in audio_paths:
try:
logger.debug("Transcribing user voice: %s", path)
transcript, note = await self._transcribe_one_clip(
path, transcribe_audio, transcribe_audio_local_fallback,
)
if transcript is not None:
successful_transcripts.append(transcript)
enriched_parts.append(note)
except Exception as e:
logger.error("Transcription error: %s", e)
enriched_parts.append(self._untranscribed_audio_note(path))
if enriched_parts:
user_text = self._prepend_media_prefix("\n\n".join(enriched_parts), user_text)
return user_text, successful_transcripts
def _pending_event_audio_paths(self, event) -> List[str]:
"""Return STT-eligible paths from a pending voice message."""
from gateway.run import _event_media_is_stt_input
return [
path for i, path in enumerate(getattr(event, "media_urls", None) or [])
if _event_media_is_stt_input(event, i)
]
async def _transcribe_pending_audio_event_once(
self, event, user_text: Optional[str] = None
) -> tuple[str | None, List[str]]:
"""Transcribe a pending audio event once and cache the result on the event: the interrupt
monitor and the pending-drain path both need it — one STT call and one echo per message."""
if hasattr(event, "_gateway_pending_stt_text"):
return event._gateway_pending_stt_text, list(getattr(event, "_gateway_pending_stt_transcripts", []) or [])
audio_paths = self._pending_event_audio_paths(event)
if not audio_paths:
return user_text if user_text is not None else (getattr(event, "text", None) or None), []
text = user_text if user_text is not None else (getattr(event, "text", "") or "")
enriched_text, successful_transcripts = await self._enrich_message_with_transcription(text, audio_paths)
event._gateway_pending_stt_text = enriched_text
event._gateway_pending_stt_transcripts = list(successful_transcripts)
return enriched_text, successful_transcripts
async def _echo_pending_stt_transcripts_once(
self, event, adapter, source, transcripts: List[str], *, metadata=None,
log_context: str = "Transcript",
) -> None:
"""Echo pending-event STT transcripts to the chat at most once. Tracked as a COUNT (not a
set — identical transcripts are distinct deliveries): ``merge_pending_message_event`` can
append a second voice note and invalidate the cache; the re-run returns earlier transcripts
as a prefix, so only the unsent tail is echoed."""
if not transcripts or not self._should_echo_stt_transcripts() or adapter is None:
return
already_echoed = int(getattr(event, "_gateway_pending_stt_echoed", 0) or 0)
event._gateway_pending_stt_echoed = max(already_echoed, len(transcripts))
await self._echo_stt_transcripts(
adapter, source, transcripts[already_echoed:], metadata=metadata, log_context=log_context,
)
async def _transcribe_and_echo_pending_voice(
self, event, adapter, source, text: str, *, log_context: str, metadata=_UNSET
) -> tuple[str, List[str]]:
"""Transcribe a pending voice event and echo transcripts once → ``(enriched_text,
transcripts)`` for ``agent.interrupt()`` or the pending-drain flow; ``(text, [])`` when there
is no STT-eligible media (caller owns the ``_build_media_placeholder`` fallback)."""
if not self._pending_event_audio_paths(event):
return text, []
try:
enriched_text, transcripts = await self._transcribe_pending_audio_event_once(event, text)
if metadata is _UNSET:
metadata = self._thread_metadata_for_source(source, self._reply_anchor_for_event(event))
await self._echo_pending_stt_transcripts_once(
event, adapter, source, transcripts, metadata=metadata, log_context=log_context
)
return enriched_text or text, transcripts
except Exception as trans_exc:
logger.warning("%s transcription failed: %s", log_context, trans_exc)
return text, []