Commit Graph

1421 Commits

Author SHA1 Message Date
Teknium 31579f781e fix(cron): transient run prompt survives the relay-fronted gateway forward
cronjob(action='run', prompt=...) context was silently dropped when the
manual run forwarded to the gateway (#96010 follow-up): POST
/api/jobs/{id}/run took no body. The forward now sends {prompt} in the
request body; the api_server validates it (length cap + strict injection
scan, same as stored prompts) and trigger_job stamps it as a transient
manual_run_prompt alongside manual_run_at. run_one_job consumes the stamp
for that single fire and mark_job_run clears it, so it never persists
into the job definition or later scheduled fires.
2026-08-27 20:53:02 -07:00
Teknium 34393c32aa feat(plugins): wire plugin platform handlers into a2a, buzz, and qqbot adapters
Platforms added to main after the original branch was cut; keeps the
source invariant (every connectable adapter calls _wire_plugin_handlers)
true, and adds qqbot to the invariant test's gateway list.
2026-08-27 07:51:37 -07:00
teknium1 272f4e4abe feat(plugins): generalize native platform handler registration to every gateway platform
ctx.register_platform_handler(platform, factory) — the generic surface for
plugins to wire native handlers into any platform adapter at connect()
time. Factories receive (native, adapter): the platform's client/app
object (PTB Application, discord.py Bot, slack_bolt AsyncApp, Teams App,
DingTalkStreamClient, aiohttp web.Application) or None for adapters with
no separate native object.

- BasePlatformAdapter._wire_plugin_handlers(native): shared, isolated
  invocation helper — a raising plugin cannot block a platform connect.
- All 27 connectable adapters call it: telegram/slack/teams/line/
  api_server/msgraph_webhook wire before their dispatch tables freeze;
  the rest hook at connect success.
- register_telegram_handler and get_telegram_handler_factories retained
  as thin back-compat aliases over the telegram bucket.
- Source-invariant test guarantees every adapter with connect() keeps
  calling the hook.
2026-08-27 07:51:37 -07:00
Tranquil-Flow 6cbb7b6115 fix(gateway): preserve exception type when error string is empty (#78183)
httpx timeout exceptions (ReadTimeout, WriteTimeout) stringify to "",
which defeats _is_timeout_error's first-line guard (if not error: return
False).  The base-layer plain-text fallback then re-sends an already-
delivered message — the user receives it twice.

Replace error=str(exc) with error=str(exc) or type(exc).__name__ at every
httpx-based adapter boundary so the existing matcher ("readtimeout",
"writetimeout") still fires.  ConnectTimeout intentionally stays
unmatched: if the connection never opened the message was not delivered,
so retry/fallback remains correct.

Applies to BlueBubbles (send + _create_chat_for_handle), WhatsApp Cloud
(text + interactive + media), QQ Bot (send chunk + keyboard + media), and
Yuanbao media handler — the same latent bug exists in every adapter that
stores error=str(exc) from an httpx call.
2026-08-27 17:21:26 +05:30
milnerrad 8e1db41041 fix(gateway): redeliver transient failures after reconnect 2026-08-26 04:49:39 -07:00
Teknium 82b32f32ef feat(terminal): wire shared-container key into profile-scoped resolver and MEDIA delivery
Follow-up on @fangliquanflq's opt-in (#84775): after the profile-scoping fix
(#94560) the container cache key is resolved in _resolve_container_task_id,
so the shared key must unify profiles there too — 'shared:<key>' for every
session of every opted-in profile AND for CLI/no-session runs. Delivery adds
the shared sandbox layout as the first translation candidate. Empty key
keeps strict per-profile isolation; SSH ignores the key entirely.
2026-08-25 04:00:27 -07:00
Teknium 15f7b7293c fix(terminal): persistent Docker containers are profile-scoped, not per-session
Commit a270c4ade's session-key fallback in _resolve_container_task_id was
added to stop cross-profile SSH environment reuse, but it wasn't backend-
gated: persistent Docker silently fragmented into one container per gateway
session, breaking the product contract (one long-lived container per profile,
shared by CLI and every session of that profile). #93950's vanishing MEDIA
attachments were downstream damage.

- persistent Docker (container_persistent: true) now keys to the profile:
  literal 'default' for the default profile (same container as CLI),
  'profile:<name>' for named profiles
- SSH and non-persistent Docker keep session scoping (the original leak fix
  and the #82731 isolation contract are untouched)
- gateway MEDIA translation follows the profile layout and keeps the legacy
  bug-window per-session sandboxes as fallback candidates, trying each until
  the file resolves — old sessions self-heal, no migration
- /root/.hermes credential-surface refusal preserved across all layouts
2026-08-25 02:30:38 -07:00
Michael Nguyen a15533b646 feat(gateway): warn when a Docker sandbox MEDIA path fails translation
De-silence the #93950 failure mode: when a container-absolute MEDIA path
under /workspace or /root cannot be resolved to a host sandbox file while
TERMINAL_ENV=docker, log the reason (no mounts / no prefix match / host
file missing) plus the delivering session key instead of only the generic
'Skipping unsafe MEDIA directive path' line upstream.
2026-08-24 23:50:23 -07:00
Michael Nguyen d4f31a8f36 fix(gateway): resolve session-scoped Docker sandboxes for MEDIA delivery (#93950)
Persistent Docker containers bind <sandboxes>/docker/<task>/{workspace,home}
where <task> is sanitize_task_id_for_path("session:<session_key>") — but the
gateway's synthetic mounts hardcoded the literal "default" sandbox
(_default_docker_workspace_host_root / _docker_persistent_home_host_root).
For any session-scoped deployment the longest-prefix match missed, the
container path fell through to a host-filesystem resolve that could not
exist, and every MEDIA attachment was silently dropped.

The post-handler delivery pipeline also runs after
_handle_message_with_agent cleared the turn's session contextvars, so even
a correct sandbox derivation consulting ambient state would collapse onto
"default". Thread the delivering session's key explicitly through
validate_media_delivery_path -> _translate_docker_container_media_path ->
the two host-root helpers (same pattern as the TTS fix for #57049/#36685).

Default-sandbox resolution and the /root/.hermes credential exclusion are
preserved; contexts without a key keep the historical behavior.
2026-08-24 23:50:23 -07:00
lkz-de cbc8d1804d fix(signal): chunk long cron deliveries instead of truncating 2026-08-24 20:03:24 -07:00
aniruddhaadak80 7befc1d2dd fix(gateway): route platform authorization reads through the profile secret scope
Under gateway.multiplex_profiles, secondary profiles are constructed
inside _profile_runtime_scope and their .env lives in the profile's
secret scope - gateway/run.py explicitly does NOT mutate os.environ with
it. Four adapters still read their AUTHORIZATION config via raw
os.getenv, so every secondary profile either (a) silently missed its own
env-only allowlists/policies (fail-closed: all DMs dropped at intake) or
(b) inherited the default profile's GATEWAY_ALLOW_ALL_USERS=true /
allowlists from the shared process env (fail-open admissions):

- weixin.py: WEIXIN_DM_POLICY / WEIXIN_ALLOWED_USERS /
  WEIXIN_GROUP_ALLOWED_USERS / WEIXIN_ALLOW_ALL_USERS +
  GATEWAY_ALLOW_ALL_USERS in _open_dm_opted_in
- yuanbao.py: YUANBAO_DM_POLICY / DM_ALLOW_FROM / GROUP_POLICY /
  GROUP_ALLOW_FROM / ALLOW_ALL_USERS (new _yb_secret helper; AccessPolicy
  hard-gates intake)
- signal.py: SIGNAL_GROUP_ALLOWED_USERS / SIGNAL_ALLOWED_USERS (new
  _sig_secret helper; empty scoped group list previously meant "drop all
  groups" silently)
- wecom/adapter.py: WECOM_DM_POLICY / WECOM_ALLOWED_USERS /
  WECOM_GROUP_POLICY / WECOM_ALLOW_ALL_USERS + GATEWAY_ALLOW_ALL_USERS -
  while credentials one line above already used _get_scoped_secret
- gateway/run.py::_own_policy_open_startup_violation: the open-policy
  startup guard validated GATEWAY_ALLOW_ALL_USERS via raw os.getenv even
  though its sibling dm/group reads already used the scoped _getenv

All reads now go through the canonical fail-closed scoped shape QQ's
_resolve_qq_secret already used (scope hit wins; unscoped single-profile
callers keep legacy os.environ behavior). Regression suite drives the
real scope contextvar across all four helpers plus the admission gates
and the startup guard, asserting both directions: profile values are
visible under multiplex, default-profile values never leak.

Fixes #93522
2026-08-24 03:20:06 -07:00
chelsealong d7e4204e77 fix(gateway): scope multiplex-profile authorization reads (weixin/yuanbao/wecom)
WEIXIN_DM_POLICY/ALLOWED_USERS/GROUP_ALLOWED_USERS, YUANBAO's equivalents,
WECOM_DM_POLICY/ALLOWED_USERS/GROUP_POLICY, and the startup guard's
GATEWAY_ALLOW_ALL_USERS check still read raw os.getenv at adapter
construction time. Under gateway.multiplex_profiles that reads the process
env instead of the per-profile secret scope, so a secondary profile either
silently drops every DM (its own env-only allowlist is invisible) or
inherits the default profile's allow-all/allowlist config.

Route these reads through the existing scoped helpers (_wx_secret,
_get_scoped_secret, gateway.authz_mixin._platform_gate_env, and
gateway.config._getenv) already used for the adjacent credential reads in
the same adapters.

Fixes #93522.
2026-08-24 03:20:06 -07:00
joaomarcos f5a9ba9ee6 perf(bluebubbles): move attachment reads off the event loop 2026-08-23 20:00:07 -07:00
loulanyue 4e8419dadb fix(gateway): honor approval scope capabilities 2026-08-23 17:45:47 -05:00
kshitij 4865194772 fix(bot-mode): review follow-ups for recoverable-archive resurrection
- Clear the accidental end stamp on resurrection (at the lineage tip):
  a surviving ws_orphan_reap/agent_close reason made a LATER deliberate
  archive auto-resurrect on the next lookup — the user could never retire
  the canonical chat. Test pins the resurrect -> deliberate-archive ->
  stays-archived cycle.
- Judge recoverability at the compression TIP: the registry row of a
  compressed lineage carries end_reason='compression', so tip-stamped
  accidents were unrecoverable through the registry row. Lineage test.
- Heal the third lookup: the api_server exact-title listing (hermes peer
  dm resolution) filtered archived rows out via list_sessions_rich and
  still failed for reap-archived canonical chats.
- Single source of truth for the recoverable set: tuple moved to
  hermes_state_common (mirroring _RESET_END_REASONS_SQL) and interpolated
  into all three recovery SQL sites — literals cannot drift.
- methods_session gate uses BOT_CHAT_TITLE (not a literal) and re-fetches
  by id after resurrection (title has no DB-level UNIQUE).
- Idempotence pinned: two consecutive profiles.list calls both resolve.
2026-08-24 03:27:11 +05:30
Teknium 6cb1085d3d fix(gateway): /p/<profile>/ on a non-multiplex gateway fails closed instead of serving the owner profile
A /p/<profile>/ URL prefix on a gateway with multiplex_profiles off was
silently ignored: the request was handled as the gateway-owning profile,
so /p/lokaj/v1/toolsets reported the OWNER's platform_toolsets (and every
other profile-owned config read — skills, capabilities, model options,
agent-run toolset resolution — resolved from the owner too). That is the
exact repro in #91583 defect 2: enabling computer_use with
'hermes -p lokaj tools enable computer_use --platform api_server' showed
enabled in lokaj's config while /p/lokaj/v1/toolsets stayed false, and
enabling it on the owner profile flipped it true.

Per-profile capability isolation is the intended design (ruling on
a different profile's config. Multiplexed gateways were already correct —
the profile-prefix middleware enters _profile_runtime_scope and every
canonical config loader honors the HERMES_HOME override contextvar
(verified empirically for load_config, get_config_path and
_load_gateway_config) — the leak was only the non-multiplex fallthrough.

Fix at the one seam both adapters share: _resolve_request_profile now
rejects (404) a prefix naming any profile other than the one the gateway
actually serves. A self-referential prefix (/p/default/ on the default
gateway, /p/lokaj/ on a gateway launched for lokaj) still falls through
so existing well-formed clients keep working. Same change in the webhook
adapter, which had the identical fallthrough. New shared helper
hermes_cli.profiles.profile_matches_home does the home comparison,
fail-closed.

Tests: tests/gateway/test_multiplex_toolsets_profile_isolation.py —
E2E-style with two real profile homes + config.yamls under a temp
HERMES_HOME, real aiohttp routing through the profile-prefix middleware:
per-profile /p/<x>/v1/toolsets isolation for both owner and secondary
(the #91583 repro asserts computer_use true under /p/lokaj only),
cross-profile key rejection, and the fail-closed non-multiplex prefix
for both adapters. Sabotage-verified: reverting the adapter change fails
the 3 fail-closed tests.

Fixes #91583 (defect 2). Repro and live validation by @kubaboski.
2026-08-23 03:57:47 -07:00
John Paul Soliva 265bdcac82 fix(api-server): a /p/<profile> prefix on a non-multiplexed gateway fails closed instead of misdelivering
The prefix is an address: the caller is naming WHICH agent the request is
for. With gateway.multiplex_profiles off, _resolve_request_profile ignored
the prefix entirely — "don't 404 a would-be valid route" — so a request
explicitly addressed to one agent was silently answered by a different one.
Observed live (Aug 2026): `hermes peer dm mini/researcher` was answered by
the mini's DEFAULT agent with no error on either side, because that host
runs one LaunchDaemon per profile and only the default daemon hosted an
api_server. A wrong-agent answer is strictly worse than an error: the
sender believes the addressee got the message.

With multiplexing off the process serves exactly one profile, so the prefix
is honored when it names that profile (peers address single-profile daemons
this way without knowing the host's topology — get_active_profile_name() is
the same identity the file already uses for model resolution) and rejected
otherwise through the existing _PROFILE_REJECTED path (404). A process that
cannot resolve its own identity rejects too: if it cannot prove who it is,
it must not answer as anyone.

Unprefixed requests are untouched, and multiplexed hosts are untouched —
the change is confined to the prefix-present, multiplexing-off branch that
previously discarded the caller's addressing.
2026-08-23 03:57:47 -07:00
Teknium 231e613d3d fix(peer): resolve hidden canonical Bot Chats in hermes peer dm
Bot Mode always hides canonical 'Bot Chat' sessions, but _find_bot_chat's
GET /api/sessions listing used the default include_hidden=False path, so
the existing hidden row was invisible, _ensure_bot_chat tried to create a
duplicate, and the peer DB's UNIQUE(title) guard rejected it — DM failed.

- api_server: GET /api/sessions now accepts an exact-title lookup
  (?title=...) and honors include_hidden=1 ONLY alongside a title filter,
  so canonical hidden rows resolve without exposing a blanket hidden
  listing on the client surface. The title needle is pushed into SQL
  (search_query) so old hidden rows outside the recency window are found.
- peer dm client: _find_bot_chat sends title + include_hidden=1; older
  peers ignore the unknown params and degrade to today's behavior.
- Clear diagnosable error on the older-peer duplicate-create rejection,
  naming the hidden canonical chat and the PATCH hidden:false workaround.
- Unit tests (hidden resolution, no duplicate create, older-peer error,
  older-peer visible fallback) + real-gateway E2E over a real state.db.

Root-cause analysis and regression recipe by @kubaboski in #91583.

Fixes #91583
2026-08-23 03:56:28 -07:00
kshitijk4poor 0b8a848754 perf(api): classify compaction rows once per message in run.completed transcript
_turn_transcript_messages pre-classified every message with
_is_compressed_summary_message (full content flatten + prefix scan), then
_message_response re-ran the same classifier inside its projection --
2x per non-summary row, 3x per summary row on every run.completed emit.
The outer guard was redundant: _message_response already yields
display_kind hidden for pure handoffs. One projection call per row now.
Surfaced by the post-merge simplify re-review of #91517/#91535.
2026-08-21 22:54:13 +05:30
kshitijk4poor 23a64a97ec fix(api): correct _handle_browser_control_frame return annotation
The frame handler returns reply dicts (heartbeat/detach acks) that the WS
reader loop sends back; the -> None annotation was the only new ty
diagnostic vs origin/main.
2026-08-21 22:33:45 +05:30
kshitijk4poor 847289864d fix(browser): make the artifact boundary compose end-to-end and scope stores per profile
Addresses both merge blockers from @andrexibiza's review of #85351:

1. HTTP-uploaded artifacts could never be consumed by broker dispatch:
   artifact_scope_key hashed (principal, session, family), the HTTP routes
   store with an EMPTY session (API-key auth has no server session) while
   broker validation carries a session-bearing ControllerScope — every
   real upload->dispatch journey died with ArtifactScopeMismatch
   (reproduced before fixing). Canonical ownership is now
   principal/transport-family (documented in the scope-key docstring);
   ids stay unguessable server-minted 32-hex and downloads one-shot.
   New composition regression: HTTP-shape upload -> registered controller
   scope -> broker artifact dispatch, mutation-checked (re-adding session
   to the key makes it fail).

2. The 'profile-scoped' artifact store was first-profile-wins process
   state: one adapter-level singleton pinned profile B to profile A's
   physical root on multiplex listeners (same frozen-handle class as
   #88734). Stores are now cached by resolved profile, and the broker
   selects the store from the controller scope's profile_id (default-slot
   fallback preserves single-profile/test behaviour). New A/B multiplex
   regression proves distinct physical roots regardless of touch order.

Also documents the advertised ticket_expires_at as best-effort wall clock
(broker enforces expiry monotonically) per review feedback.
2026-08-21 22:33:45 +05:30
kshitijk4poor 13f209d4fd refactor(browser): dedupe auth-flow names and sentinel identity
- Rename the broker's TicketInvalid to ControllerTicketInvalid: the same
  exception name already exists in hermes_cli/dashboard_auth/ws_tickets.py
  and BOTH are caught in the same WS auth flow this feature touches — two
  unrelated same-named exception types in one blast radius invited a wrong
  except clause.
- Import the 'server-internal' sentinel identity from its canonical
  definition (ws_tickets.INTERNAL_USER_ID/INTERNAL_PROVIDER) instead of
  re-declaring the strings; drift would have silently broken the
  internal-peer exclusion in _is_authenticated_identity.

Surfaced during review of PR #85351.
2026-08-21 22:33:45 +05:30
kshitijk4poor 45078eb941 fix(browser): offload broker lock acquisition off the event loop
attach/disconnect/detach acquire a per-controller threading.Lock that a
worker-thread dispatch can hold for up to 10s while blocking on the event
loop to transmit its command frame (run_coroutine_threadsafe +
result(timeout=10)). Acquiring that lock synchronously from loop context
(controller WS finally, frame handler, gateway WS teardown) could park the
ENTIRE gateway event loop behind the send bridge — a deterministic
multi-second global stall whenever controller teardown raced an in-flight
command. All loop-context broker calls now go through asyncio.to_thread,
matching the existing offload pattern for _close_sessions_for_transport.

Surfaced during review of PR #85351.
2026-08-21 22:33:45 +05:30
abundantbeing 1977c3d2eb feat(browser): add scoped artifact endpoints, broker permission gates, and companion journal 2026-08-21 22:33:45 +05:30
abundantbeing 095a1d078c fix(browser): preserve controller work across reconnects
Treat unexpected controller transport loss as recoverable until each command's original deadline. Same-identity reconnects refresh transport and capability state, flush deferred cancels before new dispatch, and can complete already-started work.

Keep explicit detach and different controller/browser identity replacement terminal, owner-gate every inbound lifecycle frame, distinguish slow in-flight WebSocket writes from real send failures, and exclude browser-control session identity from shared shell snapshots.
2026-08-21 22:33:45 +05:30
abundantbeing d524cc9a16 fix(browser): harden extension controller routing
Keep extension control opt-in and preserve existing browser backends unless an exact server-bound controller is available. Centralize protocol and capability admission across API and dashboard transports, make selected-controller results authoritative, bypass stale availability caches only inside bound requests, and serialize structured results for the existing tool contract.

Add a real browser_snapshot route-table/WebSocket E2E, strict admission and ownership regressions, public configuration and protocol documentation, and tests proving feature-off/no-controller compatibility.
2026-08-21 22:33:45 +05:30
abundantbeing c9fd5223f6 feat(browser): enable extension controller actions 2026-08-21 22:33:45 +05:30
abundantbeing 5df1d0e113 feat(browser): add authenticated control broker 2026-08-21 22:33:45 +05:30
kshitijk4poor b2c4f1f376 refactor(api): reuse _COMPACTION_INTERNAL_FIELDS from compaction_display
The 7-key internal-fields tuple was inlined twice (agent/compaction_display.py
and _project_client_message); a drift between the copies would silently leak
one internal field class through the API projection. Surfaced during review
of PR #85442.
2026-08-21 21:53:11 +05:30
abundantbeing a2a23a8f7e fix(clients): hide compaction carriers across surfaces 2026-08-21 21:53:11 +05:30
abundantbeing 97e32d49ac fix(api): hide compaction scaffolding from clients
Project client-visible session messages through the canonical compaction classifier. Hide standalone handoffs, unwrap merged carriers to their authentic prior-tail content, strip inherited internal fields, and keep model-facing recovery history unchanged.
2026-08-21 21:53:11 +05:30
Teknium 2d59cb4386 fix(api-server): 'max' and 'ultra' reasoning efforts are no longer silently ignored on API/browser requests
_request_reasoning_config() whitelisted none..xhigh, so a client sending
max or ultra (valid /reasoning + config.yaml levels) fell through to the
default effort with no error. The server now accepts the full internal
ladder (hermes_constants.VALID_REASONING_EFFORTS); per-provider wire
clamping happens downstream via agent.reasoning_effort, same as every
other entry surface. Salvages the api_server hunk of #78216 (credit
@snowzlmbot); the un-clamping half of that PR was rejected separately.
2026-08-19 22:56:14 -07:00
victor-kyriazakos 3683e70043 feat(relay): live-card ops — native draft streaming + task cards over the relay (gateway half) (#85796)
* feat(relay): live-card ops — native draft streaming + task cards over the relay (gateway half)

NS-658. Three additive ops within contract v1, emitted only when the
connector's negotiated descriptor advertises them:

  {op: draft, chat_id, draft_id, content, final, metadata}
  {op: task_card, chat_id, card_id, chunks, metadata}
  {op: task_card_stop, chat_id, card_id, metadata}

The gateway side is deliberately dumb: no platform API knowledge, no new
config keys. Slack mechanics (chat.startStream/appendStream/stopStream,
per-workspace feature-gate cache, send+edit fallback) live connector-side
where the platform adapter lives in the relay model.

Semantic bridge: base send_draft is Telegram-shaped (draft clears; final
is a separate send). Slack native streaming makes the stream THE message.
The adapter tracks the open draft per chat and converts the turn-final
send() into draft(final=true) so the connector seals the stream instead
of posting a duplicate; the stream ts returns as the message identity.
A failed frame disarms interception so the edit-based fallback's real
send goes through untouched.

BEHAVIOR CHANGE (deliberate): relay supports_draft_streaming() now
requires the descriptor flag AND the draft op. Flag-only was a latent
lie — send_draft inherited NotImplementedError, so a connector setting
the flag without the op would have crashed the stream consumer's draft
path. supported_ops stays fail-open for legacy (pre-contract) ops;
draft/task_card did not exist pre-contract and must not fail open.

Task cards ride #85476's adapter-agnostic TurnRunner seam (hasattr on
send_native_task_card_progress); supports_native_task_cards() is the
descriptor probe. Connector half + E2E harness pair follow in the gg
repo.

* fix(relay): expose native_task_cards_enabled() on the relay adapter

Live-canary finding (Alice, staging): the TurnRunner's task-card lane
probes adapter.native_task_cards_enabled() (the native Slack adapter's
opt-in contract). The relay adapter only offered
supports_native_task_cards(), so the hasattr gate failed silently and
tool progress stayed on the text path — draft streaming worked, cards
never rendered. Alias it to the descriptor probe.

* fix(relay): match task-card methods to the TurnRunner's native keyword contract

Live-canary finding #2 (Alice, staging): gateway/run.py's card lane calls
send/stop_native_task_card_progress with the NATIVE Slack adapter's
signature (tasks/title/reply_to/metadata/fallback_text, keyword-only) —
PR 85796's relay methods took a positional card_id, so every call raised
TypeError('unexpected keyword argument reply_to') in the progress task,
repeatedly killing the card publisher (and the retry loop resent the
final delivery 4-5x). Card id now derives per turn thread
(turn:<reply_to>), thread_ts anchored like draft; title/fallback_text
accepted for parity, not forwarded (plan-mode stream renders chunks).

* fix(relay): one draft stream per turn for stream-is-the-message adapters

Live-canary finding #4 (Alice, staging): the stream consumer bumps
draft_id at every tool boundary so Telegram-shaped drafts animate each
text segment as a fresh preview. On relay Slack NATIVE streaming a new
draft_id opens a brand-new chat.startStream — the user saw one frozen
message per segment (stuck streaming cursor ▉, never sealed: only the
LAST stream gets the final=true seal) plus the real final; 5-6 cumulative
snapshots per turn. Adapters that mark draft_stream_is_message keep ONE
stream per turn: tool progress lives in the native task card, and the
connector's suffix-delta falls back to whole-text append on prefix
mismatch, so segments append cleanly. Telegram-shaped drafts keep the
per-segment bump.

* fix(relay): don't seal the native stream at tool boundaries — only the turn-final does

Live-canary finding #5 (Alice; supersedes the incomplete #4 which was
necessary but not sufficient). Root cause CONFIRMED by integration trace
(test_live_cards_flow_trace.py, real consumer semantics + real adapter +
stub transport): at every tool boundary the consumer calls
_send_or_edit(finalize=True), which skips the draft path and issues a
real send(); the relay adapter's seal-interception converts THAT into
draft(final=true) — sealing the stream once per segment. Timeline showed
3 seals for a 3-segment turn: exactly the frozen cumulative ▉ snapshots
seen live (the replaced stream never gets stopStream, keeping its cursor).

Fix: for draft_stream_is_message adapters, a segment-break finalize
(finalize=True, is_turn_final=False) stays ON the draft path as another
cumulative frame; only got_done (is_turn_final=True) falls through to
send() and seals. Telegram-shaped platforms unchanged. Trace test now
pins the invariant: ONE user-visible message per turn.

* fix(relay): strip the text cursor from native draft frames

Live-canary finding #6 (Alice) — the ACTUAL duplicate-content mechanism,
confirmed by full-flow scan of both sides' code + logs. The consumer
appends its text cursor (▉) to every non-final display_text tick. The
connector's stream sender diffs CUMULATIVE frames via prefix check:
'abc▉'.startsWith → 'abc def▉' is NEVER a prefix match (the cursor sits
mid-string), so deltaFor falls back to whole-text append on EVERY tick —
chat.appendStream stacks each full cumulative snapshot (cursor included)
into the ONE stream message. Exactly the observed thread: repeated
blocks, each ending in a frozen ▉, growing per tick.

Fixes #4/#5 were real (one stream per turn now) but this was the last
mechanism standing. Native streams render their own typing indicator, so
the text cursor is pure noise on this path: strip it from draft frames.
Prefix check now holds; every tick appends only its true suffix delta.

* fix(relay): seal-interception covers EVERY egress door, not just send()

Live-canary finding #7 (Alice): one duplication remained after #6 — the
stream froze mid-word with the live indicator (never sealed) and the
final posted as a separate message. Log receipt: 'Queued follow-up:
final text delivery confirmed; delivering explicit media before
continuing' — the turn's final went out via the DELIVERY RESOLVER lane
(gateway/delivery.py), which calls send_for_platform() DIRECTLY,
bypassing send() and its seal-interception. The open stream never
absorbed the final; it arrived as a plain 'send' op → chat.postMessage.

Fix: hoist the open-draft check to the top of send() (ahead of the
explicit-platform branch) AND add it to send_for_platform() — an open
native stream absorbs the turn-final regardless of which egress door it
arrives through. The stream IS the message.

* fix(relay): failed seal falls back to plain send (PR 85796 AI-review point 1)

A turn-final seal that fails at the transport must never swallow the
final answer: the stream consumer has already disabled the draft
transport for the run, so a failed _seal_open_draft returning
success=False meant the user got NOTHING. Both seal-interception sites
(send + send_for_platform) now fall through to the regular plain-send
path on seal failure, with a warning receipt. Also mitigates AI-review
point 2 (sticky _open_draft_by_chat after an abandoned turn): a stale
entry's failed seal no longer blocks the next turn's delivery.

* fix(relay): arm seal-interception optimistically; never disarm on ambiguous failure (audit G-D1)

Deep-audit defect G-D1 (HIGH): the outbound leg is at-most-once on the
wire but its ack channel is lossy — send_outbound timeout (30s) and
WS-drop 'failures' frequently mean the frame WAS delivered and the
connector stream is open. send_draft popped _open_draft_by_chat on any
failure, disarming seal-interception while the connector stream lived:
the turn-final went out as a plain send → orphaned mid-word stream +
complete duplicate final (intermittent; needs a drop/timeout inside the
draft window).

Fix: arm the entry BEFORE the transport call and keep it armed on
failure/exception. Safe in every case: sealing a non-existent stream
opens+seals a single complete message connector-side, and a truly failed
seal already falls back to plain send at both interception sites.
Stale-entry damage is self-healing (one warning + plain send).

* fix(relay): gateway-side sealed-draft tombstone — G-D1 arming must not resurrect sealed streams

Regression fix on G-D1 (live: 'worse than before' — escalating frozen
prefixes). Optimistic arming had no seal-awareness: a straggler frame
arriving AFTER the seal re-armed _open_draft_by_chat for the already-
sealed draft_id; the next send was converted to draft(final=true) on the
tombstoned connector key, which CLEARED the connector tombstone (final
frame = new-turn signal), re-opened a stream with cumulative content,
and left it frozen — repeating per straggler: 4-5 escalating frozen
snapshots. Mirror the connector: _sealed_draft_by_chat records the
sealed draft_id per chat (tombstoned BEFORE the seal's transport call);
send_draft for a sealed draft_id is a success no-op (content already in
the sealed message) and never arms. A new turn's fresh draft_id arms
normally.

* fix(relay): key stream/card state per (chat, turn anchor) — parallel turns must not collide (finding #10)

Live finding #10 (Alice; three concurrent turns in one flat DM): all
coordination state was keyed per CHAT on a one-active-turn assumption.
Three parallel turns produced: turn B's task card merged into turn A's
(both were card 'turn:root' — reply_to is None in flat DMs), B left
cardless, and _open/_sealed_draft_by_chat clobbered across writers (3x
duplicate finals on the last turn). Per-turn machinery was correct;
the keys were not.

Fix: _draft_key(chat, metadata) = chat + the turn's thread anchor
(inbound stamps thread_ts = event.thread_ts or ts on every top-level
message, so each turn has one even in flat DMs). draft arming, seal
tombstones, both interception sites, and the task-card id all derive
from the same anchor. New trace test pins two interleaved turns:
distinct cards, own-stream seals, no leaked plain send, no cross-turn
tombstone drops (289 tests green).

* fix(gateway): preserve cumulative native stream across tools

* fix(gateway): consumer-declared final — the seal carries the true final

Three composed fixes for the Slack live-cards duplicate-final class:

1. finish(final_text): TurnRunner passes the completed final_response
   (verifier footer, completion explainer included) as the authoritative
   finalize payload. The native-stream seal delivers the TRUE final, so
   post-stream mutation no longer forks a corrective plain send (#11).

2. Interim-send contract: commentary and segment-tail sends carry a
   gateway-internal _interim_send marker; relay seal-interception skips
   them at both egress doors. A mid-turn interim send can no longer seal
   the live stream and orphan the real final into a duplicate.

3. Queued-follow-up lane reconciles an unconfirmed final by EDITING the
   consumer's delivered message in place (sealed stream = regular
   message, chat.update live-verified); plain send only as fallback.
   This was the actual duplicate lane in the parallel canaries — every
   duplicated turn logged 'final stream delivery not confirmed; sending
   first response' (subagent-completion queued inbound), not parallelism.

Also: draft frames stay prefix-stable gateway-side (no fence-closing, no
segment state reset, no commentary reset for stream-is-the-message
adapters; MagicMock-safe 'is True' guards).

* test+docs: streaming-contract coverage completeness + maintenance guidelines

Coverage: two gaps closed on the consumer-declared-final contract —
(1) send_for_platform (the delivery-resolver egress door) honors the
_interim_send contract: no seal, marker stripped before the wire;
(2) finish(final_text) on a turn that never streamed does not adopt the
final (delivery ownership stays with the gateway's normal send path for
non-streaming models / tool-only turns).

Docs: AGENTS.md 'Known Pitfalls' gains the streaming delivery contract —
the four invariants of stream-is-the-message adapters (prefix-stable
frames, consumer-declared final, interim-send marker, reconcile-by-edit),
each traced to its live incident, plus the live-probed Slack streaming
API ground truth and the MagicMock 'is True' guard-style note.

* fix(relay): seal transport failure must never silently lose the final (review B1)

Two halves of one silent-loss path, live-probed on the review branch:

1. adapter: _seal_open_draft did not catch transport exceptions. A socket
   drop at seal time raised out of send(), skipping the fail-open plain
   send entirely. Now: retry the SAME idempotent final frame once (the
   connector's sealed-key tombstone returns the original stream ts for a
   repeated final — a retry can never open a second stream or duplicate),
   then report failure so the caller's fail-open path runs.

2. consumer: the turn-final retry (elif not _already_sent) called
   _send_or_edit with finalize=False, which re-entered the DRAFT-FRAME
   branch. Its no-op dedupe compared the adopted final against the last
   unsealed frame, matched, and returned True with ZERO transport calls —
   final_response_sent went green, delivered_final_matches reconciled,
   the gateway suppressed its fallback, and the user never received the
   answer. finalize=True keeps this retry out of the draft branch.

Regression suite: tests/gateway/test_relay_seal_failure.py (3 tests).
Mutation evidence in follow-up verification: reverting either half sends
the suite red.

* fix(relay): draft ids unique across gateway incarnations (review B3)

The relay connector tombstones sealed streams by (channel, draft_id) and
keeps up to 512 of them; they outlive the gateway process. Relay gateways
are disposable BY DESIGN (scale-to-zero), and _draft_id_counter restarted
at zero every incarnation — so the first turns after every scale-from-zero
in a recently-active channel replayed already-sealed wire identities. The
connector answered those frames straight out of the old tombstone: zero
Slack API calls, the OLD message ts returned as the new turn's identity,
the new answer silently dropped while gateway-side flags recorded success.

Seed the counter from wall-clock milliseconds at process start. Ids stay
plain ints within the existing contract op; incarnations cannot overlap
for realistic turn counts and restart gaps.

Regression: tests/gateway/test_draft_id_restart_uniqueness.py — the seed
test fails on the old code (seed 0 is not epoch-scale).

* fix(relay): stream/card state keyed per TURN, not per thread anchor (review B2)

The thread anchor is the wrong coordination identity — simultaneously:

- too coarse: two parallel turns replying INSIDE ONE Slack thread share
  thread_ts. Live-probed on the review branch: turn A's final sealed turn
  B's stream with A's content while A's own stream stayed open, and B's
  final degraded to a plain send.
- too fragile: a flat DM with no thread metadata degraded to the bare
  chat id, re-creating the original finding-#10 collision the anchor was
  meant to fix.

_draft_key now prefers the triggering inbound message id (message_id /
reply_to_message_id — per-turn by construction; the gateway's Slack
thread metadata and the consumer's send path both stamp it), falling back
to the thread anchor, then the bare chat. The consumer stamps the same
reply_to_message_id on draft frames so frames and the turn-final resolve
to one key. Task-card ids share the derivation via _card_key (one helper
for send AND stop, so the stop always hits the stream the send opened).

Legacy resolver-lane callers with placement-only metadata still seal via
_match_open_draft's fallback — but ONLY when exactly one stream is open.
With several open, an identity-less send stays a plain send: a duplicate
message is recoverable, sealing someone else's stream is not.

Regression: tests/gateway/relay/test_relay_turn_keying.py (7 tests).

* fix(relay): stream-is-the-message is a Slack semantic, gate it on the descriptor (review B4)

draft_stream_is_message was hardcoded True on the relay adapter class,
i.e. for EVERY relay platform. The base send_draft contract is
Telegram-shaped — the draft clears client-side and the final arrives as
a separate real send that becomes the history message. With the flag
forced on, any non-Slack connector advertising the draft op had its
turn-final intercepted into draft(final=true): probed on the review
branch with a telegram descriptor, the op stream was
[draft(final=false), draft(final=true)] and NO send — no history message
would ever be posted.

Gate the flag on the negotiated descriptor platform (slack), and skip
arming seal-interception entirely when it is off. A future platform with
genuine stream-is-the-message native streaming should advertise it via
the descriptor rather than widening the platform check by guesswork.

Regression: tests/gateway/relay/test_relay_stream_semantics_gating.py
(4 tests: gating both ways, telegram final is a real send, slack final
still seals).

* fix(gateway): mark every mid-turn status lane interim — heartbeats must not seal the stream (review B5)

Seal-interception treats the first unmarked send to an armed (chat, turn)
key as the turn-final. The consumer's own interim lanes (commentary, tail
flush) carry _interim_send, but four gateway-side lanes that fire DURING
a streaming turn did not:

- long-running heartbeat (default every 180s — probed live: at 3 minutes
  it sealed the live stream with '⏳ Working — 3 min', the real final
  posted as a duplicate, and later frames were silently swallowed by the
  seal tombstone)
- inactivity warning
- plain-text approval fallback (button lane failed)
- background-review notice

Add _interim_metadata() beside _non_conversational_metadata and wrap all
four call sites. The marker is gateway-internal; the relay adapter strips
it before the wire (existing behavior, pinned by test).

Note for follow-up: the opt-out shape remains fragile — any FUTURE
unmarked mid-turn send lane re-creates this bug. Inverting the contract
(explicitly mark the one turn-final send) is the durable fix but touches
every adapter's final-delivery path; deliberately kept out of this
review-fix series.

Regression: tests/gateway/test_interim_send_lanes.py (4 tests).

* fix(gateway): interrupted/incomplete turns must not adopt the diagnostic as the stream final (review B6)

The finish(final_text) adoption gate checked only 'not failed', but the
interrupt/abort returns in agent/conversation_loop.py are
{completed: False, interrupted: True, final_response: 'Operation
interrupted during …'} with NO failed key. Adopting that diagnostic:

1. sealed the user's streamed partial answer over with the interrupt
   text (stream-is-the-message: the seal rewrites the whole message), and
2. recorded the diagnostic as the turn-final payload, so
   delivered_final_matches reconciled and the gateway suppressed its own
   error-delivery path — the diagnostic became the ONLY thing delivered.

Enumerated all 27 final_response-bearing return shapes in
conversation_loop.py: every non-happy-path shape carries completed:
False (several with a diagnostic final_response and neither failed nor
interrupted — retry exhaustion, truncation, codex-incomplete); the happy
path routes through turn_finalizer.finalize_turn (completed=True). Gate
is therefore: not failed AND not interrupted AND completed is not False.
Results lacking the completed key entirely (older callers/test doubles)
keep the previous behavior.

Regression: tests/gateway/test_stream_final_adoption_gate.py (6 tests,
incl. a source-level pin on the run.py call site).

* fix(relay): task-card transport failures degrade to failed SendResults (review B7)

send_native_task_card_progress and stop_native_task_card_progress let
transport exceptions escape. The stop runs inside the progress loop's
finally block on the turn-cleanup path, and the post-cancel awaits in
gateway/run.py caught only CancelledError — a socket drop during a card
publish/stop therefore aborted cleanup BEFORE the final-delivery
bookkeeping ran.

Three layers, outermost defends any adapter:
- both adapter methods catch transport exceptions and return failed
  SendResults (progress is advisory; the TurnRunner's text fallback
  already handles failure results)
- the progress loop's finally wraps the stop (best-effort; the connector
  seals orphaned card streams on its own via recycling/eviction)
- the cleanup awaits log-and-continue on non-cancellation errors so
  final-delivery bookkeeping always runs

Regression: tests/gateway/relay/test_relay_task_card_failures.py.

* fix(relay): a dying turn seals its native stream instead of orphaning it (review B8)

Stale-generation exits (/new, /stop mid-stream) and cancellations
returned from the consumer's run() with the native stream still open:

- the Slack message kept its live streaming indicator forever (the
  cancellation best-effort edit only runs when _message_id exists, and
  the native draft path deliberately keeps it None);
- the adapter's armed interception state survived the turn, so the next
  turn on the same key could inherit it and seal a dead draft_id.

New adapter op abandon_open_draft(chat, content): seals in place with
the text already on screen (the consumer passes its last delivered
frame) — the seal adds nothing and claims nothing; delivery flags are
never set, so the gateway's normal paths still own whatever happens
next. Best-effort by contract (failure reported, never raised); the
connector reaps truly orphaned streams via recycling/eviction.

The consumer calls it from both death paths: the stale-generation early
return and the CancelledError handler.

Regression: tests/gateway/test_stream_abandon_on_turn_death.py (4 tests,
incl. the next-turn-inheritance hazard).

* fix(relay): bound the draft/seal coordination dicts (review M1)

_sealed_draft_by_chat's key embeds a per-turn identity, so every
completed turn wrote a permanent entry — unbounded growth for the life
of a long-running gateway process (the docstring said 'one entry per
chat', which stopped being true when the key gained the turn anchor).
_open_draft_by_chat could grow the same way via abandoned entries.

FIFO-evict both at 512 entries — the same idiom as the sibling bounded
cache (_auto_thread_by_chat, capped at 256) and the same size as the
connector's own tombstone store. The straggler window the tombstone
exists for is seconds long; FIFO is more than enough.

Regression: tests/gateway/relay/test_relay_state_bounds.py.

* fix(relay): explicit connector rejection disarms interception; exceptions stay armed (review P3)

The G-D1 optimistic-arming change silently dropped disarm-on-failure
entirely: after an EXPLICIT connector rejection (success=False result —
not a transport ambiguity), interception stayed armed even though the
stream consumer disables the draft transport on that failure and falls
back to edit-based streaming. Its turn-final would then be converted
into a seal on a stream the connector just told us is unusable.
test_draft_failure_result_propagates claimed to cover this ('must NOT
leave seal-interception armed') but passed for an unrelated reason: the
stub's canned failure also failed the SEAL, whose fail-open path did the
plain send.

Split the two semantics and pin each honestly:
- explicit rejection (result success=False): disarm — turn-final is a
  real send (test_draft_failure_result_propagates, now testing what its
  comment says)
- transport exception: ambiguous, stay armed — turn-final still seals
  (test_draft_transport_exception_keeps_interception_armed, the G-D1
  contract)

Also corrects commit ba3a24a's claim ('a failed frame disarms
interception so the edit-based fallback's real send goes through
untouched') to hold again for the rejection case it described.

* fix(relay): lost acks are ambiguous, not rejections — on the RESULT channel too (review r2, finding 1)

The production ws transport does not raise on ack timeout — it returns
{"success": False, "error": "relay outbound timed out"}. The round-1
ambiguity handling keyed entirely on the exception channel, so the shape
production actually produces was misclassified as a definite connector
rejection. Probed on the head:

- lost SEAL ack: skipped the idempotent retry, fell straight to a plain
  send — duplicate final whenever the seal had actually applied;
- lost FRAME ack: the round-1 disarm-on-rejection fired — interception
  disarmed, frozen native stream beside a plain final. This re-created
  the original G-D1 ambiguous-ack defect on the result channel.

Contract now spans both channels:

- transport: the ack-timeout branch tags ambiguous=True. The fail-fast
  branches (closing / not connected) never sent anything and stay
  unmarked — they are definite non-delivery.
- adapter frame path: ambiguous results keep interception armed (same
  as exceptions); only definite rejections disarm.
- adapter seal path: one shared _attempt() classifier — exception and
  ambiguous result both mean "unknown"; the SAME idempotent frame is
  retried once (connector tombstone returns the original stream ts for
  a repeated final). Only after both attempts stay ambiguous does the
  caller's fail-open plain send run: a possible duplicate after double
  ack loss beats a silent loss, and double ack loss on one socket
  almost always means the transport is down for the plain send too.

Regression: tests/gateway/relay/test_relay_ack_ambiguity.py (6 tests,
incl. a source-of-truth check that the transport tags the timeout branch
and leaves fail-fast branches unmarked).

* fix(relay): stream semantics + draft capability resolve per CHAT, not per primary (review r2, finding 2)

One RelayAdapter fronts N platforms (Phase 1.5): descriptors accumulate
per platform on the transport and egress is tagged per chat — but the
round-1 gate keyed draft_stream_is_message and supports_draft_streaming()
off the PRIMARY scalar descriptor. Probed on the head:

- Slack primary + Telegram chat: the Telegram chat's turn-final was
  intercepted into draft(final=true) — no real Telegram history message;
- Telegram primary + Slack chat: the Slack chat was denied native
  streaming entirely.

Resolve both through _descriptor_for_chat — the same per-chat machinery
max_message_length already uses (added for the identical class of bug:
the primary's 39000-char cap over-sending into Discord 400s):

- new stream_is_message_for_chat(chat_id) on the adapter; arming and
  NotImplementedError gating use it. The class attribute remains as the
  single-platform value and legacy-probe fallback.
- supports_draft_streaming() gains an optional chat_id kwarg (base
  signature updated; single-platform adapters ignore it). The consumer
  passes chat_id with a TypeError fallback for out-of-tree adapters.
- the consumer's four draft_stream_is_message reads collapse into one
  _stream_is_message() helper that prefers the per-chat probe
  (class-resolved, MagicMock-safe) over the attribute.

Platform-name inference ("slack") stays deliberate: a descriptor-level
semantic field is the right eventual contract but is a cross-repo wire
change — noted for the gg follow-up so future platforms advertise the
semantic explicitly.

Regression: tests/gateway/relay/test_relay_multiplatform_semantics.py
(5 tests: both starvation directions, scalar fallback, per-chat
capability gate).

* fix(gateway): split delivery + authoritative footer reconciles by suffix, not full resend (review r2, finding 3)

The _FINAL_TEXT adoption guard refuses wholesale adoption on split turns
— correct (#78541: sealed heads would repeat inside the tail) but it was
absolute: a post-split verifier footer never entered the ledger,
delivered_final_matches() reported a mismatch, and the gateway resent
the ENTIRE body+footer after the split chunks (the #11 duplicate class,
one level up).

When the authoritative final strictly prefix-extends the split ledger,
the missing suffix is the only undelivered content: append it to the
live tail and the ledger, so the finalize carries it and the recorded
payload reconciles. Non-prefix rewrites keep the full-resend fallback —
a rewrite cannot be patched onto sealed heads.

Regression: tests/gateway/test_split_final_suffix_reconcile.py (3 tests:
suffix rides the tail + reconciles, rewrite still mismatches, unsplit
adoption unchanged).

* fix(relay): cancellation mid-seal restores open state so abandon can close the stream (review r2, finding 4)

_seal_open_draft pops the open entry and writes the local tombstone
BEFORE awaiting transport I/O — correct ordering for the straggler race,
but CancelledError is not an Exception: a cancel during the await
bypassed all failure handling, leaving the remote stream live (visible
streaming indicator until connector eviction) while the local state said
'nothing open'. The consumer's abandon pass — added for exactly this
turn-death case — found nothing to close and no-oped.

On CancelledError: restore the open entry, drop the premature tombstone
(only if it is still ours), re-raise. The abandon path then seals the
stream in place with the on-screen text.

Regression: tests/gateway/relay/test_relay_seal_cancellation.py (2
tests: state restoration, and end-to-end cancel→abandon→remote seal).

* fix(relay): thread anchors are placement, not turn identity — revive the placement-only fallback (review r2, finding 5)

_match_open_draft's single-open-stream fallback was dead for its primary
intended callers: metadata carrying thread_ts/thread_id (placement-only
resolver lanes) was classified as having 'turn identity', so those sends
never reached the fallback — probed: a plain final posted beside the
still-open turn-keyed stream.

Only per-turn MESSAGE ids are identity now. Thread-anchored and bare
callers share the fallback: absorb into the chat's open stream when
EXACTLY one is open; stay a plain send when several are (duplicate is
recoverable, wrong-stream seal is not). Callers WITH a message id whose
key misses never fall back — their identity is authoritative and a miss
means the stream belongs to a different turn.

Regression: 4 new tests in test_relay_turn_keying.py (thread-anchored
seal, both ambiguous-stay-plain shapes, id-mismatch never steals).

* fix(relay): random process nonce for draft-id seeding (review r2, follow-up 6)

The epoch-millisecond seed (round-1 B3 fix) mitigates the restart-replay
class but is not a uniqueness guarantee: two gateways starting in the
same millisecond, a forked process inheriting the class state, or a
clock step backwards can all mint colliding wire identities against the
connector's per-(channel, draft_id) tombstone store.

Seed from secrets.randbits(49) instead: collision probability negligible,
no clock dependence, and ids + realistic per-process turn counts stay
comfortably inside the connector's JS number range (draft_id?: number,
2^53). Regression test now spawns two real interpreters and asserts
their seeds differ — the exact scale-to-zero restart shape, and both
start within the same second so a clock-locked seed would fail it.

* fix(relay): stamp per-turn Slack egress identity — cache is fallback only (R3-5)

The connector (gateway-gateway#210) fills chat.startStream's
recipient_user_id / recipient_team_id — required by Slack when
streaming to a channel — from metadata.user_id / metadata.scope_id.
The gateway stamped only slack_team_id per-turn and left user_id (and
scope_id) to RelayAdapter._with_scope, whose per-chat caches are keyed
on chat_id alone and overwritten by every inbound message: with users
U1 and U2 running overlapping turns in one channel, U2's arrival
overwrote the cache before U1's stream opened, and U1's stream carried
U2 as recipient_user_id.

_thread_metadata_for_source now stamps scope_id and user_id from the
turn's OWN source (setdefault — explicit values win), so identity is
turn-scoped data on the wire. _with_scope is unchanged and fill-only:
the caches keep serving restart/synthetic sends that carry no per-turn
identity, which is all they were ever safe for.

Mutation evidence: reverting the run.py hunk sends
test_thread_metadata_stamps_per_turn_user_and_scope and
test_concurrent_turns_carry_their_own_identity red; restore returns
green. The _with_scope fill-only tests pass on both trees (existing
correct behavior, now pinned against regression).

---------

Co-authored-by: Ben Barclay <ben@nousresearch.com>
2026-08-20 15:31:21 +10:00
69k4xmdfm2-blip 21260c3281 fix(gateway): carry the profile in adapter-derived session keys (#88404)
Adapter ingress derives a session key BEFORE the runner stamps
source.profile in _make_profile_message_handler, so the namespace fell
back to the active profile and every bot in a multiplexed gateway
produced agent:main:<platform>:<chat>. A Telegram private chat reports
the user's own id as chat.id, identical for every bot, so two profiles
sharing one human collapsed onto a single lane: _pending_text_batches,
_active_sessions, the busy-session guard and _post_delivery_callbacks are
all keyed on that string. A day of production logs across two bots shows
60 flushes, none carrying the secondary profile's namespace.

set_owner_profile records credential ownership on the adapter and
_session_key_profile resolves the namespace as source.profile ->
_owner_profile -> the session store's resolver, so a secondary adapter
keys into its own namespace even before the source is stamped. Stamped
sources keep priority, so relay/connector ingress, which routes per event
rather than per credential, is unchanged. _configure_profile_adapter
installs the owner alongside the other handlers, covering startup and
reconnect.

Every candidate is type-checked as a non-blank str, and every attribute
read goes through getattr: adapters are routinely built without
BasePlatformAdapter.__init__, and a duck-typed session store returns a
truthy non-string that would otherwise be interpolated into the key as
agent:<MagicMock ...>:.

Also routes the four call sites that passed no profile at all (feishu
media batches, raft, slack _session_key_for_source, telegram photo
batches) through the same resolver.

test_multiplex_busy_input_mode's secondary-adapter busy case seeded
_active_sessions with the unstamped agent:main: key, asserting the
pre-fix collapse. It now seeds the lane the profile-owned adapter
actually derives.

A primary adapter has no owner and an unstamped source, so it resolves
exactly as before; with multiplex_profiles off the resolver returns None
and every key is byte-identical to today's.
2026-08-19 02:00:16 -07:00
k0rnacki 1352439036 fix(api_server): recovery net must never recover the advertised virtual alias
The #35314 empty-dispatch recovery cache accepts any previously
resolved model string, including the advertised virtual model
(hermes-agent by default) — which is never a dispatchable model.
Guard both directions: never store the alias in _last_resolved_model,
and never serve a cached value equal to it. The alias now survives at
most one turn and cannot propagate across turns through the cache.

Class bug behind #79101; complements the session-row symptom fixes
(#72739, #79102).
2026-08-17 16:54:01 -07:00
Andy Lo-A-Foe 152b1940d7 fix(api-server): stop persisting the virtual model alias as a session's model
Salvaged from #72739 (net diff onto current main; the read-side legacy-row
guard now flows through the shared _stored_session_model() helper that
landed in #88751, keeping one resolver for both chat sites).

POST /api/sessions persisted the advertised virtual alias (hermes-agent)
whenever the request omitted model or echoed the alias back; later turns
replayed it upstream as a real model id and every turn on the session
failed. Null the alias in _session_runtime_request_from_body() — shared by
session create, chat, chat-stream, and model-lock — and stop the
create-handler's raw-body fallback from bypassing that normalization.
2026-08-17 16:54:01 -07:00
Teknium 189de16280 fix(peer): live-verified cross-gateway bot DMs — session shape + virtual-model replay
Two bugs found by a REAL two-gateway live test (two isolated HERMES_HOMEs,
bravo running the api_server platform, alpha's agent autonomously running
`hermes peer dm` from its Bot Chat protocol; reply relayed correctly and
persisted in bravo's canonical Bot Chat):

1. peer dm parsed the session-create response flat, but api_server wraps
   the row: {"object": "hermes.session", "session": {...}} — every first DM
   to a fresh peer failed with "Peer did not return a session id" (and the
   orphaned Bot Chat then 400'd retries with duplicate-title). Parse the
   wrapped shape; the test fake now mirrors the real response shape so this
   class can't pass green again.

2. api_server: a session created with no model persists the advertised
   virtual model ("hermes-agent") on the row; session chat then replayed it
   as a REAL model id and the provider 400'd ("hermes-agent is not a valid
   model ID"). _request_agent_overrides already filters the virtual model
   for per-request bodies — apply the same filter to the stored session
   model at both chat sites (sync + stream), so it means "gateway default"
   exactly like the request-body path.

Live E2E transcript (bravo's Bot Chat, via /api/sessions/{id}/messages):
  user:      Message from 🤖 alpha (@alpha): What is your callsign?
  assistant: CALLSIGN-BRAVO-7
peer cmd unit suite 10/10 with the corrected fake.
2026-08-17 16:34:22 -07:00
Shannon Sands f755ed5e90 fix(gateway): surface multiplex profile failures (OOF-3) 2026-08-16 20:32:09 -07:00
Shannon Sands c127d0c3b1 fix(gateway): attribute scoped credential lock conflicts to the owning profile (OOF-3)
Scoped credential locks (Telegram bot token, Discord bot token, etc.) are
machine-global, but the conflict error only reported the holder's PID:

    Telegram bot token already in use (PID 559). Stop the other gateway first.

On multi-profile hosts (e.g. hosted instances running 13 profiles), a bare
PID gives the operator no way to tell WHICH profile owns the credential —
the exact failure mode observed on zerocool-9781, where the 'default'
profile was misconfigured with the same bot token as 'lead-gen-outreach'
and logged an unattributable conflict every ~5 minutes (4,602 rows).

Fix:
- acquire_scoped_lock() now stamps a 'profile' label on lock records,
  inferred from the process HERMES_HOME (<root>/profiles/<name> layouts,
  'default' for the root home). Omitted when not inferable.
- New scoped_lock_owner_label() resolves the owning profile from a lock
  record: prefers the explicit field, falls back to inferring from the
  persisted hermes_home for locks written before the field existed.
  Labels are validated against the profile-id grammar before use (lock
  files are plain JSON on disk and the label flows into log lines and a
  suggested CLI command).
- _acquire_platform_lock() conflict message now names the owning profile
  and gives the correct remedy:

    Telegram bot token already in use by the 'lead-gen-outreach' profile
    gateway (PID 559). Stop that gateway first
    (hermes --profile lead-gen-outreach gateway stop).

  Records with no attribution signal keep the original PID-only wording.

Testing:
- New TestScopedLockOwnerLabel suite covering label inference (named,
  Docker, root/default, unknown layouts), grammar validation, explicit-
  field preference, hermes_home fallback, and legacy/malformed records.
- acquire_scoped_lock tests for profile stamping and omission.
- Adapter-level tests for profile-attributed, legacy-home-inferred, and
  PID-only conflict messages.
- 76/76 targeted gateway tests pass; broad gateway suite failures are
  baseline-identical (verified via git stash comparison). Ruff clean.
2026-08-16 20:32:09 -07:00
liuhao1024 8dc427949f fix(gateway): accept CJK full-width punctuation as MEDIA path terminators
MEDIA_TAG_CLEANUP_RE (and MEDIA_EXTENSIONLESS_TAG_RE) only recognized
ASCII terminators after a MEDIA:<path> tag. Chinese-language agent
output naturally writes MEDIA:D:\...\zhibao.pdf(782.6 KB)or ...pdf:内容 —
the full-width punctuation failed the trailing lookahead and the
attachment was silently dropped (cron even reported 'delivered') (#88038).

Both lookaheads now accept a CJK full-width terminator set (()〈〉《》:,。;
!?、curly quotes【】) alongside the ASCII set. The #68773 adjacent-tag
splitting guard is covered by a regression test.
2026-08-16 19:54:42 -07:00
webtecnica b7936c892d fix(gateway): catch BaseException in _process_message_background to notify on SystemExit
The fire-and-forget handler only caught asyncio.CancelledError and
Exception, so a SystemExit/KeyboardInterrupt escaping a turn (e.g. a
plugin calling sys.exit() in a tool call or summary-LLM path) skipped
the user-facing failure notification and surfaced only as 'Task
exception was never retrieved' — radio silence for the user.

Catch BaseException instead; send the failure notification first, then
re-raise SystemExit/KeyboardInterrupt to preserve shutdown semantics
for the loop's own signal handling. Other BaseExceptions stay contained.

Closes #86651
2026-08-16 01:51:10 -07:00
liuhao1024 f13f3401a1 fix(webhook): authenticate Linear deliveries via linear-signature HMAC (#87348) 2026-08-16 01:45:37 -07:00
Nicolas Formenton c75d835555 feat(desktop): mark a session as unread/read with a persisted watermark 2026-08-15 01:21:40 -07:00
Teknium fbaea9bddc feat(sessions): generic 'hidden' session flag (sidebar-hide, still resumable) (#86797)
* feat(sessions): generic 'hidden' session flag (sidebar-hide, still resumable)

Adds a source-orthogonal, archive-orthogonal 'hidden' session flag meaning
'don't show in the global Sessions sidebar, but stay fully resumable by the
surface that owns it'. Mirrors the existing archived/pinned capability end to
end, so it's a generic widening (any plugin that owns its own session lifecycle
- kanban, Bot Mode, future plugins - can keep its sessions out of the shared
recents list) rather than a per-plugin special-case.

- Schema: hidden INTEGER NOT NULL DEFAULT 0 on sessions (additive; lands on
  existing DBs via the declarative _reconcile_columns ADD COLUMN path, same as
  archived/pinned - no version-gated migration).
- DB: SessionDB.set_session_hidden(session_id, hidden) (clones set_session_pinned
  incl. the compression-lineage recursive CTE); list_sessions_rich gains
  include_hidden=False, appending 's.hidden = 0' by default so hidden rows drop
  from every listing path (and the REST sidebar endpoints inherit it with no
  change).
- Gateway: session.set_hidden RPC (mirrors session.title); session.create accepts
  hidden=true, deferred via pending_hidden and applied in _ensure_session_db_row
  when the row is lazily created (mirrors pending_title).
- REST parity: PATCH /api/sessions/{id} accepts+bool-validates 'hidden' ->
  set_session_hidden; _session_response exposes it.

Enables Hermes-Bot-Mode to hide canonical 'Bot Chat' sessions from the sidebar
(NousResearch/Hermes-Bot-Mode#46) WITHOUT retagging source (which would mis-set
the agent platform). Bot Chats keep source=desktop. Gateway RPC needs a
SERVE-backend restart to take effect live. 1 focused test (default-exclude /
include_hidden / unhide round-trip).

* fix: teach lost-and-found recovery about the 55-column sessions layout

Adding the 'hidden' column makes the current sessions table 55 columns. The
SQLite lost-and-found recovery classifier keys off the physical field count
(SESSIONS_LAYOUT_NFIELDS) to identify a salvaged sessions row, so a recovered
current-layout row (nfield=55) would otherwise be unrecognized and dropped.
Add 55 to the frozenset (54/52 stay as historical prefixes) and update the
column-count assertions + synthetic current-layout insert in the recovery test.

---------

Co-authored-by: Teknium <teknium1@users.noreply.github.com>
2026-08-15 00:31:37 -07:00
joaomarcos 39e480c051 fix(state): close leaked SessionDB connections on exception paths (#83226)
SessionDB could leave native SQLite handles open when construction failed
partway through schema/pragma/FTS/repair/lock/interrupt handling. Other
short-lived callers (MCP reads/polling, session search, reactions, trace
upload, insights, shutdown recovery) opened temporary SessionDB handles
without a complete ownership boundary. API-server profile caches and
RetainDB shutdown had similar late-close races. Under sustained load this
exhausted file descriptors (EMFILE).

- Close partially initialized SessionDB connections on every constructor
  exception path via a finally block guarded by an initialization-complete
  flag.
- Close temporary/cross-profile SessionDB handles in finally blocks across
  CLI, MCP, search, trace, reactions, insights, and recovery paths.
- Add API-server per-profile cache ownership and disconnect cleanup.
- Make RetainDB writer-queue shutdown exception-safe: track connections per
  thread, close on worker exit, reject new enqueues after shutdown starts,
  and sweep any connections left by short-lived threads.
- Add regression coverage for constructor failures, worker-thread readers,
  API disconnect failures, shutdown recovery, RetainDB late enqueue, and
  foreign-loop async clients.

Salvage notes: the original PR's per-thread WAL-reader ownership changes
were superseded by main's read-connection pool (permits + checkout/return);
its cron timeout-abandon fix is credited separately to #72822's earlier
identical fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-14 21:41:26 -07:00
Teknium 480342232a fix(gateway): close leaked poller sockets in weixin/email adapters (#79889)
On macOS (256 soft fd limit), routing the weixin/email pollers through a
local HTTP proxy leaked one TCP socket per failed poll/connect cycle
until the gateway hit `[Errno 24] Too many open files` and crashed
(launchd respawn loop). Live capture showed 216 of 256 fds pinned on
connections to the proxy, ~214 of them abandoned.

Code-side gaps fixed:

- email adapter, `connect()`: no try/finally around the IMAP test
  connection — a failure in login/ID/select/search abandoned the
  connected socket with no owner. Every reconnect-watcher retry builds
  a fresh adapter, so each retry against an unreachable/proxied host
  leaked another fd. Teardown now runs in `finally`.
- email adapter, IMAP teardown: `imaplib.IMAP4.logout()` only swallows
  `OSError` internally; on a broken connection `LOGOUT` raises
  `IMAP4.abort` before the internal `shutdown()`, leaving the socket
  open. New `_close_imap()` helper chases a failed `logout()` with an
  unconditional `shutdown()`; used in `connect()` and
  `_fetch_new_messages()`.
- weixin adapter: repeated poll failures through a proxy strand
  sockets in the aiohttp connector where the tight keepalive reaper
  never sees them. The poll loop now recycles its ClientSession
  (swap-then-close, safe for concurrent `_process_message` tasks)
  after each MAX_CONSECUTIVE_FAILURES streak, tearing down the
  connector and every socket it holds.

Targeted tests: tests/gateway/test_poller_fd_lifecycle.py (9 tests).

Reported by @EthanHunter1229 with measured fd captures.
2026-08-14 21:38:54 -07:00
ygd58 525ca9ca1c fix(gateway): match the profile-namespaced session key in the clarify bypass lookup
Fixes #82975.

The adapter-level clarify reply bypass in gateway/platforms/base.py's
handle_message() built its session_key via build_session_key(...)
without a profile= argument, defaulting to the legacy agent:main
namespace. The runner registers pending clarifies under
SessionStore._generate_session_key()'s key, which DOES include
profile=self._resolve_profile_for_key(source). Under a named-profile
multiplex these diverge, so the bypass lookup at
clarify_gateway.get_pending_for_session(session_key, ...) misses --
the user's answer to a pending clarify() gets routed to the adapter's
busy-session queue instead of resolving it. The turn then hangs until
the clarify's 3600s timeout, with no inbound message: log line and no
"Gateway intercepted clarify text response" log line, matching the
reported Telegram symptom exactly.

Verified the divergence directly: _resolve_profile_for_key() returns
None when multiplex_profiles is off (default) -- byte-identical to
the prior implicit profile=None, so this only changes behavior for
multiplexed deployments, matching the issue's exact reported scope.

Fixed by using the same self._session_store._resolve_profile_for_key()
the runner's key generator calls, guarded with getattr() + a None
fallback since _session_store is set via a setter and can be unset
for adapters that never call set_session_store() -- preserving prior
behavior for any such adapter rather than introducing a new crash.

Added a regression test alongside the existing bypass coverage: with a
mocked session_store configured for profile multiplexing, a clarify
registered under the profile-namespaced key must still be found and
resolved (not routed to the busy queue). Verified as a genuine
regression by reverting the fix and confirming the new test fails
with the exact reported symptom (the message handler never gets
awaited -- the clarify lookup misses).

21/21 pass across the five directly related clarify test files;
16/16 across the broader multiplex/clarify-progress test files (no
regression).
2026-08-14 20:57:37 -07:00
Evgenii 1a8625abee fix(cron): harden gateway fire admission and provider compatibility
- The gateway api_server fire webhook acknowledges 202 only after a
  durable claim + execution row exist (admission failure stays retryable
  as 503; a live claim answers 200 duplicate), then dispatches the
  claimed snapshot with the live runner adapters (delivery parity with
  the built-in ticker, including relay-fronted and E2EE platforms).
- Legacy single-phase providers (a documented fire_due override without
  split hooks) keep being driven through their own hook. Capability
  detection now credits claim_fire AND fire_claimed overrides, so
  Chronos is correctly classified split-aware (its re-arm lives in
  fire_claimed; the redundant fire_due passthrough override is removed).
- Multi-profile dashboards fail closed for external providers: an
  unscoped reconcile would disarm other profiles' armed one-shots in the
  shared NAS registry.
- Manual runs (cronjob run) carry the owner-bearing claimed snapshot
  through every entry point, composing with upstream's manual-run
  heartbeat (#76502) and background dispatch.

Note: current main moved the dashboard NAS webhook to a pure
forward-to-gateway design (the gateway owns execution and live
adapters), so the dashboard-side claim/tracking machinery from earlier
revisions of this PR is dropped; the durable admission contract lives in
the gateway webhook path.
2026-08-14 20:46:50 -07:00
Teknium 75736cd3a4 fix: don't double-count session-stream turns in the shutdown drain
The session chat stream registered its wrapper task in _active_run_tasks,
but that turn is already counted by active_agent_work_count() via
_inflight_agent_runs (_run_agent) — the drain saw 2 for one turn
(test_session_chat_sse_turn_is_interrupted). Keep only the agent-ref
registration; run-scoped steer control doesn't need the task entry.
2026-08-13 09:34:08 -07:00
Jon Komet 001bcb908e feat(api): steer active runs
Adds POST /v1/runs/{run_id}/steer and bridges Browser-Extension/WebUI
session chat streams into the active run registry so live runs on those
surfaces are steerable too.

- steer accepted only while run status is exactly 'running'; stop/stopping/
  terminal states return 409 run_not_accepting_steer even while cooperative
  shutdown retains the agent reference
- session SSE disconnect/cancellation interrupts and drains the executor-
  backed run instead of cancelling only the async wrapper; control refs stay
  registered until the turn actually exits
- undelivered steer text (accepted after the final response) is preserved as
  pending_steer on the terminal run.completed event/status so clients can
  replay it as the next user turn instead of losing it
- docs for the endpoint, next-tool-boundary delivery, acceptance-vs-delivery
  semantics

Salvaged from PR #54466 by @abundantbeing.
2026-08-13 09:34:08 -07:00