Fence owner discovery by profile, lease and loopback endpoint, and hand the authenticated URL to the existing Ink transport without acquiring a competing lease. Distinguish lease age from turn activity in unsupported-owner recovery.
Client slice only: requires the integration runtime to advertise shared_runtime_url and provide the session-attach handshake. Classic CLI attachment remains an integration gap.
_drain_queued_prompt attaches the transport pinned to the queued envelope, but the client that queued the prompt may have disconnected while the prompt sat in the queue. Attaching it then pins a dead peer into the session's fan-out, where it costs one failed write before the fan-out prunes it. That is not a regression — the line this replaced rebound the whole slot to that same dead transport, which cost the session for the entire drained turn rather than one write — but the guard is one condition, so it goes in.
_transport_is_dead is the predicate already used by the reaper and the orphan check: the parked drop sentinel, or a transport carrying _closed. Drain semantics are otherwise unchanged. The prompt still runs, and the clients already attached keep their stream; only the dead peer's attachment is skipped.
tests/tui_gateway/test_multi_client_fanout.py pins that the drained prompt still dispatches while the disconnected queuer stays out of the slot, alongside the existing case where a live queuer is attached and the single-client control where the queuer takes an unoccupied slot.
Raised by the automated review on #86784.
subagent.steer resolves authority by comparing the request's context-bound transport with the session's transport slot. Once a session mirrors to more than one client that slot holds a FanoutTransport, so the comparison fails for every client, the peer that commissioned the subagent included, and every steer is rejected. Fan-out without this check ships that regression, and no existing test catches it because the suite only exercises single-client sessions.
Authority now asks whether the request's transport is attached to the session, directly or through the fan-out. The single-client case is unchanged: a bare slot still compares by identity.
This widens authority. Any client attached to a mirrored session can steer that session's subagents, not only the peer that commissioned them. Narrowing it back to the commissioning peer requires recording that peer per subagent, which this change does not do. tests/tui_gateway/test_multi_client_fanout.py pins the commissioning peer's authority inside a fan-out, pins the widened case, and keeps a single-client control in which an unattached client is still refused.
The four browser.controller.* handlers gate on the session transport slot exactly as subagent.steer did, through one shared gate in the _controller_method decorator, and the conversion missed them. Once a session mirrors to more than one client the slot holds a FanoutTransport, which is identical to no peer's WSTransport, so browser.controller.register, .result, .heartbeat and .detach all answer "session is not owned by this transport" for every client, the peer that registered the controller included. Browser control is therefore unusable on any mirrored session. No existing test catches it because the browser-control suite only exercises single-client sessions.
All four now ask the same question steer asks: is the request's transport attached to this session, directly or through the fan-out. The single-client case is unchanged, because a bare slot still compares by identity.
Only registration widens. On .result, .heartbeat and .detach the broker's is_owner check sits below the session gate and compares controller.owner is owner against the transport recorded at attach time (gateway/browser_control_broker.py), so a mirrored peer that did not register the controller is still refused there, now with "controller is not owned by this transport" instead of the session message. browser.controller.register has no such check, so any client attached to a mirrored session may register a controller for it; the broker's principal lane keeps that inside one authenticated identity, and a second identity in the same lane hard-replaces the first. tests/tui_gateway/test_multi_client_fanout.py pins the registering peer's access inside a fan-out, pins the widened and broker-refused cases, and keeps a single-client control in which an unattached client is still refused on all three scope-gated handlers.
A session held exactly one transport, and prompt.submit, session.resume, session.activate, and the queued-prompt drain all rebound that slot. A second client therefore took the stream away from the first: the earlier client stopped receiving the turn it was already rendering, and either client disconnecting parked the whole session on the drop sentinel.
FanoutTransport goes in the same slot and satisfies the same Transport protocol, so write_json and every other reader of the slot are unchanged. It delivers each frame to a snapshot of its peers, concurrently when more than one peer is attached and the caller is not on an event loop, and prunes any peer that returns False or raises. A dead client is dropped; a slow one costs the emitter at most one write timeout per frame rather than one per peer. Request/response RPCs are unaffected: they still answer on the request's context-bound transport, so a client only ever sees replies to its own calls.
The rebind sites become attach sites through _attach_session_transport, whose ladder keeps the single-client shape identical. The same object already in the slot is a no-op; an empty, stdio, or parked slot is taken outright; only the arrival of a second live client wraps both. The queued-prompt drain is included because it pinned the drained turn to the queuer and silenced everyone else. A non-peer newcomer such as stdio or the drop sentinel never displaces a live client, so an activate dispatched without a bound websocket cannot silence the socket that owns the session.
Disconnect detaches first. A session that retains another client keeps streaming and is neither parked nor reaped, and only the clientless ones follow the existing close_on_disconnect and park-sentinel path, so a single-client disconnect, the orphan reaper, and its grace window behave as before. _ws_session_is_orphaned is unchanged: it still asks whether the drop sentinel is in the slot, and a fan-out is never the sentinel, so a session that still has a peer is never reported as orphaned.
Attach performs no entitlement check: any authenticated peer may mirror any session.
The fan-out architecture follows the approach in #40822 by @OmarB97.
What the slot's later history forces. _close_sessions_for_transport drops the #83716 rebind-to-the-most-recent-surviving-viewer, which fan-out membership subsumes — a pop-out window is a peer, so a session that still shows in one is never returned as clientless — and keeps the #77129 revalidation before parking, now expressed as a liveness check under _session_transport_lock so it is race-free against attach and detach. _transport_is_live_peer defers its last answer to _transport_is_dead: a socket that already latched _closed is a departed client, and admitting it would keep a session out of both the park and the reap. _transport_is_dead also learns the fan-out: a FanoutTransport with no live peer is dead, so a session whose peers were all pruned by failed writes cannot outlive the TTL and LRU reapers. The upstream test that pinned the #83716 rebind, test_close_transport_rebinds_session_to_remaining_viewer, is re-expressed in fan-out terms: both windows attached, the pop-out closes, the session stays with the main window unparked and still receiving frames.
`key_cmd` (#86891) authenticates a provider with a SHORT-LIVED bearer minted
by a command — SSO/OIDC brokers, cloud IAM, internal auth proxies. The
request path has honoured it since it landed, but the picker resolved probe
credentials from `api_key`/`key_env` ONLY, so a key_cmd provider probed
`/v1/models` with an EMPTY key.
Against an authenticated endpoint the probe 401s, discovery returns nothing,
and the provider falls back to its single configured default model. The
picker shows ONE model, indistinguishable from an endpoint that genuinely
serves one — while inference keeps working, because that path mints
correctly. Reproduced against a LiteLLM gateway behind Entra OIDC: 0 models
discovered with an empty key, 26 with the minted token.
Both picker probe sites already funnel through `_entry_credentials()`, so
the fix lands in one place: it now reports a `cmd:<key_cmd>` identity, and
each site falls back to `resolve_probe_token()` after api_key/key_env. An
explicit static key still wins, so existing configs are unaffected.
The identity is keyed on the COMMAND, never the minted token: the token
rotates on every refresh, so keying on its value would change the group
fingerprint constantly and force a re-probe on every open. Two entries on
one URL with different helpers still get distinct rows.
`resolve_probe_token()` lives in agent.command_token_source, which already
owns key_cmd minting, and shares the CommandTokenSource cache with the
request path — a cache read, not a fresh sign-in. Fail-closed: a helper
needing an interactive sign-in degrades to today's empty-key behaviour
rather than taking down every other provider's row.
`_model_flow_named_custom` (the `hermes model` setup flow) is the sibling
path — it builds its own `Authorization: Bearer` from the same incomplete
resolution — and is fixed the same way, with one ordering constraint: the
value persisted to config.yaml is computed BEFORE the mint, so a short-lived
bearer can never be written back to shadow the key_cmd meant to re-mint it.
Tests drive the real code paths and assert on the credential each probe
receives rather than on function source, so a semantics-preserving refactor
does not fail them. Verified they fail with the fix reverted.
Complete the type-array normalization salvaged from #55643: stringify mixed
union enum metadata, preserve existing anyOf constraints, and keep array
items and object properties/required on the corresponding typed branches.
Exercise real native request serialization over loopback and Google SDK
validation with a scalar control; no live Google credentials were available.
The enum-compatibility check evaluated `[...] in {...}` on an array `type`,
raising TypeError: unhashable type: 'list' and aborting translation of the whole
tool catalog rather than the one offending tool.
Addresses both review points:
- Reuses tools.schema_sanitizer._normalize_type_array instead of picking the
first non-null member, so a real union becomes an anyOf of single-type
branches and no branch is dropped.
- Derives the type outside the key loop and sets nullable after it, so the
flag implied by "null" in the array beats an input nullable: false whichever
key the producer emitted first. Both orders are pinned by a parametrized test.
Since #90674 hermes-ink restores the shifted letter's case for CSI-u and
modifyOtherKeys input, so Cmd+Shift+Z reaches the composer as inp 'Z' with
key.shift set. The redo branch compared inp === 'z' and missed, falling
through to the printable path and inserting a literal "Z". The legacy
raw-byte path (ESC Z) already delivered 'Z', so the binding was latently
case-sensitive on both paths.
Compare with inp.toLowerCase(), matching the copy/paste chords a few lines
above. Linux Ctrl+Shift+Z is unaffected (ctrl chords keep the lowercase
key name).
The test drives the real TextInput through renderSync with kitty CSI-u
bytes (super+z, then super+shift+z) and asserts the redo lands and no "Z"
is inserted; it fails on the unfixed tree with "aZ".
Ghostty (and any terminal reporting modified letters via the kitty CSI-u
or xterm modifyOtherKeys protocols) sends Shift+R as a lowercase keycode
with a shift modifier. keycodeToName() lowercases the printable ASCII
range, so the composer received 'r' instead of 'R' — uppercase input was
silently destroyed. The shift flag was parsed correctly but discarded at
the input layer.
Re-apply shift to a single lowercase letter in inputForSpecialSequence so
the inserted text is case-restored while keybinding consumers still see
the lowercase canonical name via key.name. Covers both the CSI-u and
modifyOtherKeys paths.
Fixes#90663
Route local producers to durable owner ingress before attempting the unowned
CLI lane. Preserve per-run/per-message IDs and receipt-first retry handling;
never fall back after ambiguous admission. Report cron admission as queued,
not completed or failed, in job status, the execution ledger and CLI/tool UX.
Native isolated Electron validation reproduces SESSION_NOT_OWNED on main for
both idle and busy owners. Fixed owner consumes idle cron, busy cron, local
DM and mounted-chat cron exactly once, keeps its lease, yields to queued
human input, and preserves the prior model-request prefix and tool schema.
Inference alone used a deterministic loopback wire stub; no paid model call.
Adapt FalconOrtiz's owner-mailbox proposal from #101564 onto the current
notification poller and topical modules. The durable mailbox is cross-process
ingress only: the existing owner admits its normal prompt turn after the
current turn and human FIFO clear. Retain immutable receipts, stable admission
identities, capability and lease fencing, compression lineage, and disable
blind recovery replay of imported turns.
The original stale server hunks and expiring receipt protocol were rebuilt
rather than cherry-picked: the current facade decomposition and durable busy
admission contract differ. Credit the earlier owner-mailbox work in #100544
and durable producer work in #100319.
Co-authored-by: fangliquanflq <fangliquan@qq.com>
Co-authored-by: 686f6c61 <github@00b.tech>
Adapt the command-position, bounded-candidate and launcher-option work from
embwl0x's #76063 to the current detection owner, then add executable basename
projection from Rohith Pariki's #104338. Parse raw quote state before applying
existing text normalization so quoted arguments do not become commands.
Cover shell payloads and literal env split-string carriers, retain path-specific
rules and whole-command globs, and document the supported normalization rather
than claiming an OS capability sandbox. Related: #104308, #76037, #76063,
#104338, #78521, #86711. No automatic closing directives: the older carriers
also contain broader case syntax and git-option work not included here.
Co-authored-by: embwl0x <embwl0x@users.noreply.github.com>
Co-authored-by: Rohith Pariki <rohithpariki@gmail.com>
Carry the owning task's notification subscriptions independently of dependency
edges, within the creation transaction. Prefer its durable session over worker
and request-local sessions while preserving explicit overrides. Cover worker
CLI create and built-in decomposition, and retain conversation route anchors.
Auto-subscribe no longer upgrades an inherited passive subscription.
Slim adaptation of Christopher-Schulze's session-precedence fix in #85687,
expanded to durable subscription provenance and sibling creation paths.
Related: #85575, #85687
Validation: strict RED/GREEN (7 failing cases before; 7 passing after), then
58 Kanban test files: 383 passed, 2 skipped. Real dispatcher-spawn subprocess
probe covers direct, linked, unlinked, explicit-session, worker CLI, built-in
children and a plain CLI negative control, with recording transport only.
Co-authored-by: Christopher <210261288+Christopher-Schulze@users.noreply.github.com>
Carry canonical scope_id and parent_chat_id through session context and reply
metadata. Stamp slash-created subscriptions with the routed source profile,
not the notifier process profile. The gateway facade only wires the topical
metadata helper and existing context binding.
Two real gateway/DB invariants reproduced missing anchors and wrong owner
before implementation. Related: #101196 and #101397.
Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
Authorize route-only profiles using the ordered canonical route matcher and
served-profile set at both claim and delivery. Preserve secondary credential
boundaries, retry denied routes, and keep scope/parent anchors plus transport
provenance on synthetic wakes. Install the destination runtime scope rather
than inheriting the notifier's scope; a removed profile cannot wake as primary.
Slim forward-port of the direction in #101196/#101397 and #93863 (#93851).
Canonical scope_id takes precedence over the guild alias and live chat cache.
Two route invariants and two runtime-scope invariants reproduced red first.
Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Salvage #82561's idle watch consumer through the existing drain, retaining
notify-off suppression and retrying failed injections. Preserve #102647's
secondary-profile ownership for reconstructed sources, preflight, synthetic
completion turns, and raw process progress/final notices. Unavailable
transports defer rather than borrowing the primary bot or losing the event.
Keep retained transport provenance and alias-aware relay resolution. Add
three invariant tests and an offline real-process/Discord lifecycle eval.
Co-authored-by: trwpang <134544126+trwpang@users.noreply.github.com>
Co-authored-by: holny <holny@foxmail.com>
Preflight source-first transport resolution and parent/API database readiness
before spending any durable delivery attempt, including every batch sibling.
Unavailable targets remain retryable; terminal targets retain their drop
semantics and stateless API completions remain durable display rows, not turns.
Minimal combined salvage of #82704 and #96086; adjacent eligibility, profile
adapter selection and idle watch-consumer work remain separate.
Co-authored-by: Screaming Sun <5552699+eventh0riz0n@users.noreply.github.com>
Co-authored-by: fangliquanflq <fangliquan@qq.com>
Salvage the busy-auth ordering fix from #95602. Gateway-owned wakes have no external identity; keep external and plugin authentication unchanged. Preserve pending human text by appending wakes through the existing FIFO instead of base-adapter text merging.
Co-authored-by: Robert Montelongo <ramontelongo22@gmail.com>
Bind settlement to the exact adapter task and event. Rejected or cancelled preparation refunds the existing claim; cancellation after the agent runner starts remains counted. Keep profile-scoped callback context and the manager replacement guard. A fire count is not outbound delivery proof. Drop departed routes rather than executing their stale schedule.
Slim accounting-invariant salvage of #93174; preserve current direct adapter dispatch instead of reviving its FIFO/inflight implementation. Prior art #92858.
Co-authored-by: Finn763 <165816600+Finn763@users.noreply.github.com>
Co-authored-by: fangliquanflq <fangliquan@qq.com>
Recover active watches from current persisted session origins and exact route
keys, reading heartbeat state off-loop in each source's profile. Failed scans
leave watches intact for the poller's next retry. Start the heartbeat poller
even when startup restores no watches.
Slim synthesis of #92660, #98310 and #98313, with earlier restart recovery
prior art from #92594. The integration poller calls restore_heartbeat_watches
on every poll, including empty registries.
Co-authored-by: chelsealong <chelsealong@126.com>
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Co-authored-by: fangliquanflq <fangliquan@qq.com>
Clear the departed schedule using the route-owned database when ending a conversation, so stale watches cannot fire after reset and resuming the archived session cannot resurrect the schedule. Compression migration already archives its parent and remains unchanged.
Adjacent lifecycle reports: #80273 by pierrenode and #80208 by 0xGr1mm. This follow-up covers durable reset cleanup, not their poller changes.
Namespace delivery markers and assign fresh keyless turn identities instead
of inferring ownership from IDs or process-local row baselines. Query only
marker existence on the canonical live compression continuation and ancestors.
Preserve raw reply IDs and exclude metadata from provider wire messages.
Expand the two existing invariants with resumed cross-chat ID collisions,
a real independent SQLite writer, reaped siblings, and archived-history
allocation controls. All 20 full-handler checkpoints and 63 targeted tests pass.
The browser and Modal fixtures replaced agent/hermes_cli with packages whose empty search paths hid newly imported production modules. Point those fake package search paths at the real package directories while retaining their explicit collaborators. This exercises real secret-scope and subprocess helpers without weakening the lifecycle or snapshot assertions.
Keep the exact hidden-directory and visible-hit assertions, but avoid an incidental process-killer token in the fixture query. The installed live-system pytest guard flattens argv and mistakes real skill plus a hermes temp path for a process-killer command. Neutral fixture text leaves that guard enabled and the search behavior unchanged.
The splice hazard the tail seed guards against is live in remote-lifecycle's
scrapeReadyPort: the remote spawn log merges stdout and stderr, yet it was
matched with the `^`-anchored READY_RE. Export one READY_IN_MERGED_OUTPUT_RE
(lookbehind token boundary — the hand-rolled `[^0-9A-Z_]` class admitted
lowercase-glued tokens) from backend-ready.ts and use it for both merged
buffers. Comments trimmed to the WHY; 6 new tests collapsed to the two
invariants (splice recovers, prose does not match) plus one remote-log case.
Both callers build the wait in the same synchronous block as
`outputTail.attach(child)`, and stream 'data' is asynchronous, so
`bufferedOutput()` is always empty at the seed and this block recovers
nothing today. Say so at the code, so nobody reads the seed as the live
recovery path for a lost READY sentinel — and so the condition that makes it
load-bearing again (any await reintroduced between attach and this call, the
pre-#100442 ordering) is written down next to the code that depends on it.
The spawn-time output tail feeds stdout AND stderr into ONE buffer, so its
contents are not line-accurate. uvicorn logs to stderr in raw chunks, and a
chunk that ends without a newline is concatenated directly onto the stdout
sentinel that follows it:
INFO Started server process [4711]HERMES_BACKEND_READY port=65238
The late-attach seed scan added for #60323 matched that buffer with the
line-anchored `_READY_RE`, so `^` never lined up and the seed silently
recovered nothing. The live stdout listener cannot cover the gap either: the
sentinel was already consumed by the tail, and flowing-mode streams never
replay chunks to late listeners. The wait then runs out its full 90s deadline
and a healthy, listening backend is SIGTERMed — while desktop.log, which reads
both streams, plainly shows the READY line. That contradiction (READY logged,
boot timed out anyway) is the reported macOS regression.
Scan the tail with a boundary-guarded pattern instead of a line-anchored one.
`port=<digits>` keeps the token unambiguous, so prose naming the sentinel and
longer identifiers ending in it still do not match. The live per-line scanner
keeps `^`: individual stream chunks ARE line-accurate there, and loosening it
would let unrelated child output settle the boot on a bogus port.
Scope: the seed only. Watching stderr on the live path is #97086's change and
is deliberately untouched here — the two compose.
Drop the 3-line facade wrapper (hermes_cli/gateway.py is already 3x the facade
threshold) and call the existing _ensure_user_systemd_env() directly under
`is_linux() and INVOCATION_ID` — the same Linux gate the process_registry seam
uses, instead of os.name == "posix". The fail-closed test now targets
_ensure_user_systemd_env() itself. Hedge the scope-unavailable error text: the
probe also returns False when systemd-run is missing or times out, so the
D-Bus diagnosis is the usual cause, not the only one.