The Aug 12 pin bump (91345435a) updated pyproject/uv.lock but not
tools/lazy_deps.py, tripping the #31817 downgrade-guard tests; the
post-VACUUM TRUNCATE fix landed without updating the checkpoint test
that pinned the old no-TRUNCATE rule. Both red on pristine main.
Adds delegation.worktree_isolation (default: false). When enabled, each
delegate_task child gets its own git worktree branched from the repo's
current HEAD under <repo>/.worktrees/subagent-<id>, its terminal session
starts there, and its goal message carries the isolation contract
(work + commit in the worktree; parent reviews/merges the branch).
- tools/subagent_worktree.py: clean-room implementation from Muse Code's
documented --subagent-worktree-isolation behavior (create per-child
worktree, finalize/inspect after run, auto-prune clean no-commit
worktrees, keep anything holding work).
- tools/delegate_tool.py: config gate + per-child setup in
_run_single_child; result entries gain a "worktree" field (path,
branch, commits, dirty, pruned) only when isolation engaged — the
default-off wire shape is byte-identical.
- Git-only + local-terminal-backend-only; non-git dirs, remote backends,
or any worktree failure degrade silently to shared-workspace behavior.
- Tests: tests/tools/test_subagent_worktree.py (15 tests, real git
repos) + E2E through _run_single_child with a real repo verified
parent-checkout isolation, branch reviewability, prune, and
default-off shape pinning.
- Docs: delegation feature page section + configuration.md key.
Port from cline/cline#12983 (the 'invalidate tokens when OAuth client
changes' invariant): tokens are minted for a specific client_id, so after
a user edits oauth.client_id / oauth.client_secret in config.yaml the old
tokens can only fail with invalid_client. Pre-registered clients are
deliberately exempt from the invalid_client auto-poison path, so the stale
tokens wedged every request until ~/.hermes/mcp-tokens/<server>.* was
wiped by hand.
_maybe_preregister_client() now compares the on-disk client.json identity
against the incoming config identity before overwriting it and discards
tokens.json + meta.json on a mismatch (with a log line pointing at
hermes mcp login). Unchanged identity is a strict no-op.
Proven live on main with an isolated-HERMES_HOME E2E probe; regression
tests sabotage-verified (fail without the wiring line).
Approval marks were emitted under a synthetic 'default' relay session:
the hook payload carried only turn_id/tool_call_id, so the observability
plugin's _session_id() fell back to 'default', parenting approval marks
to a session scope that never closes — and close-time exporters never
shipped them. The audit board's approval tables stayed empty while
approvals were demonstrably firing (staging 2026-08-10).
Bind session_id in set_current_observability_context at both dispatch
sites (model_tools tool dispatch, plugins pre-tool-call approval gate)
and forward it on every approval hook. Explicit session_id in a hook
payload still wins; unbound contexts omit it (legacy behavior).
Adds a pre_transcription transform hook (prompt/language/model mutable,
file_path read-only, last-writer-wins per the transform_* convention)
fired before any STT backend, threads prompt to faster-whisper
(initial_prompt) and OpenAI/Groq/Mistral/DeepInfra (prompt), adds an
optional stt.prompt config key on the same plumbing, and keeps the
no-hook dispatch path byte-identical. Fixes#64168.
Documents the new surface for users: a "Transcription prompt
(vocabulary hints)" subsection in the configuration guide (composition
order, per-provider support matrix, length contract, privacy warning),
a pre_transcription entry in the hooks reference, and the mirrored row
in the plugins hook table.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AG6LyYMvHC2o6HbVUozmVR
Route plugin target parsing, validation, and host-driven delivery through PlatformEntry across CLI and cron while preserving the host-only send_message policy.
The write_file / patch file tools hard-denied ~/.ssh/config as a
"protected system/credential file", while the terminal tool only
*asked* for approval on ~/.ssh writes. That inconsistency meant a write
to ~/.ssh/config was refused via write_file but succeeded via terminal
after an approval prompt -- the same operation flip-flopping between
denied and OK depending on which tool ran it.
The SSH client config carries no private-key material, and editing it
(host aliases, ProxyJump, VS Code Remote-SSH targets) is a routine,
user-initiated task. It CAN carry ProxyCommand / Match exec directives
that run commands, so a free write is still inappropriate -- approval,
not a flat refusal, is the right policy, matching what the terminal tool
already does.
Changes:
- agent/file_safety.py: remove ~/.ssh/config from the flat credential
deny; add build_write_approval_paths() + is_write_approval_required(),
and short-circuit it out of the ~/.ssh/ prefix deny so the file is
allowed at the classifier layer. Private keys, authorized_keys, and
everything else under ~/.ssh/ stay hard-denied.
- tools/file_tools.py: _check_approval_required_write() routes ssh config
writes through the shared _run_approval_gate (once/session/always,
honors --yolo, fail-closed with no human), wired into write_file_tool
and patch_tool right after the protected-instruction gate.
- Non-interactive consumers fail closed: the ACP file bridge
(copilot_acp_client) rejects approval-required paths outright, and the
TTS output-path picker refuses them as before.
- Docs + tests updated (security.md exception note;
TestSshConfigApprovalGate covers config approval-gated, keys still
hard-denied).
/simplify-code findings on the full PR diff:
- _is_usable_python had the same sticky-failure bug the previous commit
fixed in _python_environment_prefix: lru_cache pinned a transient
probe failure (fork pressure, timeout) as False forever, silently
locking project mode to sys.executable. Both probes now share a
success-only bounded dict cache via _cache_probe_result() with FIFO
eviction at _PROBE_CACHE_MAX (the old < cap guard stopped caching new
entries instead of evicting, re-probing entry 33+ on every call).
- The hermes-root-omitted logger.info fired on every external-env call
in project mode; now deduped once per interpreter path per process
(matching the tirith/mcp warn-once convention).
- Regression test: _is_usable_python probe failures are retried, not
cached (mutation-verified).
Follow-up to the salvaged #81201 commits:
- Short-circuit _uses_hermes_python_environment when the child IS the
running interpreter (path or realpath match). The default strict-mode
path no longer spawns a probe subprocess at all, and a flaky probe of
sys.executable can never drop the hermes root from PYTHONPATH
(protects the test_repo_root_modules_are_importable invariant). The
realpath leg also covers uv-style venvs whose bin/python resolves to
the same binary.
- Stop caching failed probes: _python_environment_prefix now uses a
success-only dict cache instead of lru_cache, so one transient
timeout under load no longer sticks for the process lifetime.
- Deduplicate the subprocess probe scaffolding shared with
_is_usable_python into _probe_python().
- Log once when the hermes root is omitted so import-behavior changes
are diagnosable from user reports.
- Tests: fail the composition tests loudly if execute_code never
reaches Popen (was vacuously passing on exceptions); assert the
staging dir is literally first in PYTHONPATH (was truthiness only);
add guards for probe-failure retry and the no-probe short-circuit.
Whole-bug-class sibling of the execute_code fix: terminal(code=...) —
the reverse confusion — fell through to command=None and failed with
'Invalid command: expected string, got NoneType', naming neither the
stray 'code' argument nor execute_code as the right tool. Mirror the
guard in _handle_terminal (verified live: the opaque NoneType error
reproduces on main). Mutation-checked: removing the guard fails the
new regression test.
Review follow-up on the salvaged handler: a non-string 'code' (int,
dict, list) reached code.strip() and surfaced as a generic
'Tool execution failed: AttributeError' — the same unrecoverable shape
the salvage exists to eliminate. Add an isinstance guard beside the
'command' check that names the received type and shows the correct
call form; narrow the docstring to what the handler actually does.
Regression test drives int/dict/list through registry.dispatch and
asserts no AttributeError leaks (mutation-checked: removing the guard
fails 3 subtests).
* fix(windows): SSH ControlMaster gating + stop hijacking the user's python
Two Windows environment-integrity fixes:
1. tools/environments/ssh.py (#73927): Windows OpenSSH has no
Unix-domain-socket ControlMaster support, so unconditionally passing
ControlPath/ControlMaster/ControlPersist failed EVERY tool call on a
Windows-hosted ssh terminal backend with 'getsockname failed: Not a
socket'. Gate the three multiplexing options behind a module-level
_SSH_MULTIPLEX = (os.name != 'nt'); the scp upload path is gated the
same way. On Windows the backend now works without connection pooling
(each command a fresh connection); POSIX behavior is unchanged. The
teardown 'ssh -O exit' is naturally inert because the socket never
exists on Windows.
2. scripts/install.ps1 (#83797): the installer put the whole
venv\Scripts directory on the user PATH, which contains python.exe /
pythonw.exe / pip.exe and so silently hijacked the 'python' command in
every terminal on the machine — unrelated projects started resolving
python to Hermes' runtime interpreter. Now copy only the launchers
(hermes.exe, hermes-acp.exe) into a dedicated $InstallDir\bin and put
THAT on PATH. Existing installs are migrated: the legacy venv\Scripts
entry is stripped from the user PATH on the next install/update. The
new bin dir is under $InstallDir (…\hermes-agent), which the uninstall
PATH sweep already matches via its \hermes-agent marker.
Updated the stale hermes_cli/update_cmd.py docstring that described the
old venv\Scripts-on-PATH layout.
Tests: SSH ControlMaster gating pinned both directions (multiplex on →
flags present; off → absent but BatchMode/StrictHostKeyChecking retained).
install.ps1 parses clean via the PowerShell AST parser.
* docs: update windows-native install docs for the bin\ launcher layout
CI (test_windows_native_docs) pins the docs and installer to the same
PATH layout. The #83797 fix moved the PATH entry from venv\Scripts to a
dedicated $InstallDir\bin holding only the hermes launchers, so update
the Windows-native guide to match: PATH-after-install section, the
install-steps list, the directory-layout table, the Get-Command
verification line, and the 'command not found' pitfall. Test now asserts
the bin\ layout and guards against a regression back to venv\Scripts on
PATH.
* fix: keep install.ps1 pure ASCII (PowerShell 5.1 codepage safety)
The two comments I added in the #83797 PATH-hijack fix used em-dashes,
tripping tests/test_install_ps1_ascii_only.py — Windows PowerShell 5.1
reads a BOM-less .ps1 in the system ANSI codepage (not UTF-8), so a
non-ASCII byte can misdecode into a stray quote and desync the parser
(issues #66994/#67000). Replace the em-dashes with ASCII '--'.
The apply phase already skips a hunk whose -/+ lines are identical
(patch_parser.py '(search_lines == replace_lines): continue'), but the
validation phase lacked the guard: such a hunk reached
fuzzy_find_and_replace, whose identical-strings error names
old_string/new_string — parameters that don't exist in patch mode — and
failed the whole atomic patch that apply would have accepted. Mirror
the apply-phase skip in validation; regression test drives a mixed
degenerate+live patch end-to-end (short text dodges the
is_already_applied >=8-char rescue).
skill_manage's patch action uses the same fuzzy_find_and_replace engine
as the file patch tool and surfaces the identical-strings error verbatim
— and unlike the file path it has NO is_already_applied no-op rescue, so
identical old/new ALWAYS errors there. Mirror the new_string description
so the schema warns before the error fires (sibling-site parity with
tools/file_tools.py PATCH_SCHEMA).
The 3-sentence identical-edit message was snapshot-asserted verbatim in
two tests. House style avoids exact-string change-detector assertions;
both tests now import the constant from tools/fuzzy_match so rewording
the message can't silently break them.
Two Windows agent-loop friction fixes:
1. tools/mcp_tool.py (#56536): shutil.which(cmd, path=env_path) reads
executable extensions from the PARENT process PATHEXT, not the MCP
subprocess env — a stdio MCP config supplying both PATH and PATHEXT
could fail to resolve a command its own env can locate, and startup
then got a bare command name. On Windows, when the first which() call
misses and the config env carries PATHEXT (any key casing), retry the
resolution with the config's PATHEXT temporarily applied.
2. skills/ + optional-skills/ (#50606): 42 SKILL.md files that declare
platforms: [.., windows] used python3 in their command examples.
python3 does not exist on native Windows (the toolchain probe in the
system prompt reports python3=missing), so every copy-pasted example
burned a failed agent turn before self-correction. Replaced the
command word python3 -> python (python3-config / python3.x version
strings untouched). python is the spelling that exists in every
Hermes-managed environment (Windows native, uv-managed venvs on all
three OSes); agents on POSIX hosts additionally see the probed
toolchain line and adapt either way.
Fixes#69472. On a Windows host every destructive native command passed
approval silently — DANGEROUS_PATTERNS were POSIX-shaped, and the
normalizer strips backslashes as shell escapes so no Windows path could
ever match a path rule. Probed live before the fix: 15 of 15 destructive
Windows commands (Remove-Item -Recurse -Force, del /s /q, iwr | iex,
taskkill /F, Format-Volume, diskpart, icacls /grant Everyone, vssadmin
delete shadows, bcdedit /set, reg delete, cipher /w, ...) sailed through
undetected.
Two changes:
1. Windows destructive tier in DANGEROUS_PATTERNS: PowerShell deletes
(bare Remove-Item -Recurse/-Force), cmd builtins with /s|/q switches,
iwr|iex remote execution (pipe and subexpression forms), taskkill /F /
Stop-Process -Force, volume/disk destruction (Format-Volume,
Clear-Disk, diskpart, format.com, cipher /w), icacls Everyone-grant /
/reset, backup destruction (vssadmin delete shadows, wbadmin delete,
bcdedit /set), reg delete / Remove-ItemProperty -Force, and service
stop/delete (Stop-Service -Force, sc stop|delete). Each pattern
requires the destructive flag so graceful/read-only usage (taskkill
/IM without /F, reg query, icacls inspect, sc query, plain del file)
does not prompt. Patterns live in the main list, not a win32-gated
tier: a Linux-hosted Hermes can drive a Windows box over SSH.
2. Windows-path detection variant in _command_detection_variants: when
the raw command contains a drive-letter/UNC backslash path, also
yield a variant with backslashes flattened to forward slashes BEFORE
normalization strips them, plus Windows spellings of the credential
path rules (Users/<u>/.ssh, AppData/{Local,Roaming}/hermes .env).
Gated on a real path shape so POSIX escape semantics are untouched.
Tests: tests/tools/test_approval_windows.py — 48 cases (27 destructive
flagged, 13 benign not flagged, 5 credential paths in both separator
spellings, 4 POSIX-escape non-regressions). The 8 pre-existing failures
under '-k approval' on this Windows host are identical on unmodified
main (ordering artifacts + known symlink cases) and unrelated.
Sweep of open Windows issues affecting day-to-day agent operation
(explicitly excluding install/setup and locale classes):
- hermes_cli/_subprocess_compat.py: new split_command_line() — Windows-
safe command-line tokenizer (posix=False + quote stripping) so
backslash paths survive. POSIX behavior unchanged (plain shlex.split).
- hermes_cli/console_engine.py (#83934): console commands like
'sessions export C:\Users\me\out.jsonl' no longer silently mangle the
path into a relative filename in the cwd.
- agent/shell_hooks.py (#78293): hook commands with backslash paths now
spawn, resolve their script path, and pass hooks doctor instead of
reporting 'not executable'. All three shlex sites routed through the
shared splitter.
- agent/prompt_builder.py (#51755): system prompt now reports
Windows (11) on Windows 11 — platform.release() returns 10 for both;
distinguish via sys.getwindowsversion().build >= 22000.
- hermes_cli/commands.py (#42016): @ autocomplete no longer crashes the
prompt_toolkit event loop when rg emits a path on a different mount
(device paths \.\nul, other drive letters) — relpath ValueError is
skipped per-entry.
- tools/browser_use_cli.py (#83884): screenshot-path detection now
matches Windows drive-letter paths (C:\... and C:/...) in addition to
POSIX; Browser Use screenshots attach on Windows.
- tools/skills_hub.py + tools/skills_guard.py (#62310): the two 'MUST
stay symmetric' skill content hashes actually agree on Windows now.
Bundle keys are normalized to POSIX separators before hashing, and the
disk digest sorts by rel-posix STRING (case-sensitive) instead of Path
objects (case-insensitive on Windows). Fixes permanent false-positive
update_available for every installed skill.
Tests: tests/tools/test_windows_agent_loop_papercuts.py — 16 cases
covering each fix, including a disk-vs-bundle hash symmetry check built
with native Windows separators and a mixed-case filename.
Efficiency review (measured with timeit probes) found two unbounded
costs on adversarial inputs:
- The masked-range rebuild copied the whole string once per range
(O(n*k)): 50k tiny heredocs took 1.7s. Replaced with a single-pass
segment join over the (sorted, non-overlapping) ranges: 152ms, and
newlines are now counted on the original command instead of
re-slicing.
- After the last '<<' occurrence no opener can start, but the scanner
still walked the remaining text per-char: one heredoc followed by a
1MB tail cost ~150ms. An rfind bound breaks out of the unit loop
once the scan passes it: 0.3ms.
Typical commands are unaffected (the '<<' fast path already returns
first). 30/30 guard tests pass; mutation check re-run on the final
stack (no-op mutation -> 11 tests fail, restore -> green).
The previous commit's regex-based stripper removed EVERY heredoc body,
which review flagged as bypassable: a fake '<<EOF' marker inside a
comment or quoted string enters the unterminated path and swallows a
later REAL background operator, and unquoted ('cat <<EOF' — expansion
runs) or shell-consumed ('bash <<'EOF'' — body IS shell) bodies are
executable content that must stay visible to the guard.
Replace it with tools/shell_heredoc.strip_inert_heredoc_bodies(), a
conservative shell-state scanner: a body is masked ONLY when every
delimiter on the opener is quoted (no expansion), every heredoc is
terminated by an exact delimiter line, the opener composes a single
command (no list/pipeline operators, no nested $()/backtick/process-
substitution scope), and the consumer is an allowlisted non-shell
interpreter (python/osascript/cat). Anything ambiguous is returned
unchanged — a false positive on exotic syntax is acceptable; hiding a
real background operator is not. Masked bodies become newlines so line
structure is preserved for MULTILINE regexes.
The helper is a standalone stdlib-only module (precedent:
tools/ansi_strip.py) because the same heredoc-as-data false-positive
class exists in the blocked-command regex checks (#83104) and the
gateway lifecycle guard (#81721/#79835, cron/lifecycle_guard.py) —
which must not import the terminal-tool module graph.
Adapted from Wolfram Ravenwolf's security-hardened rework of #63788
(69c7663c6de6b6cb05bf99203fa39673efe01ccf); test scenarios for the
bypass cases derive from his suite.
Co-authored-by: Wolfram Ravenwolf <github.com@wolfram.ravenwolf.de>
_strip_quotes documented that it stripped heredoc bodies but only handled
single/double/backtick quotes. As a result _foreground_background_guidance
scanned heredoc body text for a backgrounding '&' and wrongly rejected valid
foreground commands whose heredoc body contained a spaced ampersand — e.g.
AppleScript string concat (osascript <<'EOF' ... "a" & b ... EOF), Python
bitwise-and, or literal UI text like 'FaceTime & Privacy'.
Add a _strip_heredocs pass (runs before quote-stripping, since a heredoc
delimiter may itself be quoted) covering <<EOF, <<-EOF, <<'EOF', <<"EOF".
The same-line tail after the opener (redirects/args) is preserved and the
opener token is blanked so a real backgrounding '&' after the heredoc is
still detected.
Adds tests/tools/test_terminal_heredoc_background_guard.py.
* fix: Windows path handling in search_files rg calls and patch escape drift
Two related Windows failures from a live session (Windows 10, git-bash
terminal backend, winget-installed native ripgrep):
1. search_files was unusable on drive-letter paths. _escape_shell_arg
rewrites C:\... to the MSYS form /c/... so bash builtins resolve it,
but rg is a native Windows binary and Hermes disables MSYS argument
conversion for its bash subprocesses (MSYS_NO_PATHCONV=1 /
MSYS2_ARG_CONV_EXCL=*, see _apply_windows_msys_bash_env_defaults) —
so nothing ever translated /c/... back and every search failed with
'The system cannot find the path specified. (os error 3)'.
Fix: new _escape_native_tool_arg emits the forward-slash NATIVE form
(C:/Users/...), which native binaries accept, bash passes through
untouched, and MSYS builds also handle. Applied to the six rg call
sites (content search, --files search x2, zero-match probe x3); the
grep fallback keeps the MSYS form since MSYS grep wants it.
2. The patch tool silently doubled backslash runs when tool-call args
arrived JSON-escaped one extra time (file had \ where old_string
had \\). Similarity strategies (context_aware) matched the region
anyway and wrote new_string verbatim, corrupting every backslash run
(reproduced: 6 backslashes on the line became 12). _detect_escape_drift
now also blocks when every backslash run in old_string is exactly twice
its counterpart in the matched region and new_string repeats the
doubling — with guardrails so exact matches, intentional backslash
edits, model-corrected new_strings, and single weak-signal runs all
still apply. Blocking returns the standard escape-drift guidance so
the model re-reads and retries with correct counts.
Tests: TestEscapeNativeToolArg (5 cases, including an end-to-end
_search_with_rg command capture) and TestBackslashDoublingDrift (6
cases). The 8 pre-existing failures in tests/tools/test_file_operations.py
on a Windows host (umask/symlink POSIX assumptions) are identical on
unmodified main and unrelated.
* fix: shell linters get native Windows paths too (node C:\c\... double-prefix)
Same class as the rg fix: LINTERS commands (python -m py_compile,
node --check, npx tsc, go vet, rustfmt) invoke native Windows binaries,
but _check_lint interpolated the MSYS /c/... form. node resolves that
as C:\c\Users\... (double-prefixed), so on Windows hosts every .js
write reported a phantom ENOENT lint failure that could mask real
syntax errors (issue #84303). Route the {file} arg through
_escape_native_tool_arg like the rg call sites.
Regression test asserts node --check receives 'C:/...' and never
'/c/...'.
* fix: warn agents off driving interactive console TUIs via pty on Windows
Driving 'gh auth login' (and other survey-style console TUIs) through a
pty background process on Windows silently hangs: these programs read
Win32 console key events via ReadConsoleInput, not the stdin byte
stream, so Enter keypresses submitted over process stdin never register.
The agent-visible symptom is a prompt frozen at 'Press Enter to open
browser...' while the user sees nothing, and a turn interrupt then kills
the process, invalidating any device code the user already entered on
github.com.
Two guidance fixes, both proven in a live session on Windows 10:
- agent/prompt_builder.py: extend _WINDOWS_BASH_SHELL_HINT to steer
agents toward non-interactive paths (flags, --with-token, config
files, curl-polled OAuth device flow) instead of answering console
prompts programmatically.
- skills/github/github-auth: document the pitfall and add the manual
OAuth device-flow procedure (curl against gh's public client_id,
poll for the token, finish with 'gh auth login --with-token'), which
succeeded first try after two interactive attempts hung.
* fix: send CRLF for Enter on Windows PTY submit; correct root cause in guidance
Review feedback (helix4u) was right on both counts:
1. Root cause correction. gh's 'Press Enter to open browser' prompt is
waitForEnter -> bufio.Scanner reading stdin, not a survey/console-API
prompt. The real bug is ours: submit_stdin appended a bare \n, and
through pywinpty/ConPTY a lone \n is not delivered as a line
terminator, so the child's blocking line read never returns. Verified
empirically against pywinpty 2.0.15 with a readline() child:
\n -> hang, \r -> line delivered, \r\n -> line delivered.
Fix: submit_stdin now appends \r\n for Windows PTY sessions (POSIX
PTYs and Popen pipes keep \n). Windows-only regression tests cover
the PTY and pipe branches.
2. Prompt hint rewritten: instead of claiming Windows console TUIs
cannot be driven, it now says to use process(submit) rather than raw
writes with bare \n, and to prefer non-interactive paths when a CLI
offers one.
3. Skill device flow rewritten as an executable script: parses the
device-code response, polls per the returned interval, handles
authorization_pending / slow_down (+5s per GitHub docs) /
expired_token / access_denied / unexpected responses, pipes the token
straight into gh without echoing it, and drops the undocumented
workflow scope (repo,read:org,gist is the documented minimum for
gh auth login --with-token). The pitfall note is narrowed to the
reproduced condition.
The Browser Use CLI became the default browser backend, but nothing
provisioned it: users without uv/uvx (field report from DongyangHe on
macOS) silently fell back to the built-in browser tools with no notice.
- install_cli() in tools/browser_use_cli.py: uv tool install browser-use
via the managed uv (bootstrapped on demand), linked into
$HERMES_HOME/bin (UV_TOOL_BIN_DIR)
- _find_cli() now also probes $HERMES_HOME/bin for browser-use/uvx —
Hermes' managed uv is not on the user's PATH
- hermes tools post_setup actually installs (Camofox standard) instead
of printing instructions
- install.sh / install.ps1 provision the CLI at install time
(best-effort, non-fatal, honors --skip-browser)
- CLI startup shows a one-line notice (24h rate-limited) when the
default backend downgraded to the built-in tools
* fix(gateway): stop frozen-preview finals and dropped idle-session delegation callbacks
Two relay-plane delivery losses from the 2026-08-09 staging incident:
1. stream_consumer: the skip-redundant-finalize branch recorded _accumulated
as the delivered turn-final payload even when the last ACKED edit was an
earlier throttled preview snapshot, so delivered_final_matches reconciled
True and the gateway suppressed the corrective final send — the user was
left with a cut-off message ending in the streaming cursor. Extracted
_mark_skip_redundant_finalize(): records the last acked wire payload
(cursor-stripped), so a preview/final mismatch now returns False and the
normal final send fires.
2. run.py: _classify_completion_target classified every ended parent session
terminal unless it ended by compression. Idle/timeout session ends are the
norm on scale-to-zero relay deployments and the chat route remains valid;
completed async delegation results were terminally dropped. Ended parents
now classify deliver unless the end was an explicit user boundary
(session_reset / user_exit / session_switch).
* fix(relay): drain in-flight outbound frames before transport teardown
disconnect() failed every pending outbound future immediately with
'relay transport closed', so a trailing finalize edit racing turn
teardown was lost even though the connector socket could still serve
it. Bounded drain grace (5s) lets in-flight requests resolve; silent
connectors still tear down promptly. asyncio.wait (not gather+wait_for)
so a timeout doesn't cancel futures owned by the fail-remaining loop.
* fix(gateway): route completion injection through the alias-aware transport resolver
Third relay-plane delivery loss from the 2026-08-09 staging incidents: a
delegation batch completed while the gateway was up, the watcher drained
the event, and delivery vanished with no log line. _inject_watch_notification
resolved its adapter with a literal p.value == platform_name scan of
self.adapters — a relay-fronted gateway registers ONE adapter under
Platform.RELAY fronting N logical platforms, so 'slack' never matched and
the injection returned None ('no gateway route'), silently dropping the
completion. The handoff path already documents this exact trap and uses
resolve_delivery_transport; the injection path now does the same (native
wins; relay eligible only when it fronts the logical platform), with the
literal scan kept as fallback for stub runners and exotic platforms.
* fix(relay): clamp disconnect drain grace to the runner's adapter-disconnect budget
Review finding (JoaoMarcos44, #82592): a fixed 5.0s drain in front of the
three 1.0s sequential teardown awaits gives an 8.0s worst case inside the
runner's 5.0s asyncio.wait_for(adapter.disconnect()) — tripping it cancels
teardown mid-drain, skips the fail-pending loop, and leaves outbound
callers blocked until _OUTBOUND_TIMEOUT_S (30s). The effective grace is
now budget - 3*TEARDOWN - margin (env-aware via the same
HERMES_GATEWAY_ADAPTER_DISCONNECT_TIMEOUT the runner reads), so the drain
can never push teardown past its caller's budget; a budget too small for
any drain disables it cleanly.
* test(gateway): pin the final-send suppression contract across a behaviour matrix
The gateway skips its own final send when the stream consumer claims the turn
final already reached the user. Every incident in that family — #71643 (stale
finalize snapshot), #78541 (payload-less multi-message split), #82656 (frozen
preview left with a visible cursor) — is the same failure: the consumer claimed
delivery for text the platform never rendered, so the corrective send was
suppressed and the answer was lost with no retry.
Each was fixed with a scenario test pinned to one branch of
GatewayStreamConsumer.run(). The got_done handler now has five sibling branches
that each set the suppression flags and record a turn-final payload, and nothing
checks them as a group: a new branch, or a new early `return True` in
_send_or_edit, can reintroduce the class without failing a test.
Pin the invariant instead of the branch — if the consumer offers the gateway any
signal it would trust, the complete final text must have reached the wire — and
assert it across {edit always / dies / never / lies} x {send always / never} x
{fresh-final on / off} x {clean / interrupted stream}.
The adapter records only frames that actually rendered, so an ACK the platform
drops does not count as delivery. 24 honest-transport scenarios hold the
invariant as a hard assertion. The 16 lying-transport scenarios are checked too;
the single combination that still violates it is reported as an expected
failure documenting the open exposure rather than asserting it away.
Refs #82656
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(gateway,relay): prime relay egress routing for synthetic injections + cap stale completion replay
Defect #4 from the 2026-08-09 staging incidents (upgrade-robustness):
after every gateway restart the durable async-delegation replay injected
completions correctly (post-741663cf1) but their replies bounced at the
connector — 'slack egress declined: target not routed to an onboarded
tenant'. The relay adapter re-attaches tenant discriminators
(metadata.scope_id / metadata.user_id) from per-chat caches warmed ONLY by
inbound traffic; synthetic turns race those cold caches on every deploy,
scale-to-zero wake, and crash recovery.
- relay adapter: prime_routing_cache() — feeds a synthetic event's
session-store origin through the same _capture_scope used for real
inbound (never raises).
- run.py injection path: prime the resolved adapter before handle_message
(duck-typed; native adapters unaffected).
- async_delegation: 48h staleness cap in restore_undelivered_completions —
a pending completion older than the cap is terminally dropped (payload
stays queryable) instead of re-run as a fresh full-context turn; the
post-restart replay of a July session burned a 102K-token context.
Also carried: JoaoMarcos44's suppression behaviour-matrix harness
(cherry-picked from #82676, authorship preserved) — 39 passed + 1 xfail
(the documented ACK-then-drop transport-honesty residue).
* test: use recent timestamps in restored-ownership fixtures
test_restore_stamps_restored_flag persisted its completion with epoch-era
toy timestamps (dispatched_at=1.0), which the new 48h replay staleness cap
correctly classifies as stale — the fixture then exercised the cap instead
of the restored-flag contract (CI slice 4 failure). Timestamps are now
now-relative; the staleness behavior itself is pinned separately in
test_relay_injection_egress_priming.py.
* fix(gateway,relay): close four review findings on the relay delivery fixes
Review follow-ups on this branch (NousResearch#82592):
1. HIGH — classifier/resolver mismatch (falsely-acknowledged loss).
_classify_completion_target now returns "deliver" for idle-ended
parents, but _resolve_async_delegation_session still dropped every
non-compression-ended pin: the durable row was acked at adapter
acceptance, then the injection died inside the pipeline with no
retry — strictly worse than the honest terminal drop on main, and
the delivery leg defect #2's fix depends on did not exist. The
resolver now retargets non-user-boundary ends (idle/timeout/
lifecycle) to the chat's current session — session_entry already IS
the routing key's current session for the same chat — while user
boundaries (session_reset / new_session / user_exit /
session_switch) stay fail-closed. Both sides share one module-level
_USER_BOUNDARY_END_REASONS so the verdict and the routing decision
cannot drift again; a coherence test asserts deliver-verdicts
resolve non-None across representative end reasons.
2. HIGH — drain clamp missed adapter-level spend. The effective drain
grace budgeted drain + 3x teardown, but RelayAdapter.disconnect
spends revocation-monitor teardown + go_idle time BEFORE the
transport drain inside the same runner wait_for; worst case still
blew the budget and cancelled teardown mid-drain (skipping the
fail-pending loop). The adapter now measures its own elapsed time
and threads the REMAINING budget into
transport.disconnect(budget_s=...); legacy/stub transports without
the keyword fall back to the no-arg signature.
3. P1 — _request_response racing disconnect() could register a future
after the fail-pending loop already ran, stranding the caller for
the full _OUTBOUND_TIMEOUT_S (30s). Fail fast with the same
"relay transport closed" error once _closing is set.
4. P1 — _build_process_event_source's last-resort reconstruction
dropped scope_id, so a scoped relay completion whose session-store
origin was unavailable primed no tenant discriminator and could
still bounce off the connector's fail-closed egress guard.
scope_id now threads through the reconstructed SessionSource, with
a warning when a scoped chat reconstructs without one.
All four: RED reproduced with the fix reverted, GREEN after; relay/
delegation delivery families pass (43 + 71 + 179 across the touched
suites); full tests/gateway run shows only failures already failing
identically on merge base 2446c8bb6 (env/dep issues).
* fix(gateway,relay): make pending-frame failure cancellation-safe; persist completion routing origin
Two remaining review findings on this branch (NousResearch#82592):
1. Cancellation could strand outbound waiters past the fail-pending
loop. transport.disconnect() failed pending futures only at the END
of the drain + three teardown awaits; a cancellation landing
mid-drain (the runner's wait_for budget, an outer cleanup deadline)
skipped the loop entirely and left registered futures unresolved —
their callers blocked until _OUTBOUND_TIMEOUT_S (30s). The budget
threading added earlier shrinks the window but is not a hard
guarantee. The fail-pending loop (and the going_idle ack failure)
now run in a `finally`, so no exit path — normal, error, or
cancelled — can leave a registered future unresolved. Idempotent:
done futures are skipped, a second disconnect() pass is a no-op.
2. Durable completions did not persist their routing origin, so the
scope_id threading in the fallback SessionSource reconstruction had
nothing to carry on the exact path it exists for (restart replay
with session store + source cache gone): the async-delegation event
producers never populated scope_id and the durable rows never
stored it. Dispatch now snapshots the originating turn's
scope_id/user_id/user_name from the session context
(_capture_routing_origin — a new HERMES_SESSION_SCOPE_ID contextvar
bound by the gateway at session-bind time alongside the existing
vars), stores them in the existing task_json payload (no schema
migration), and re-attaches them to all three completion-event
shapes (live single, live batch, crash-recovery rebuild). The
gateway's fallback reconstruction then primes both discriminators
after a restart.
Tests: cancellation mid-drain -> every pending future resolves with
"relay transport closed" (mutation: moving the loop out of the finally
goes RED); second-pass disconnect idempotence; end-to-end
dispatch -> owner-death recovery -> event carries scope_id -> fallback
SessionSource primes it (mutations: dropping the dispatch capture or
the task_json persistence both go RED); live completion event carries
the origin. 94 passed + 1 xfailed across the delivery/delegation
suites; tests/tools delegation family 73 passed (2 collection errors
pre-existing on merge base 2446c8bb6).
---------
Co-authored-by: joaomarcos <joaomarcosdias444@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Ben Barclay <ben@nousresearch.com>
request_review on a running task under a live claim now requires the
caller to prove ownership (expected_run_id, the unchanged worker path)
or pass an explicit force=True override (CLI --force; dashboard human
actions pass force=True) instead of silently clearing claim_lock /
worker_pid of a live run.
Failures now carry distinct diagnostic reasons via with_reason=True
(mirroring request_changes' tuple pattern): live-claim refusal,
malformed re-review provenance, unsatisfied parents, unknown task, and
CAS miss. Tool/CLI handlers surface the specific reason instead of the
generic 'unknown id or not in running/ready'.
Regression tests: live-claim refusal + force/worker paths; malformed
provenance gets a distinct reason and explicit reviewer= recovers.
Add a non-terminal "review" status so a worker that finished implementation
can hand off for human review without abusing kanban_block. The old
kanban_block(reason="review-required: ...") convention routed the handoff
through the unblock-loop breaker, so a normal review -> changes -> review
cycle was falsely escalated to triage.
- kanban_db: request_review (running/ready -> review, non-block, emits
review_requested), reopen_review_task (review -> ready/todo, review_reopened),
complete_task accepts review -> done, and a review_dispatch gate (default off,
shared by the dispatcher loop and the gateway health probe).
- kanban_request_review worker tool + `request-review` / `reopen-review` CLI
verbs; tool wired through toolsets, EXPOSED_TOOLS, _POLISHED_TOOLS.
- Gateway notifier wakes the origin subscriber on review_requested and
block_loop_detected; the subscription survives until done/archived, so every
review cycle re-notifies.
- Dashboard PATCH + bulk route the review transitions (request_review /
reopen_review_task) and render the review column.
- goals.py goal-loop and KANBAN_GUIDANCE recognize review as a terminator.
- Docs (reference tables, user guide, AGENTS.md, zh-Hans mirrors) + tests.
needs_input / failed are unchanged: they still route through kanban_block,
still count toward block_recurrences, and still escalate to triage.
An unset browser.backend ("") now resolves to Browser Use mode whenever
the browser-use CLI is runnable (installed binary or uvx); otherwise the
built-in browser tools are kept so browsing never silently breaks.
Camofox setups always keep the built-in tools (no CDP surface), and
backend: off (including YAML 1.1 bare off -> False) forces the built-in
stack. hermes tools row highlighting follows the same effective-mode
resolution, and tests/tools/ pins CLI discovery off so host uvx installs
can't flip built-in-browser tests.
'Binary file - use appropriate tools' names a recovery the model may
not have — in a file-only toolset it thrashed for 41 turns / 178 tool
calls / 1.5M tokens on a PNG-behind-.txt (readtool eval, qwen3.8-max)
hunting for tools that did not exist. Name the type instead: 25 magic
signatures (images, archives, executables, media, SQLite), ftyp check
for ISO media, size in human units. 'Binary file (PNG image data,
4.1 KB) - cannot display as text.' answers what-is-this in one read.
Both ShellFileOperations refusal sites (read_file + read_file_raw) use
the shared describe_binary_file(); the extension-based guard keeps its
extension message (an extension is a claim; only sniffed content earns
a type name).
The Hugging Face agent-harness registry matches standard-var values
EXACTLY against the harness id. Our registry id is 'hermes-agent'
(huggingface.js agent-harnesses.ts), so AI_AGENT=hermes was counted as
'unknown' — fixed at both entry points.
Remote terminal backends (Docker/SSH/Modal/Daytona/Singularity/Vercel)
never inherit the Hermes process env, and the cross-session leak guard
deliberately strips HERMES_SESSION_* from subprocess envs in engaged
multi-session hosts — so hf/huggingface_hub traffic from those shells was
unattributable. _wrap_command now exports AI_AGENT/HERMES_AGENT inside
every wrapped command with ${VAR:-default} semantics (outer harness is
never clobbered), and the snapshot dump excludes both names so a baked
value can never shadow a later outer harness.
E2E: verified against real huggingface_hub 1.27.0 detect_agent() with a
cached registry — 'hermes-agent' detected via AI_AGENT and via
HERMES_SESSION_ID; old 'hermes' value reproduced the 'unknown' bug.
Reframe (per review): browser.backend: browser-use is now a DRIVER over
whatever browser source is configured, not a competing backend choice.
- browser_exec resolves its CDP endpoint through the same chain the
built-in tools use: BU_* env override > BROWSER_CDP_URL/browser.cdp_url
(/browser connect) > the configured cloud provider via browser_tool's
_get_session_info() — sharing the per-task session cache, expiry
replacement, inactivity reaper, and atexit cleanup instead of
duplicating them. Live-validated against Browserbase (session created,
driven, reaped) and gateway-provisioned Browser Use cloud browsers.
- Direct-API Browser Use configs skip provider resolution (the CLI talks
to their cloud natively via BU_AUTOSPAWN); the Nous-gateway variant
resolves through the provider, so subscribers get CLI mode without a
raw BROWSER_USE_API_KEY.
- Camofox: only true fallback — Firefox-based, custom HTTP API, no CDP
surface (its own health probes fail on CDP-schema calls). Active
Camofox setups keep the built-in browser tools even with
backend: browser-use set.
- hermes tools picker: provider rows and the Browser Use row are no
longer mutually exclusive; selecting a provider keeps the driver
choice, and both rows highlight when composed.
- Docs updated for driver-over-source semantics.
Camofox is selected via CAMOFOX_URL env var, not browser.cloud_provider —
so a Camofox user with a stray BROWSER_USE_API_KEY in .env matched the
legacy-migration predicate (cloud_provider unset + key present) and got
silently flipped into CLI mode, losing browser_* / Camofox entirely
(browser_exec cannot drive Camofox: its HTTP API exposes no CDP endpoint,
and the browser-use harness is CDP-only against Chromium).
is_legacy_browser_use_cloud_config() now defers to is_camofox_mode().
Follow-ups on the salvaged Browser Use CLI integration (PR #66476):
- browser_exec runs model-written Python on the host. Strip it at
tool-definition time for sessions whose resolved toolsets exclude
'terminal' so terminal-less surfaces (locked-down messaging configs)
don't silently regain host code execution through the browser toolset.
Session-level gate in model_tools, not a check_fn (check_fn results are
TTL-cached process-wide across sessions).
- Replace the live 'browser-use skill' schema fetch with a pinned helpers
digest: no third-party version-drifting text in the prompt, byte-stable
schema across machines. A/B benchmarked (108 runs, opus-4.8 + kimi-k3,
6 multi-step web tasks x 3 arms x 3 reps): pinned digest matches the
full skill dump 36/36 vs 36/36 at ~equal tokens; both cut total task
tokens ~60% vs the legacy browser_* toolset.
- Docs note for the terminal gate; contributor mapping for salvage.
Teknium