`hermes update` runs in the pre-pull interpreter. The auto-restart phase
imports freshly-pulled gateway source, which resolves sibling imports
against the OLD sys.modules cache — so any update where an already-cached
module gained a new export ImportErrored the whole phase and left the
gateway serving pre-update code (2026-08-20 field failure: new gateway.py
needs cli_output.line_input, cached cli_output predates it).
Class fix replacing the per-symptom _UPDATE_RUNTIME_RELOAD_MODULES
approach: _purge_stale_hermes_modules() evicts every cached module under
the Hermes package prefixes (hermes_cli/gateway/tools/tui_gateway/agent)
right before the restart phase, so later lazy imports rebuild a
self-consistent module graph from the updated checkout. The updater's own
executing modules are exempt (purging them buys nothing; reload-in-place
is the unsafe op, and we never reload). Root-segment check spares
prefix-lookalike packages. Best-effort, never raises.
5 new tests incl. an end-to-end repro of the field failure shape
(stale module missing symbol -> ImportError -> purge -> import resolves).
The desktop updater ran `hermes update --yes`, which auto-restored any
uncommitted source-tree edits onto the freshly updated checkout. On dirty
from-source installs this silently carried local modifications across every
update and could break the rebuilt app (field report: Windows update handoff
leaving the app 'crashed').
New `hermes update --keep-stash`: local changes are still autostashed so the
update can proceed, but are never re-applied — they stay parked in git stash
with printed recovery guidance. Both desktop handoff scripts (windows.ps1,
posix.sh) now pass it, probing `update --help` first so older installed
backends without the flag keep working. Failure paths are unchanged (stash
preserved, no restore); updates.non_interactive_local_changes: discard still
wins.
Tests: park/restore/failure-path coverage incl. a sabotage-verified
regression test; docs updated.
Builds on @fattchris resolve_turn_limit salvage (#67696): flips the default
from a numeric cap to unlimited across all construction paths (CLI, agent_init,
run_agent subagents), adds inf/infinity/null to the unlimited spellings, and
sets DEFAULT_CONFIG agent.max_turns to null. The turn cap caused more problems
than it solved (silent mid-task truncation).
Previously agent.max_turns only accepted positive integers. Setting it to
'none', 'unlimited', or 0 — all natural ways to say 'no limit' — either
crashed int() or was silently skipped by `or` checks, falling back to 90.
This adds resolve_turn_limit() in hermes_cli/config.py as the single
normalization point. It accepts:
- int/float → int(raw) (floats truncated)
- numeric string ('120') → int(raw)
- 'none'/'unlimited'/'infinite'/'∞'/'-1'/'0' (case-insensitive,
whitespace-tolerant) → sys.maxsize sentinel
- YAML None/null → default (90)
- bool/list/dict/garbage → default (with debug log)
All config-reading sites (cli.py, gateway/run.py, cron/scheduler.py) now
call this instead of bare int(), so agent.max_turns: none in config.yaml
becomes a first-class supported spelling of 'unlimited'.
The sentinel (sys.maxsize) survives the str()→int() round-trip through
the HERMES_MAX_ITERATIONS env-var bridge in gateway/run.py and works in
every <, >=, remaining = max - used comparison without requiring call
sites to learn about a special value.
Includes 38 tests covering the full spelling table, the str→int env-var
round-trip, and sentinel properties.
Every long-lived Hermes process is now positively identifiable so reapers
never have to guess lineage from PPID archaeology or cmdline shape:
- hermes_cli/process_identity.py (new): HERMES_SPAWN tag build/parse,
spawn-ledger.json self-registration keyed on (pid, create_time) — PID
reuse cannot forge the pair — with #89298-style corrupt-file quarantine,
and a kill-on-close job-object self-attach (BREAKAWAY_OK preserved for
the existing CREATE_BREAKAWAY_FROM_JOB escape hatches).
- serve/dashboard (web_server.py) and the gateway entry point register
themselves at startup and attach to the job; Desktop legacy
HERMES_PARENT_PID/winms marker reused as spawner identity so lineage
works with every Desktop version.
- Desktop stamps HERMES_SPAWN on backend spawns (parent-process-identity.ts).
- hermes update gets a positive-identity rung ahead of the heuristic ones:
_ledger_reapable_backend_pids reaps holders the ledger PROVES are orphaned
backends (purpose reapable + recorded spawner provably dead) in ANY update
context. Ledger-unknown holders fall through to the existing rungs.
22 new tests, sabotage-verified.
The no-live-shim probe defaulted to True (proceed with the reap) when
_venv_scripts_dir() returned None or the concurrent-instance detection
raised. Flip the default and the except-arm to False so an unverifiable
shim state keeps the updater refusing, matching the fail-closed contract
stated in the PR. Also fix the docstring: the hand-off gate lives in the
caller, not a function parameter.
Field incident (2026-08-20): a Windows Desktop update hand-off
(update --yes --gateway --force) left a swarm of per-profile serve
backends (mr-tester, probe-inherit, turqoise, clippy, maroon, …) holding
cryptography/_rust.pyd. Some still had a live parent (the tearing-down
Electron process, or the venv launcher->worker two-hop chain mid-exit),
so the strict orphan-only reap (_orphaned_desktop_backend_pids, which
bails the instant ANY holder has a live parent) disqualified the whole
set and the venv-holder guard dead-ended. The user saw a ~12-minute hang,
force-closed, and the half-done state stranded bot sessions.
New rung: _handoff_reapable_backend_pids reaps surviving Hermes
serve/dashboard backends from this venv — live parent or not — but ONLY
in the hand-off context the caller gates on: args.gateway AND the
update-incomplete marker present AND no live hermes.exe shim. In that
window nothing legitimate supervises or respawns a serve backend (the
Desktop tree-kills its backends and parks any relaunch behind the marker,
#50238), so a surviving backend is a leak, not a race. A non-backend
holder (operator REPL, stray script) still disqualifies the whole set;
psutil-unavailable returns None (keep refusing). Wired as the final rung
before the existing dead-end, after the orphan-only reap.
`hermes update` on Windows detached on every run, including the
`Already up to date!` no-op that never touches the venv. emozilla hit
the visible half: the shim exits, PowerShell takes the console back,
and a child prints the result under a fresh prompt — it reads as a
frozen update. The invisible half is worse: the hand-off sat ahead of
the fetch, so it also carried off the stash and branch-switch
questions, which #90205 then had to answer by closing stdin. Nobody
who mods Hermes got asked about their local changes again.
The shim lock is real and the child is still required — a launcher
holds venv\Scripts\hermes.exe open without FILE_SHARE_DELETE for the
whole command, so the quarantine rename is refused and uv fails with
os error 32. A parent that waits deadlocks against the handle it is
itself holding, and Windows has no exec to escape with.
But that lock only binds one step. Move the hand-off to the dependency
sync boundary, beside the native-module deferral that solves the same
"this process holds a file the sync must replace" problem — and for
the reason that placement already exists (#86735: a preflight ahead of
the fetch re-bricked the flow it was meant to protect). Everything
before the sync now runs foreground in the user's console: the
preflight, the stash question, the branch switch, git pull. An
up-to-date run never hands off at all.
Deferring to the next launch cannot substitute here the way it does
for a mapped .pyd: every future `hermes` launch is also the shim, so
the marker would defer forever. The child re-runs the update to keep
the node/web/lazy-refresh tail, and takes the sync it was spawned for
rather than the up-to-date early return.
'hermes --version' (and -V) now prints the full version report — banner
version line with upstream SHA, install directory, authoritative install
method, Python and OpenAI SDK versions, and update status — making the
separate 'hermes version' subcommand redundant. The subcommand is removed.
- _startup_fast.print_fast_version_info() is now THE canonical version
printer: static lines print instantly from stdlib probes, then the
banner label, install-method resolver, and update check lazy-import
after the first line is on screen (each degrades gracefully).
- main.py _print_version_info() delegates to it (used by /version in the
CLI chat surface and the --version flag path); the old duplicate
implementation is deleted.
- hermes_cli/subcommands/version.py removed; parser wiring, subcommand
sets, console-engine extraction entry, and tests updated. Hermes
Console keeps a 'version' command wired to the shared printer.
- Termux fast paths now include update status too (previously
check_updates=False).
- Docs/i18n, CONTRIBUTING, SECURITY, and nix checks updated to
'hermes --version'.
A GitHub-side HTTP 429 during 'hermes update' printed only
'Failed to fetch updates from origin.' — and the curl
'unable to access ... returned error: 429' shape even matched the
network-error branch, blaming the user's connection for a GitHub
outage.
- new _classify_fetch_failure(): 429/rate-limit -> 'GitHub is rate
limiting requests or having an outage — try again in 5 minutes';
5xx -> outage message with githubstatus.com; ordered BEFORE the
generic 'unable to access' network check
- both fetch-failure sites (update apply + --check) now share the
classifier via _print_fetch_failure(), and both always print the
first raw stderr line so the wire error stays diagnosable
- tests: classifier matrix + E2E against a live local HTTP server
returning 429 through real git
Fixes#89287
When the chosen/keyed backend fails a web_search or web_extract call
(bad key, upstream outage, 5xx, raised exception), that single call
retries on the keyless free-tier ring instead of erroring. The next
call attempts the chosen backend again — no sticky failover, no state.
Resolves the keyed half of #78984/#32159 (keyless half landed in the
ring PR).
- tools/web_tools.py: _rescue_eligible (keyed ring vendors + non-ring
backends eligible; keyless-mode calls excluded — they already walked
the ring), _rescue_search/_rescue_extract (search annotates
rescued_from + backend_error naming the original failure and the
retry-next-call semantics; extract rescues only whole-batch failures,
partial failures pass through untouched; rescue failure preserves the
ORIGINAL backend error with the rescue note appended)
- both dispatchers wrap the provider call: failure-results AND raised
exceptions rescue; ineligible paths re-raise unchanged
- web.keyless_rescue config key (default true; implicitly off when
keyless_fallback is off); docs updated
Live E2E: keyed Tavily with an invalid key 401'd and the call was
served by the real ring with the rescue annotation; a second call
re-attempted Tavily first (statelessness proven); whole-batch extract
rescue returned real page content. 13 new tests; 67 green across the
keyless suites.
Fresh installs with zero web credentials now rotate web_search/
web_extract across FIVE vendors' public free tiers — Exa, Parallel,
Tavily, Firecrawl, Keenable — instead of a 2-vendor 50/50 split, with
next-in-line ring failover on rate limits (multi-hop until a vendor
serves or the ring is exhausted; served_by marks the actual vendor).
- plugins/web/keenable/: new bundled provider (search via /v1/search,
fetch via /v1/fetch; keyed Bearer or keyless with the mandatory
X-Keenable-Title app header). Credit: integration proposed by
Ilya Gusev (Keenable) in #49758; Free/Paid picker rows included.
- keyless_mcp: tavily/firecrawl/keenable keyless search+extract
wrappers, _KEYLESS_RING + per-process round-robin cursor (seeded by
the random session id, advances per unpinned request), pinned-vendor
entry (pin = start there; rotation off), paid-pinned vendors excluded
from the ring entirely.
- Tavily/Firecrawl providers route keyless traffic through the ring;
both are now default-on ring members (no longer selection-gated).
- web_tools/registry: keenable in backend sets, auto-detect, availability
probes; _keyless_preference() delegates to the ring cursor.
- KEENABLE_API_KEY in OPTIONAL_ENV_VARS; docs updated (ring semantics).
Live E2E: all 10 vendorXcapability paths (5 search + 5 extract) served
real results keyless; rotation cycled all five vendors over 5 dispatch
calls; double-throttle failover walked exa->parallel->tavily.
The startup pruner is deliberately conservative (unattended, pre-banner),
so real installs accumulate what it can never touch: trees preserved for
untracked-only scratch, and orphaned local branches beyond the two
auto-generated prefixes it deletes. A measured multi-agent box: 35 trees /
15GB / 244 local branches, 120 of them fully merged.
New attended surface (hermes_cli/worktree_gc.py + worktree_cmd.py):
- hermes worktree list — audit every tree: age, size, verdict, reason,
plus deletable-branch count
- hermes worktree prune [--dry-run|--trees-only|--branches-only]
- /worktree prune [--dry-run] — same engine in-session; never touches the
session's own active tree
- startup escalation: one WARNING when .worktrees/ exceeds 10 trees or
5GB, naming the reclaim commands (silence is how boxes hit 15GB)
Safety invariants (shared with the startup pruner via cli.py primitives):
tracked modifications and unique unpushed commits never deleted at any
age; live-locked trees untouched; branch deletion gated on worktree
removal success; untracked-only scratch ARCHIVED to
~/.hermes/archive/worktree-prune/ before its tree is reaped.
Branch GC is content-gated, not name-gated: any local branch fully merged
or git-cherry patch-equivalent upstream is safe to delete (rebase merges
rewrite SHAs, so --merged alone misses the dominant leak); unique-commit,
checked-out, protected, and stale-base (>50 ahead) branches are kept.
Classification is parallel (8 workers) — 244 branches audit in ~64s live.
git timeouts degrade to keep (returncode 124) instead of crashing the
audit — live-verified failure on a 746MB .git repo.
16 behavior-contract tests against real git fixtures; live dry-run on the
production repo: 12 trees reclaimable, 120 branches deletable, 0 false
positives among kept trees.
Post-review cleanup on the salvage: update the three alias-installer docstrings to mention lock twins (and fix ctrl-enter's stale 'stock maps none of these' claim - stock maps the tilde form to plain ControlM, which the overwrite fixes); skip the never-emitted modifier-1 tilde forms in _install_paired; name the twins-only idiom as _lock_twins() and use it at the legacy-nav + PUA sites; route the Esc loop through _lock_variants; hoist the per-base table lookup out of the lock loop in the legacy-nav section.
Follow-up to the salvaged #89676 + #90291 lock-bit fixes: extract a shared _lock_variants() helper and cover the sites both PRs missed - install_shift_enter_alias / install_ctrl_enter_alias / install_cmd_backspace_alias CSI-u spellings, legacy CSI-letter and CSI-tilde navigation twins derived from the existing table for ALL modifiers 1-16 (not just plain/shift), plain F1-F4 SS3 fallback, unmodified CSI-u keys (Tab/Enter/Space/Backspace), and kitty PUA functional keys (keypad, F13-F24, Ignore range) under lock bits. 8 new tests.
With the kitty keyboard protocol push active (CSI >1u disambiguate), kitty
encodes lock-key state into the CSI modifier field of function keys: a
plain Down with NumLock on arrives as ESC[1;129B (NumLock), ESC[1;65B
(CapsLock), or ESC[1;193B (both) instead of the legacy ESC[B. Stock
prompt_toolkit maps none of these, so the parser fires Escape and
inserts the remainder as literal text in the input line.
03bf85d83 restored the kitty push and completed the extended-key alias
table but left these lock-bit variants unmapped, so kitty + NumLock
still leaks [1;129A/B for arrow/nav keys.
Map modifier 129/65/193 (plain) and 130/66/194 (+shift) for arrows,
Home/End, Insert/Delete/PageUp/PageDown, and CSI-u Enter/Tab/Backspace/
Space to their plain keys.
kitty and ghostty OR the CapsLock (64) / NumLock (128) state into the
CSI-u modifier parameter. With NumLock on, Ctrl+C arrives as
ESC[99;133u (5 + 128) instead of ESC[99;5u; the alias table had no
entry for it, so every key combo leaked as literal text like
[127;133u (#89651). Install every CSI-u alias with the lock-bit
variants (+64/+128/+192); the xterm modifyOtherKeys encoding never
carries lock bits, so the ESC[27;N;CP~ form is left untouched. The
Esc-key registration now covers modifier 1 as well (1+128=129 is a
lone Esc with NumLock on).
Salvaged from #78434 by @Slobaka (also the issue reporter; earlier than
the competing #78436). hermes doctor no longer paints a green web check
when the explicitly selected provider cannot initialize — web splits
into per-capability rows (web search / web extract) resolved through
the same registry resolvers the dispatchers use, with readiness from a
true availability probe (_provider_is_ready).
Keyless-tier integration on top of the salvage:
- _provider_is_ready counts is_keyless_available() as ready — keyless
mode is a working state, not a misconfiguration (zero-config installs
and selected-keyless Tavily/Firecrawl show ok, not warn)
- Tavily/Firecrawl gain is_keyless_available() (True only when
explicitly selected — they stay out of the zero-config fallback)
- doctor triggers plugin discovery before reading the registry (fresh
doctor processes saw an empty registry and warned on everything)
E2E: searxng-selected-without-URL warns (the #78412 repro);
zero-config, tavily-keyless, firecrawl-keyless all read ok;
parallel pinned paid without a key warns.
- Updated the Tavily API key description to clarify that it is optional and keyless access is supported.
- Modified the Tavily plugin and provider to handle requests with or without an API key, using Bearer authentication when the key is provided.
- Enhanced documentation to reflect the new keyless functionality and updated environment variable descriptions.
- Added tests to ensure correct behavior for both keyed and keyless requests.
Both the wire path and the picker only consulted the catalog's
`mandatory` flag, so a route the Portal lists as accepting no reasoning
parameter at all still got sent a disable, and still offered a Thinking
toggle in the model picker.
For a route it serves, the aggregator's own catalog outranks the
models.dev inference: `supports_reasoning: false` now suppresses the
disable on the wire and drops reasoning controls from the picker
entirely, so there is no disable left to describe.
Portal reasoning capabilities were held only in memory, so a process that
had not yet fetched them answered "unknown" — and on that answer the Nous
profile drops the disable rather than risk a 400. A short-lived process
(`hermes -p`, a cron job, a freshly booted gateway) is always in that
state, so every one of those runs silently ignored "thinking off" and
billed the user for reasoning they had turned off.
The parsed catalog is now mirrored to `cache/reasoning_caps.json`, keyed
by the URL it came from, and hydrated on a cold lookup without touching
the network. Every picker and pricing fetch already pulls that same
document, so they seed the mirror for free.
The catalog URL itself now resolves through the same ladder as the rest
of the Nous catalog reads (`NOUS_INFERENCE_BASE_URL` → credential base →
production) instead of being pinned to production, which had a staging
profile deciding the reasoning-mandatory question from prod's answers.
Keying the mirror by URL keeps those deployments apart.
The picker offered an off switch for every reasoning model, including routes
whose upstream answers a disable with HTTP 400 — so "thinking off" was a
control that could not work. Carry the catalog's mandatory verdict through
model.options as can_disable_reasoning and hide the toggle when it is false.
Effort levels are left alone. The catalog's supported_efforts under-reports
what the Portal serves (z-ai/glm-5.3 publishes max, high, low yet honors
minimal at its lowest thinking), so filtering the scale by it would hide
levels that work.
The Portal serves OpenRouter's catalog schema, so the existing parser and
cache-only tri-state contract carry over unchanged. Only the HTTP fetch is
generalized across the two catalogs; each keeps its own cache because they
list different models.
The Portal 403s a catalog read with no User-Agent, so the shared fetch now
sends one.
Widens #90327's line_input() to the whole bug class: all 46 bare input()
free-text prompt sites across the setup wizards (model_setup_flows, setup,
config, gateway, auth, auth_commands, plugins_cmd, skills_hub, bundles,
setup_whatsapp_cloud, main) now route through line_input(), and the shared
cli_output.prompt() / setup.prompt() helpers do too — so every CLI wizard
gets cursor editing, not just the custom model prompt.
Redirected stdin and missing prompt_toolkit keep builtin input() behavior.
E2E: real PTY with raw escape bytes through line_input, cli_output.prompt,
and setup.prompt (arrows + Ctrl+A/E edit correctly); redirected-stdin
fallback verified.
The #89503/#70058/#74295/#87279 bug class kept regenerating because every
transport and provider profile hand-rolled its own effort translation map
(9 sites, 4 distinct policies). New agent/reasoning_effort.py is the single
source of truth:
- EFFORT_LADDER: canonical low->high ordering (superset check against
VALID_REASONING_EFFORTS pinned by test)
- clamp_effort(): one policy — supported passes verbatim, otherwise nearest
WEAKER supported level (never escalate, never invert the ladder), floor
when nothing weaker, 'none' never a degradation target, declared
vendor-documented overrides win, bespoke names pass through
- declared wire vocabularies as data: OpenAI-compat, Codex Responses,
xAI (4.6/legacy), Actual relays, Kimi K3/K2, TokenHub, GLM-5.2,
DeepSeek V4, Ollama Cloud, Meta, Solar
Converted sites (all behavior-preserving except noted):
- chat_completions chokepoint, Kimi + TokenHub paths
- codex transport (backend branches now pick a declared set)
- auxiliary_client Responses path
- hermes_cli.models clamp_reasoning_effort_to_supported -> thin wrapper
- plugins: kimi-coding, zai, opencode-zen, deepseek, ollama-cloud,
meta-ai, upstage, custom (copilot already routes via the wrapper)
Behavior fixes the shared policy surfaces:
- ollama-cloud/opencode-go 'minimal' now degrades to 'low' instead of
being dropped (drop left the server default = MORE thinking than asked)
New tests: ladder contract (every configurable level is clamped by every
declared wire set; monotonicity across the full ladder for every set).
/api/model/options probed Copilot auth via `gh auth token` four separate
times per payload build. When gh has no credential store for the backend's
HOME (fresh profile, desktop-spawned backend, CI), each probe blocks its
full 5s subprocess timeout on keyring/D-Bus, so every open of the Desktop
Models or Providers settings page took 20s — past the renderer's 15s IPC
budget, painting 'Error invoking remote method hermes:api: Timed out'.
Fix: cache the gh-CLI probe result (hit or miss) for 5 minutes with an
invalidation hook, feed gh stdin=DEVNULL, and disable gh interactive
prompts/update notifier in the probe env.
Measured on the failing profile: 20.5s -> 5.3s cold (one bounded probe),
0.03s warm.
Teknium