Adoption of a second home only wrapped the file handlers that existed at
adoption time. A later setup_logging(hermes_home=<known>, mode="gateway")
skipped _adopt_secondary_home (the home is already served) and appended a
bare _ManagedRotatingFileHandler for gateway.log, which carries no home
filter and so took every profile's gateway records. When a router is
already queued, the new handler is now wrapped in a _ProfileRoutingFileHandler
over the union of the live routers' homes.
Review finding: setup_logging(A); setup_logging(B); setup_logging(A, mode="gateway") wrote gw-b into A's gateway.log.
setup_logging(hermes_home=X) for a home other than the one this process already
logs for added another queued file handler with no home filter. In a dashboard
or serve backend that builds agents for several profiles, every profile's
records were therefore written into every served profile's agent.log and
errors.log — the same lines, byte for byte — and with profile routing already
on (multiplexed gateway, Desktop cron ticker) a duplicate writer for the
secondary home sat on top of the router. Adopt the new home into profile
routing instead: enable it for the union of homes when no router exists yet,
widen the live routers otherwise, and let _add_rotating_handler recognise a
router that already covers a home.
The first compaction of a session runs check_compression_model_feasibility()
inside compress_context() BEFORE _announce_compression_start(). That probe is
network-bound (live model catalog / provider lookups; connect timeouts stack up
through proxies and slow remote gateways), so every automatic entrypoint that
reaches compress_context without its own pre-emit — the post-tool gate in
agent/turn_preflight.py::compress_after_tool_results, overflow recovery,
manual /compress — left the client with no `kind="compacting"` status for the
whole probe. On Desktop that is a bare working-row spinner with no
"Summarizing thread" label (#111294).
Move the announcement ahead of the probe in the one choke point so every
caller is covered, and retire the announced phase (force_terminal) when the
probe's hard rejection propagates so the client never stays "compacting" for
an attempt that never started.
Live repro: /tmp probe driving compress_after_tool_results with a 2 s
feasibility stand-in — before: first compacting status at t=2.29 s (after the
block); after: t=0.13 s.
`_is_secret_config_key` matched only an exact set plus four suffixes, so
credentials that `hermes config set` routes to .env but end in bare `_KEY`
(FAL_KEY, VOICE_TOOLS_OPENAI_KEY, API_SERVER_KEY) or AWS_SECRET_ACCESS_KEY
printed raw with redaction on. Every `_is_env_config_key`-routed key is now a
credential unless its suffix names a non-secret shape (_URL/_HOST/_USER/_ID/
_DOMAIN/_SCHEME); `_key`/`_access_key` join the leaf suffixes. Header names are
folded `-`→`_` before matching so `mcp_servers.<s>.headers.X-API-Key` masks on
get and on the set echo. Bare `auth` leaves the exact set: `mcp_servers.<s>.auth`
is the documented `oauth` mode enum and rendered as `***`. Unresolved `${VAR}`
placeholders are printed as-is so the operator can see which env var the config
references.
Review finding: `config get` printed FAL_KEY / AWS_SECRET_ACCESS_KEY / X-API-Key raw, masked `auth: oauth`, and hid `${VAR}` placeholders.
`hermes config get providers`, `config get providers.<p>.api_key`, `config get
<PROVIDER>_API_KEY` (the .env-routed branch) and `config get mcp_servers.<s>.env.X_API_KEY`
all printed the full credential. The agent runs this command from sessions whose transcripts
persist and get forwarded (a Gemini key surfaced in a Discord DM log), so `print` output is a
leak path the logging redactor never sees.
`get_config_value` now applies the structural masker used by `config show` before printing,
honouring `security.redact_secrets` (default on), with a `--raw` flag for operators/scripts
that need the real value. `_is_secret_config_key` extends the exact-name set with the same
`*_API_KEY / *_TOKEN / *_SECRET / *_PASSWORD` suffixes `_is_env_config_key` already routes to
.env, so env-map leaves under `mcp_servers.*.env` mask too, and the `config set` echo uses the
same predicate.
Slim redo of #84153 by @webtecnica (same direction: mask in get_config_value; dropped the
redact_url_query_params re-export and the separate redaction-enabled reader in favour of
agent.redact._redact_enabled, which already resolves the profile-scoped policy).
Fixes#110758Fixes#84106
apply_wal_with_fallback returns via _apply_delete_for_wal_reset_bug on
WAL-reset-vulnerable SQLite builds (Debian 12 / Ubuntu 22.04 system Pythons,
the reporter's pinned image) before the #110848 existing-WAL cross-VM check,
so exactly the deployment class the fix targets still got zero startup signal
while doctor flagged it. Share one helper between both early-return paths and
key the once-per-process dedupe on the DB path instead of the label so a
gateway serving several profiles hears about each database. The docker doc
remedy now uses the image's python3 (it ships libsqlite3 but no sqlite3 shell).
Review finding: vulnerable-SQLite early return skipped the cross-VM ERROR; dedupe per label; sqlite3 shell absent from image.
d8dcdfd620 (v2026.9.14) made apply_wal_with_fallback refuse to ENABLE WAL
on a fresh database whose directory is on a cross-VM bind mount (virtiofs/9p),
but a database that was already WAL on such a mount kept WAL — correctly, we
never live-downgrade under other openers — and emitted nothing. The operator
in #110848 ran exactly that shape (Podman applehv virtiofs bind mount) and got
"database disk image is malformed" within a minute with no prior signal.
- apply_wal_with_fallback: in the on-disk-WAL branch, log a once-per-process
ERROR ('cross_vm_fs_existing_wal') when the DB file is on a cross-VM
filesystem, naming the two remedies (offline PRAGMA journal_mode=DELETE
after stopping every process + database.journal_mode: delete, or move the
database to a native/named volume). The fresh-DB refusal is unchanged.
- hermes doctor: _report_database_journal_modes flags a WAL database on a
cross-VM filesystem with check_warn and the same remedy (ranked above the
WAL-reset exposure warning; the exposure bookkeeping is kept).
- docs: docker.md gains "Filesystem requirements for state.db in containers";
configuration.md's database comment no longer implies operators must set
delete by hand on virtiofs.
Detection stays /proc/self/mountinfo-based (runs inside the Linux container
on macOS/Windows hosts). locking_mode=EXCLUSIVE is deliberately not adopted:
gateway, cron and workers open state.db concurrently.
Fixes#110848
The narrowed CREATE TRIGGER IF NOT EXISTS DDL only reaches fresh stores by
itself; _execute_ddl_skipping_settled_triggers drops and recreates any trigger
whose stored body differs, which is what upgrades existing stores. Pin that
path: install the pre-narrowing trigger text, reopen, assert the narrowed body
before measuring the reaction write.
The reopen backfill guard `child.started_at >= parent.ended_at` compared against
the parent's CURRENT end boundary. A genuine markerless legacy reset child whose
parent was later reopened and re-ended has started_at earlier than that second
boundary, so it was no longer frozen with `_reset_from`; once end_reason cleared
it dropped out of /sessions as ephemeral — the multi-cycle gateway-peer shape the
issue describes. Comparing against the parent's started_at still rejects
children that predate the parent while keeping every earlier-boundary reset
child; the marker and source exclusions are unchanged.
Review finding: cycled parent's earlier reset child lost its `_reset_from` stamp and vanished from the session list on reopen.
The positive case (a markerless legacy reset child is still frozen with _reset_from on
reopen) is already covered by tests/gateway/test_resume_command.py::
test_sessions_full_keeps_legacy_reset_child_after_parent_resume, so the third test was a
change-detector duplicate.
Redaction ran before glob derivation, so a mined `GITHUB_TOKEN=ghp_… git push`
became the pattern `GITHUB_TOKEN=*** git *` and `--apply` persisted it to
config.yaml. In the permanent-allowlist matcher `***` is three fnmatch
wildcards, so that entry pre-approved any `GITHUB_TOKEN=… git …` command
(`sudo git push --force`, `chmod -R 777 /etc git x`) ahead of the dangerous
command detector. Globs now come from the raw normalized command; when the
redacted form would yield a different glob the command is proposed under its
dangerous-class key instead. Redaction stays for example rendering only.
Review finding: masked `***` inside a persisted glob widened the allowlist to arbitrary `KEY=… git …` commands.
- Move the `agent.redact` import out of the per-record loop in build_proposals.
- Keep the two display-boundary invariant tests (rendered `e.g.` line, --json
examples), drop the unit-level duplicate that asserted the same masking.
- website/docs: state that mined examples are masked at display time only;
state.db itself is unchanged.
- contributors/emails: map kokhlo's commit email.
Mined terminal commands can carry credentials (URL userinfo, env
assignments, bearer tokens). Patterns and examples are printed to the
operator and can be persisted into config.yaml, so pass them through
redact_sensitive_text like every other display boundary. Danger
classification still sees the raw command.
_mcp_registry_scope() became profile-keyed for served profiles with the
multiplex flag off, but check_fn_cache_scope() still returned None in that
mode, so the process-wide availability cache stayed keyed (fn, None) across
profiles. A served profile whose mcp__x__* check_fn now correctly resolves to
its own (absent) connection cached False for the TTL window and the launch
profile that owns the live connection lost its tools for that window.
Both sites now call one helper, agent.secret_scope.serves_routed_profile()
(multiplex on, or a HERMES_HOME override naming a home other than the
process home), so the registry scope and the cache key can no longer drift.
Review finding: served profile B's check_fn verdict shadowed the launch profile's live mcp tools via the unscoped check_fn cache.
A dashboard/desktop backend (and the per-profile cron ticker) serves sessions of
several profiles through the HERMES_HOME contextvar override while
gateway.multiplex_profiles stays off. _mcp_registry_scope() keyed every MCP
connection by the bare server name in that mode, so the first profile to
discover `zernio` owned the only connection and every later served profile —
including one whose config carries a different Authorization header — called
the server through it and got the other account's data back (#111151).
The registry scope now follows the served home: a routed profile (an override
naming a home other than the process home) gets the same per-profile overlay
the multiplexer uses, so a same-named server with other credentials is a
separate connection, discovery for profile B is a connect candidate instead of
"already connected", and status/tool views stay per profile. Single-profile
processes (no override) keep bare keys, byte-identical to before.
Fixes#111151
Credit: #111158 by @KoNit-K located the inert flag on hermes_cli surfaces; its
fix (activating fail-closed multiplex secret scoping from config.yaml on the
dashboard) is not taken — the connection-key seam, not the secret-scope mode,
is what leaks the connection, and flipping the process-wide mode from the
dashboard would change credential resolution for every code path in it.
Binding every saved URI made a URI the user explicitly set to "Never"
match (match=5) a valid fill target — wider than the manager's own policy.
Filter those out; all other URIs keep exact-origin semantics.
Review finding: match=5 (Never) URIs became fill origins.
1Password/Bitwarden items can carry several websites, but both backends
collapsed the item's urls[]/uris[] to the first origin that normalizes,
so browser_vault_fill refused every other explicitly saved origin with
origin_mismatch. Reordering the URLs in the manager just moved which
single origin worked.
VaultItemMeta now carries allowed_origins (every normalized, deduped
web origin; origin stays the first/primary one). Fill matching stays
exact-origin against that list — no wildcard, parent-domain or subdomain
inference — and the in-page synchronous check pins the origin actually
matched via build_fill_js(expected_origin=page_origin). App URIs such as
androidapp:// never widen the fill set.
_repair_object_shape recursed over every dict value as a schema node,
including the properties map itself. When one of the map's keys was
literally named properties/required, the missing-type heuristic fired on
the map and injected a bogus "type": "object" string as a parameter,
400ing the whole tool array on strict providers.
Recurse into map values only for the mapping-valued keywords
(properties, patternProperties, $defs, definitions, dependentSchemas),
matching the in-tree precedent in _rewrite_local_refs.
Fixes#110530
Two holes in the forced retry. With an explicit live-agent api_key the
retry dropped it and re-resolved from the singleton/pool, so a 401 on pool
entry B rendered account A's usage — the cross-account leak the surrounding
comments say the tiering prevents. The retry now force-refreshes the pool
entry that issued the key (or the singleton when it IS the singleton token)
and fails open when neither matches.
In a pool-only setup (empty singleton) resolve_codex_runtime_credentials
returned the pool token before consulting force_refresh, so the retry resent
the identical revoked bearer. The pool-only branch now rotates the pool
entry through try_refresh_matching when force_refresh is set.
Also: when the forced refresh itself raises inside redeem_codex_reset_credit,
surface the original 401 (re-login hint) instead of the refresh error.
Review finding: explicit-key force refresh re-resolved another account; pool-only tier ignored force_refresh.
The raw_codex branch of _resolve_openai_codex_branch (main agent built without
explicit creds via agent_init._routed_client_kwargs, and the mid-turn fallback
chain) still hardcoded the official Codex endpoint while the pooled/aux/singleton
paths honoured the override, so a proxy user's main agent silently bypassed it.
Hoist the profile-scoped read into _codex_base_url_override and use it in both
builders; the Cloudflare identity headers follow the resolved base_url.
Review finding: raw_codex client ignored HERMES_CODEX_BASE_URL while pooled/aux/singleton honoured it.
_to_async_client rebuilds default_headers from scratch and dropped the blank
Authorization set by _create_openai_client, so every async aux call (the path
aux tasks actually take) for a free-tier OpenCode model still shipped the
keyless placeholder as a bearer and 401'd. Re-apply the same keyless policy on
the async twin.
Review finding: fourth client builder (_to_async_client) missed the keyless header policy.
A free slug (deepseek-v4-flash-free) re-resolved under the paid `opencode` profile — via
`/model --provider opencode`, a fallback switch, or a config default — resolves to the
keyless placeholder, but the primary client builder blanked the Authorization header only
when `agent.provider == "opencode-free"`. The placeholder then went on the wire as a
bearer, the Zen relay 401'd every request, and the credential pool (size 0) had nothing to
rotate: the loop reported in #110831.
Key the header policy on the placeholder as well as the provider, matching agent_init and
auxiliary_client which already do so.
build_model_state dropped every inventory row whose slug name matched a
named `providers:` entry and relabelled the current provider `custom:<key>`
unconditionally. When a `providers:` key shadows a canonical provider name
(providers.openrouter: -> proxy with a models: list), a session genuinely
running on openrouter.ai lost all canonical rows and its current id became
custom:openrouter:..., which resolves to the proxy base_url — picking the
"current" row silently re-routed the session to a different endpoint.
Only rows flagged is_user_defined are replaced by the named catalogs now,
and the current provider is promoted to custom:<key> only when the session
base_url matches that entry's api_url or no canonical row for the raw key
exists (the reported relay/MixedCase class keeps its custom:<key> id).
Review finding: named entry shadowing a canonical provider dropped canonical rows and re-routed the current session to the proxy.
Stored tool call ids are minted per turn (terminal:0, terminal:1…), so the
same id recurs on later turns of one session. The Responses adapter replayed
each pair verbatim, putting the same call_id on the wire N times; strict
validators (opencode Console, OpenAI) reject the request with 400
"Duplicate function_call_output for call_id" and every retry fails the same
way (#102629, #111231 bug 1).
Every occurrence past the first now gets a `_dup<n>` wire id and the matching
tool output pops the id its function_call was given, in call order, so pairs
stay intact and stored history is untouched.
Re-applied onto the facade split (helpers moved into _replay_tool_call_items /
_tool_output_items) from PR #102634.
With no fallback_chain (or an exhausted one) the ladder tail returned the
_RERAISE_ORIGINAL sentinel and call_llm's bare `raise` re-raised the ORIGINAL
error. After a 401 that the refresh rung already healed, the retry's actionable
"404 requires available credits" was hidden behind "401 Unauthorized", steering
the user to re-login instead of top-up. Raise the narrowed `first_err` from the
ladder tail instead and drop the sentinel; the pool-rotation rung shares the
same tail so it is covered by the same change.
Review finding: exhausted no-chain path re-raised the healed 401 instead of the retry's 404 credits error.
Review feedback on the fall-through fix: the accept boundary and the
retry-fails-with-a-connection-error path through the credential rung were
reachable but untested, and two comments described behavior the code does
not have. Tests and comments only, no behavior change.
Both auth-refresh rungs in the auxiliary recovery ladder performed their
post-refresh retry with a bare `yield` inside a `return` statement, so an
exception from that retry escaped `_aux_recovery_ladder` instead of being
converted into `(None, err)` by `_rung()`. The ladder therefore never reached
`_ladder_provider_fallback` and the configured
`auxiliary.<task>.fallback_chain` was silently skipped.
Observable case: a stale Nous runtime token makes the auxiliary request return
401; the ladder refreshes the credential and retries; when the retried model is
out of credits the resulting 404 escapes the ladder and the task fails (context
compression degrades to truncation) even though a healthy fallback_chain is
configured. With a fresh token the same 404 takes the payment path, which
already falls through correctly.
Guard both rungs with `_rung()` - the helper the sibling payment rung and the
credential-pool rung below already use - so a failed retry resumes the ladder.
Copilot enterprise custom models (BYOK) expose catalog ids shaped
owner/sub/model. normalize_copilot_model_id() only tried the full id and the
id minus its FIRST segment, and returned the stripped form when neither
matched the (unreachable) catalog - corrupting those ids into sub/model,
which the CLI's second normalization pass then stripped again to model. The
Copilot API answered HTTP 400 model_not_supported on every call.
Only accept the strip guess when the remainder is itself a flat id: a result
that still contains "/" cannot be a Copilot id.
Fixes#110597
_is_structured_output_rejection matched several phrasings for a provider refusing the
structured-output field, but not gateways that validate the body with a strict pydantic
model and reject the OBJECT-form json_schema by shape:
HTTP 422: {"detail":[{"loc":["body","response_format","json_schema"],
"msg":"str type expected","type":"type_error.str"}]}
422 already passes the status check; only the phrasing was missing, so the
one-retry-without-response_format rung never engaged and the auxiliary task failed
hard (title generation left 'HTTP 422' in the session title). Treat the shape error as
a rejection: the field is what the provider refuses and the remedy is identical.
Fixes#110631
_is_unsupported_parameter_error matched "does not support" but not the contraction Bedrock
Converse returns for xAI Grok ("This model doesn't support the temperature field"), nor the
inference-profile Claude wording ("`temperature` is deprecated for this model"), so the aux
ladder's retry-without-temperature rung never fired and the ValidationException surfaced on
title/vision/compression calls (#111043, second half).
Bedrock-hosted xAI Grok (us./global. inference-profile prefixes) rejects
temperature/topP in Converse with a hard 400 — the same restriction as
Claude Opus 4.6+, but _forbids_sampling_params is Claude-only, so Grok
needs its own denylist gate in build_converse_kwargs.
Fixes#111043
`hermes profile create <name> --clone` copies whatever `hermes import-agent`
had pulled into the source profile, but leaves import-sync.json behind, so
the clone can never run `import-agent --sync` itself: its imported skills and
memories freeze at clone time.
`--sync-imports` (with --clone / --clone-from) also copies the manifest. It
is deliberately narrow: the manifest points at EXTERNAL Claude Code / Codex
trees, never at the source profile, so both profiles remain independent
islands (root AGENTS.md ruling) — config.yaml, SOUL.md and skills are still
one-off copies. Opt-in, one-directional, explicit; --clone-all already
carries the file as part of the full copy. Refused without a clone source.
faulthandler.register(SIGUSR2, ..., chain=True) writes the stack dump and
then re-raises the signal to its previous handler. SIGUSR2's default
disposition is "terminate", so the diagnostic hook added for #70344 kills the
very process an operator is trying to inspect (rc = -12), which is what the
#110437 reporter hit while introspecting a long-lived gateway. Nothing else
in the gateway installs a SIGUSR2 handler, so there is nothing to chain to.
Test: a child interpreter runs the real _start_install_faulthandler, receives
SIGUSR2, and must still be alive with a dump in gateway_faulthandler.log.
Red on origin/main (rc=-12), green with chain=False. Docs: the stack-dump
signal is now documented next to the event-loop watchdog.
#89322 fixed the bridge-local normalizeWhatsAppId, but bridge.js has since
moved its id handling to bridge_helpers.js::normalizeWhatsAppId, which still
turned `<user>:<device>@lid` into the malformed `<user>@<device>@lid` for
mentionedJid / quoted participant / reaction keys, and the Python side
(gateway/platforms/whatsapp_common.py::_normalize_whatsapp_id) did the same
':'->'@' swap on botIds. Drop the local duplicate in bridge.js, import the
helper, and strip the `:<device>` suffix on both layers so the bot's own ids
compare equal to the bare ids WhatsApp sends for mentions and quotes.
One invariant test: device-qualified botIds match a bare mentionedId and a
bare quotedParticipant; a plain group message still does not trigger.
Drop tests/gateway/test_slack_conversational_senders.py (12 parametrized
cases driving _handle_slack_message end-to-end for allow_bots policies and
canvas mentions — the allow_bots matrix is already covered by the existing
bot-filter tests in test_slack.py) and fold the remaining coverage into two
_prefilter_inbound tests: every housekeeping subtype is dropped, and every
conversational subtype (absent, file_share, thread_broadcast, me_message,
document_mention, bot_message under allow_bots=all) still passes. Distinct
ts per event: the prefilter dedups by (team, ts) before the subtype gate.
Adds the contributor email mapping for the second author.
Drop the unreachable handle_message assertion in the drop tests
(_prefilter_inbound never calls it) and state the deliberate
file_comment drop decision in the allowlist comment.
Housekeeping subtypes (channel_join/leave/topic/name/purpose,
convert_to_private/public, pins, deletions) are not a person speaking,
yet _prefilter_inbound only rejected message_changed/message_deleted,
so each of them started a full agent turn in free-response channels.
Replace the denylist with an allowlist: a message passes when subtype
is absent, file_share, thread_broadcast or me_message; everything else
is dropped. Fixes#110778.
`_clarify_callback_sync` decided "no answer arrived" by testing whether the
response text starts with '[' (the shape of the timeout / undeliverable
sentinels). A real answer can start with '[' too — a "[A] staging" choice
label picked by number, or "[urgent] ..." free text after Other — so the
clarify resolved and the agent got the answer, yet the Slack card was
rewritten to "This prompt expired" and typing was never re-armed.
`_clarify_send_then_wait` now returns `(response, answered)` and the runner
branches on that flag only.
The Slack click handler popped the retire entry as soon as Other was
clicked, but Other is not terminal: the clarify stays pending for typed
text, so a later timeout or /new reset found nothing to retire and the card
stayed stuck on "Awaiting typed answer". The entry is now popped only on a
terminal outcome (a choice click, or Other on an already-dead entry).
A typed answer to a native card (numeric pick, or text after Other) never
reaches the click handler, so the card kept its buttons forever; the
TEXT_RESOLVED intercept now retires it with the answer.
Review finding: '[' prefix mistaken for the timeout sentinel; Other click dropped the retire entry; typed answers never rewrote the card.