The salvaged #88217 test replaced Path.stat on the class with a lambda
taking one positional arg; pathlib.exists() passes follow_symlinks= and
pytest's own tmp_path teardown crashed with INTERNALERROR TypeError.
Delegate to the real stat for every path other than state.db.
Review follow-ups on the live-inode import:
- A `.db` member is now page-restored into the live file, but a paired
`-wal`/`-shm`/`-journal` member from an old or hand-built archive still went
through the rename publish — installing a foreign WAL beside the restored
database (and over a live sidecar's inode). Skip them; current backups never
ship them (_EXCLUDED_SUFFIXES), now shared as _SQLITE_SIDECAR_SUFFIXES.
- restore_quick_snapshot ignored _safe_restore_db's False and counted a
refused restore as success; honour it like run_import does.
- _safe_restore_db docstring described the pre-#90950 unconditional fallback.
`hermes import` published every zip member, including `state.db`, with
`_extract_member_atomically` — a rename that swaps the file's inode. Any
gateway, dashboard, or WebUI process holding the database open keeps its
descriptor on the now-unlinked inode: it goes on serving pre-import pages
and writing sessions no other process can see, while the sidecar WAL left
beside the new file describes the database that was just unlinked. Nothing
raises, so the import prints "Import complete" and the sessions are simply
absent from the database everyone opens next.
The live-safe path already exists: `/snapshot restore` has routed `.db`
files through `_safe_restore_db()` since #65942, writing snapshot pages
into the existing file so every open connection converges. `hermes import`
— the disaster-recovery path, reached by users who already lost something
once — never got that treatment.
Route `.db` members through it. A target that does not exist yet has no
holders and no inode worth preserving, so it keeps the ordinary atomic
publish. A refused or failed live-safe restore now raises, so the import
reports a skipped file instead of counting a silent success, and the
existing database is left untouched.
Importing an older backup over newer work stays allowed but no longer
silent: the summary reports the session/message counts the import replaced,
the same before/after evidence `restore_cron_jobs_if_emptied` uses for
`cron/jobs.json`.
Closes#100960
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ShTVU941HYygvypY9JMYE
(cherry picked from commit 8ff260a312341cb85bdf7cdd570de5ff888c6013)
The gateway broadcast and turn-failure explanation printed a literal
~/.hermes/state.db; now that the line is a command the operator is meant to
run as-is, interpolate _default_db_path() so profile / HERMES_HOME installs are
pointed at the store that actually failed. Also: split a comment that a merge
fused onto the logger line in session_lost_and_found.py, and fix an inverted
test docstring.
Review blocker on e62940d: every state-db guidance site printed
hermes sessions recover --source <db>
but cmd_sessions rejects that shape with exit 2 ("--output is required
unless --inspect-only is used") before any snapshot is taken — the user
follows the instruction during a corruption incident and gets nothing.
All five state-db sites now print the established two-stage operator
contract (the same shape `sessions repair` failure output and
docs/state-db-recovery.md already use):
hermes sessions recover --source <db> --inspect-only
hermes sessions recover --source <db> --output recovered-state.db
with the stop-the-gateway precondition stated for the gateway/turn
banners, and --inspect-only leading in the hermes_state refusal strings
(inspection before writing anything).
New TestEmittedCommandsSatisfyCliContract dispatches the exact emitted
flag shapes through the real cmd_sessions and asserts they pass the
contract gate (rc != 2) on a scratch DB, plus a premise test pinning
that the v1 no-flag shape is still rejected with rc 2 — so a guidance
string can never again pass a source-substring test while the command
it prints deterministically fails.
Noted for merge order: #101423 and #101168 also touch
hermes_cli/session_recovery.py. They are complementary recovery-integrity
work, not duplicates of this guidance/gate fix; whichever lands second
should rebase and rerun the lost_and_found + session-recovery suites.
(cherry picked from commit 34dc59a284509e76a0342c36d03a2a437aa8a3b9)
Refs #100368. The forensics thread established that a sqlite3 CLI with
the WAL-reset opener bug (fixed 3.51.3+ / backports 3.50.7 / 3.44.6;
Debian/Ubuntu system shells 3.45.1/3.46.1 are in the vulnerable band)
unlinks the live -wal/-shm pair when pointed at a live state.db whose
writer's DMS lock has been cancelled, splitting the store into two
concurrent generations whose acknowledged writes vanish while both
report integrity_check ok. Hermes' own corruption banners instructed
exactly that command.
- gateway corruption broadcast, run_agent corrupt-cause explanation,
hermes_state repair-budget and forensic-backup refusals, and the
kanban manual-recovery hint now route operators to
`hermes sessions recover --source <db>` (which snapshots the damaged
bundle before any shell touches it) and warn against a raw sqlite3
shell on the live file
- find_sqlite3_cli() now refuses a WAL-reset-vulnerable shell for the
page-level salvage lane even on the snapshot, reusing the canonical
gate from hermes_cli.sqlite_runtime so the embedded runtime and the
salvage shell can never disagree
- find_sqlite3_cli_refusal() records why a shell was refused so the
lost_and_found lane can tell the operator exactly what to install
instead of a generic "not found"
- regression tests cover the version gate (vulnerable/fixed matrix, the
mirror check), every refusal reason, and each guidance site
Test plan:
- scripts/run_tests.sh tests/hermes_cli/test_sqlite3_cli_salvage_gate.py
tests/test_state_db_repair_loop_cap.py
tests/run_agent/test_corruption_recovery_guidance.py
tests/hermes_cli/test_session_recovery_lost_and_found.py
tests/hermes_cli/test_session_recovery.py tests/test_sqlite_wal_reset_gate.py
tests/hermes_cli/test_sqlite_runtime.py - 91 passed, 1 skipped locally
(cherry picked from commit e62940d1021e80e9b7d6423ced1cbdfe7dd0c37d)
- Register meta-ai as a live-first picker provider so the /v1/models catalog
leads the picker; new models appear without a PR
- Override fetch_models to exclude non-chat models (muse-image-*, muse-voice-*)
from the picker; new chat model families pass through automatically
- Slim fallback_models to a single safety-net entry (muse-spark-1.2), shown
only when the live fetch fails
- Make data-policy contributor warning model-generic (not hardcoded to 1.2)
so it covers any future -contributor model
- Update test assertion to match generic warning text
LOCAL ONLY — pre-launch, not for push.
The salvaged #95487 adds an expression index over sessions.last_activity_at.
Two web_server tests emulate a legacy store with ALTER TABLE ... DROP COLUMN
last_activity_at, which SQLite refuses while an index references the
column ('error in index ... after drop column'). Drop the index first —
the same adjustment the PR already made in test_schema_read_probe.py. A
real pre-column store has neither the column nor the index, so the healed
path under test is unchanged.
Each TUI instance spawned three stdio MCP server copies: one in the
CLI wrapper, one in tui_gateway.entry, one in the slash worker. The
wrapper's copy is dead weight — _launch_tui blocks in subprocess.call
until the TUI exits, so its registered MCP tools are never invoked,
yet the server process (35-85 MB) lives for the whole session.
Root cause: _is_tui_chat_launch() only detected --tui / HERMES_TUI=1,
so bare `hermes` with display.interface: tui fell through to
background MCP discovery in the wrapper while the TUI gateway
(spawned moments later) ran a second discovery.
Fix: _is_tui_chat_launch() now consults _resolve_use_tui() — the exact
TUI-vs-classic decision cmd_chat makes — for chat commands only
(command in {None, "chat"}), leaving mcp serve / gateway / acp / cron
discovery behavior untouched.
Verified: unit tests (RED->GREEN); E2E with a canary stdio MCP server
in a scratch HERMES_HOME counted 2 spawned copies pre-fix vs 1
post-fix (gateway's only), and the wrapper's RSS dropped ~43 MB.
Related: #71928 (same per-process duplication class), #11115 (lazy
non-core discovery).
Follow-up on the #101437 salvage (#101415). `_own_live_lease_ids()` snapshots
the server's session records under `_sessions_lock`, but the registry file
lock is taken afterwards, and a sibling session attaches its lease to its
record only after `try_acquire_active_session` has already written the entry.
A finalize racing that acquire would read the brand-new lease as an orphan
and drop it. Entries this process wrote inside a 30 s grace window are kept
regardless of the snapshot; real orphans are minutes old.
The referenced-script walk in cron/lifecycle_guard.py capped each file
(1 MiB) and the recursion depth (8) but not the walk: a command referencing
hundreds of scripts, or one enormous shlex token, held the GIL for minutes
on every gateway terminal call (#78398).
Add a per-walk _LifecycleScanBudget (bytes, lines, longest line, unique
paths, remote reads) charged BEFORE any text reaches shlex, and cap each
referenced read at the remaining byte budget so an oversized file is never
read whole. Exhaustion fails closed (the existing contract for one oversized
file) and is logged at WARNING so operators can tell it from a genuine
lifecycle block. Limits are sized so real wrapper graphs never hit them:
a 200-script benign graph is allowed and a restart hidden behind it is
still caught.
tools/terminal_tool.py gates its optional launchctl pre-scan (which also
tokenizes) on the same budget; the full guard still runs afterwards.
Redesigned from #83821 by @Riccardo-Vecchi, which introduced the budget
idea but blocked benign wide graphs (64-path cap) and bundled a suffix
classification change that is left out here.
Refs #78398
The plausibility gate excluded stub rows by a hard-coded title literal that
session_lost_and_found builds inline at two sites; rewording either would have
turned every stub into a mis-mapped row. One STUB_TITLE_PREFIX constant.
Follow-ups on the #101423 salvage (#101409):
- `stub_missing_parent_sessions` legitimately writes `started_at = 0.0` when
no timestamped message survived; a salvage where only stubs remain is
depleted, not mis-mapped. Stub rows leave the sessions denominator.
- Reuse `_EPOCH_LOW` from session_lost_and_found instead of a second
2001-epoch constant; collapse the two per-table blocks into one loop.
- Tests: a real upgraded-layout source (started_at physically appended) with
page 1 zeroed goes through the real `.recover` lane and the report comes
back `verified: False`; a stub-only output is not flagged; a mapped row
with a NULL title (the mis-mapped shape) still counts as mapped.
The lost_and_found salvage lane maps cells positionally onto the
destination template's declared column order, but a source upgraded
via ALTER TABLE has its columns in the order they were added, which
differs from SCHEMA_SQL whenever a column was inserted mid-definition
(#101409). Every row still inserts, so integrity/FK/FTS/count checks
stay green and the report ends up verified: true — while all 1,875
sessions in the reporter's DB got started_at = 0.0 with counters and
URLs shifted into the wrong columns.
Add a semantic plausibility gate to the salvage lane: when every
salvaged sessions.started_at or messages.timestamp is NULL or before
2001-09, the cells were mapped onto the wrong columns and the
recovery is reported with errors and healthy: false, so verified no
longer claims a mis-mapped output. A partially damaged column (torn
cells on some rows) does not trip the gate — only a systematic
violation does.
Fixing the positional mapping itself needs a historical-layouts
table derived from schema-version history; that design decision is
left to maintainers (suggested fix 2 in the issue).
With servers configured but none selected by -t/--toolsets, discovery still
paid the ~260ms mcp SDK import before discovering it had nothing to spawn.
Filter first; the test now asserts the SDK probe is not reached.
The cherry-picked commit added the allowed_mcp_names filter to
discover_mcp_tools(). Since then CLI startup grew a second discovery path —
start_background_mcp_discovery / the deferred desktop start in
hermes_cli/mcp_startup.py — so wiring the filter only into the inline call
would leave `hermes chat -t terminal` (the default backgrounded path) still
spawning every server.
Store the filter once in mcp_startup (set_mcp_server_filter, called from
_prepare_agent_startup from args.toolsets; `all`/`*`/empty clears it) and
have both the inline and the background discovery honor it. The unfiltered
call shape is unchanged so zero-arg test stubs keep working.
Dropped from the original PR: the atexit/SIGINT/SIGTERM oneshot MCP reap —
main already does this in _cleanup_oneshot_runtime() -> shutdown_mcp_servers().
E2E (3 configured stdio servers, real subprocesses, 5 runs median):
no filter 3 spawned / 2.0 s; `-t terminal` 0 spawned / 1 ms.
Findings from the efficiency review pass on the two salvages, applied as one
small follow-up:
- copilot_auth: check the negative cache BEFORE taking the per-fingerprint
exchange lock. During the 60 s post-failure window, dashboard polls now
raise immediately instead of parking an executor thread behind the
in-flight holder (up to ~50 s) to learn the same answer. Test hangs
without the check (timeout 124), passes with it.
- buzz _localize_inbound_media: download_path.read_bytes() was still
evaluated on the loop as the argument to the offloaded cache call — up to
the 128 MiB inbound cap. Read off the loop too.
- test_list_credential_pool_keeps_loop_responsive: 0.5 s block / 0.25 s
threshold (2x margin) so runner descheduling cannot false-fail it while a
real regression still trips it.
Follow-up to the off-loop move: once the credential-pool handlers run on
worker threads, the dashboard's periodic /api/credentials/pool polls can
overlap, and during a DNS outage each poll would have started its own
exchange and abandoned its own hung resolver thread.
- Per-fingerprint threading.Lock around the exchange: concurrent callers
wait on the one in-flight attempt, then hit the positive or negative
cache (bounded worker count, no duplicate network calls).
- _urlopen_bounded: when the hard cap fires and the abandoned worker later
succeeds, close the HTTPResponse instead of leaking the socket.
- Tests (none shipped with the original PR): hard cap + late-close,
single-flight success and failure paths, and the pool endpoint running
off-loop / keeping the loop responsive under a 200 ms blocking read.
The module already binds run_in_threadpool (used by list_profiles_endpoint)
and every sibling router uses the same starlette helper; the nine new
loop.run_in_executor(None, _run) sites now go through that alias so the
file has one offload idiom. Behaviour-identical (both hand the callable to
a worker thread).
Also sweeps the one endpoint the PR left synchronous:
update_profile_model_endpoint's _write_profile_model reads and rewrites
the profile's config.yaml on the event loop.
Two assertions per offloaded site:
- a loop probe, where the stubbed callee records whether an event loop is
running in its own thread — the idiom already used by
tests/hermes_cli/test_cron_dashboard_off_loop.py; and
- a concurrency proof, where the stubbed callee blocks on a threading.Event
while an unrelated request is timed. On the unfixed handlers that request
waits out the whole block; served off the loop it returns in
milliseconds.
The concurrency proof needs a single event loop across requests, so the
client fixture enters the TestClient context manager: that pins one
blocking portal for the whole fixture, where a bare TestClient(app) would
spin up a fresh loop per request and pass even unfixed.
Also covers the status-code mapping through the executor hop (404 on a
missing profile, 400 on a rename collision, 404 from the resolve that stays
on the loop ahead of describe-auto) and the _MISSING sentinel cases: a
desktop.json holding `null` still reports exists=true, an absent one
reports exists=false, and an empty SOUL.md is still distinguishable from a
missing one.
The client fixtures read web_server._SESSION_TOKEN from the module rather
than pinning a literal. web_server resolves that token once at import, so
whichever test file imports it first fixes the value for the session and a
later monkeypatch.setenv is silently ignored — two files hardcoding
different tokens would 401 depending on collection order.
Extends the off-loop suite to the third and last blocking method on the
`UpstreamAdapter` contract, the 401/429 rotation.
As with the two existing pairs, the primary assertion is **thread identity**,
not latency: a latency assertion measured by an HTTP client on the blocked
loop is vacuous, because the client's own timer cannot advance until the
block ends and it therefore reports a fast response on provably frozen code.
* `test_get_retry_credential_runs_off_the_event_loop` records
`threading.get_ident()` inside the fake adapter and compares it to the
loop thread, and checks the rotation still works end to end (rejected
bearer forwarded first, rotated bearer second).
* `test_event_loop_keeps_running_while_the_retry_credential_resolves`
samples a loop-side heartbeat counter from inside the stalled adapter. On
the unfixed handler it records exactly 0 loop iterations across a 0.5s
rotation.
* `test_retry_credential_failure_still_returns_the_upstream_rejection`
guards the error contract the change must leave alone: a raising rotation
is still swallowed and the upstream's own 401 is streamed back, with no
second forward.
A new `_build_rejecting_upstream` harness drives the `status in {401, 429}`
branch by rejecting every bearer except the rotated one.
Extends `test_proxy_off_loop.py` with the `/health` half, using the same
two-assertion shape as the credential tests:
- `test_is_authenticated_runs_off_the_event_loop` compares the thread the
adapter's `is_authenticated` ran on against the loop thread. Before the
fix they are the same ident.
- `test_event_loop_keeps_running_while_health_resolves_auth_state` reads a
loop-side heartbeat counter sampled by the adapter across its own stall.
Before the fix exactly 0 iterations run across 0.5s.
Both also assert the response is unchanged (`200`, `authenticated: true`),
so the offload cannot quietly alter what `/health` reports.
Adds `tests/hermes_cli/test_proxy_off_loop.py`, mirroring the harness in
`test_proxy.py`: the proxy and a fake upstream run as real aiohttp
servers on ephemeral ports under a single `asyncio.run`, guarded by
`pytest.importorskip("aiohttp")` — no pytest-aiohttp dependency.
The primary assertion is thread identity, not latency. A latency
assertion measured with an HTTP client on the blocked loop is vacuous:
the client's own timer cannot advance until the block ends, so it reports
a fast response on code that was provably frozen.
- `test_get_credential_runs_off_the_event_loop` records
`threading.get_ident()` inside the adapter and compares it to the loop
thread. Before the fix both are the same ident.
- `test_event_loop_keeps_running_while_credentials_resolve` runs a
heartbeat task on the loop and has the adapter sample its counter on
entry and exit, so the reading is taken from the loop rather than
through a client that shares it. Before the fix exactly 0 iterations
run across a 0.5s stall; after it, ~50.
- `test_credential_failure_still_maps_to_401` pins the error contract
across the change of call form. It is deliberately not in the
red-before set — it guards behaviour the fix must leave alone.
GitHub answers anonymous fetches with HTTP 401 during outages (and for
renamed/private repos). git then prompts `Username for 'https://github.com':`
on the inherited terminal and `hermes update` sits there — users read it as
Hermes demanding a GitHub login.
Every network git call in the updater (fetch/pull/push, apply + --check +
fork sync) now runs with GIT_TERMINAL_PROMPT=0 / stdin=DEVNULL, so the 401
fails fast into the fetch-failure classifier, which now reports it as a
GitHub-side rejection (likely outage) rather than blaming the user's
credentials. Credential helpers/askpass are left configured so private-fork
origins still authenticate.
Live repro: PTY-attached update --check against a 401 origin hung 15s+ on
the prompt before; exits rc=1 in 0.2s with the diagnosis after.
Same class as #73751 (@Frowtek, pre-main.py decomposition); passive banner
half salvaged from #101421 (@RobbertC5).
Addresses review feedback on #84928.
The tick interval was exposed as HERMES_NOUS_KEEPALIVE_INTERVAL_SECONDS.
AGENTS.md reserves .env for credentials and puts behavioural thresholds in
config.yaml, so the knob moves to `nous.keepalive_interval_seconds`,
following the existing `vertex:` section's precedent for non-secret
provider settings. The env var is dropped rather than bridged: it was never
released, so nothing depends on it. Adding a key to a new section is handled
by the deep-merge, so no _config_version bump is required.
test_keepalive_interval_fits_inside_the_token_lifetime asserted
`900 < 899 * 4 - 120`, which is true for any realistic interval and could
never fail. It also tested the wrong value: the configured constant is only
a ceiling, while the schedule that ships is the derived tick. Replaced with
an assertion over the derived tick for each observed lifetime, which does
fail if the derivation constants regress -- verified against both
TICKS_PER_LIFETIME=1 and MIN_INTERVAL_SECONDS=5000.
Also adds coverage for an unreadable config.yaml, which must fall back to
the module default rather than take the keepalive thread down.
pytest tests/hermes_cli/test_nous_auth_keepalive.py -> 9 passed
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up to the interval fix: ticking faster narrowed the gap but did not
close it, and the hardcoded interval was wrong for short-lifetime accounts.
Two independent problems, both needed:
1. Lifetime is not a constant. Installs have been observed issuing ~3594s
and ~899s (see #35752, which reports expires_in: 899 while this account
reports 3594). Any hardcoded interval is right for one and wrong for the
other. The tick now derives from the lifetime the server actually issued,
capped by the configured interval and floored at 60s. The access token and
the invoke agent key carry separate lifetimes, so the shorter one governs.
2. Ticking faster alone never closes the gap. The refresh only fires once a
credential is within ACCESS_TOKEN_REFRESH_SKEW_SECONDS (120s) of expiry,
so a tick spaced wider than that window steps straight over it. With a 900s
tick against a 3594s lifetime the last tick before expiry still saw 894s
remaining, declined to refresh, and the credential died 6s before the next
one. The keepalive now asks "will this outlive my next tick?" (tick + skew)
rather than reusing the request path's bare skew.
Measured against both observed lifetimes:
lifetime 3594s: was never refreshed proactively; now refreshes 900s early
lifetime 899s: was never refreshed proactively; now refreshes 227s early
The lifetime is re-read every pass rather than cached, since it can change
when the account, plan, or server-side policy does -- exactly the case the
keepalive exists to cover.
min_access_ttl_seconds is threaded through as an optional parameter, so the
request path keeps its existing 120s behaviour and only the keepalive widens
the window.
Nous Portal access tokens carry a one-hour lifetime, and the keepalive only
refreshes once a token is within ACCESS_TOKEN_REFRESH_SKEW_SECONDS (120s) of
expiry. The tick interval was 6 hours, so it could only land inside that
2-minute window by coincidence. In practice every hour rolled over untouched
and the next inference call paid a 401 plus a re-auth round trip.
Observed on a production install: 71 "refreshed Nous runtime credentials
after 401, retrying" events across current logs.
Drop the default interval to 15 minutes, which gives four ticks per token
lifetime and leaves ample margin under the TTL-minus-skew ceiling of 3480s.
Add HERMES_NOUS_KEEPALIVE_INTERVAL_SECONDS so the interval is tunable without
a source edit, matching the existing HERMES_NOUS_TIMEOUT_SECONDS convention.
Zero still disables the thread.
Callers pass no interval, so the default is what actually shipped; the
signature now resolves at call time rather than binding the constant at
import.
Cuts the 41 contributor tests down to 8 pinning the before/after contracts
(out-of-tree provider resolves end to end, copilot-acp unchanged, broken
plugin falls through, flat-install discovery + non-provider kinds untouched).
Adds the create_client hook and process_* fields to the model-provider
plugin developer guide.
An external-process provider is an agent CLI Hermes drives over stdio rather
than an HTTP endpoint. Three things about it were spelled out for one vendor,
and each was a hard stop for any other:
* ``resolve_provider()`` gates on ``PROVIDER_REGISTRY``. Its auto-extend from
``providers/`` covered api-key providers only, so an external-process profile
never entered it and ``hermes -m <that provider>`` died with "Unknown
provider" before a client was ever built.
* ``resolve_runtime_provider()`` keyed the external-process branch on the
literal ``"copilot-acp"``, so anything else silently fell through to the
OpenRouter default instead of its own runtime.
* ``resolve_external_process_provider_credentials()`` hardcoded the binary
(``copilot``), the argv (``--acp --stdio``), the env var names and the
placeholder api_key — so a third-party provider would have been handed
another vendor's CLI.
Now the profile carries what only the provider knows — ``process_command``,
``process_args``, ``process_command_env_vars``, ``process_args_env_var`` — and
the three core paths key on ``auth_type == "external_process"`` instead of a
name. copilot-acp's values move into its profile verbatim, so
``HERMES_COPILOT_ACP_COMMAND`` / ``COPILOT_CLI_PATH`` /
``HERMES_COPILOT_ACP_ARGS`` and its ``copilot-acp`` api_key placeholder behave
exactly as before; the new tests assert that alongside the out-of-tree case at
every step.
The error for a missing binary now names the provider and its own env override
instead of telling every user to install GitHub Copilot CLI.
Co-Authored-By: Junie <junie@jetbrains.com>
Three gaps between the copilot-acp picker row and what the user's
subscription actually serves (reported: picker showed the stale curated
list while the Copilot CLI offered Sonnet 5 / Opus 5 / GPT-5.6):
1. _resolve_copilot_catalog_api_key() never looked at the Copilot CLI's
own token store (~/.copilot/config.json copilotTokens). A user whose
only credential is 'copilot login' got no catalog key, the live fetch
401'd, and copilot-acp silently fell back to the stale curated list.
Add it as resolution source 3, JSONC-tolerant, with each candidate
validated and exchanged like pool entries.
2. The existing credential-pool branch unpacked exchange_copilot_token()
into two names, but it returns (api_token, expires_at, base_url) —
the ValueError was swallowed by the enclosing except, disabling that
entire resolution path. Latent since the base_url return was added.
3. GitHub now returns model_picker_enabled: false for EVERY model on
some accounts/token types, so honoring the flag rejected the whole
live catalog. Treat the flag as a display hint: when it empties the
result, refilter without it (chat/endpoint checks still exclude
embeddings and non-chat rows).
Verified live: catalog resolves 44 models for a copilot-login-only
account, matching the CLI's own picker (claude-sonnet-5, claude-opus-5,
gpt-5.6-sol/terra, gemini, kimi).
Two follow-up gaps found by actually running 'copilot login' end-to-end:
1. The CLI (without an OS keychain) stores its token in
~/.copilot/config.json under copilotTokens — a JSONC file with
//-comment header lines. Add it as an auth-evidence source in
_external_process_auth_evidence(), parsed comment-tolerantly and
counting only a non-empty copilotTokens map (config.json exists after
first launch even when logged out).
2. The desktop chat picker requests explicit_only rows, and
_filter_explicit_provider_rows() dropped copilot-acp because a CLI
login leaves no trace in active_provider, model.provider, or env vars
— exactly the Anthropic-OAuth carve-out case. Keep external_process
rows when their CLI credentials are verified (auth_verified), while
still dropping ambient executable-on-PATH-only rows so the filter's
narrower contract holds.
Net effect: after 'copilot login', copilot-acp appears in the desktop
picker and the Accounts card reads signed in; a machine with only the
binary installed keeps today's hidden-until-configured behavior.
Pin the fix class from the previous commits: auth_type-based dispatch in
get_auth_status(), positive-only auth_verified semantics (supported env
token yes, classic ghp_* PAT no, populated hosts.json yes, empty store no),
and the Accounts-tab cli_command (valid 'copilot login' default, configured
executable substitution, non-external providers untouched).
Review follow-up: the sweeper is right that the fixture's premise leaked.
It clears the tokens and both command variables, but get_auth_status()
treats an `acp+tcp://` base URL as configured on its own — no executable
required — so on a host that sets COPILOT_ACP_BASE_URL the
missing-executable test was answering a question about the host instead
of about the code.
Verified by handing the test the hostile value it was vulnerable to:
with COPILOT_ACP_BASE_URL=acp+tcp://127.0.0.1:9999 in the environment,
test_copilot_acp_hidden_when_executable_missing fails before this commit
and all three tests pass after it.
The overlay loop in list_authenticated_providers() checks every way a
provider might be authenticated — env keys, the auth store, the
credential pool, even Claude Code's external token files — but never
asks the one question that matters for an external_process provider:
does the executable resolve? copilot-acp has no key or token by design
(the spawned `copilot --acp --stdio` brings its own auth), so has_creds
stayed False and the filter dropped it from every picker. Funny enough,
five lines further down the same loop has a dedicated copilot-acp
branch for fetching its model ids — it just never got a chance to run.
Availability now comes from get_auth_status(), the same source
`hermes model` and the auth status endpoints already use, so the CLI
and GUI agree on what 'configured' means for external-process
providers.
Fixes#63662
re_register_config_hooks() cleared the entire process-global idempotence
set on every force-reload, so a profile-local plugin force-reload dropped
another live profile's ledger key without touching its still-registered
callback — the next registration call for that profile then appended a
duplicate. Scope the clear to the reloading profile's own home, and give
outbound webhooks the same force-reload restoration shell hooks already
had, since unload() wipes both from the shared _hooks dict.
Follow-up to #85245: on a profile-resolution exception the facade fell
through to `_authorization_adapter(platform, None)` — the DEFAULT bot —
which is exactly the fallthrough the profile-aware lookup exists to
prevent. Return `adapter_not_registered` instead. Tests now drive the
real `GatewayAuthorizationMixin` ladder (no hand-written stand-in) and
are trimmed to one positive + one parametrized fail-closed case.
Co-authored-by: pierrenode <298902573+pierrenode@users.noreply.github.com>
PlatformActions._resolve_adapter() unconditionally read
runner.adapters — the default profile's adapter registry — with no
awareness of multiplex/Team-Gateway secondary profiles. Every other
adapter-resolution path in this codebase (GatewayAuthorizationMixin.
_authorization_adapter, and the plugin message-injection path that
uses it) is careful about this split: a secondary profile's adapters
live in runner._profile_adapters[profile], and a stamped secondary
profile with no registry entry must fail closed rather than fall back
to the default profile's adapter of the same platform — falling back
would send replies out the wrong bot.
ctx.platform_actions had none of that: a plugin scoped to one profile
calling add_reaction/set_thread_title in a Team-Gateway deployment
would silently act through the DEFAULT profile's adapter/bot identity
instead of its own profile's, whenever the default profile also ran
that platform.
Route _resolve_adapter() through the same _authorization_adapter
lookup, resolving the calling profile via
hermes_cli.profiles.get_active_profile_name() (which reflects the
per-task HERMES_HOME override multiplex profiles already propagate).
Falls back to the old bare runner.adapters lookup only when the
runner predates _authorization_adapter (defensive, not expected in
practice).
The Windows-only test still modelled the old in-place pack (corrupt exe already
at release/win-unpacked + a .bak to roll back to). Under stage-and-swap the pack
writes into -c.directories.output=<staging>; the integrity gate runs there and a
failure discards staging while the live tree is never touched. Verified on Linux
with sys.platform patched to win32 inside the test; sabotage (skip the discard)
fails it.
`hermes update` → `hermes desktop --build-only` → `npm run pack` packed
electron-builder's output IN PLACE: before-pack.mjs wipes
`release/<platform>-unpacked` (or the mac `Hermes.app`) before the Electron
unpack/asar/rename, so any failure after that point — corrupt cached zip,
blocked download, missing dep, disk full — left the user with NO app and the
update reporting "partially complete" over an empty release/ (#86443).
Fix the class, not the predicate: cmd_gui now passes
`-c.directories.output=apps/desktop/.staging-<pid>-<ts>` to the pack, runs
the existing verification (packaged-exe probe, macOS re-sign, Windows PE
integrity gate) against the STAGED tree, and only then promotes it:
`release/<unpacked>` → `.previous`, `<staging>/<unpacked>` → `release/<unpacked>`,
drop `.previous`. A rename failure between the two steps restores `.previous`.
On any failure the staging dir is removed and the live app is untouched.
- `_purge_electron_build_cache` / `_ensure_desktop_exe_launchable` /
`_desktop_macos_relaunchable_fixup` take the output dir so the corrupt-zip
retry purge and the integrity self-heal only ever clear the staging tree,
never `release/*-unpacked`.
- `.gitignore` the staging dir so a killed build cannot dirty the checkout.
- Docs: updating.md describes the stage-and-swap Desktop rebuild step.
Live repro (real `_rebuild_desktop_after_update` → real `hermes desktop
--build-only` subprocess, fake npm whose pack wipes appOutDir then fails):
before — `release/linux-unpacked/hermes` gone after the failed rebuild;
after — marker intact, no `.staging-*` left, rebuild returns False; a
passing pack swaps the new app into `release/`.
Closes#86443
Co-authored-by: AIalliAI <285906080+AIalliAI@users.noreply.github.com>
Co-authored-by: deathxdefeat <deathxdefeat@users.noreply.github.com>
`hermes gateway status`, `hermes gateway list`, `hermes profile list/show`
and the dashboard profiles payload keyed liveness off the profile's own
gateway.pid / gateway_state.json, so a satellite profile served by the
default multiplexer (gateway.multiplex_profiles) showed "not running"
even though the multiplexer is its live inbound process.
Reuse the single lookup the start guard and cron liveness already share —
named_profile_served_by_running_multiplexer() — with an optional
profile_name so list surfaces can ask about any profile, and OR it into
gateway_running for named profiles. Default profile and unserved named
profiles are unchanged.
Salvage of #69118 rebased onto the shared helper (which post-dates it).
Co-authored-by: Isaac Dobson <isaac@dobsonheadlights.com>
Co-authored-by: Mushisushi28 <133449918+Mushisushi28@users.noreply.github.com>
Under gateway.multiplex_profiles only the DEFAULT profile's api_server is
bound; secondaries share it via /p/<profile>/ mirrors. _gateway_fire_endpoint
read the port from the TARGET profile's config.yaml/.env and then prefixed
the mirror path, so a secondary with its own API_SERVER_PORT produced a URL
nothing listens on (connection refused on every Chronos fire).
Multiplex is now detected first (config.yaml + the GATEWAY_MULTIPLEX_PROFILES
override via gateway.config._env_multiplex_profiles_override — same
semantics as the gateway loader), and in that mode the port is resolved
from the default root's config/.env with the fallback logged. Per-profile
gateway topology is unchanged.
Salvaged from PR #84755 (@bergusdz), with env-override parity restored and
the silent except replaced by a debug log.
`cmd_dashboard` started the background MCP discovery thread before importing
`hermes_cli.web_server`. The thread's first act is the ~350ms `mcp` SDK
import, which holds the GIL against the main thread's own web_server import,
so the HERMES_BACKEND_READY sentinel — and every renderer paint behind it —
moved ~300ms later on every Desktop cold start with any MCP server configured.
Desktop `serve` (headless + HERMES_DESKTOP=1) now arms discovery one second
after the sentinel instead. Starting it AT the bind was measured to give back
most of the gain (the renderer's WebSocket connect + first hydration reads
contend on the same loop). An agent build inside that window pulls the
deferred start forward itself via `wait_for_mcp_discovery`, so the bounded
join and the late-binding tool refresh behave exactly as before. Dashboard
and non-Desktop `serve` keep the eager pre-import ordering.
Minimal reimplementation of the MCP-deferral slice of #96751 by @helix4u;
the plugin-route deferral / 503 middleware / cron-after-bind slices were
measured at ~0-10ms each and are not taken.
Co-authored-by: Gille <4317663+helix4u@users.noreply.github.com>
`_env_expand_match` read `os.environ` directly, so under a multiplexed
gateway every secondary profile whose config.yaml carried
`${MATRIX_ACCESS_TOKEN}` (or `${env:...}`) expanded to the DEFAULT
profile's token loaded at startup -- each profile "had" the credential and
one inbound message fanned out across all of them. This is the residual
half of #84079 the secondary credential gate cannot see (the expanded
token is non-empty).
Add `_env_ref_lookup`: outside a secret scope it is the same
`os.environ.get`; inside a scope it goes through `get_secret`, which is
authoritative under multiplexing and an environ overlay otherwise -- the
same policy `gateway.config._getenv` and `get_env_value` already follow.
The cache env-snapshot (#58514) uses the same lookup so a scoped load is
not served another scope's cached expansion.
kshitijk4poor