Commit Graph

5346 Commits

Author SHA1 Message Date
kshitijk4poor 42e1aa39fc fix(serve): widen fd-1 sentinel write to BACKEND_PORT_IN_USE sibling site
The same stdout redirect that rerouted the READY sentinel (#96282) also
reroutes the machine-parsed BACKEND_PORT_IN_USE sentinel printed by
_report_port_in_use() — both preflight and probe-to-bind-race callers run
after tui_gateway.server's sys.stdout=sys.stderr swap. Extract the fd-1
write into _write_machine_sentinel_line() and use it at both sentinel
sites; human-facing hint lines stay on print().
2026-08-27 17:11:56 +05:30
Kitson Kelly f2dd32d3e5 fix(serve): announce READY sentinel on fd 1, not the redirected sys.stdout
Since 6d4e851d8 the serve startup path imports tui_gateway.server (for the
flush-on-SIGTERM handlers) before the READY sentinel is printed. That module
redirects sys.stdout to sys.stderr at import time, so the
HERMES_(BACKEND|DASHBOARD)_READY port=<n> sentinel landed on stderr while the
Electron desktop spawn watches child.stdout only — the desktop timed out
after 90s and killed a perfectly healthy backend (issue #96282).

Write the sentinel to the real stdout file descriptor (fd 1 is untouched by
the Python-level redirect), with a print() fallback.

Adds a regression test that captures stdout/stderr separately — the existing
E2E suite merges them, which is exactly how this slipped past CI.
2026-08-27 17:11:56 +05:30
Teknium ab80a38514 fix(models): delist openrouter/elephant-alpha (no longer served by OpenRouter)
Live /api/v1/models probe (2026-08-27) confirms the id is gone from the
catalog, so the curated picker entry was a dead pick. Manifest
regenerated. No provider-agnostic metadata existed for the slug.

Delist credit: @orouge97 flagged this in PR #80036.
2026-08-27 04:37:31 -07:00
Adolanium a9611f3c6f feat(models): add GLM-5.3-Flash to z.ai and OpenCode Go pickers
OpenRouter and Nous already list z-ai/glm-5.3-flash (#95621). The
native z.ai picker, OpenCode Go/Zen fallbacks, setup wizard, and
Coding Plan probes did not. Context still resolves through the
existing glm-5.3 1M key.
2026-08-27 04:14:31 -07:00
fangliquanflq 091cc0e8be fix(hermes_cli): scope hook timeouts and fail closed on pre_tool_call
Allowlist hot-path hooks for abandon-on-timeout, keep subagent_stop on the caller thread, suppress re-fires of hung callbacks, and block tools when pre_tool_call times out.
2026-08-27 16:13:45 +05:30
kshitijk4poor 1e5fb70fb5 fix(cron): widen the lock-first liveness check to 'hermes cron status'
Sibling site of the salvaged #95947 fix (same file): cron_status
declared 'Gateway is not running — cron jobs will NOT fire' from a bare
find_gateway_pids() miss even while the runtime lock proved the gateway
alive. Now the not-running verdict requires both the scan AND the lock
to read dead; when only the lock answers, the pid line falls back to
the recorded gateway pid (or is omitted).

Two regression tests pin the false-alarm suppression and the genuine
not-running warning.
2026-08-27 16:10:57 +05:30
kshitijk4poor f2d043eb08 test(cron): pin lock-first liveness + harden lock-probe failure
Follow-ups to the salvaged #95947 cron commit:

- Wrap the lock probe in its own try/except: a crashing probe is
  'unknown', not 'dead' — the pid scan still decides instead of the
  whole tri-state collapsing to None.
- Regression tests (shape adapted from #94155 by @liuhao1024): lock
  held + empty pid scan -> alive (the reported false alarm); lock
  inactive -> pid-scan fallback both ways; crashing lock probe still
  falls back.
- patch_liveness now pins the lock probe inactive by default so the
  pre-existing pid-scan tests stay deterministic on machines where a
  real gateway holds the real lock.
- contributors mapping for magnus.lundstedt@infidyne.com.

Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
2026-08-27 16:10:57 +05:30
Magnus Lundstedt b71c3cc13c fix(cron): trust the gateway runtime lock for builtin-ticker liveness
_builtin_gateway_liveness() decides whether the builtin cron ticker can fire by
PID-scanning via find_gateway_pids(). That scan can transiently return empty even
while the gateway is up (e.g. just after a restart), so the in-gateway cronjob tool
emits a false 'Gateway is not running — jobs won't fire' while jobs are firing on time.

Prefer the gateway runtime lock: it is held for exactly the gateway's lifetime (a
reliable liveness signal), and inside the gateway process it short-circuits to True,
so the in-gateway check can never false-alarm. Fall back to the PID scan only when the
lock reads inactive (the external-CLI path).
2026-08-27 16:10:57 +05:30
Teknium 9b44273c05 fix: follow-up for salvaged PRs #93250 + #96234
- move minimax/minimax-m3:free into the Free tier section (house
  convention: :free SKUs group together, matching glm-5.2:free and the
  nemotron :free entries) and regenerate model-catalog.json
- add Inkling family context length (1,048,576 — OpenRouter live
  metadata, 2026-08-27) to DEFAULT_CONTEXT_LENGTHS; new family slug
  otherwise fell through to no entry
- add Inkling to the reasoning stale-timeout floor table (300s tier,
  same as Grok reasoning / Ox Alpha; OpenRouter marks the family as
  reasoning-capable)
- widen the floor matcher's right-anchor separator class to include
  ':' so OpenRouter SKU suffixes (:free/:batch/:nitro) inherit the
  family floor — inkling:free previously missed the inkling entry
- regression tests for the inkling floor + ':' separator
2026-08-27 03:24:20 -07:00
Eric Zhang 51df117def feat(models): add Inkling free models to OpenRouter catalog 2026-08-27 03:24:20 -07:00
kshitij 6607f70673 feat(models): add minimax/minimax-m3:free to OpenRouter picker (#96234)
The OpenRouter model picker builds its list from a curated set of model
IDs, then filters against OpenRouter's live catalog. minimax/minimax-m3:free
exists on OpenRouter (free tier, 1M context, tool-calling) but was missing
from both the in-repo fallback list and the remote catalog manifest.

Add it to OPENROUTER_MODELS and website/static/api/model-catalog.json so
the free variant surfaces in the picker alongside the paid one.
2026-08-27 09:34:48 +00:00
Teknium 6d4e851d80 fix(serve): bounded flush-on-SIGTERM + periodic incremental session flush
A hermes serve killed mid-update lost every un-flushed in-memory session
(#94724 item 2, reported by @ruangraung): the next RPC failed with
'session-scoped RPC rejected: not in memory (detached/reaped runtime)'
and no store held the transcript. #95576 made serves survive future
updates; this closes the kill path itself:

- install chaining SIGTERM/SIGINT handlers (hermes serve / dashboard
  startup, before uvicorn's capture_signals) that first persist
  in-memory session transcripts to state.db — bounded by
  HERMES_TUI_EXIT_FLUSH_BUDGET_S (default 5s, daemon worker + join) so
  a hung SQLite write can never block exit
- _shutdown_sessions (atexit) runs the same bounded flush FIRST, before
  the slow per-session teardown a supervisor may SIGKILL mid-way
- the idle-reaper scan piggybacks a periodic incremental flush
  (marker-deduped agent._persist_session, running sessions skipped) so
  even a SIGKILL loses at most one flush interval — no new timer
  subsystem

Refs #94724
2026-08-27 02:18:16 -07:00
Teknium 01f7ce5b76 feat(sessions): one-shot single-match owner backfill for legacy NULL-profile rows (#94724)
POST /api/sessions/owner-backfill stamps a store's own serving-profile
identity onto its pre-#95407 'profile_name = NULL' session rows. Single
match by construction (each profile's state.db belongs to exactly one
profile), idempotent, one-shot-per-row, never overwrites a non-NULL
owner, and reports the stamped count for logging.

Refs #94724
2026-08-27 02:17:56 -07:00
kshitijk4poor 93a29d110d fix(hermes_cli): surface fail-closed config write refusals cleanly
Follow-ups to the salvaged #71385 guard (which raises RuntimeError from
require_readable_config_before_write on unparseable / non-mapping YAML):

- config_command: catch RuntimeError for set/unset and print a clean
  one-line error + exit(1) instead of a raw traceback on the primary
  'hermes config set/unset' CLI path.
- console_engine._capture_output: convert escaping RuntimeError into a
  ConsoleCommandError so 'hermes console' and the dashboard console
  report the refusal instead of crashing the REPL/websocket session.
- _warn_config_parse_failure: add a dedicated 'refuse-write' wording
  branch — the old fallthrough claimed 'falling back to default config'
  even though the write was refused and the file preserved.
- approval_mode: update the stale SystemExit-only comment.
- Regression tests for the console path and both config_command paths.
2026-08-27 13:27:24 +05:30
fangliquanflq 77d4d23fbc fix(hermes_cli): stop config set/unset from wiping user overrides on invalid YAML
Fail closed when config.yaml is unparseable or non-mapping before set/unset writes, reuse the readable-config guard to return the parsed mapping, and cover refuse/empty-mapping paths with regression tests.
2026-08-27 13:27:24 +05:30
Gille 6defe7eb6c fix(config): preserve lossy decimal values as strings 2026-08-27 11:49:58 +05:30
kshitijk4poor 9f05b06589 Revert "Merge pull request #94245 from kshitijk4poor/feat/gw-event-replay"
This reverts commit df7d7f6e8d, reversing
changes made to 1a66134404.
2026-08-27 11:26:57 +05:30
kshitij df7d7f6e8d Merge pull request #94245 from kshitijk4poor/feat/gw-event-replay
feat(gateway): slim WS-only server — remove FastAPI/uvicorn from desktop boot path
2026-08-27 11:22:41 +05:30
Cursor Agent 8246c4f92a fix(cli): repair interrupted update fleet restart
An interrupted hermes update after git pull advanced HEAD never
restarted running gateways, and the next update said "Already up to
date" and skipped the fleet. Persist a HERMES_HOME fleet_restart_pending
marker after HEAD moves, clear it only when restart completes (or
nothing was running), and catch up on the next hermes update even when
git is current — also when latest.json records a stale runtime SHA.

Co-authored-by: GokayAI <gokay-ai@users.noreply.github.com>
2026-08-26 21:38:20 -07:00
fangliquanflq cb54576b1a fix(gateway): isolate control routes from default executor 2026-08-26 21:38:20 -07:00
Teknium e4451ec6e5 feat(browser): close-with-approval flow for Windows real-profile (toggle arms, agent asks, blocked if still locked) [proof do-not-merge]
Refines the Windows path per three requirements:
1. Only when the toggle is set — closing is offered only if
   browser.real_profile_autoclose is on.
2. Blocked when locked — snapshot_real_profile NEVER kills; a locked profile
   always returns the [profile-locked] signal and the copy is refused. A later
   attempt that is still locked blocks again (no loop, no auto-kill).
3. Ask approval to close — closing is an explicit, user-approved step:
    (new CLI subcommand) runs
   close_browser_holding_profile only when the agent has the user's OK. The
   locked error tells the agent to ask first, then run it, then retry.

- browser_connect: snapshot blocks with _PROFILE_LOCKED_PREFIX (autoclose-armed
  message offers the close; off message says fully-quit); no in-snapshot kill.
- main.py:  subcommand (identity+binding-verified
  tree kill via close_browser_holding_profile); added to _BUILTIN_SUBCOMMANDS.
- browser_tool: surfaces the locked signal + the exact approved-close command.
- Docs/config: toggle arms + agent asks + blocked-if-still-locked.

Tests: snapshot blocks-not-kills with autoclose on AND off; process matcher
identity/binding. 73 real-profile tests pass. Windows live E2E (proof): locked
blocks fast without killing → approved close terminates Chrome → snapshot then
copies a valid DB; autoclose-off blocks with quit guidance.
2026-08-26 19:25:33 -07:00
Teknium 9e9e1b2245 feat(browser): consented auto-close of a running browser for Windows real-profile [proof workflow do-not-merge]
Live Windows CI proved copy-while-running is impossible (Chrome opens the cookie
DB deny-all). So to make Windows actually WORK — not just fail cleanly — add
opt-in auto-close: browser.real_profile_autoclose (default false). When the
profile is locked and consent is on, snapshot_real_profile terminates the
browser process tree bound to THAT user-data-dir (psutil, identity+binding
verified like the daemon reaper — browser binary AND this exact --user-data-dir
in cmdline, fail-closed on ambiguity), waits for the lock to release, then
snapshots. Destructive (loses unsaved tabs) so it's off by default and the agent
asks first; the fail-fast message names the option. No effect on POSIX.

- close_browser_holding_profile: graceful terminate → kill → poll until the
  cookie DB is openable again (bounded); reports relaunch/tray failure clearly.
- _processes_holding_profile: identity+binding matcher (never kills an
  unrelated same-name process on a different dir).
- Config key + docs admonition.

Tests: autoclose closes-then-snapshots, autoclose-failure-reports, fail-fast
names the option, process-matcher identity/binding. 74 real-profile tests pass.

Windows live E2E (PROOF workflow, reverted before merge): autoclose-off fails
fast <30s; autoclose-on terminates real Chrome, lock releases, valid cookie DB
copied.
2026-08-26 19:25:33 -07:00
Teknium 931bf613b1 fix(browser): Windows real-profile fails fast when the browser is running
Live windows-latest proof settled it: a running Chrome opens its cookie DB
deny-all (even CreateFile with FILE_SHARE_READ|WRITE|DELETE fails; sqlite
mode=ro/immutable/nolock all 'unable to open'), so copy-while-running is
impossible on Windows without VSS/admin — and the prior code HUNG ~24min on the
locked file.

Fix: a fast up-front lock probe (_profile_is_locked: one open() of the active
profile's cookie DB; PermissionError = locked) runs BEFORE any copy in
snapshot_real_profile. If locked, bail immediately with 'fully quit the browser
(incl. background/tray) and retry, or turn browser.use_real_profile off'. Never
hangs, never a silent signed-out copy. POSIX has no mandatory locking so the
probe never trips there — copy-while-running still works on macOS/Linux.

Docs: admonition stating Windows needs the browser fully closed (background
apps included); the live-drive-while-running path is #95669.

Tests: lock-probe unit coverage (readable/no-db/PermissionError), snapshot
fails-fast-no-copytree when locked. Windows live E2E asserts the fast-fail
contract (returns <30s with the quit message) + the read-strategy diagnostic.
2026-08-26 19:25:33 -07:00
Teknium 42046b452b fix(browser): copy real-profile auth DBs lock-aware (Windows 'file in use')
On Windows a running Chrome holds Cookies / Login Data / Web Data with an
exclusive lock, so the raw file copy the snapshot used raised WinError 32
('being used by another process') and the best-effort skip left a signed-out
copy — the reported profile-cloning failure.

Fix: copy the SQLite auth DBs via SQLite's online-backup API (read-only
connection + Connection.backup()), which reads a consistent COMMITTED snapshot
while the writer holds the lock. Non-DB files (Preferences, Local State) stay a
plain copy. If even the online-backup can't read a DB, snapshot_real_profile now
FAILS CLOSED with an actionable 'close <browser> and retry' message instead of
launching a silently signed-out session.

- _copy_auth_file: sqlite-backup for Cookies/Login Data/Web Data, raw copy
  otherwise, raw-copy fallback if backup fails.
- Drop -journal/-wal/-shm sidecars from the auth set + snapshot ignore: the
  backed-up DB is self-contained; a stale sidecar next to it corrupts it.
- Fresh copytree excludes the auth DBs (raw copytree of a locked file raises on
  Windows); they're always mirrored lock-aware afterward.
- _mirror_profile_auth returns the count of DBs it could not copy so the caller
  can fail closed.

Tests: locked-DB copied-via-backup (open write txn = live-lock analog, 42
committed rows, uncommitted excluded, no journal sidecar), _copy_auth_file DB vs
plain, fail-closed when unreadable. 192 browser tests pass. Live: 68 real
cookies copied through the backup path and the session launches.
2026-08-26 19:25:33 -07:00
Teknium 6e854595e2 fix(browser): real-profile review round 3 — overlay-ordering race, torn-copy marker, consent cleanup, active-only copy
Addresses the round-3 findings from @Adolanium + @kshitijk4poor on #95620:

1. Overlay-before-reuse race (blocker): _real_profile_cdp ran snapshot_real_profile
   BEFORE the session-reuse check, so a cold resolve that ends in reuse rewrote
   Cookies/Login Data under a live Chromium holding the user-data-dir open (torn
   DBs, locked txns, phantom logouts). Now: resolve copy dir as a PATH, probe
   reuse first, return early on a hit; snapshot/overlay only on the relaunch
   path when no live browser owns the dir.
2. Torn first copy poisoned freshness forever: freshness keyed on isdir(Default),
   so a half-written copy (disk full / Ctrl+C) was treated as populated and only
   ever got auth overlays. Now gated on a .hermes-snapshot-complete marker
   written only after a full copy succeeds; a torn copy is rebuilt from scratch.
3. Consent revocation left copied credentials on disk: turning use_real_profile
   off now deletes ~/.hermes/browser-profile/ on next browser use
   (cleanup_real_profile_snapshots), so cookies/logins don't outlive consent.
4. Stale non-active profile copies: only the ACTIVE profile (last_used) is copied
   into the copy's Default now — other Chrome profiles are never snapshotted
   (smaller copy, no stale credential dirs lingering).
5. Docs/config/desktop wording aligned to actual behavior (active-profile only,
   refresh on fresh session, consent-off cleanup).

Tests: overlay-skipped-on-reuse + overlay-runs-on-relaunch, done-marker gating +
torn-copy rebuild, active-only copy, consent-off cleanup (removes store +
idempotent + triggered from _real_profile_cdp). 206 browser + 222
backup/file_safety pass. Live: reuse skips re-snapshot; direct launch on the
active-only copy loads the real signed-in Gmail inbox.
2026-08-26 19:25:33 -07:00
Teknium 7fb52c14ba fix(browser): real-profile review round 2 — last_used profile, sidecar isolation, macOS26 parser, perms, lightpanda
Addresses the five findings from @kshitijk4poor + @GottZ on #95620:

1. macOS 26 LSHandlers parser returned a version number ('7559.97') from the
   nested LSHandlerPreferredVersions block instead of the bundle id — detection
   returned None on a machine whose default IS Chrome. Strip the nested block
   before the role regex.
2. Wrong profile launched (the LinkedIn/Gmail 'logged out' bug): Chrome opens
   Default, but the session lives in Local State profile.last_used (e.g.
   'Profile 6'). Resolve last_used and mirror its auth files into the copy's
   Default on both fresh and refresh paths, so the launched browser is signed in.
3. Private-URL sidecar carried the real cookie jar to arbitrary LAN hosts:
   _create_local_session gains allow_real_profile (default True); the
   force_local sidecar passes False → always a throwaway profile, and a
   real-profile resolve failure no longer breaks private-URL routing.
4. Snapshot permissions were set once (fresh only): now secure the snapshot dir
   AND its browser-profile parent on every consented launch.
5. browser.engine=lightpanda + consent gave an unactionable error: guard with
   _using_lightpanda_engine() before detection, naming the setting and the fix.

Tests: last_used mirroring (fresh+refresh+fallback), sidecar throwaway + error
isolation, macOS26 parser + detect, perms-on-refresh, lightpanda guard. 198
browser tests pass. Live: Profile-6 cookie DB lands in copy Default (file-level);
real Gmail (Default profile) still signed in.
2026-08-26 19:25:33 -07:00
Teknium f1d05ce7d8 fix(browser): real-profile snapshot is a first-class secret store + preserve channel identity
Addresses two P1 review blockers (kshitij / @kxee) on the real-profile feature:

Credential-store lifecycle for ~/.hermes/browser-profile/ (copied Cookies/
Login Data):
- exclude the singular 'browser-profile' dir from backup AND import
  (_EXCLUDED_DIRS drives both) — was silently archiving cookies/logins
- add a browser-profile/ directory-PREFIX read-deny to agent/file_safety.py,
  same class as auth.json / mcp-tokens
- secure the snapshot dir through the canonical hermes_cli.config._secure_dir
  (honors managed/NixOS group-share + HERMES_UID/GID), not a bespoke chmod

Channel identity (#95549 invariant — never normalize Beta/Dev/Canary to
stable, which would drive a different account's profile):
- detect recognized pre-release channels FIRST (Win ProgIds, macOS bundle ids,
  Linux .desktop) and return UNSUPPORTED_CHANNEL
- macOS bundle match is now EXACT (was startswith); Linux/Win channel-before-
  stable ordering; real_profile_data_dir/chromium_executable reject the sentinel
- _real_profile_cdp fails closed with a channel-specific message, never snapshots

Tests: channel-not-normalized (linux/darwin/windows), wrong-principal fail-closed,
backup exclusion, read-guard block/allow, snapshot dir secured. 187 browser +
222 backup/file_safety pass. Live re-verified: real Gmail inbox still loads.
2026-08-26 19:25:33 -07:00
Teknium 1f4d095fd8 feat(browser): real-profile browsing via agent-browser copy + browser-use CDP
Copy the user's default-Chromium profile (auth state only) into a managed
snapshot, launch Hermes' packaged Chromium on it via agent-browser, and hand
the CDP endpoint to the Browser Use CLI (and built-in tools) to drive. The
snapshot is a non-default dir, so it sidesteps Chrome 136+'s default-profile
remote-debugging block and never contends with the user's running browser;
launched without mock-keychain switches so keyring-encrypted cookies decrypt.

- consent-gated browser_exec 'local' arg (schema only appears with consent)
- fail-closed on non-Chromium default / snapshot failure
- stale-session guard: reuse only when the live session is on our copy dir
- snapshot excludes extensions/service-workers (renderer wedge) + caches
2026-08-26 19:25:33 -07:00
Jan-Stefan Janetzky 7e2c2b1b08 fix(browser): resolve snap and Flatpak Chromium profiles on Linux
real_profile_data_dir hard-wired Linux to $XDG_CONFIG_HOME/<name>, and the
xdg fragment map only knew the native package names. Ubuntu's default snap
Chromium (xdg reports chromium_chromium.desktop, profile under
~/snap/chromium/common/chromium) and Flatpak builds (~/.var/app/<id>/config/…)
therefore ended in 'profile directory was not found' for a browser the user
runs every day, and Flatpak Chrome (com.google.Chrome.desktop) was reported as
'not a supported Chromium browser'.

Try the native, snap and Flatpak locations and return the first that exists;
fall back to the native path so the error message still names a concrete
directory. Map the Flatpak application ids in the xdg lookup.

Tests cover the xdg names for all four browsers in native and Flatpak form,
and the directory preference order with a temp HOME.
2026-08-26 19:25:33 -07:00
Jan-Stefan Janetzky f5e6028417 fix(browser): read the macOS https handler per entry and drop the installed-browser fallback
_detect_default_darwin matched a Chromium bundle id and the literal 'https'
anywhere in the whole LSHandlers dump, so a browser registered for ftp or a
content type was reported as the https default, and map order decided ties.
When nothing matched it fell back to the first installed Chromium app — with
Safari or Firefox as the actual default that drove a browser the user never
consented to, contradicting the docstring, the config comment and the desktop
copy ('a non-Chromium default fails with a clear message').

Parse the dump entry by entry, take the LSHandlerRoleAll/Viewer of the entry
whose LSHandlerURLScheme is https, and fail closed on anything else — an empty
handler list is what macOS stores while Safari is still the implicit default.

Tests feed real 'defaults read' output shapes instead of patching the detector
(reviewer fixture from the PR discussion: Safari on https, Chrome on ftp).
2026-08-26 19:25:33 -07:00
Jan-Stefan Janetzky c41562cf0e fix(browser): pass encoding to detector subprocesses and accept local_browser in test spies
The two default-browser detectors call subprocess.run(text=True) without an
explicit encoding, which the Windows-footgun linter (and its full-repo test,
tests/scripts/test_footgun_subprocess_encoding.py) rejects. Pass
encoding='utf-8', errors='replace' like the rest of the tree.

Two existing tests replace browser_navigate / _navigation_session_key with
positional-only lambdas; both callables now receive local_browser= from the
registry handler and browser_navigate, so the spies raised TypeError. Accept
the keyword with its default.

Fixes the three CI failures on the PR head (Windows footguns lint,
test_browser_extension_router_wiring x2, test_browser_open_timeout).
2026-08-26 19:25:33 -07:00
Teknium 830e4a29be feat(browser): consent-gated real default-Chromium profile for local browsing + local_browser arg 2026-08-26 19:25:33 -07:00
Teknium 71823be9da fix(update): stop counting the Windows resume token as a fleet runtime
Fixes #93406 (residual). _fleet_probe_expected_runtimes counted the
_windows_gateway_resume pause/resume token (profiles/unmapped entries)
as an 'expected fleet rows' signal. The token is pause/resume
bookkeeping, not a runtime inventory, and its entries have no rows
collect_fleet_versions() can return: unmapped Scheduled-Task gateways
never publish gateway_state.json, and a resumed profile gateway
relaunches detached and may not republish within the probe window. So
every Windows update that paused a gateway set _fleet_rows_expected,
the verification loop silently waited out its polling window (~14 min
wall clock with the retry loop on user reports), printed 'Fleet version
check returned no rows', and exited 1 for an update that succeeded.

Expected-runtimes now keys only on row-capable signals: restart-phase
bookkeeping, the pre-restart PID snapshot, and the pre-update plan
inventory -- which already cover any genuinely live pre-update Windows
gateway.

Counterfactual proof: tests/hermes_cli/test_update_fleet_probe_resume_token.py
fails on the pre-fix predicate (token-only => True) and passes with the
fix; the row-capable signals are pinned unchanged.
2026-08-26 18:23:33 -07:00
Teknium e1e72f109c fix(mcp): register stdio MCP helper children in the spawn ledger and reap orphans (#61514)
Stdio MCP helper subprocesses (npx/binary servers) never import Hermes
code, so they could not self-register in the machine spawn ledger and an
unclean parent exit left them running invisibly forever.

- process_identity.register_child(pid, purpose): ledger mirror of
  register_self for spawned children — records the CHILD (pid,
  create_time) with this process as spawner. Refuses pid-only entries a
  PID reuse could forge. Writes go through the single _append_entry
  path under _LEDGER_LOCK (prune + atomic tmp/replace unchanged).
- 'mcp-helper' added to REAPABLE_PURPOSES so the updater's
  _ledger_reapable_backend_pids rung flows helpers through its existing
  spawner_is_dead gate (live spawner => never reaped).
- tools/mcp_tool.py: best-effort register_child(pid, 'mcp-helper') at
  the post-spawn PID capture; never breaks MCP startup.
- reap_orphaned_mcp_helpers(): startup sweep mirroring
  _reap_orphaned_desktop_local_serves but ledger-driven — kills only
  helpers whose recorded spawner is PROVABLY dead, with a create_time
  re-check at kill time. Wired next to the desktop serve reap in
  web_server.py.
2026-08-26 17:42:58 -07:00
fred0m 53057f2bc4 fix(update): run config migration on the 'Already up to date' repair path (#91360)
A failed update attempt can pull fresh code onto disk and then die before
the config-migration block (e.g. a PyPI timeout during the dependency
sync). The desktop hand-off retries; the retry takes the commit_count == 0
branch, repairs deps, prints 'Already up to date!' and returns early -
skipping _run_config_check_fresh / migrate_config entirely. The fresh
code (requiring a newer _config_version) then refuses to start against
the old config until 'hermes doctor --fix' is run.

Fix: _maybe_migrate_config_on_current() mirrors the version_bump_only
handling (silent, non-interactive) and is called on both repair-path
completion points before claiming success.

Also: scripts/desktop-update/posix.sh no longer retries when the update
was deliberately SKIPPED (checkout parked on a non-target branch) -, the
retry is deterministic and only wastes time. Uses a dedicated non-
colliding exit code (8) and an honest message instead of 'Update failed'.

New tests: tests/hermes_cli/test_update_config_migration_on_current.py
(5 cases: migrate-when-behind, noop-current, noop-ahead, warning re-
surface, silent check failure).
2026-08-26 17:42:50 -07:00
loulanyue bf5ff51078 fix(update): check and apply config migrations on current checkout / retry paths (#91360)
When an update was interrupted or failed mid-install (e.g. dependency install
timeout) after pulling new code, the subsequent update run takes the
'commit_count == 0' path and early-returned without checking or migrating
the configuration. Fresh code requiring a newer config version would fail to
boot on the next run.

Extract _check_and_apply_config_migration and invoke it across all update
completion paths (normal update, current checkout / node repair, and python
dependency repair).
2026-08-26 17:42:50 -07:00
Casey 790e1eb6bd fix(update): pause SCM-supervised Windows gateway services before venv mutation
On Windows installs where the gateway runs as an SCM service (WinSW,
NSSM, sc.exe create), the existing pause machinery kills the gateway
process directly — and the service wrapper's failure ladder resurrects
it within seconds, re-taking the venv file locks mid-update. The update
then dies partway through dependency sync with access-denied errors.

This extends _pause_windows_gateways_for_update() to detect when a
gateway's process tree is owned by a running SCM service, and to stop
the SERVICE through sc.exe instead of killing the child:

- gateway/status.py: expose service-ownership discovery for gateway
  runtimes (find_windows_gateway_services maps validated gateway PIDs
  through process ancestry to running SCM service PIDs, with
  create-time identity checks against PID reuse).
- hermes_cli/update_cmd.py: stop verified services via sc.exe before
  venv mutation and restart them afterward. Stops wait for a stable
  SCM 'stopped' state AND for the original descendant processes to
  exit (service 'Stopped' is not proof the child released its
  handles). Failure to prove ownership, stop a service, or restart it
  fails closed; rollback restores attempted services, and rollback
  failures are surfaced rather than swallowed.
- Fail-closed throughout: unreadable identities, ambiguous ancestry,
  or a service that will not reach a stable state abort the update
  before any file mutation.

Complements #37039 (gateway-only concurrent instances no longer abort):
that fix lets the update proceed past the gate; this one makes the
pause actually stick when the gateway is service-supervised.

Note: tests/gateway/test_status.py::TestReadProcessCmdlinePsFallback::
test_ps_fallback_when_proc_unavailable fails on Windows on current main
before this change as well (POSIX ps fallback asserted on a platform
without it); all other touched suites pass (155 passed, 5 skipped).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-26 16:45:31 -07:00
Teknium b2c011364e fix(update): conservative outcomes + serve-ledger coverage for fresh restart recovery
Salvage adjustments to PR #94392 per review:

- Narrow the supervisor claim to the systemd-VERIFIED path only. The fresh
  recovery child now probes 'systemctl --user is-active' after each relaunch;
  only an observed-active systemd unit is reported 'verified'. A relaunch that
  merely exited 0 is labelled 'relaunch_attempted', never counts as supervisor
  coverage, and never clears gateway_fleet_restart_incomplete.
- Serve-owned runtimes (serve/dashboard entries from the spawn ledger, per the
  update_inventory serve collector) are no longer silently skipped: the
  recovery pass records them (and manual gateways) as skipped-with-reason in
  the recovery result and the persisted update receipt.
- Receipt fresh_recovery persists the conservative vocabulary
  (requested/verified/relaunch_attempted/failed/skipped); 'succeeded' is gone.
- Added an end-to-end test that drives the real recovery module in a genuinely
  fresh interpreter (sitecustomize shim intercepts the grandchild
  'gateway restart' and systemctl probes).
2026-08-26 16:45:26 -07:00
joaomarcos f9135c189c fix(update): persist fresh recovery outcome 2026-08-26 16:45:26 -07:00
joaomarcos ccdd7f41ed fix(update): persist per-profile recovery outcomes 2026-08-26 16:45:26 -07:00
joaomarcos f0045c5383 fix(update): verify fresh restart recovery results 2026-08-26 16:45:26 -07:00
joaomarcos 5609ccbece fix(update): recover aborted gateway restart in a fresh process 2026-08-26 16:45:26 -07:00
Teknium df3d41ee67 fix(update): sweep aborted-fetch tmp_pack debris before it corrupts the pack directory (#93732)
Every git fetch that dies mid-transfer (timeout, HTTP 429, dropped
line) strands a tmp_pack_* file in .git/objects/pack, and git never
cleans them. The banner's background update check is the main generator
on flaky lines — several aborted fetches a day — and the reporter's
install accumulated hundreds of files / 6.0 GB over 9 days until the
pack directory corrupted outright and every update check hung or
failed permanently.

clear_stale_tmp_packs() in gitlock.py sweeps tmp_pack_/tmp_idx_/
tmp_rev_/tmp_mtimes_ debris with the exact safety contract the lock
sweep already uses: only files past the 10-minute age floor, never
while any git process runs, never raises, real pack-*.pack/.idx files
untouchable by construction (prefix match). Wired into all three
fetch-adjacent sites: _cmd_update_check, the update apply path, and
the banner's passive check (generator = janitor).

Live E2E: 300 aged tmp_pack files (the reported scale-shape) swept
from a real repo; an in-flight fresh tmp and ancient real packs
survived; fsck clean and a real fetch round-trip succeeded after.
2026-08-26 16:45:22 -07:00
chelsealong 2812d6121b fix(cli): note pre_restart_pids' per-PID data model gap, pin the matching-start_time path
Addresses the two follow-up notes from review: document that
pre_restart_pids is a bare PID set (not (pid, start_time) pairs), so a
recycled PID from one gateway landing in another's stale record could
still mislabel it as down; and add a companion test asserting a
matching start_time still yields the live/current row.
2026-08-26 16:14:32 -07:00
chelsealong 5d7ed70eef fix(cli): guard the post-update fleet check against PID reuse
collect_fleet_versions()'s gateway_state.json fallback path only checked
_pid_exists(pid) to decide whether a recorded gateway was still running.
On Windows, a paused gateway's PID can be recycled by an unrelated
process spawned during the update's own churn (npm/git/python
subprocesses) before the record is refreshed, so the dead gateway's
stale code_sha still gets compared against HEAD and reported STALE for a
PID that no longer belongs to it (#93258).

Switch to runtime_status_pid_is_live(), the existing (pid, start_time)
PID-reuse guard already used elsewhere in gateway/status.py, so a
recycled PID is treated the same as a dead one (DOWN row, or no row, per
the existing rollout-safety rules) instead of a false STALE.
2026-08-26 16:14:32 -07:00
pierrenode de2a9de788 fix(update): feed Windows gateway relaunch outcome into fleet reconciliation
#91277 Phase 2's plan-vs-execution reconciliation (match_runtime_outcomes)
cross-checks every runtime collect_runtime_inventory() saw against
restarted_services / relaunched_profiles / externally_supervised_profiles /
killed_pids — the systemd/launchd restart phase's bookkeeping. That
inventory is cross-platform (control-socket / PID-file based), so it
includes Windows gateways too, but Windows's own pause/resume mechanism
(_pause_windows_gateways_for_update / _resume_windows_gateways_after_update)
never wrote into any of that bookkeeping.

Result: a Windows gateway that was correctly stopped and relaunched by
_resume_windows_gateways_after_update was still classified "unaccounted" by
the reconciliation (the plan saw it and no bookkeeping mentions it) —
report_unaccounted_runtimes() escalates that into sys.exit(1), and in
gateway_mode also writes ".update_exit_code"="1". Every successful
`hermes update` on Windows with a running gateway reported itself as
failed, unconditionally (the sys.exit(1) is not gated to gateway_mode).

_resume_windows_gateways_after_update now records the profiles it
successfully relaunched onto the resume token; _cmd_update_impl merges
that into the shared relaunched_profiles list right before reconciliation
runs. A profile whose relaunch genuinely fails is deliberately left off
the list, so it still surfaces as unaccounted — Windows has no watcher to
recover a failed relaunch, so that escalation is the correct signal.

Regression tests exercise _resume_windows_gateways_after_update directly
(records successes, omits failures) and reproduce the reconciliation-level
bug end to end: the same plan row resolves "unaccounted" without the merge
and "restarted" with it. Mutation-verified: with the fix reverted, three of
the four new tests fail (KeyError on the token / wrong outcome).
2026-08-26 16:14:27 -07:00
Teknium 39a5aa91ed fix(serve): serve Desktop token page at / in headless mode (#94227)
The Electron shell boots by fetching / and extracting
window.__HERMES_SESSION_TOKEN__ to authenticate /api/ws
(dashboard-token.ts adoptServedDashboardToken). Headless serve 404'd
every path, so when the renderer's spawn token drifted from the
backend's live token — e.g. hermes update replaced the backend and the
env pin no longer matched — the renderer had no way to adopt the served
token, the WebSocket handshake failed, and the primary window
white-screened (#95575).

Serve a minimal token-only HTML page at the exact root path in
mount_spa()'s headless branch, matching the renderer's extraction regex.
Gate it on app.state.auth_required read at request time: a gated
(non-loopback / remote public_url) serve keeps returning the 404 JSON so
the session token never leaks past the loopback boundary. Every other
path stays 404 JSON — the SPA remains unserved.

Regression tests: TestHeadlessServeTokenPage (3 cases) — verified to
fail against the pre-fix headless branch.
2026-08-26 15:51:22 -07:00
pierrenode 6766732620 fix(memory-setup): route .env writer through save_env_value's validation gate
hermes_cli/memory_setup.py::_write_env_vars() wrote provider-controlled
.env entries with a direct Path.write_text() + post-hoc chmod, bypassing
the denylist/regex/CRLF-stripping/atomic-replace validation that
hermes_cli/config.py::save_env_value() already provides for every other
.env writer in the codebase. A malicious or buggy memory-provider plugin
declaring a crafted env-var name/value in its setup schema could inject
arbitrary lines into .env.

Routes memory-provider env writes through save_env_value(), and fixes a
regression this surfaced in plugins/memory/supermemory/__init__.py::
post_setup(), which called the old two-parameter _write_env_vars(env_path,
values) signature — restores the caller via context-local
hermes_constants.set_hermes_home_override()/reset_hermes_home_override()
instead of a removed env_path parameter, so explicit HERMES_HOME overrides
during setup still resolve correctly.

Adds test_env_file_created_with_secure_permissions, guarded on Windows
(POSIX mode bits aren't enforced there, mirroring the existing skip in
test_openviking_provider.py / test_supermemory_provider.py) since
save_env_value's atomic-replace path creates the temp file at 0o600 before
writing content, closing the TOCTOU window the old direct-write + chmod
implementation had.
2026-08-26 15:48:39 -07:00
AlexGabbia b3e477f304 fix(update): wait for resumed Windows gateway before failing fleet check
The post-update fleet version check slept 2s and probed once. On Windows the
resume path relaunches the gateway detached, and it needs ~10s to boot (the
Telegram polling reconnect) before it stamps gateway_state.json or answers the
control socket. That race reported "no rows" for a healthy resume, exited 1,
and triggered a full retry that re-killed the gateway the first attempt had
just started — leaving it down and surfacing "Update failed (exit 1)".

Poll a bounded window (up to 30s) for the resumed gateway to publish its
identity, and only treat a persistently empty snapshot as verification
failure. The fail-closed contract from #93406 is preserved: a gateway that
genuinely never comes back still exits 1.
2026-08-26 15:05:32 -07:00
Teknium 4860978115 feat(update): image/package-managed installs refuse in-place updates through one shared gate (#91277 Phase 3)
Every surface that can start an in-place mutation — hermes update
(apply), update --check, and the dashboard's update endpoint — now
routes through evaluate_update_admission(): the baked image-provenance
marker first (authoritative; a bind-mounted checkout inside a container
looks like git to the heuristics while the filesystem is an immutable
image), then the pre-existing docker/nix/apt heuristics verbatim.

A refusal prints the real update command for the deployment kind,
records a 'refused' receipt (fleet tooling sees 'not updatable in
place, use <cmd>' instead of a silent non-update), and exits 2 on CLI
surfaces — distinct from exit-1 errors. The dashboard response keeps
the per-kind error codes its UI already keys on. collect_runtime
inventory()'s updatable_in_place also honors the marker, so --plan and
receipts report image-managed truthfully even with a bind-mounted
checkout.

Live E2E (real hermes update subprocesses, real marker file): apply and
--check both refuse exit-2 with docker-pull guidance, receipts land as
refused/image-marker, an in-place corrupted marker still refuses
(fail-closed), removing the marker admits the git checkout.
2026-08-26 11:41:04 -07:00