Commit Graph

27517 Commits

Author SHA1 Message Date
hermes-seaeye[bot] 97f3229dfd fmt(js): npm run fix on merge (#101836)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-09-03 03:17:48 +00:00
Jeffrey Quesnelle 22e6f9a428 Merge pull request #101823 from NousResearch/ethie/local-enable
enable local models by default on win32 and darwin
2026-09-02 23:08:17 -04:00
ethernet 75aaf56904 enable local models by default on win32 and darwin 2026-09-02 22:40:22 -04:00
Jeffrey Quesnelle 48c0c3a873 Merge pull request #96633 from afourniernv/chore/relay-0.8.0
fix(relay): upgrade to 0.8.3
2026-09-02 22:03:27 -04:00
kshitijk4poor 593aa74c61 test(state): pin that a v28 install still runs the trigram cron-exclusion migration
The salvaged #101266 gated its rebuild on current_version < 27; main had
already reached SCHEMA_VERSION 28 (column-reconciliation bumps), so on any
existing install the gate would never fire and cron rows would stay in the
trigram index forever. The cherry-pick resolution renumbers to v29; this
test seeds a v28 database and asserts the migration runs. Mutation-checked
against the original < 27 gate.
2026-09-03 05:08:22 +05:30
fangliquanflq ea65fcd980 perf(state): exclude cron sessions from trigram FTS 2026-09-03 05:08:22 +05:30
fangliquanflq 57162d0cc1 fix(state): bound FTS indexing for large tool results 2026-09-03 05:03:04 +05:30
Royalaid 48a00349bd test(search): make ignored-tree probe fixture portable 2026-09-03 04:56:59 +05:30
Royalaid 906cd5f443 fix(search): prune heavy trees from zero-match probe 2026-09-03 04:56:59 +05:30
Royalaid efd7277f0f fix(search): propagate order through code sandbox 2026-09-03 04:56:59 +05:30
Royalaid 73e116608e test(interrupt): prove publisher lock contention 2026-09-03 04:56:59 +05:30
Royalaid f81214307b test(interrupt): harden publisher cleanup 2026-09-03 04:56:59 +05:30
Royalaid f88bdb7086 test(interrupt): clear publisher thread state 2026-09-03 04:56:59 +05:30
Royalaid 4e5499c62e fix(search): close admission cancellation race 2026-09-03 04:56:59 +05:30
Royalaid 9e47ac17bd fix(search): serialize filename walks by root 2026-09-03 04:56:59 +05:30
Royalaid af98771b45 fix(search): normalize dash-prefixed find roots 2026-09-03 04:56:59 +05:30
Royalaid 3c864fe331 fix(search): terminate rg root options 2026-09-03 04:56:59 +05:30
Royalaid 092b355f90 fix(search): scope macOS globs and mark bounded totals 2026-09-03 04:56:59 +05:30
Royalaid 4c189cfd41 fix(search): accept bounded rg SIGPIPE and full SemVer 2026-09-03 04:56:59 +05:30
Royalaid 6d9022a55e fix(search): harden bounded fallback handling 2026-09-03 04:56:59 +05:30
Royalaid e11d91339d fix(search): fail closed and preserve explicit roots 2026-09-03 04:56:59 +05:30
Royalaid 06086b95e1 fix(search): unify fallback scans and reject partial errors 2026-09-03 04:56:59 +05:30
Royalaid 3880e4af62 fix(search): close ordering and resolver gaps 2026-09-03 04:56:59 +05:30
Royalaid 413fd2a1fc feat(search): add fast discovery file ordering 2026-09-03 04:56:59 +05:30
Royalaid aa446092d0 fix(file-ops): normalize Windows find fallback paths 2026-09-03 04:56:59 +05:30
Royalaid d21cfad3d1 test(file-ops): run subprocess helper through Git Bash 2026-09-03 04:56:59 +05:30
Christopher cf86c47624 fix(agent): evict stale screenshot payloads before send
Call the existing keep-newest vision retirement on the per-call
api_messages clone after sanitization so OpenAI-style tool-result
screenshots are not re-uploaded on every later turn.
2026-09-03 04:51:51 +05:30
Adolanium 7caee2898b perf(agent): stop rebuilding the streamed reply text on every delta
`_record_streamed_assistant_text` grew the turn's visible text with `+=`
on an attribute. Python only grows a string in place when the target is a
local variable, so this copied the whole text on every delta.

The loop runs once per streamed token, so a reply of length N costs about
N squared in copying. A 200 KB answer arriving in 4-character deltas
moves several billion characters and burns seconds of CPU in the loop the
file itself calls the hottest one in the agent.

The text is now kept as a list of pieces and joined when read. Reading
happens at turn end and on interrupt, not per delta, so the whole turn is
linear in the length of the reply. `_fire_stream_delta` used to join on
every token just to ask if the text was empty. That check now looks at
the parts list.

`_current_streamed_assistant_text` becomes a property over that list, so
the seven readers and the call sites that clear it between turns keep
working unchanged. Reading does not collapse the pieces, because a delta
landing between the join and the write back would be lost.

Measured with 8-character deltas: adding 20k deltas to an already long
text took 3.1 times as long as the first 20k before, and 0.9 times after.
2026-09-03 04:46:41 +05:30
John Paul Soliva 1a451bfaeb fix(mcp): pick the Windows launcher, not the sh script, from an npx cache
Review follow-up, and a real platform bug. On Windows npm's `.bin` holds three
siblings per binary — the extensionless sh script, `<name>.cmd` and
`<name>.ps1`. Spawning the sh one from a Windows process fails, and
`os.access(X_OK)` there is effectively an existence check, so the previous
candidate test could not tell them apart: it would have picked a file that
cannot run and broken server startup that works today via npx.

Select by extension instead (`.cmd`, then `.exe`), the same precedence
hermes_constants._candidate_node_command_names already uses for npm/npx/node,
and fall back to npx when no launcher is present.

The platform branch moved into `_npx_bin_candidates(..., windows=...)` so it is
testable by injection. Patching `os.name` instead took pytest's own traceback
formatting down with an INTERNALERROR — a test that cannot report its own
failure is worse than no test.

Also from review: an `npx pkg -y` shape (flag AFTER the spec) now falls back to
npx, since those args are forwarded verbatim and would hand the server a flag
npx would have consumed. And the structural OSV-ordering guard reports a rename
explicitly instead of raising a bare ValueError that reads like a broken test.
2026-09-03 04:40:44 +05:30
John Paul Soliva 11ed840431 perf(mcp): skip npx's resident parent when the package is already cached
`npx` resolves a package and then FORKS: it stays alive as the real MCP
server's parent for the whole process lifetime while doing no work. Measured
on a 4-agent host, that is ~48 MB of private memory per stdio MCP server —
and it buys nothing here, because Hermes already wraps the child in its own
parent-death watchdog, so npx's supervision is a second parent nobody reads.
The process tree for one server is watchdog -> `npm exec <pkg>` -> node.

When the package is already in npx's cache, spawn its binary directly and
drop the middle process. A cache miss changes nothing: the command stays
`npx`, so a cold machine still installs on first run.

The swap deliberately happens AFTER the OSV malware preflight.
`_infer_ecosystem` keys off the command basename being `npx`/`uvx`/`pipx`, so
rewriting first makes `check_package_for_malware` return None and silently
turns the gate into a no-op. Two tests pin that ordering — one behavioural,
one structural — because a future edit that moved the swap earlier would
disable the malware check without failing any test of either piece alone.
This is also why doing it in code beats the config workaround of pointing
`command` straight at the cached path: that loses the preflight.

Conservative by construction — falls back to npx for a version-pinned spec
(npx owns that resolution), an ambiguous or absent `bin` map, a missing or
non-executable binary, an unreadable cache entry, or no cache at all.

Measured on a Mac mini running four agent gateways plus a dashboard, ten
stdio MCP servers total: resident `npm exec` parents 5 -> 0, tracked
footprint 1531 MB -> 1324 MB across 30 -> 25 processes, free memory
464 MB -> 697 MB on a host that was actively swapping. Every MCP server kept
working; tool calls verified against a live Linear server afterwards.
2026-09-03 04:40:44 +05:30
RelaxJonh 6fdd93ab45 fix(tools): release per-task file registry state (#86514)
Clear file-operation trackers when a terminal task ends and remove per-path
locks after the last holder or waiter exits. This prevents long-lived gateway
processes from retaining file state indefinitely while preserving sibling-write
tracking and path-level serialization.

Fixes #86514
2026-09-03 04:34:45 +05:30
nftpoetrist c7429f60ca fix(state): close the SessionDB lock gate's blind spot on its own mixin files
test_no_locked_readers_gate.py (#97676) parses hermes_state.py's SessionDB
class body with ast and flags any method that holds the writer lock
around a pure-read query — Pattern C, where every concurrent turn's
persistence convoys behind an unrelated read. #97676 converted 39 such
methods and closed detection blind spots for alias/variable-SQL readers.

But SessionDB is declared as
`class SessionDB(SessionSearchMixin, SessionSchemaMixin, SessionPortabilityMixin)`,
and the gate only ever opened hermes_state.py — it never parsed the three
mixin files those base classes are defined in, so a locked reader
declared there was structurally invisible to the scanner regardless of
how good the alias/variable-SQL detection got.

Applying the gate's exact scanning logic to the three mixin files
directly turns up 9 genuine pure-read methods still holding the writer
lock, none in #97676's converted list:

- hermes_state_search.py: _fts_teardown_trash_step, fts_optimize_available,
  optimize_fts_storage, list_recent_user_messages
- hermes_state_portability.py: distinct_session_cwds, list_cron_job_runs,
  _get_session_rich_rows_batch, list_skill_scaffolded_sessions,
  get_first_assistant_text

_get_session_rich_rows_batch is a hot path: it backs list_sessions_rich's
compression-tip resolution and the web server's session-search hydration
across every gateway install — its own docstring already claims "same
read-your-writes guarantee as list_sessions_rich", but list_sessions_rich
was already using _read_ctx() (its guarantee comes from flush_token_counts()
before the read, not from holding the writer lock) while this method's
implementation never caught up to match.

Converted all 9 to `with self._read_ctx() as conn:`, the exact pattern
#97676 used, verified each is a genuine pure read with no hidden writes
by tracing every helper call it makes.

Extended the gate itself (_ALL_STATE_SOURCES) to scan all three mixin
files under their own class names, plus hermes_state.py, so this blind
spot can't silently reopen. Added a regression test
(test_scan_all_state_sources_visits_every_mixin_file) that plants a
synthetic violation in a mixin-shaped file and asserts the scanner still
catches it — a change that reverts the file list back to one file passes
the existing sabotage test but fails this one.

Mutation-verified: with the gate's new scope but the old (unconverted)
mixin sources, test_no_locked_pure_readers fails and names all 9 real
violations with correct file/line. Restored the fix; it passes clean.
2026-09-03 04:29:06 +05:30
kshitijk4poor af964412d9 test(web): drop idx_sessions_effective_activity before emulating a pre-column store
The salvaged #95487 adds an expression index over sessions.last_activity_at.
Two web_server tests emulate a legacy store with ALTER TABLE ... DROP COLUMN
last_activity_at, which SQLite refuses while an index references the
column ('error in index ... after drop column'). Drop the index first —
the same adjustment the PR already made in test_schema_read_probe.py. A
real pre-column store has neither the column nor the index, so the healed
path under test is unchanged.
2026-09-03 04:23:35 +05:30
Jason Wu 98a6e4a90e fix(session-search): bound recent-session browsing
Preselect indexed recent candidates before rich hydration, cap and deduplicate compression lineage traversal, and interrupt sustained SQLite work through a cooperative progress deadline. Fail closed when the bounded browse API is unavailable and cover legacy-schema reconciliation plus malformed lineage cases.
2026-09-03 04:23:35 +05:30
fangliquanflq 6064668c8f perf(usage): prefer bundled pricing before metadata fetch 2026-09-03 04:00:51 +05:30
kshitijk4poor eab28066c3 test(state): point the repair-guard docstring at the scan-then-probe admission
The referenced WAL-index-lock docstring on _live_writer_holds_db was removed
by the extraction; describe hermes_state_holders.live_writer_holds_db instead.
2026-09-03 03:50:43 +05:30
kshitijk4poor 5e01b8fa7a refactor(state): one canonical-path helper for holder scans
Follow-up on the #97330 salvage (#97329): main grew `_canonical_sqlite_path`
after the PR's base; the holder module re-implemented it as a closure. Export
`canonical_sqlite_path` from hermes_state_holders and alias the legacy name in
hermes_state. Drop the unused `_read_proc_argv` / `_looks_like_hermes`
re-exports (no in-tree callers).
2026-09-03 03:50:43 +05:30
Benjamin PERRY 06d7b77b1c fix(state): block repair on deleted SQLite holders
Move live-holder inspection into a bounded helper and make repair fail closed when a process still owns the state database, including deleted WAL/SHM descriptors and ambiguous procfs reads.

Preserve the contributor lineage from the original four-commit review train while presenting one coherent release object on current main.

Co-authored-by: fangliquanflq <fangliquan@qq.com>
Co-authored-by: ciabata <296402666+ciabata-git@users.noreply.github.com>
2026-09-03 03:50:43 +05:30
Mike Smith 5edc0c492b fix(cli): skip wrapper-side MCP discovery when chat launches the TUI
Each TUI instance spawned three stdio MCP server copies: one in the
CLI wrapper, one in tui_gateway.entry, one in the slash worker. The
wrapper's copy is dead weight — _launch_tui blocks in subprocess.call
until the TUI exits, so its registered MCP tools are never invoked,
yet the server process (35-85 MB) lives for the whole session.

Root cause: _is_tui_chat_launch() only detected --tui / HERMES_TUI=1,
so bare `hermes` with display.interface: tui fell through to
background MCP discovery in the wrapper while the TUI gateway
(spawned moments later) ran a second discovery.

Fix: _is_tui_chat_launch() now consults _resolve_use_tui() — the exact
TUI-vs-classic decision cmd_chat makes — for chat commands only
(command in {None, "chat"}), leaving mcp serve / gateway / acp / cron
discovery behavior untouched.

Verified: unit tests (RED->GREEN); E2E with a canary stdio MCP server
in a scratch HERMES_HOME counted 2 spawned copies pre-fix vs 1
post-fix (gateway's only), and the wrapper's RSS dropped ~43 MB.

Related: #71928 (same per-process duplication class), #11115 (lazy
non-core discovery).
2026-09-03 03:48:39 +05:30
kshitijk4poor 234fff6153 fix(sessions): never reap an own lease younger than the vouch snapshot
Follow-up on the #101437 salvage (#101415). `_own_live_lease_ids()` snapshots
the server's session records under `_sessions_lock`, but the registry file
lock is taken afterwards, and a sibling session attaches its lease to its
record only after `try_acquire_active_session` has already written the entry.
A finalize racing that acquire would read the brand-new lease as an orphan
and drop it. Entries this process wrote inside a 30 s grace window are kept
regardless of the snapshot; real orphans are minutes old.
2026-09-03 03:48:31 +05:30
fangliquanflq 2c4474cc69 fix(gateway): reclaim own orphaned session leases 2026-09-03 03:48:31 +05:30
kshitijk4poor 60ee3790e0 fix(gateway): publish degraded status when polling dies mid-session, via a public seam
Follow-ups on the #101406 salvage (#101391):

- The issue's measured incident was a healthy connect followed by silent
  polling loss; nothing republished for 11 h. `_schedule_polling_recovery`
  (the single entry to the recovery ladder) now publishes "retrying" while
  the adapter is running; `_record_polling_progress` already flips it back.
- `BasePlatformAdapter.send_path_degraded` property (default False) replaces
  the runner's `getattr(adapter, "_send_path_degraded")` reach into a
  plugin-private attribute; Telegram overrides it. `_mark_connected()` reads
  the property instead of taking a `degraded=` kwarg, and `_mark_degraded()`
  + `DEGRADED_STATUS_MESSAGE` replace three copies of the same write/string.
- The startup connect stamp (sibling of the reconnect stamp) honours the
  same flag.
- Tests: mid-session death publishes retrying; pre-connect death does not;
  progress never flips while not running / fatal; runner reconnect stamp
  honours the adapter flag.
2026-09-03 03:47:38 +05:30
chelsealong ce52ae21a7 fix(telegram): do not publish connected for a degraded, unconfirmed polling path
connect() intentionally returns True when Telegram polling starts in
degraded mode (or a reconnect's require_progress=False skips the strict
readiness gate), so background recovery can retry without failing
gateway startup. But _mark_connected() published platform_state
"connected" unconditionally, and the reconnect watcher in gateway/run.py
stamped "connected" again right after -- so gateway_state.json was
indistinguishable from a healthy adapter for as long as recovery took
(observed ~11h on one seat).

_send_path_degraded already tracks exactly this (set at polling
generation start, cleared on the first confirmed getUpdates round-trip),
so:
- BasePlatformAdapter._mark_connected() takes a `degraded` flag and
  publishes "retrying" instead of "connected" when set.
- TelegramAdapter passes its current _send_path_degraded into
  _mark_connected().
- The gateway/run.py reconnect watcher checks the same flag before
  overwriting the adapter's own status write.
- _record_polling_progress() republishes "connected" the moment
  polling actually proves progress, so the state does not stay wedged
  at "retrying" until the next disconnect.

Fixes #101391
2026-09-03 03:47:38 +05:30
kshitijk4poor 013e3aff0c refactor(tui-gateway): reuse the shared surrogate sanitizer for WS frames
_sanitize_ws_text now delegates to agent.message_sanitization._sanitize_surrogates
(the same U+FFFD replacement gateway/run.py, oneshot.py, and the chat helpers
use) instead of a second encode/decode round-trip implementation.
2026-09-03 03:47:26 +05:30
686f6c61 0e5ff99aeb fix(tui-gateway): keep the WebSocket open when a frame has a lone surrogate
Starlette encodes send_text as UTF-8. A lone UTF-16 surrogate in a
status payload used to UnicodeEncodeError and latch the whole Desktop
connection closed. Sanitize the frame and skip only that send.
2026-09-03 03:47:26 +05:30
kshitij 6441d3a916 Merge pull request #101652 from kshitijk4poor/salvage/83821-lifecycle-scan-budget
perf(cron): bound the lifecycle guard's whole-walk scan work (#78398)
2026-09-03 03:38:15 +05:30
kshitijk4poor e4c1e56d5d perf(cron): 1 MiB walk budget; document the independent pre-scan budget
Final-diff review measured the worst text the 4 MiB walk budget still
admitted at ~20 s of shlex (cost is linear in admitted bytes, ~7 lex passes
per line, so the line cap was not the binding knob). 1 MiB brings it to
~4 s while a 200-script wrapper graph (~5 KB) and a ~1 MB build script are
still allowed.

lifecycle_scan_root_within_budget: state that it uses a fresh budget,
independent of the full guard's walk (review point on #83821).
2026-09-03 03:33:24 +05:30
kshitijk4poor e1a791b9f6 refactor(cron): fold review findings into the lifecycle scan budget
- one _capped_read_limit() for local and remote reads (the file's history
  is local/remote divergence bugs: #76762, #77703)
- line cap 256 KiB -> 64 KiB: shlex is quadratic on a single token, so the
  worst text the budget still admits drops from ~60 s to ~17 s of lexing;
  real shell lines are far below either
- fix a comment that described an encode the code does not do; trim the
  header comment to the non-obvious WHY
2026-09-03 03:33:24 +05:30
kshitijk4poor d99eed7d83 perf(cron): bound the lifecycle guard's whole-walk scan work
The referenced-script walk in cron/lifecycle_guard.py capped each file
(1 MiB) and the recursion depth (8) but not the walk: a command referencing
hundreds of scripts, or one enormous shlex token, held the GIL for minutes
on every gateway terminal call (#78398).

Add a per-walk _LifecycleScanBudget (bytes, lines, longest line, unique
paths, remote reads) charged BEFORE any text reaches shlex, and cap each
referenced read at the remaining byte budget so an oversized file is never
read whole. Exhaustion fails closed (the existing contract for one oversized
file) and is logged at WARNING so operators can tell it from a genuine
lifecycle block. Limits are sized so real wrapper graphs never hit them:
a 200-script benign graph is allowed and a restart hidden behind it is
still caught.

tools/terminal_tool.py gates its optional launchctl pre-scan (which also
tokenizes) on the same budget; the full guard still runs afterwards.

Redesigned from #83821 by @Riccardo-Vecchi, which introduced the budget
idea but blocked benign wide graphs (64-path cap) and bundled a suffix
classification change that is left out here.

Refs #78398
2026-09-03 03:33:24 +05:30
kshitijk4poor 62f2c82f2b fix(terminal): &> after a backgrounded compound is a redirect, not a terminator
Follow-up on the #42278 salvage (#98222): `a && b & &>/dev/null c` is valid
bash but rewrote to `a && { b & } &>/dev/null c`, which binds the redirect to
the brace group and orphans `c`. Treat a suffix starting with `&>` as command
text and insert the `;` separator. Adds bash -n coverage for that shape, for
the `;;` case-arm terminator (must NOT gain a separator), a multi-line mixed
form, and separator idempotence.
2026-09-03 03:32:32 +05:30