Commit Graph

21936 Commits

Author SHA1 Message Date
hermes-seaeye[bot] 07ed2fdb60 fmt(js): npm run fix on merge (#85010)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-08-13 03:33:45 +00:00
Brooklyn Nicholson 8018f9e016 test(desktop): shipped sidebar defaults now include the preview line
The inbox card's preview ships on, so the default row-meta contract is
['preview', 'updated'] — the reset test pins the new shipped view.
2026-08-12 22:24:37 -05:00
Brooklyn Nicholson 8403595f78 style(desktop): satisfy jsx-prop sort on the recents section card prop 2026-08-12 22:24:37 -05:00
Brooklyn Nicholson 92620e4ad3 feat(desktop): live task progress on inbox cards
An inbox card whose session is working through a todo list shows its "X/Y"
fraction at the far right of the footer line, updating live as items
complete. The count projects the runtime-keyed todo map to stored session
ids through the same lineage-alias fallback the working/attention
projections use, emits pre-rendered strings so unchanged rows never repaint,
and skips cancelled items on both sides of the fraction.
2026-08-12 22:24:37 -05:00
Brooklyn Nicholson c96e0daba5 feat(desktop): marquee clipped card titles on hover
Hovering a truncated inbox-card title glides the clipped tail into view —
one direction at constant speed, a short hold at each end, then a snap back
to the start. Overflow is measured on pointerenter and the animation arms
only when the text actually clips, so short titles never wiggle. State lives
in DOM attributes and CSS variables, so a hover never re-renders the
memoized row, and the blanket reduced-motion override already disables it.
2026-08-12 22:24:37 -05:00
Brooklyn Nicholson 45f663b746 feat(desktop): opt-in inbox-style session cards in the sidebar
A new "Inbox style" toggle in the sidebar filter menu renders the flat
recents list as cards: a workspace header line (project when it resolves,
else the cwd leaf, else Home) with the age at its right edge, the title
grouped with a one-line last-message preview, and a model + size footer.
The preview line ships on by default and has its own Show-menu toggle,
offered only while Inbox style is active — the one-line row has nowhere
to put it.

A render variant, deliberately not a grouping — it composes with whichever
grouping is active and only the flat recents list opts in; pinned, project,
and messaging surfaces keep the one-line row. Spacing hangs off a single
--card-gap variable; the title/preview pair is one grouped cell with its own
tighter internal gap. The age and kebab sit in flow inside the header line
rather than a full-height side column, so title, preview, and footer span
the card's entire width.

The card's project label reads through a selector that resolves the label
string, so tree polls with fresh atom identity repaint only rows whose label
actually changed.
2026-08-12 22:24:37 -05:00
Brooklyn Nicholson b6ed6542a1 fix(desktop): stop double scrollbar gutters shaving the recents list
The virtualized recents list sat inside a wrapper that was itself a themed
scroller, so two 4px classic-scrollbar gutters stacked and every row ended
8px short of the sidebar edge the pinned list reaches. Drop the wrapper's
scroller when the virtual list owns scrolling, use the overlay scrollbar on
the virtual scroller (zero gutter, native fade), and neutralize both axes of
the wrapper overflow — `overflow-y-visible` next to `overflow-x-hidden`
computes to `auto` and still reserved a gutter.
2026-08-12 22:24:37 -05:00
Brooklyn Nicholson fb206fd12b fix(desktop): idle sessions without a project color get a visible dot
The idle dot variant had no background of its own, so a settled session
outside any project rendered a literally invisible dot — the row read as
missing its status indicator next to its neighbours. Fall back to the
faintest filled grey; a project color still wins when there is one.
2026-08-12 22:24:37 -05:00
Teknium e3983f91eb feat(plugins): capability-gated ctx.platform_actions facade (#64176)
Minimal v1 platform action surface for plugins, routed through the live
gateway adapter registry — the sanctioned alternative to monkeypatching an
adapter:

- ctx.platform_actions.add_reaction(platform, chat_id, message_id, emoji)
- ctx.platform_actions.set_thread_title(platform, chat_id, thread_id, title)

Gated behind a new 'gateway.platform_actions' capability in
CAPABILITY_REGISTRY (legacy key plugins.entries.<id>.allow_platform_actions,
default OFF), re-checked on every call via plugin_capability_granted (the
#84912 consent registry). Verbs validate the adapter exists and is connected,
return structured {ok, error, detail} results with stable error codes, and
never raise into hook dispatch. Every action is audit-logged with plugin id,
verb, platform, and outcome.

Telegram routes to _set_reaction / rename_dm_topic; Discord to
fetch_message().add_reaction / rename_thread. No adapter handles or raw SDK
objects are exposed.

Docs: plugins.md platform-actions section with the security note and the
explicit raw-SDK-not-shipped statement.
2026-08-12 20:10:51 -07:00
Teknium 3b7c940208 feat(gateway): more normalized gateway_platform_event types (#64176)
Extend the normalized-envelope pipeline shipped in #82063 with new event
types, each with its own versioned, event-local payload contract:

- Telegram: message_edited (edited_message updates; editor-identity auth
  extraction, forum topic thread_id, bounded text/caption, ISO edited_at)
- Discord: message_edited, message_deleted, thread_created, thread_renamed
  (on_message_edit/delete, on_thread_create/update fire-sites with has_hook
  no-subscriber fast-paths, bot-authored events dropped, rename-only
  filtering on thread updates)

All events flow through the same gateway-owned post-auth boundary; malformed
or unauthorized events drop, fail closed. Raw SDK payload access is
deliberately NOT shipped (round-2 correction: needs its own
gateway.raw_events capability and design).

The Discord fire-site machinery (no-subscriber fast-path, observer isolation,
connect-time wiring) adapts the observer-hook design from PR #62584
(@paoloantinori) onto the normalized-envelope contract; PR #36875's raw
telegram update hook is superseded by the same correction.

Docs: hooks.md gains per-event payload contract tables.

Co-authored-by: Paolo Antinori <pantinor@redhat.com>
2026-08-12 20:10:51 -07:00
Teknium fe8b44dac4 fix(ci): sync lazy_deps SDK pins + update WAL vacuum test contract
The Aug 12 pin bump (91345435a) updated pyproject/uv.lock but not
tools/lazy_deps.py, tripping the #31817 downgrade-guard tests; the
post-VACUUM TRUNCATE fix landed without updating the checkpoint test
that pinned the old no-TRUNCATE rule. Both red on pristine main.
2026-08-12 20:02:07 -07:00
Teknium 46e20083d8 feat(plugins): plugin packs — declarative, shareable plugin sets (#64166)
Adds hermes-pack.yaml: a single YAML file pinning a set of plugins to
exact 40-char commit SHAs with optional non-secret plugins.entries
config seeds and a declared (not yet installed) skills list.

CLI:
- hermes plugins pack install <path|https-url> [--force]: mandatory
  review screen (plugins + refs + declared capabilities), one summary
  confirmation, then fan-out through the existing pinned install path.
  Per-plugin capability consent rides the standard #64228 flow — a pack
  never bulk-grants. Partial failures reported per plugin; non-zero
  exit when any fail. Interactive only (no --yes in v1).
- hermes plugins pack export [--enabled-only] [--name]: pack YAML on
  stdout from install metadata (repo + exact SHA); local-only plugins
  become warning comments; secrets/capability grants stripped.
- hermes plugins pack show <path|url>: dry-run view.

Supply chain: refs must be exact 40-char SHAs (tags/branches rejected
naming the entry, same rule as the community index); config seeds
reject secret-shaped, capability, and allow_* keys; bare names resolve
through the community index; https-only URL fetch with size cap.

Tests: tests/hermes_cli/test_plugin_packs.py (36) — parse/validate,
SHA enforcement, mocked install fan-out, consent-per-plugin assertion,
export round-trip + sanitization, partial-failure exit code, parser
wiring. No live network.

Docs: user-guide plugins.md packs section (notes packs build on the
manifest v2 fields per #64165) + cli-commands.md rows.

Closes #64166
2026-08-12 19:56:44 -07:00
Teknium f4c2c263f0 docs(plugins): note autostash behavior on plugins update 2026-08-12 19:44:50 -07:00
Teknium eb214ad148 Inspired by Factory Droid: plugin updates autostash local changes
Factory Droid v0.188.0 (Aug 4, 2026): 'Updating a plugin marketplace now
succeeds when its checkout has local changes instead of failing.'

Hermes had the same failure: users who tweak an installed plugin in place
(config constants, small patches) hit 'Your local changes ... would be
overwritten by merge' on every 'hermes plugins update <name>' and the
dashboard update path — the plugin becomes permanently un-updatable
until they hand-run git.

_git_pull_plugin_dir() now autostashes before the pull and re-applies
after, reusing the ref-compared stash discipline hermes update already
uses for the main checkout (PR #70161):

- clean tree → identical single pull, no behavior change
- dirty tree → stash push --include-untracked (ref-compared so 'nothing
  saved' aborts before touching the checkout), pull, stash apply
- clean re-apply → drop the stash entry, note in output
- conflicted re-apply → reset to the updated revision (plugin stays
  importable, no conflict markers on disk) and KEEP the stash entry
  with recovery instructions
- failed pull with a stash → restore the user's edits before reporting

Covers both callers: cmd_update (CLI) and dashboard_update_user_plugin.
Real-git E2E tests for all four paths + sabotage-verified (tests fail
on the old single-pull implementation).
2026-08-12 19:44:50 -07:00
Teknium 6ee58f4088 Inspired by Muse Code: opt-in git worktree isolation for delegated subagents
Adds delegation.worktree_isolation (default: false). When enabled, each
delegate_task child gets its own git worktree branched from the repo's
current HEAD under <repo>/.worktrees/subagent-<id>, its terminal session
starts there, and its goal message carries the isolation contract
(work + commit in the worktree; parent reviews/merges the branch).

- tools/subagent_worktree.py: clean-room implementation from Muse Code's
  documented --subagent-worktree-isolation behavior (create per-child
  worktree, finalize/inspect after run, auto-prune clean no-commit
  worktrees, keep anything holding work).
- tools/delegate_tool.py: config gate + per-child setup in
  _run_single_child; result entries gain a "worktree" field (path,
  branch, commits, dirty, pruned) only when isolation engaged — the
  default-off wire shape is byte-identical.
- Git-only + local-terminal-backend-only; non-git dirs, remote backends,
  or any worktree failure degrade silently to shared-workspace behavior.
- Tests: tests/tools/test_subagent_worktree.py (15 tests, real git
  repos) + E2E through _run_single_child with a real repo verified
  parent-checkout isolation, branch reviewability, prune, and
  default-off shape pinning.
- Docs: delegation feature page section + configuration.md key.
2026-08-12 19:44:45 -07:00
Hermes Agent f508c6e40a Inspired by Perplexity Computer: session-librarian skill — prompt-driven session library management
Adds a bundled productivity skill that lets Hermes organize the user's own
session library conversationally: find sessions by topic via session_search,
summarize goals/decisions from bookends, rename them meaningfully, propose
archives/prunes with a mandatory plan-first + dry-run discipline, and split
requests into parallel workstreams via delegate_task.

Inspired by Perplexity Computer's session management by prompt (changelog
07/27/26): find/summarize past sessions, fork focused follow-ups, rename,
pin/archive with plan-first confirmation, and fan one request out into
parallel per-task sessions.
2026-08-12 19:44:39 -07:00
Teknium 314968f5fb Port from PrimeIntellect-ai/prime-agent#1258: derive OpenRouter reasoning support and effort levels from catalog metadata
OpenRouter's /v1/models entries advertise reasoning capability
(supported_parameters + reasoning.mandatory/supported_efforts). Use that
metadata as the primary gate in _supports_reasoning_extra_body instead of
the hand-maintained vendor-prefix allowlist, which went stale one vendor at
a time (nvidia/ missing -> #75386). Also clamp the requested effort to the
nearest LOWER catalog-supported level in the OpenRouter profile so ultra/max
against a high-capped route no longer 4xxes.

Cache-only on the hot path: capabilities parse for free out of the existing
fetch_openrouter_models() payload, a background warmer covers cold starts,
and unknown models/offline catalogs fall back to the static prefix list
unchanged.
2026-08-12 19:44:32 -07:00
Teknium 25363985e9 fix(skills): shorten blocked-page-recovery description to authoring hardline (60 chars) 2026-08-12 19:44:25 -07:00
Teknium 537722bf65 Port from code-yeongyu/oh-my-openagent#6662: blocked-page-recovery research skill
omo's ultimate-browsing engine added a 'surrogate retrieval tier' (PR #6662):
when a page fetch is blocked by a WAF/paywall/rate-limit, it falls back to
third-party copies (Wayback, archive.today, Jina Reader) with strict
provenance labeling and validators that reject fake successes (dead Google
Cache interstitials, AMP redirect stubs, rate-limit bodies).

Hermes adaptation: a bundled research skill + stdlib-only script instead of
a Python sub-engine — zero core-tool footprint, per the footprint ladder.
Clean-room implementation (their repo is Sustainable Use License; nothing
copied), keeping the good ideas: provenance contract (snapshot vs live),
body validation over status codes, domain rotation for archive.today,
API-first pivot guidance, and explicit skip of proxy relays (MITM).

E2E tested: recovered a real 486KB Wayback snapshot with timestamp;
validators reject redirect stubs, interstitial titles, and sub-floor bodies.
2026-08-12 19:44:25 -07:00
Teknium 3eac116b9d fix(mcp): invalidate OAuth tokens when the configured client changes
Port from cline/cline#12983 (the 'invalidate tokens when OAuth client
changes' invariant): tokens are minted for a specific client_id, so after
a user edits oauth.client_id / oauth.client_secret in config.yaml the old
tokens can only fail with invalid_client. Pre-registered clients are
deliberately exempt from the invalid_client auto-poison path, so the stale
tokens wedged every request until ~/.hermes/mcp-tokens/<server>.* was
wiped by hand.

_maybe_preregister_client() now compares the on-disk client.json identity
against the incoming config identity before overwriting it and discards
tokens.json + meta.json on a mismatch (with a log line pointing at
hermes mcp login). Unchanged identity is a strict no-op.

Proven live on main with an isolated-HERMES_HOME E2E probe; regression
tests sabotage-verified (fail without the wiring line).
2026-08-12 19:44:17 -07:00
Hermes Agent 8d9e18d40b Inspired by Perplexity Computer: Model Council mode for Mixture of Agents
Adds a 'council' synthesis style to MoA (per preset via synthesis_style,
one-shot via the new /council command on CLI + gateway). Reference models
answer independently; the aggregator chairs the deliberation and produces
a user-facing report of consensus, per-model disagreements (with the
differing assumptions behind them), unique contributions, and a
recommendation with an explicit confidence level.

Inspired by Perplexity's Model Council rollout to Perplexity Computer
(changelog 08/04/26): pick a board of 2-8 models, run them independently,
synthesize where they agree/disagree and what each uniquely surfaces.
2026-08-12 19:44:09 -07:00
Teknium 1965fde394 test: update Windows shell-hook flag test for the Popen-based spawn
_spawn() now uses subprocess.Popen + communicate() instead of
subprocess.run(); the windows_only creationflags assertion mocks Popen
accordingly and additionally pins that the POSIX-only process_group
kwarg never reaches a Windows spawn.
2026-08-12 19:44:04 -07:00
Teknium 3b9d1b3cde fix(hooks): kill the whole process tree when a shell hook times out
Port from openai/codex#37527: Terminate timed-out hook process trees.

A shell hook that forked helpers (scanners, watchers, "cmd &") and then hit
its timeout left those descendants running forever — subprocess.run() only
kills the direct child. Worse, descendants holding the inherited pipe write
ends could stall run()'s post-kill communicate() drain.

- agent/shell_hooks.py _spawn(): spawn hooks in their own process group on
  POSIX (process_group=0, Python >=3.11); on timeout/error, reap the whole
  tree via the shared kill_process_tree() helper, then drain bounded (1s).
  Hooks that complete in time keep their descendants, so intentionally
  detached helpers survive successful runs (mirrors codex semantics).
- hermes_cli/_subprocess_compat.py: rename _kill_git_process_tree ->
  kill_process_tree (it was never git-specific; taskkill /T /F on Windows,
  ownership-gated os.killpg on POSIX). Backward-compat alias retained.
- tests/agent/test_shell_hooks_tree_kill.py: real-subprocess regression
  tests (descendant killed on timeout, preserved on success, own-group
  spawn, fast-path contract, fail-open). Sabotage-verified: reverting the
  process_group spawn fails exactly the two new behavior tests.

Gap proven live on main first: a forking hook timed out at 2s and its
descendant survived; same probe against this branch shows it reaped.
2026-08-12 19:44:04 -07:00
Teknium 91345435a1 chore(deps): bump platform SDK pins — PTB 22.8, slack-bolt 1.30.0, mautrix 0.21.1
Weekly platform API scout, Aug 12 2026:
- python-telegram-bot 22.6 -> 22.8: full Bot API 9.6 + 10.0 support
  (poll persistent ids, guest mode types, live photos). No hermes code
  uses the deprecated surfaces (positional InputMedia* filename,
  correct_option_id, InputPollOption.de_json) — verified by grep.
- slack-bolt 1.29.0 -> 1.30.0: assistant DM suggested-prompts widening;
  no breaking changes (Bolt Python is unaffected by the Bolt JS v5 /
  Node SDK major-version wave).
- mautrix 0.21.1: corrupted-invite safety, /messages response parsing
  fix, custom join-rule enum values.

Validated: uv lock regenerated; real-SDK install (uv sync) then
tests/gateway telegram (590 passed) + slack/matrix (662 passed) suites
against the upgraded packages.
2026-08-12 19:43:53 -07:00
Teknium 97c06dcfd7 fix(sessions): probe sqlite3 CLI for .recover capability, not just PATH presence
Ubuntu CI (and other distro builds) ship a sqlite3 shell compiled without
the sqlite_dbpage virtual table that .recover requires, so PATH presence
alone let the lane attempt and fail with 'no such table: sqlite_dbpage'.
find_sqlite3_cli() now probes .recover on a scratch DB once; the test skip
gate uses the same probe, and the no-CLI guidance names the capability
requirement.
2026-08-12 19:43:47 -07:00
Teknium 6dad74596e fix(sessions): recover budget exhaustion + lost_and_found last-resort lane
Fixes #80205: when one ordered rowid-edge probe failed,
_salvage_rowid_bounds() substituted the whole SQLite rowid domain and
_copy_table_salvage() burned the 10,000-query budget bisecting a
synthetic tail that could not contain rows, silently omitting readable
boundary rows (field case: message 76882 of 76882). Two-part fix:

* _probe_populated_edge(): gallop outward from the surviving edge with
  doubling offsets; a clean 'no rows beyond X' probe caps the domain in
  O(log range) queries instead of exhausting the budget on it.
* exact-key singleton salvage: a one-row range scan must advance the
  cursor past the hit into the damaged sibling page to prove exhaustion,
  which discards the already-produced row; 'WHERE rowid = ?' stops at
  the hit, recovering the boundary row exactly like sqlite3 .recover.
* the strict-path refusal now points users at --allow-partial.

New last-resort lane for --allow-partial when the sessions/messages
table schemas themselves are unreadable (previously a hard refusal even
though page-level salvage recovers the rows fine). If a sqlite3 CLI is
on PATH, shell out to '.recover --ignore-freelist' into a scratch
lost_and_found DB, then heuristically map rows back into a fresh
SessionDB-schema database (hermes_cli/session_lost_and_found.py):
classification keyed on nfield counts + sentinel columns (session ids
matching ^\d{8}_\d{6}_, roles in user/assistant/tool/system, known
source strings), covering the current 54-col sessions layout, the
52-col historical layout, a 14-col legacy identity-only salvage,
rowid-alias messages rows and 18-col session_model_usage rows. Missing
parent sessions are stubbed (children are never deleted for FK
cleanup), FTS is rebuilt at the end, and output is labeled BEST-EFFORT
everywhere. Without the CLI the error names the sqlite3 requirement
with actionable guidance. Mirrors a successful manual recovery of a
real corrupt state.db (2026-08-12), and this lane was validated against
that preserved file: 32 sessions / 7 messages / 4 usage rows mapped,
integrity_check ok, opens via SessionDB.

Also fixes #72291: the source-fingerprint 'bundle changed while it was
being copied' error now enumerates that the parent interactive CLI
session itself counts as a Hermes process and suggests a fresh shell or
an immutable snapshot.

Tests use real physical page corruption (flipped b-tree/schema header
bytes), skip the CLI-dependent path cleanly when sqlite3 is absent, and
keep the mapper unit tests binary-independent via a synthetic
lost_and_found DB. Sabotage-verified: reverting the fixes makes the
regression tests fail with the exact field failure shape.
2026-08-12 19:43:47 -07:00
Teknium 7d0b5a332c chore: AUTHOR_MAP for zhouou6@users.noreply.github.com → shali10 2026-08-12 19:43:41 -07:00
zhouou6 66d7a39ea6 fix(state): self-heal 'file is not a database' write connections + retry transient EIO on journal-mode probe
Salvaged remainder of PR #82280 (state.db hardening rollup):

- Runtime connection corruption: a sibling process replacing/truncating
  the backing file breaks the live write connection — every subsequent
  write raises 'file is not a database' and the gateway wedges
  permanently (messages pile up in memory). Add a bounded one-shot
  reconnect on the write path: close the broken connection, reopen the
  DB file (re-running WAL activation + schema reconciliation), retry
  the failed write once.
- _on_disk_journal_mode: retry transient 'disk i/o error' (virtualized
  block devices) a few times before returning None, so a one-shot EIO
  doesn't push callers onto the fail-closed unknown-mode branch.

The rollup's write-lock machinery, checkpoint-strategy changes, and
repair serialization are intentionally NOT included — superseded by
PRs #84277 and #69609, or wrong-direction per the POSIX
lock-cancellation findings (#71724 lineage).
2026-08-12 19:43:41 -07:00
Aldo 9cf4fe5513 fix(state): bound WAL growth and checkpoint after VACUUM
`sessions optimize` could consume several GB of disk instead of freeing
any, filling the host to 100% on exactly the large databases it exists to
shrink.

Two causes, both in the WAL lifecycle:

1. No `journal_size_limit`. SQLite defaults to -1 (unlimited), so after a
   checkpoint the WAL is reused in place and never truncated —
   `state.db-wal` permanently keeps the high-water mark of the largest
   transaction ever run. `hermes_cli/kanban_db.py` already bounds its WAL
   with `wal_autocheckpoint=100`; the session store, by far the larger
   database, had no equivalent.

2. `vacuum()` checkpoints BEFORE `VACUUM` but not after. VACUUM rewrites
   every page through the WAL, so the pre-checkpoint does nothing about
   the slack VACUUM itself creates.

Measured on a 3.0 GB state.db: `hermes sessions optimize` reported
"3143.9 MB -> 3155.1 MB (reclaimed -11.2 MB)" while leaving a 3.07 GB
state.db-wal behind. Free space fell from 6.9 GB to 772 MB (100% full)
and stayed there. A manual `PRAGMA wal_checkpoint(TRUNCATE)` recovered
the full 3.07 GB, confirming it was slack, not data.

Fix: set `journal_size_limit` (64 MiB) when enabling WAL, and truncate
the WAL again after VACUUM. Both are best-effort and never raise — a
failure costs disk slack and must not stop the DB from opening.

Tests assert the contract (limit is a finite positive bound; VACUUM does
not leave an oversized WAL) rather than pinning the byte count, which is
a tunable. They skip where WAL is unavailable — including hosts where
Hermes falls back to journal_mode=DELETE due to the SQLite 3.50.4
WAL-reset bug.

Verified: 462 passed / 3 skipped in tests/test_hermes_state.py, and
_apply_wal_size_limit flips a real WAL database from -1 to 67108864.
Tested on Linux (aarch64, Python 3.11).
2026-08-12 19:43:35 -07:00
Teknium ea6f4e33c3 chore(contributors): map ernst-bablick email for PR #69609 salvage 2026-08-12 19:43:29 -07:00
Teknium d724bd0376 fix(state): make a refused pre-repair backup a hard stop (#69603)
The Aug 2026 incident in #69603 documented a fail-open: when the
pre-repair backup was refused (another same-process handle open),
repair_state_db_schema() recorded backup_path=None and proceeded —
leaving the writable_schema surgery, FTS-schema deletion, REINDEX and
VACUUM strategies reachable against the only remaining copy of the
damaged DB.

_backup_db_file() now returns (path, reason) and the repair path treats
any refused/failed backup as an unconditional hard stop: abort before
the first mutating strategy and surface the reason in report['error'].
Explicit backup=False (CLI --no-backup) is unchanged — that is the
operator opting out, not a silent failure.

Three new tests: refusal hard-stops with source bytes untouched,
OS-level copy failure hard-stops with the reason surfaced, and
backup=False still repairs.
2026-08-12 19:43:29 -07:00
Ernst Bablick 923d86e099 fix(state): serialize state.db schema surgery across processes
`repair_state_db_schema()` performs `PRAGMA writable_schema=ON` +
`sqlite_master` surgery + `VACUUM` on a private connection. The only guard
around it is `_repair_attempt_lock`, a `threading.Lock`, whose docstring
claims it "serialises concurrent web_server / gateway opens" — but a
threading lock covers threads inside one interpreter, not processes.

A normal host runs four independent processes against the same state.db:
the gateway service, the Desktop app's own `hermes serve` backend (it
spawns one per launch, not a thin client), interactive CLI sessions, and
the TUI slash worker. When two of them hit a malformed DB, both entered
the critical section and each ran the full surgery while the other was
mid-rewrite. Observed as a repair/re-corrupt cascade: the DB is repaired,
then re-corrupts minutes later, repeatedly.

Two fixes:

1. Wrap the surgery in a bounded `flock` on `<db>.repair.lock`. `flock` is
   the right primitive — the kernel drops it when the holder dies, so a
   crashed repairer cannot wedge future repairs the way a pidfile would.
   The acquire is bounded (#36644's failure shape) and, unlike the kanban
   init lock, a caller that times out must NOT proceed: here "proceed
   anyway" is exactly the unsafe interleaving. It re-probes instead, and
   reports success if the holder already healed the file.

   Under the lock, the existing `_db_opens_cleanly()` check becomes a
   double-check: a queued process finds the DB healthy and returns
   `already_healthy` rather than re-running surgery on a repaired DB.

2. Bump the schema cookie after direct `sqlite_master` edits. Ordinary DDL
   bumps it for free and every other connection compares it before running
   a prepared statement — that is how they learn to drop a cached schema.
   Editing `sqlite_master` under `writable_schema=ON` does not, so live
   connections in other processes kept writing `messages` rows through
   triggers into `messages_fts*` shadow tables the surgery had just
   deleted. SQLite's writable_schema docs call out incrementing
   `schema_version` as the required companion to such an edit.

Tests: four new cases in tests/test_state_db_malformed_repair.py, all
using real child processes and a real flock. All four fail on main and
pass with this change; the concurrency case asserts exactly one
`malformed-backup-*` file is produced by two simultaneous repairers
(two on main). Full state suite: 558 passed.

Complements #43742, which makes the *in-process* claim loser retry rather
than raise; it explicitly leaves `repair_state_db_schema()` unchanged and
does nothing cross-process. The two are independent and compose.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-12 19:43:29 -07:00
Hermes Agent 12a7a46578 chore: map hermes-agent@nousresearch.com contributor email 2026-08-12 19:43:22 -07:00
Hermes Agent 4354a07c34 fix(kanban): PASSIVE not TRUNCATE for the dispatcher WAL checkpoint
Follow-up to the state.db PASSIVE checkpoint salvage (PR #84277,
#45383/#80255/#44795): the kanban dispatcher's periodic explicit
checkpoint still used TRUNCATE on the shared kanban.db. The dispatch
flock only serializes dispatchers — CLI kanban commands in other
processes write to the same board without it, so the TRUNCATE races
live writers exactly like the state.db close() path did.

Switch it to PASSIVE and bound the -wal file with
journal_size_limit=8MiB set at connection init (SQLite trims the file
on the writer's natural post-checkpoint reset), since PASSIVE never
truncates.

tests/hermes_cli/test_kanban_db_repair.py updated to assert PASSIVE
and reject TRUNCATE. Remaining TRUNCATE call sites are test fixtures
operating on private temp DBs (sole opener), which is the legitimate
use.
2026-08-12 19:43:22 -07:00
lkz-de ba80f3b86d fix(state): PASSIVE not TRUNCATE for all state.db checkpoints (#45383)
SessionDB.close() ran `PRAGMA wal_checkpoint(TRUNCATE)`. Every cron
run_agent opens and closes its own transient SessionDB, so on a busy
fleet this fired a full WAL reset many times an hour, racing the
gateway's long-lived writer on a large WAL database and tearing hot
B-tree pages -- structurally the same corruption this module's own
periodic checkpoint was already switched to PASSIVE to avoid (#45383).
Only close() and two manual-maintenance paths still used TRUNCATE.

Route every checkpoint on the shared state.db through PASSIVE:
  - close()                    (hermes_state.py)
  - pre-VACUUM in vacuum()     (hermes_state.py)
  - post-optimize-storage      (hermes_state_search.py)

PASSIVE never resets/truncates the WAL and never takes the exclusive
checkpoint lock, so it cannot lose a transient closer's race with the
live writer. The WAL is instead bounded by `journal_size_limit` and the
writer's natural post-checkpoint reset. TRUNCATE belongs only on a
sole-opener/quiescent connection (e.g. offline maintenance); this change
does not try to detect that -- PASSIVE is the safe default.

Diagnosed as the root cause of three state.db B-tree corruptions in
2026-08: damage localized to the hottest-written pages (gateway_routing
and the sessions indexes), with whole zero-filled pages still live and
off the freelist -- the checkpoint/reset-race signature, not disk or
application SQL.

Tests: tests/test_wal_checkpoint_strategy.py now asserts PASSIVE at
close(), before vacuum(), and after optimize_fts_storage() VACUUM;
tests/test_hermes_state.py asserts close() likewise. Focused run:
226 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-12 19:43:22 -07:00
Bartok9 aec7fb3ce6 test(relay): pin isolated plugin managers as discovered
Hook queries now lazy-discover plugins (delivery parity, #64178). The
relay direct-runtime tests build a bare PluginManager to prove zero
plugins are involved; mark it discovered so the parity path doesn't
populate it from the real plugin tree mid-test.
2026-08-12 19:40:59 -07:00
Bartok9 8be9c76f8c fix(plugins): hook delivery parity + symmetric force-reload (#64178)
Salvaged from PR #64188 (@Bartok9), re-reviewed against the #64229
ownership ledger (landed in #84923).

Delivery parity (survived):
- Module-level invoke_hook/invoke_middleware/has_hook/has_middleware
  lazily run plugin discovery via _delivery_manager(), so surfaces that
  never import model_tools (dashboards, TUI slash workers, query mode,
  cron, gateway platform events) deliver plugin callbacks instead of
  silently dropping them (#50776, #67597, #67890, #50937).
- _delivery_manager() joins any in-flight background discovery first and
  tolerates test doubles that monkeypatch get_plugin_manager().

Symmetric force-reload (survived):
- agent/shell_hooks.py gains re_register_config_hooks(); the force branch
  of discover_and_load() calls it after a successful sweep, restoring
  config.yaml shell hooks that the ledger-driven unload wiped but cannot
  restore (they are config-owned, not plugin-owned) (#60036).
- unload(plugin=None) now sweeps pre-ledger _plugin_tool_names entries
  out of the process-global tools.registry, mirroring the platform-name
  sweep that already existed, so zombie tools cannot survive a force
  reload in long-lived pre-ledger processes (#60050).

Superseded by the ownership ledger (dropped from #64188):
- _unload_global_plugin_registrations() bulk tool/platform teardown —
  the ledger's reverse-order handle disposal with previous-entry
  restoration covers it more precisely.
- tools.registry/platform_registry displaced-entry LIFO restore stacks —
  the ledger's restore_registration() identity-checked previous-entry
  restoration made them redundant.
- Discovery serialization lock + double-checked singleton — main already
  has _discovery_lock on every discover/unload path and a keyed,
  lock-guarded per-home manager cache (#24714 concern is covered).

Fixes tracked under #64178 (#50776, #60036, #60050, #24714, #67798,
#50937, #67597, #67890, #31480 — #31480 already handled on main by
_parse_hooks_block warn+suggest).
2026-08-12 19:40:59 -07:00
Victor Kyriazakos eac1e25127 fix(observability): parent marks to the live turn scope, not the session
Scope events export when their OWNING scope closes. Turn scopes close
every turn; session scopes close only at session end. Marks were attached
to the session handle, so a long-lived conversation — a Slack thread open
all day, the normal enterprise case — emitted no approval or turn marks
for hours, and none at all if the process died first. Audit dashboards
showed an empty approval table while approvals were demonstrably firing;
the operator had to end the session to see anything.

Attach marks to the live turn handle when one exists for the mark's
session (active_turn already validates live/same-profile/same-session/
unreleased), falling back to the session handle otherwise — correct for
session-level events like session.end and for marks emitted outside a
turn. Parentage semantics are unchanged: the turn is a child of the
session, so the session tree is identical, only export cadence changes
from per-session to per-turn.
2026-08-12 19:20:03 -07:00
Victor Kyriazakos 15959d8259 fix(observability): forward Hermes session id on approval hooks
Approval marks were emitted under a synthetic 'default' relay session:
the hook payload carried only turn_id/tool_call_id, so the observability
plugin's _session_id() fell back to 'default', parenting approval marks
to a session scope that never closes — and close-time exporters never
shipped them. The audit board's approval tables stayed empty while
approvals were demonstrably firing (staging 2026-08-10).

Bind session_id in set_current_observability_context at both dispatch
sites (model_tools tool dispatch, plugins pre-tool-call approval gate)
and forward it on every approval hook. Explicit session_id in a hook
payload still wins; unbound contexts omit it (legacy behavior).
2026-08-12 19:20:03 -07:00
Victor Kyriazakos 24be384bb8 fix(relay): bound the interpreter-shutdown fallback lane; unwedge test fakes at teardown
CI caught the file hanging AFTER '6 passed in 4.32s' until the runner's
300s SIGKILL. Two defects, same class the PR fixes:

1. The executor-refused (interpreter shutdown) fallback ran the native
   call UNBOUNDED on the calling thread — a wedged pipeline would block
   process exit forever. Now runs on a bounded daemon exit-thread with
   the same timeout/abandon semantics as the executor lane.
2. The wedge tests left daemon workers parked on Event.wait() and live
   sessions registered on the atexit shutdown hook; exit re-ran the
   wedged pops (bounded, 10s each) and the per-file runner timed out.
   Autouse teardown now releases every wedge and drains each runtime.

Canonical runner: 4.4s (was 300s file-timeout kill). Bare pytest was a
false green for this class — it exits before atexit replay cost shows.
2026-08-12 19:19:54 -07:00
Victor Kyriazakos d607f0cafb fix(relay): bound native scope lifecycle operations so a wedged pipeline cannot block the agent
The NeMo Relay native binding's scope.pop/push are synchronous and
unbounded ('returns after the scope is closed successfully'). When the
native pipeline cannot make progress, the session coordinator's turn and
session finalization block forever inside run_conversation: delegated
children finish their turns but never return, and delegation batches die
on the stall watchdog. Proven live 2026-08-10 on the staging fleet — a
falsification probe (plugin disabled, identical config) completed the
same delegation batch that wedged with the plugin active.

Bound every scope lifecycle operation that gates turn/session completion
(session push, turn push, turn pop, logical-LLM pops, session pop,
subscriber flush) by running the native call on a shared
DaemonThreadPoolExecutor and honoring a 10s result timeout. On breach a
TimeoutError propagates into each call site's existing exception
handling — warn, retain the unclosed-prefix diagnostics, continue — so
the worst case is one lost span, never a blocked agent. timeout=None
preserves byte-identical synchronous behavior for all other callers, and
interpreter-shutdown paths fall back to the synchronous call so the
atexit flush still exports.

Observability must never block the product.
2026-08-12 19:19:54 -07:00
Teknium 11310068c6 feat(plugins): pre_command observer hook + capability-gated ctx.call_mcp (#64204)
Part A — pre_command observer hook (observer-first per #64182 ground rule 3):
- New VALID_HOOKS event `pre_command`: fires when a recognized slash command
  is about to be dispatched, BEFORE the handler runs, on both surfaces:
  - CLI: cli.py process_command (right after alias resolution)
  - Gateway: gateway/run.py _handle_message cold-path canonical dispatch
- Payload: surface ('cli'|'gateway'), command (canonical), alias_used,
  args_raw, session_key, platform. Return values IGNORED in v1; a plugin
  returning a directive-shaped dict gets a debug log so future
  block/rewrite adopters are discoverable (#64231 taxonomy).
- Deliberately NOT fired on the gateway running-agent intercept path
  (/stop, /approve, busy_policy dispatch during an active run): those are
  control-plane escape hatches on an in-flight run and must stay outside
  plugin observation/veto reach.
- fire_pre_command_hook() helper never raises, so broken plugin infra can
  never break command dispatch.

Part B — ctx.call_mcp (capability-gated, default-off, ground rule 4):
- PluginContext.call_mcp(server, tool, arguments, timeout=30): synchronous,
  callable from plugin hooks/tools, routes through the EXISTING native MCP
  client machinery (tools.mcp_tool._make_tool_handler: background loop,
  trust-tier gates, circuit breaker, reconnect) — never a parallel client.
- Gate: plugins.entries.<id>.mcp_allowlist (list of server names).
  Absent key / unreadable config / non-list value => default-deny.
  Unlisted server raises PermissionError naming the exact config key.
  TODO seam left for the #64228 declared-capability model.
- Bounded: timeout clamped to 1-600s and forwarded to the MCP loop call;
  results capped at 64KB with truncation marker; stable
  {ok, result|error, structuredContent?, truncated?} envelope.

Tests (transport mocked, no live MCP servers):
- tests/hermes_cli/test_pre_command_hook.py: both surfaces fire, canonical
  alias reporting (/exit->quit, /q->queue), hook-before-handler ordering,
  control-plane exclusion, hook failure non-fatal, observer-only directive
  handling.
- tests/hermes_cli/test_plugin_call_mcp.py: default-deny (absent entry,
  unreadable config, non-list, '*'), allowlist enforced per-server,
  denied calls never touch transport, timeout forwarding/clamping,
  result truncation, error/structuredContent envelopes.

Docs: hooks.md shipped-catalog row for pre_command; plugins.md
"Calling MCP servers from plugins" section with the security note.

Closes #64204
2026-08-12 19:16:59 -07:00
Teknium e9bf8a7844 test(plugins): assert per-hook stream ordering, not cross-thread interleaving
The streaming-hook dispatcher runs one worker per callback; delivery
order is FIFO per hook, never across hooks. Two tests pinned a global
start->delta->delta->end interleaving that three concurrent workers
don't guarantee, flaking CI twice within an hour of #84924 landing.
Also wait for the full event count before shutdown so late deltas
aren't dropped mid-assert.
2026-08-12 19:15:02 -07:00
Teknium 4be8bd0816 ci: retrigger — previous pull_request run failed with zero jobs (transient workflow materialization) 2026-08-12 19:13:32 -07:00
Teknium b9542f8e1f fix(plugins): preserve force-path platform sweep + scoped plugin-source listing after rebase
Rebase over the capability-model merge dropped two behaviors the tests
pin: (1) unload_all must still unregister every _plugin_platform_names
entry from the global platform registry (pre-ledger state has no
handles); (2) list_plugin_sources() must see profile-scoped
registrations — scoped entries are plugin-registered by definition.
2026-08-12 19:13:32 -07:00
Teknium 2219747990 feat(plugins): widen ownership ledger to all registration surfaces
Extends the salvaged #64229 ledger (PR #76490) to cover the registries
added on main since the PR was cut, and lands the remaining Phase 0
lifecycle pieces:

- register_system_prompt_section and register_approval_transport now
  record ownership handles, so unload/force-reload removes plugin
  system prompt sections and approval transports too
- ctx.on_unload(callback): plugin cleanup callbacks run through the
  reverse-order ledger walk, exception-isolated
- ctx.spawn_task(coro): supervised background asyncio tasks tracked in
  the ledger and cancelled on unload
- document the #65593 multi-profile constraint on the ledger (keyed per
  manager/(hermes_home, plugin_id); identity-conditional restores) with
  a TODO(#64178) for full profile keying of remaining global slots

Part of #64229; prerequisite for #64178.
2026-08-12 19:13:32 -07:00
doncazper 85020f2238 fix(plugins): isolate ownership by profile 2026-08-12 19:13:32 -07:00
terry197913 4e1b2e436c fix: scope plugin manager by resolved hermes home (keyed cache)
fix: remove .codegraph artifacts from commit
2026-08-12 19:13:32 -07:00
doncazper 22af80bcfd feat(plugins): add ownership ledger unload lifecycle 2026-08-12 19:13:32 -07:00
brooklyn! 03d2c0e144 Merge pull request #84966 from NousResearch/bb/todo-truncate
fix(desktop): stop status-stack todos truncating way before the row edge
2026-08-12 21:12:41 -05:00