_recall_config() previously read all settings (recall_limit, score_threshold,
recall_resources, etc.) exclusively from environment variables. This forced
users to store behavioural configuration in .env, violating the Hermes
convention that .env is for secrets only.
The infrastructure to load config.yaml -> memory.openviking was already in
place via _load_hermes_openviking_config(), but _recall_config() never
called it.
Fix: call _load_hermes_openviking_config() and pass its values as the
default parameter to _env_int/_env_float/_env_bool. Env vars still override
config.yaml values, preserving backward compatibility.
Closes#62540
(cherry picked from commit 6aadf1256835745e0302aa3d3b5ae0660b368637)
OpenViking is_available() only consulted env vars and use_ovcli_config, so an
endpoint saved to config.yaml (e.g. by the Dashboard) reported needs_config;
_resolve_connection_settings() likewise never folded config.yaml's non-secret
fields into its chain. RetainDB initialize() read base_url/project from the
environment only, ignoring the values the Dashboard writes to config.yaml.
Both now resolve non-secret fields as env -> (ovcli ->) config.yaml -> default;
secrets still come from the environment. Adds regression tests for both.
Fixes#68209
(cherry picked from commit dca57915b97b5705b30927a062e1d0f2f23d3841)
Review feedback: the previous test called _mark_session_committed
directly, so it verified the guard's behavior but not the wiring that
sets it — a future break in the commit_memory_session -> same-id
compression-boundary path would not be caught.
Add a lifecycle regression that drives the real sequence: on_session_end
commits through the actual path, on_session_switch(same id,
reason="compression") crosses the boundary, sync_turn records a genuinely
new turn, and a second on_session_end must produce a second commit POST.
Without the fix it fails showing exactly one commit call, which is the
reported data loss: every turn after the first compression is dropped.
The rotation and /undo tests stay as scope guards.
(cherry picked from commit 0ca5a330630a30b105cbbc32e8a23f2c5ffe0eab)
`_committed_session_ids` is a permanent per-sid latch, and
`_session_needs_commit` checks it before the turn counter by design — a
racing sync_turn can re-increment `_turn_count` after commit+reset, so
the guard must win to stop a double-commit.
That is correct for a session being left behind. It is wrong for one
that keeps its id. `compress_context()` commits before rewriting the
transcript in both modes, and with `compression.in_place: true` (the
default) `on_session_switch` receives the same id and does not rotate.
The latch then rejects every later commit for a still-live session — the
next compression, /new, normal session end, startup recovery — so every
post-compression turn is silently never extracted.
Rotation mode is unaffected because a fresh child id is minted and
starts clean, which is what confirms the latch's intent was only ever to
dedupe the departing id.
Clear the latch when compression completes without rotation. Turns
arriving after that point are genuinely new, and this is a defined
moment rather than a race. The rotation path is untouched, so the old
id stays latched and its _finalize_session_async still dedupes against
the compression commit.
Fixes#74695
(cherry picked from commit d1e5c3dc33ef0d43d021662674e1a7cd5e43eecd)
The provider used to disable OpenViking permanently when the server was
unreachable. That was fixed: `_ensure_client()` now reconnects lazily,
with a 30s cooldown gate in `_ensure_client_locked`.
Only one of the seven user-facing warnings was updated to match. The
other six still told the user memory was "disabled for this Hermes run",
which is no longer true — every one of those paths is retried on the next
access. A user who reads the old message has no reason to retry, which is
very likely how #5721 ("never recovers") came to be filed against
behaviour that already recovers.
All six sites were traced to confirm none is terminal for the run: the
`initialize()`-time and waiter-thread failures never arm `_failed_refresh`
(only line 2439 does), so they retry on the very next access with no
cooldown at all.
The replacement wording deliberately omits the "(after cooldown)"
parenthetical used at the already-correct site — that detail is only
accurate where `_failed_refresh` was just armed. The neutral phrasing is
true at all six.
Also promotes two clause separators to periods to avoid "…; …disabled;"
collisions.
(cherry picked from commit 8346403a4b97af503d26b0f7905ff513828d821e)
`_start_local_openviking_server()` spawned `openviking-server`
unconditionally. Both callers — `initialize()` and the runtime
unreachable handler — reach it from a health probe, and that probe can
time out client-side while the server is up and serving. The spawned
process then loses the data-directory lock and exits immediately with
`DataDirectoryLocked`; because the probe keeps timing out, the cycle
repeats every cooldown window (~5 min observed).
The existing 30s `_failed_refresh` cooldown paces the loop but cannot
stop it, since it expires while the underlying condition persists.
Probe the target host:port before spawning and treat an occupied port as
already-started. This guards both call sites at their single convergence
point. The probe deliberately tests only that a listener owns the port —
enough to know a second server would lose the lock — and says nothing
about that listener's health.
The parse/probe now precedes the PATH lookup, so a reachable server is
reported as running even when `openviking-server` is not on PATH.
Fixes#74846
(cherry picked from commit b49427d85fd6628eb4a7fe099e5c390c5c4cc935)
## Summary
- Normalize OpenViking endpoints through `is_always_blocked_url` and fall back to the default local endpoint when poisoned.
- Keep intentional loopback / LAN self-host working.
- Add focused unit tests.
## Salvage / credit
Memory-provider endpoint floor sibling of RetainDB/Supermemory always-blocked hardening (avoids over-broad #4984-style private-IP bans).
(cherry picked from commit 8fa607d0aedb8c5fca398d7f112b1b25ade54fa2)
Simplify-pass fold: to_drop names come from the literal update_names\nallowlist via IN binding, so the [A-Za-z0-9_]+ fullmatch could never\nfail — and if it somehow did, its `continue` would miscount (the\nskipped trigger stayed in len(to_drop)/the log while CREATE TRIGGER\nIF NOT EXISTS silently kept the broad variant). Delete the guard and\nits function-local re import; keep the invariant as a comment.
_ensure_fts_cjk_schema never raises on OperationalError; post-condition
after dropping messages_fts_cjk_update now requires a narrowed UPDATE
trigger or durable fts_cjk_stale + unavailable. Covers the production
soft-fail path the raise-only handler missed.
Retarget #73639 onto the SessionDB mixin split (hermes_state_common /
hermes_state_schema). Fresh installs create UPDATE OF content/tool_*
triggers; existing broad AFTER UPDATE triggers are inspected and
replaced under schema init without an FTS rebuild (WHEN clauses already
guarded content correctness; OF skips non-content status writes that
saturated disk I/O on large state.db).
Tests: tests/test_fts_update_of_narrowing.py (4)
CI-caught: cron-jobs-section had an extra blank line between sorted imports; use-message-stream's visibility-flush effect assigns flushHandleRef.current=null inside a useEffect (legitimate timer-clear, not an atom mirror) — eslint-disable-next-line per the rule's documented convention.
Two findings from the simplify pass on the final trio diff:
- status-pulse: one pause controller + one aligned period timer shared by all StatusPulse instances (ref-counted), instead of N x (document/window/bridge listeners + unsynchronized 5s wakes) — a sidebar can show dozens of pulsing dots. Pause still cancels in-flight animations so the compositor sleeps immediately.
- cron-jobs-section: the runs-peek effect created its interval even while the pane was hidden (callback no-oped but the timer still woke the renderer every 8s/60s per expanded job). Early-return when hidden — visibility is already in the dep array, so becoming visible restarts load + timer.
Partial pick of the surviving renderer hunks from #75395 (perf commit
6502e441d plus fixup 3fbbc9c1d): gate the 500ms subagent now-ticker and
the cron sidebar 1s ticker/run-poll on usePaneVisible, and skip the
legacy floating-pet poll while the document is hidden. Dropped hunks
(electron/main.ts, vitest.setup.ts/config) intentionally excluded.
E2E guard for the salvaged PR #71755: database.cache_size/mmap_size/
temp_store from config.yaml must reach the writer connection, the
read-only cross-profile attach, and the WAL per-thread reader — and a
default install (no database: keys) must keep byte-identical SQLite
defaults on every connection type. Also covers integer-coercion
rejection of garbage values for the three new keys.
cache_size uses -16000 (not the doc example -2000) because -2000 is
SQLite's compiled-in default and would not discriminate a regression.
Addresses review from @teknium1 on PR #71755:
- Extended apply_database_pragmas() to handle cache_size, mmap_size,
and temp_store from config.yaml (alongside existing wal_autocheckpoint
and journal_size_limit). No hardcoded defaults — all values are
opt-in via config.yaml, avoiding policy conflicts with other PRs.
- Applied to ALL connection types: writer (_connect_and_init),
read_only cross-profile attach, and WAL per-thread readers
(_get_read_conn). Previously PRAGMAs only ran on the writer path.
- Removed inline PRAGMAs from _connect_and_init — single source of
truth in apply_database_pragmas().
- Documented config keys with examples in function docstring.
Cross-PR interaction fix: #77714 (salvage of #71775) changed
_available_entries to return (available, pending_refresh) while #77631
(salvage of #67642) added next_available_at() which still truthiness-
tests the bare return. A non-empty tuple is always truthy — even
([], []) — so the reset-aware gate silently returned None ('no wait
info') for every exhausted pool, disabling the feature #77631 shipped.
Unpack the tuple and test the available list.
Also adapts the lock-probe test for the RLock introduced by #77714
(same-thread non-blocking acquire always succeeds on an RLock; probe
from a helper thread instead).
Simplify-pass finding: _listing_group_label already falls back to 'other' for empty source names, and _classify_source guarantees source_name=='' only when source=='other' — both legs were dead by construction. Aligns the summary path's grouping with the listing path.
```
Error: optimization failed: no such table: messages_fts_trigram
No data was lost. Re-run to resume.
```
on any install where the trigram FTS index is legitimately absent. The failure is
deterministic — re-running can never make progress, because the crash happens at the same
point every time — so the database is permanently stuck on the legacy high-footprint FTS
layout with no supported way forward.
Observed on a 5.4 GB production `state.db`. After the fix the same database optimized
successfully and shrank to 3.3 GB.
The trigram index is absent whenever the runtime cannot maintain it. On a SQLite build
without the `trigram` tokenizer, `_ensure_fts_schema()` returns `False`, so `__init__`
leaves `self._trigram_available = False` and no `messages_fts_trigram` table on disk. This
is a **supported degraded runtime**, not damage — CJK/substring search falls back to
`LIKE` and everything else works normally. `_is_fts5_unavailable_error()` and
`_warn_trigram_unavailable()` exist specifically to make this path graceful.
Two code paths write the boundary sweep for the deferred FTS rebuild, and only one of them
respects that flag:
| Function | Trigram `INSERT` guarded? |
|---|---|
| `fts_rebuild_step()` | ✅ `if include_trigram:` where `include_trigram = self._trigram_available` |
| `_fts_rebuild_finish()` | ❌ unconditional |
`_fts_rebuild_finish()` runs the boundary sweep at the *end* of the backfill. Its
unguarded `INSERT INTO messages_fts_trigram …` raises `OperationalError`, which propagates
out of `optimize_fts_storage()` and aborts the entire optimization — *after* the backfill
has already completed. Hence the characteristic output showing 100% progress immediately
before the error:
```
Rebuilding index: 100% (909,671/909,671)
Error: optimization failed: no such table: messages_fts_trigram
```
There is a second, quieter consequence. The teardown phase that reclaims the demoted
`fts_v22_trash_*` shadow tables runs *after* the backfill phase in
`optimize_fts_storage()`. Because the crash happens before teardown is ever reached, those
tables are never emptied or dropped — so the space the migration was supposed to reclaim
stays allocated indefinitely, and the leftover trash tables look (misleadingly) like
evidence of a half-finished migration.
Build a populated v23 database, set the deferred-rebuild markers, then reopen it on a
runtime where `_ensure_fts_schema('messages_fts_trigram', …)` returns `False` (exactly
what a SQLite build without the trigram tokenizer produces) and call
`optimize_fts_storage()`:
```
[precondition] trigram absent, _trigram_available=False, rebuild pending ✓
RED ✗ optimize_fts_storage raised OperationalError: no such table: messages_fts_trigram
```
With this patch applied, unchanged harness:
```
optimize_fts_storage returned {'ok': True, 'vacuumed': None}
GREEN ✓ optimize ok; markers cleared; base FTS 'zebra' -> 200 hits
```
Full harness and transcripts in `TEST-EVIDENCE.md`.
Gate the sweep on `self._trigram_available`, exactly as `fts_rebuild_step()` already does:
```python
include_trigram = self._trigram_available
def _do(conn):
...
if include_trigram:
conn.execute("INSERT INTO messages_fts_trigram(...) ...")
```
The base `messages_fts` sweep and the marker cleanup are untouched, so the rebuild still
finalizes correctly and the index remains complete for every row it is responsible for.
The fix does not disable or weaken search to dodge the error — the regression tests assert
that base FTS still returns results afterwards.
`TestFtsRebuildFinishWithoutTrigram` in `tests/test_hermes_state.py`:
- `test_rebuild_finish_skips_trigram_when_unavailable` — drives `_fts_rebuild_finish()`
directly on a trigram-less runtime; asserts it completes, clears both rebuild markers,
and leaves base FTS searchable.
- `test_optimize_fts_storage_succeeds_without_trigram` — end-to-end through the public
`optimize_fts_storage()` entry point; asserts `ok=True`, markers cleared, search intact.
Both use the existing `_NoTrigramConnection` helper already in the file. Both fail on
`main` with `no such table: messages_fts_trigram` and pass with this patch.
`tests/test_hermes_state.py` passes in full (463 tests → 465 with these two). `ruff` clean.
This PR is the crash only.
A companion PR narrows `_db_opens_cleanly()` so that
`hermes sessions repair --check-only` stops reporting a write-broken FTS schema as
healthy — the gap that makes this class of problem hard to diagnose in the first place.
The two are independent and can land in either order.
Review folds on the #71775 salvage (dossier findings 1+2):
- self._lock becomes an RLock and the mutation primitives
(_replace_entry, _persist) are now self-locking, so the deferred
single-use-token refresh path — which deliberately runs its
cross-process flock + OAuth network I/O OUTSIDE the pool lock —
still serializes its pool mutations against concurrent
select()/rotation. In-lock callers re-acquire reentrantly.
- Dropped _refresh_pending_entries' redundant second _replace_entry:
_refresh_entry already merges the refreshed entry internally.
Adds tests/agent/test_credential_pool_deferred_refresh.py pinning both
invariants: select() must NOT hold the lock during the refresh window
(the PR's whole point), and the post-refresh mutations MUST contend on
the lock (blocking-thread probe).
select() and acquire_lease() held self._lock during the entire
_available_entries() loop, which for openai-codex and xai-oauth providers
includes a cross-process file lock (_auth_store_lock) plus OAuth token
refresh HTTP POST. The lock timeout can exceed 20 seconds, blocking all
credential pool consumers across every gateway thread and subagent.
Collect single-use-token refresh entries under the lock, then execute the
refreshes outside it. On success the refreshed entry is merged back into
the pool and re-selected. Non-single-use providers (anthropic, nous)
continue refreshing inside the lock since their refresh is a simple HTTP
POST with no cross-process coordination.
Review fold on the #67642 salvage: next_available_at() called
_available_entries() — which prunes DEAD entries, syncs tokens, and
persists — and iterated self._entries with no lock, racing concurrent
select()/rotation exactly as has_available()'s comment warns. Wrap the
method body in self._lock and pin it with a non-blocking-acquire probe
test.
restore_primary_runtime retries the primary every turn once the 60s
transient cooldown clears. For subscription-window limits (Claude
Pro/Max 5h windows, Codex weekly caps) the reset is hours or days away,
so every retry is a guaranteed failure costing two provider switches
and two prompt-cache invalidations per turn.
Add CredentialPool.next_available_at() (earliest reset across exhausted
entries; None when available now or no reset info) and gate the restore
on it: skip while the primary's pool says nobody can serve, restore on
the first turn after the reset elapses. Fail-open: any gate error or
missing reset info falls through to the existing per-turn retry, so
recovery can never be later than today. Cross-provider fallbacks
consult the PRIMARY's pool (not the attached fallback pool), reusing
the loaded pool for the existing rebind to keep auth reads at one per
restore.
Every hashed bundle chunk under /assets/ was served with no caching
directives, so each dashboard load re-fetched (or at best revalidated)
every JS/CSS chunk. Those filenames carry a Vite content hash — the
bytes behind a given URL can never change; a rebuild mints new
filenames referenced by a freshly served index.html.
Mark them Cache-Control: public, max-age=31536000, immutable:
- the /assets StaticFiles mount, via a subclass that stamps the header
on 200s only (404s stay uncached — a rebuild can create the file),
- serve_css, preserving its X-Forwarded-Prefix url() rewrites for
/fonts/, /fonts-terminal/, /ds-assets/, /assets/.
index.html keeps no-store, no-cache, must-revalidate — it is the
mutable entry point that binds users to the current hashes.
The original PR also added hand-rolled per-request gzip compression of
asset responses; that part is deliberately dropped. This server is a
localhost-default dashboard backend: compressing every response on the
CPU to save loopback bandwidth is a pessimization, and callers that
front it with a real proxy already get compression there.
Salvaged from PR #28543 (idea by @sea-monsters; gzip groups dropped as
described above).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The sessionStorage seed set loading=false whenever any cache existed, which
defeats App.tsx's load-bearing pluginsLoading gate: with a cached manifest
that declares tab.override === "/chat", the persistent ChatPage host must
NOT mount before plugins resolve, or it spawns a PTY and gets yanked when
the override plugin takes over the route.
Seed loading=false from the cache only when no cached manifest overrides
/chat (canSeedLoadedFromCache); manifests are still seeded either way so
plugin routes register synchronously on refresh. Adds focused tests for
the gate, including the /chat-override case.
- Plugin manifests are now cached in sessionStorage on fetch.
- On refresh, plugin routes are registered synchronously from cache, preventing unwanted redirects to /sessions.
- Removes the !pluginsLoading guard from the catch-all route in App.tsx, as plugin routes are now always available on first render.
- Background fetch always updates the cache and routes, so new/removed plugins are reflected after reload.
- Resolves the race condition where plugin pages would redirect to /sessions on hard refresh.
le=100 would 422 real desktop callers: sessions-settings fetches
archived at limit=200, the command palette lists at 200, and the
electron remote-merge over-fetches limit+offset (exceeds 100 at
offset>=81, and its .catch(()=>null) silently drops remote sessions).
Clamp must sit above real client maxima. New test pins limit=200 w/
offset.
Re-derivation of aydnOktay's twin clamp PRs onto current main (the
session-list endpoints moved into web_routers/; the analytics endpoints
gained asyncio.to_thread wrappers since the originals):
- limit le=100 on /api/sessions, /api/sessions/search and the
/api/profiles/sessions fan-out (one unbounded request could drag every
session row + correlated-subquery preview work out of SQLite, times
every profile's state.db on the fan-out).
- days ge=1 le=365 on /api/analytics/usage + /api/analytics/models
(huge or non-positive values force full-history InsightsEngine work or
inverted windows; the UI only offers 7/30/90 presets).
FastAPI Query bounds reject at the validation layer (422). 8 new tests;
both clamp classes mutation-checked (clamp removed -> its tests fail).
The 2-line alias had zero production consumers (web_server calls
get_usage_breakdown directly). Tests rewired onto the real API; the
contracts they pin are unchanged. Stale test docstring fixed.
Follow-up to #77652: each runFlush registered a fresh requestAnimationFrame and never cancelled it. Chromium parks rAF callbacks for hidden renderers, so a long hidden stream at the 33ms floor accumulates thousands of parked closures that all fire in the first frame on refocus (all but one no-oping through the stale-frame guard). Track the pending handle, cancel it before requesting a new one (only the newest flush's measurement matters), and cancel on unmount.
On dashboard-only sessions nothing else executes check_fn warmers (they
live only in the tool-schema build), so the hub's read-only cache lookup
would report auth_required=False forever. On a cache miss, schedule a
deduplicated daemon-thread probe off the request path; the short hub TTL
surfaces the verdict on the next fetch.