Some bundled CPython runtime builds strip stdlib ThreadPoolExecutor's
copy_context() propagation, so work submitted to the daemon pool runs in a
bare context. Under the multiplexed gateway this dropped the profile
secret scope in pool workers: the context-compression timeout fence
resolved auxiliary provider keys (SURPLUS_API_KEY) with
UnscopedSecretError, silently degrading LLM compression to lossy
deterministic summaries and driving re-read loops in affected sessions.
Restore stdlib semantics in submit() by snapshotting the caller's context
and running the callable inside it (a no-op re-application on runtimes
that already propagate). Mirrors the gateway's
_run_in_executor_with_context pattern.
Tests: daemon pool worker sees caller contextvars; scoped get_secret works
in a daemon-pool worker under multiplex while scoped misses still fail
closed (no env leak).
- generate() now passes kwargs.get("model") into _resolve_model(), so the
user's hermes tools pick (forwarded by the dispatcher as top-level
image_gen.model) is honored instead of silently dropped (#55893 class;
matches xai/krea/openrouter).
- Setup schema badge "internal" -> "paid" to match every other paid
image backend in the hermes tools picker.
- Tests: caller-model precedence, unknown caller model falls through,
model kwarg reaches the API payload, badge contract.
Adds a bundled image-generation backend for the Meta Model API
(https://api.meta.ai/v1), which is OpenAI-compatible. Exposes the
muse-image-1.0 model via the standard image_generate tool. This is the
image-gen companion to the already-bundled meta-ai chat provider
(plugins/model-providers/meta-ai, PR #88565).
- plugins/image_gen/meta-ai/ — provider registered as `meta-ai`, matching
the chat provider's id. Reuses the openai SDK pointed at Meta's base URL.
- Auth mirrors the chat provider: MODEL_API_KEY (Meta's documented var),
with META_API_KEY / META_MODEL_API_KEY aliases and a META_BASE_URL
override.
- Text-to-image only for now (capabilities gated); base64 (WebP) and URL
responses both handled and saved under $HERMES_HOME/cache/images/.
- Auto-loads as `kind: backend` and appears in `hermes tools` with no
central list edits, matching the other bundled providers.
- tests/plugins/image_gen/test_meta_ai_provider.py — 27 tests (metadata,
auth-alias resolution, base-url override, model resolution, generate
paths incl. b64 save, aspect mapping, URL caching, error handling).
- docs: image-generation feature page + provider-plugin built-in list.
Desktop's cold resume (defer_history + omit_messages, transcript paged over
REST) only ever holds the live tip segment in memory, but session.resume
bounded it against the FULL compression lineage (sessions.max_resume_messages,
default 20000). A Bot Chat with 85 compaction segments / ~29k lineage rows
behind a ~700-row tip was refused at 20001, sent zero model prompts, and sat on
"Waking up default…" forever — the healthiest possible session shape, rejected
by a guard sized for in-memory materialization.
- hermes_state: one `_resume_lineage_ids` definition shared by the resume
readers (get_resume_conversations, get_ancestor_display_prefix) and the
guard (assert_resume_safe / get_resume_message_count). Guard grows
`tip_only=` and names the scope it counted; the branch-aware lineage the
readers already used is now what the guard counts too (a /branch copy was
being counted against its parent's rows).
- tui_gateway session.resume: deferred, omit_messages and lazy resumes are
bounded by the tip; only the full in-memory lineage resume keeps the
lineage-wide bound. Deferred hydration falls back to tip-only history when
the lineage exceeds the limit instead of loading the rows the guard refused.
- CLI mid-setup tip-only path routes through the same guard instead of
borrowing assert_export_safe.
- docs: sessions.max_resume_messages / max_export_messages documented with the
per-surface scope.
Live repro (real SessionDB fixture, 85 segments / 29,226 lineage rows / 666 tip
rows, real tui_gateway.server.handle_request): before — deferred resume ->
4130; after — ok, hydrated history=666 prefix=0; the non-deferred full resume
still returns 4130 on the same fixture.
When a watchdog (TTFB / stream-idle / stale-call) force-closes a Codex
Responses request, the worker thread can still be draining SSE frames.
`_consume_codex_event_stream` returns `status=terminal_status`, which defaults
to `"completed"`, and its only truncation guard is
`if not saw_terminal and not output`. A mid-stream kill leaves
`saw_terminal=False` but `output`/text non-empty, so the partial text came back
as a `finish_reason=stop` response and got persisted as a finished assistant
turn — a long reply just stops mid-sentence with no error surfaced.
Observed as a long generation dying at `1. Create (6/6)` and never emitting its
end marker, with the truncated text already stored in state.db.
Fix: publish a per-request retirement token so the worker can tell it has been
retired.
- `agent/chat_completion_helpers.py`: `interruptible_api_call` installs
`agent._active_codex_stream_request_token` before handing off to the worker
(codex_responses only) and clears it at all four kill sites plus the worker's
own `finally`. Retirement is cleared BEFORE `_close_request_client_once`,
which can raise — every other call site wraps it in try/except, and a leaked
token would let a later worker mistake itself for the owning attempt. The
request-local `_codex_request_retired` mirror also swallows the transport
error our own force-close causes, so the worker's local error cannot replace
the watchdog's retryable TimeoutError (same split as `_request_cancelled`).
- `agent/codex_runtime.py`: `run_codex_stream` captures the token and raises
`TimeoutError` from `interrupt_check` when it no longer owns the request —
raising rather than breaking, because a break returns the partial `final`.
The four stream callbacks also drop post-retirement frames so an abandoned
attempt cannot stream tokens into the live turn's bubble (the gateway caches
AIAgent instances per session).
`TimeoutError` is not an httpx / ConnectionError / RuntimeError subclass, so it
passes through the four `except` clauses around the consume call untouched.
No token installed (auxiliary callers such as `handle_max_iterations` drive
`_run_codex_stream` directly) means every check passes — behavior unchanged.
Tests: 5 new cases. Retirement raises instead of returning partial output;
post-retirement deltas stop reaching callbacks; the no-token path keeps its
existing terminal-frame tolerance; the watchdog installs and clears the token;
non-codex api_modes install nothing. A `_LazyCreateStream` helper is needed
because `_FakeCreateStream` materializes events in __init__, which would run
the retirement side effect before consumption starts.
The conversation loop has forced stream=True for every turn — subagents
included — since #3120 (always-prefer-streaming for liveness health
checking). Self-hosted OpenAI-compatible backends with broken streaming
tool-call paths (e.g. vLLM --tool-call-parser qwen3_xml + reasoning
parser + MTP) can leak tool-call markup into plain text and return zero
tool_calls, so delegated tasks silently no-op instead of executing.
model.streaming was never a real config key, so users could not opt out.
Seed agent._disable_streaming from model.streaming: false at init; the
loop already routes that flag to the non-streaming path (the same path
used when a provider rejects streaming at runtime). Default stays
streaming-on, preserving #3120's behavior for everyone else. Orthogonal
to display.streaming (token rendering).
Tests: config->flag seeding (patched loader + real config.yaml E2E),
legacy string model section, multi-agent config propagation.
Follow-up to the salvaged #96044 hunk: drop the 'or callable(...)' arm —
callable(MagicMock) is True, which would have flipped stubbed sessions
into the fast-fail race the surrounding comment explicitly routes to the
plain-await path. inspect.iscoroutinefunction alone reproduces the old
isawaitable(call) split exactly (real async def / AsyncMock -> race,
MagicMock -> plain await) without creating the leaked coroutine.
The fast-fail gate probed the stdio child watcher by CALLING it —
inspect.isawaitable(_watch_children()) — creating a fresh coroutine on
every stdio MCP tool call that was never awaited (RuntimeWarning spam +
gc churn). Inspect the function instead of invoking it.
Salvaged (unique hunk only) from PR #96044; the bundled
_stdio_children_dead polarity fix was already on main via #94339.
CI's deep pytest tmp_path pushes state/gateway.loop-tick.<pid>.sock past
the sockaddr_un limit; the producer swallows the OSError into
loop_tick_socket=False and the POSIX arm test fails falsely. Use a
mkdtemp under the temp root, same as test_update_wedged_gateway.py.
asyncio.start_unix_server does not exist on Windows, so the ungated
call raised AttributeError on every gateway start — a warning +
traceback in the logs each time, with the witness permanently absent
(the documented deliberate fail-safe). Gate the server creation inside
the existing os.name == "posix" block, mirroring the stale-socket
sweep above it, and log the non-POSIX skip at debug. POSIX behavior
and the two-witness liveness contract are unchanged.
Closes#96956
Desktop "Read replies aloud" / voice conversation, TUI and CLI /voice tts
now hold a lease on the TTS engine. Acquiring pre-loads the configured
provider (piper/kittentts model into the same LRU slot synthesis reads;
lazily-installed cloud SDKs), so the first spoken reply no longer pays the
model load as dead air. Releasing the last lease across surfaces unloads
resident local models.
- tools/tts_tool.py: warm_tts_provider / release_tts_provider /
acquire_tts_lease / release_tts_lease over a _LOCAL_TTS_MODEL_CACHES
registry; piper/kittentts loaders extracted so warm-up and synthesis
share one resolution path.
- web_server: POST /api/audio/tts-lease (profile-scoped, off-loop,
failures reported in body never as HTTP errors).
- tui_gateway voice.toggle + cli.py /voice tts|on|off wire the lease.
- desktop: lib/tts-lease.ts (dedupe, per-lease serialization, latest
intent wins) driven from useComposerVoice; setTtsLease API client.
- docs: features/tts.md section.
Live (real piper, isolated HERMES_HOME): first synthesis 988ms cold →
92ms after the toggle warmed the engine; release drops the model.
should_use_direct_api_call() contexts (gateway cron turns #62151, delegate_task
children #60203) were short-circuited onto the NON-streaming wire because the
interrupt worker wedges inside their nested thread pools. That dropped every
liveness property streaming provides: edge proxies kill the silent POST
(z.ai HTTP 524 — three retries later the child dies as "max_iterations"), and
the non-stream stale watchdog cannot tell a reasoning model's thinking phase
from a hung provider, so children die at exactly stale_timeout (#100260).
Keep those contexts on interruptible_streaming_api_call. The request now runs
INLINE on the conversation thread (no worker → the deadlock class stays
closed) while the existing poll loop — 30s heartbeat, stale-stream detector,
cross-thread interrupt abort — moves onto a monitor thread that only ever
aborts sockets, never dispatches (same shape as direct_api_call's watchdog
timer). Interactive sessions are unchanged: worker + poll loop as before.
should_use_direct_api_call() itself is untouched; only what it routes to.
Live A/B (real SSE server, real AIAgent.run_conversation):
before: subagent/cron wire stream=None, request on conversation thread
after: subagent/cron wire stream=True, request on conversation thread
cli unchanged (stream=True, spawned worker)
inline stale detector kills a one-chunk-then-silence stream at budget;
AIAgent.interrupt() from another thread unwinds the inline stream in 0.6s.
Co-authored-by: Expri-commits <184641533+Expri-commits@users.noreply.github.com>
A wall-clock step-back (NTP) can leave the marker's mtime in the future,
which would push the idle window out by the step size. Clamp with
min(mtime, now); _last_inbound_at has the same exposure but is at least
bounded by process uptime. Review comment on #100830.
The Anthropic long-context 429 handler restarts on row count alone,
the same shape #100614 fixed in the generic overflow handler. Arm the
same provider-overflow recovery flag there so the rebuilt request is
measured against the reduced window before the provider is retried.
The 413 (byte-scored) and output-cap (max_tokens) handlers are a
different yardstick and are left as-is.
Collapses the three test-refinement commits from #100614 (423f7f8703,
157db13c76, ffa72dd67f): model recovery pressure by provider-call state,
assert the rebuilt-oversized retry fails closed, keep the compacted
history compressible (user+assistant summary rows).
The `${{ false && (...) }}` if-expression on the reusable-workflow
e2e-desktop job made GitHub's workflow parser fail at startup (annotation:
"An unexpected error has occurred"), so every ci.yaml run on main and
every PR dispatched 0 jobs. Discriminator on this branch: pre-24f5a60
blob → 26 jobs; paren-relocated && form → 0; bare if: false → 25.
Lane stays disabled; comment documents the re-enable line.
session-tile-owner-route.test.ts asserted against the TEXT of
session-tile.tsx, so it passed on a broken implementation whose call site
merely looked right, and failed on this refactor, which changed nothing the
tile actually does. AGENTS.md bans the pattern outright.
Extracted the ladder as tileOwnerRoute() and replaced the three regex
assertions with six that call it: tile route wins, row falls back, hint
falls back, targetProfile carries through, a bare profile narrows away, an
untagged session stays ambient. 12s of source-matching becomes 1.1s of
behavior.
The coalescing key now carries the owner, so two backends that both expose a
session called `parent` get their own create instead of the second caller
receiving the first's child. Both creates are held open, which is the only
state the key guards — a sequential version passes even with the owner
stripped out.
Co-authored-by: Ahmett101 <ahmet.tunc@gmail.com>
TileChat re-renders per streamed token, and the owner ladder spread three
session arrays before scanning them on each one. Subscribe to the atoms it
actually reads and memoise the lookup on them, so it recomputes when the
tile store or a session list changes rather than per frame.
branchStoredSession looked its parent up in $sessions alone, and
branchCurrentSession did the same. A conversation reachable through a
profile-scoped project tree has no row there, and when it appears in both
places the flat Recents copy is the ownerless one — so the lookup returned
the row that cannot route, and the branch created its child on whichever
backend happened to be active.
cachedSessionRow spans Recents, cron, messaging and the project tree, and
prefers the self-describing candidate. One ladder, used by both branch
entry points and by resolveStoredSession.
Co-authored-by: evan-bradford <evan-bradford@users.noreply.github.com>
A scoped projects.tree / projects.project_sessions response is built from
ONE profile's state.db, so the request scope is authoritative even for
legacy rows whose persisted profile_name is NULL. Without the stamp those
rows reach the renderer ownerless, and every owner lookup off them — the
branch path included — falls back to whichever backend is active.
Co-authored-by: evan-bradford <evan-bradford@users.noreply.github.com>
Review follow-up on #93959:
1. Partial-failure window: if the row commits but the transcript copy or
title write fails, the durable-but-empty child defeated the lazy
first-prompt fallback (_ensure_session_db_row is INSERT OR IGNORE), so
the renderer fail-latched on a transcript-less session again. The seed
block now compensates: delete just this child so the lazy path can
retry cleanly. Disk-full is exempt — deleting data on a full disk makes
things worse.
2. Silent degradation: the best-effort catch now logs at WARNING with
exc_info instead of DEBUG, so a regression in this user-facing path is
observable without enabling debug logs.
Tests: compensation deletes the half-written row and preserves
pending_title; disk-full keeps the row and surfaces the WARNING.
Desktop branch creation hung on an infinite spinner and lost the branch
on restart. Root cause: the renderer branches via session.create with
parent_session_id + a seeded transcript, but session.create defers the
DB row to the first prompt (the draft-hygiene contract). The renderer's
post-create resume then re-fetches the fresh child through REST and
defer_history hydration — both read the DB. An unpersisted child 404s
and hydrates empty, the client fail-latch (sessionShouldHaveTranscript +
empty messages) refuses to bind a "transcript-less" session, and the
user sees a spinner forever; on restart the rowless child vanishes and
the optimistic "Draft: Branch N" entry disappears with it.
A seeded branch is explicit user intent, not an abandoned draft.
session.create now persists the child immediately when both
parent_session_id AND seeded history are present:
- Row created in the PARENT's profile-scoped state.db, stamped with
_branched_from + parent_session_id (same shape as TUI /branch).
- Seeded transcript copied via append_messages_batch so REST prefetch
and defer_history hydration find it on the first read.
- Title assigned from get_next_title_in_lineage(parent) and cleared
from pending_title — the branch lands in the parent's lineage instead
of falling back to a message-preview name.
Persistence is best-effort: a broken DB logs and lets create succeed,
leaving the lazy first-prompt path as fallback. Plain drafts keep the
lazy-row contract unchanged.
Fixes#93959
Routing the create and stamping the optimistic row still left a
remote-owned branch child flickering into "Couldn't open this session —
Session keeps losing its backend runtime right after resuming". The RPCs
were right and every resume succeeded; the OWNING SOCKET was the
casualty. Three gaps, one cause — nothing durable named the owner:
- openSessionTile persisted ownerRoute only for workspaceMode==='bots',
so the branch tile pinned nothing in the gateway keep-set
(openTileGatewayScopes / foregroundSessionScopes). The pruner closed
the owner socket, the backend orphan-reaped the draft runtime,
session.reclaimed unbound the tile, resume re-armed and succeeded on a
fresh socket the next recompute closed again — until the resume-storm
breaker (#93892) latched the error card at TILE_RESUME_STORM_LIMIT.
Persist the route for sessions-mode tiles whose opener knows the exact
owner, and stop a route-less re-scope from clobbering it.
- forkBranch, unlike both sibling routed creates in the same file, never
called setSessionOwnerHint/holdSessionOwnerUntilForeground — so in the
gap between session.branch returning and the tile publication landing,
no keep-set rung named the owner and a prune could reap the just-minted
draft runtime before the first prompt. Add both, mirroring the
siblings; the hold retires once the tile's own route covers the scope.
- resetTileRuntimeBindings preserved cross-connection runtimes only for
bot tabs, so a flapping sibling connection (an SSH source re-dialing)
dropped the branch tile's healthy binding on every reconnect, re-arming
resume each time — the same storm by another path. Preserve any
owner-routed tile; the owner's own reconnect still rebinds.
Also keep open tiles' rows in sessionsToKeep: a branch child is a draft
the aggregator cannot return until its first turn persists it, so the
next background refresh silently dropped the optimistic "draft: branch
#N" row and the sidebar showed no trace of the branch until first send.
Each fix verified RED by reverting its line. Live acceptance against a
remote-owned parent branched from another connection: draft row visible
immediately and stable across refreshes, tile carries the owner route,
first turn accepted and completed by the owning backend, and no
storm/error card through the full 120s storm window — before the fixes
the card latched at ~20s.
Routing the branch create to the parent's owning connection was only half the
job. The child then landed in the sidebar as a row that lied about who owned
it, so the chat pane spun forever on "draft: branch #1" and never hydrated —
the create was right, the row was wrong.
upsertOptimisticSession stamps the row's profile from $activeGatewayProfile and
omits connection_id entirely when no owner is passed (utils.ts:1318-1342), and
it also skips setSessionOwnerHint. The branch call site passed no owner, so the
child got NEITHER a row tag NOR a hint. resumeSession's owner ladder starts at
`capturedOwner || getSessionOwnerHint(storedSessionId)` and forkBranch calls it
without a capturedOwner, so the missing hint alone was enough to send the
resume to whichever backend happened to be active. Pass the parent's route as
the owner argument, restoring both mechanisms. The two sibling routed creates
in this file already did exactly this.
The tile path had the same defect one rung further out. A branch of a session
that is not the open chat opens a tile instead of resuming, and
SessionTileChrome resolved its owner from the tile route alone. openSessionTile
is called for a branch child with no workspaceScope, and session-states.ts only
persists a tile ownerRoute in bots mode, so that tile had no owner at all and
its model + composer RPCs fell back to the ambient socket. Use the same
tile-route-then-row ladder its sibling in session-tile-actions.ts already uses,
resolved per render so it cannot go stale against the tile store, the
recents/cron/messaging rows, or the hint map, with only the resulting identity
memoised on primitives.
An untagged parent row still reproduces the previous ambient behaviour exactly,
so single-connection users are unaffected.
Verified end to end against two real gateways: a session owned by a remote
connection, branched through the actual sidebar context menu in a running dev
app. The remote gateway served the create (ws closed ... messages=11
detached_sessions=1) and the resulting row polled stable at connection_id =
the remote for the full 8s window. Before the fix the same gesture produced a
row with no connection_id.
Branching a session owned by one connection while another is active created
the child on the wrong backend — or nowhere — while the sidebar still painted
an optimistic row. That row pointed at an id no backend owned, so hydration
retried, exhausted, and armed the stranded-session overlay: "Couldn't load
this session. The connection to this session failed and automatic retries
gave up." Retry re-ran the same mis-route, so it never recovered.
branchStoredSession and branchCurrentSession resolved only the parent's
PROFILE and called ensureGatewayProfile(profile), then dispatched through the
ambient requestGateway. A profile name does not identify a backend once
several connections expose the same name, so both the parent transcript read
and the session.create/session.branch RPC landed on whichever socket happened
to be active. removeSession, twelve lines away, already routed by
(connectionId, profile) via SessionOwnerScope — branch simply never got the
same treatment.
Reuse that existing contract: derive the exact owner from the parent row with
sessionOwnerRouteFromRow, activate it with ensureGatewayAgent, and dispatch
via requestGatewayForAgent. getAllSessionMessages takes the same owner scope
so the transcript read cannot silently come back empty and abort the branch
as "nothing to branch" before any create is attempted. Both arms of forkBranch
(session.branch for the open chat, session.create for a sidebar right-click)
are covered.
An untagged parent row — the single-backend case — keeps the previous
profile-only path exactly, so behaviour is unchanged for users with one
connection.
Tests: three call-site regressions asserting the create rides the owning
(connection, profile) socket, that the transcript read carries the same owner
scope, and that an untagged parent still uses the ambient socket. Plus an
integration test that mocks nothing inside the router — the real
requestGatewayForAgent runs against a fake Electron bridge and transport, so
a regression that re-collapses a registry route onto the ambient socket fails
even if the call-site assertions still pass.
The sidebar reports a profile it could not scan as HTTP 200 with an empty
page and errors=[{profile}]. The renderer merges that page keeping only
working, pinned, and selected rows, so every idle Yesterday / This-week
session disappears until a later scan succeeds — and the 5s coalescing cache
then serves the same empty payload back for the rest of its TTL.
Carry the previous rows forward for exactly the profiles named in errors[],
keyed by profile::id so a twin id in another profile is never stitched in.
Profiles that scanned cleanly are still authoritative, so a genuinely empty
page with no errors still clears the list. Per-profile usage and truncation
flags follow the same rule rather than zeroing under a list that was kept.
The legacy per-slice fallback stamps errors on the slice that actually
failed, so a cron read failure can no longer blank recents.
Part of #73847
Part of #88528
Co-authored-by: AKAZIK-py <AKAZIK-py@users.noreply.github.com>
A concurrent WAL checkpoint / reset / frame-flush can surface SQLITE_IOERR
to a reader on a perfectly healthy database: a mode=ro connection cannot
perform the WAL recovery the read needs, because recovery writes the -shm
index and read-only mode refuses. The window is millisecond-scale.
Today that one-shot error escapes the SessionDB read-only constructor, and
GET /api/sessions turns it into a 500 the desktop reads as an authoritative
empty list.
Retry it, bounded, in the constructor so every read-only opener is covered —
the sidebar poll, cross-profile aggregation, recall, browse — rather than at
one route. A persistent IOERR still exhausts the budget and propagates.
Remaining transient failures answer 503, so the client keeps the list it has.
On the write path, BEGIN IMMEDIATE can hit the same transient IOERR before
the callback runs. That one is safe to retry on the same connection because
nothing has been mutated; once the callback starts, settlement is unknown and
the error propagates. Never close()+reopen to heal it — close() cancels this
process's POSIX advisory locks on the file for every sibling connection, and
a list poll's reader must stay disposable so a replaced state.db is observed
and the pre-repair forensic backup stays reachable.
Fixes#100436
Co-authored-by: rkfshakti <rkfshakti@users.noreply.github.com>
Co-authored-by: AKAZIK-py <AKAZIK-py@users.noreply.github.com>
The everything-flow's client leg read `$updateStatus.get() ?? await
checkUpdates()`, so a cached row always won. That row can be up to a
poll interval (30 minutes) old and is captured before the backend leg
runs, so a cached "already current" skipped the client apply entirely —
the stale-GUI gap the flow exists to close.
Re-check first and fall back to the pre-flow snapshot when the live
check can't answer. `checkUpdates()` resolves with an error status
rather than rejecting and overwrites the atom with it, so the snapshot
is taken before any leg runs.
Co-authored-by: Dhana <227747512+whoisdhana@users.noreply.github.com>
The update entry points chose their target from the connection mode, so
every surface in remote mode acted on the backend — including the ones
showing the client's own status.
The macOS "Check for Updates…" app-menu item is the clearest case: it
sits next to "About Hermes" and is the OS-standard way to update THIS
app, but on a Mac connected to a remote Linux backend it checked the
Linux box. The backend was already current, so the action reported
nothing and did nothing, and the desktop app drifted months behind with
no error and no updater log to explain it. The update-available toast
had the same split: a client check raised it, clicking it opened the
backend's overlay, which has no target switcher and no way back.
Surfaces bound to one target now name it; only genuinely generic
commands still take the connection-mode default, so the command
palette's remote-mode backend target and the everything-flow are
unchanged.
Co-authored-by: BerneYue <14088768+yuexiongHNU@users.noreply.github.com>
Co-authored-by: David Metcalfe <80915+DavidMetcalfe@users.noreply.github.com>
Co-authored-by: clayduncan <234173110+clayduncan@users.noreply.github.com>
Co-authored-by: Dhana <227747512+whoisdhana@users.noreply.github.com>
When the bundle was swapped under a running process, the About banner sent the
user to the installer — a download and a reinstall for a state that a plain
restart repairs, and the reason reinstalling never helped these reports.
Report bundleSwapPending on hermes:version and give that case its own copy and
a "Restart Hermes" button. It gets its own headline too: reusing "App build out
of date" over a body that says the app is already installed repeats the
contradiction with the Updates card that the banner is supposed to resolve. The
installer link stays for the genuinely-stale-bundle case.
Packaged builds only — a dev `--build-only` rewrites the stamp under a running
`npm start`, and that is a rebuild the developer asked for, not a torn install.
Co-authored-by: tk-pkm111 <133480534+tk-pkm111@users.noreply.github.com>
A user who reopens Hermes while an update is running lands on the boot gate,
which is what it is for. But the updater swaps the packaged bundle on disk
after `hermes update` exits, and its `open` leg only focuses this already-
running process, so nothing ever loads the new build. The parked instance then
passes the gate and boots the new runtime under the old renderer — the "App
build out of date" banner immediately after a fully successful update, over an
Updates card that says "You're on the latest version" and so offers no remedy.
Compare the install stamp this process loaded at boot with the one on disk when
the gate clears. On positive proof of a swap — different commit, or a different
builtAt at the same commit — relaunch instead of starting a backend. Detection
fails quiet like bundle-skew, so a swap that never happened (the Windows
locked-binary case) is unchanged. A one-shot argv flag makes a relaunch loop
impossible and a 15s failsafe falls back to the old behavior.
Co-authored-by: tk-pkm111 <133480534+tk-pkm111@users.noreply.github.com>
Co-authored-by: aeonsong <aeonsong@users.noreply.github.com>
detectBundleSkew() trusted `git rev-list --count <stamp>..HEAD -- apps/desktop`
outright, which claims skew in two states where the install is not torn.
Ancestry: `A..HEAD` only measures how far HEAD is ahead of A when A is an
ancestor of HEAD. A ZIP-fallback update rewrites the tree onto a synthetic
root, so the stamp still resolves but is unreachable; the range degenerates to
HEAD's own history and reports a permanent >= 1 while apps/desktop is
byte-identical. Ask `merge-base --is-ancestor` first and go quiet unless it
answers yes.
Scope: the pathspec counted every file under apps/desktop/, so a docs- or
e2e-only commit produced a banner promising missing UI features that do not
exist. Count only the paths that reach the shipped app.
Co-authored-by: jackulau <jackulau@users.noreply.github.com>
Co-authored-by: kokhlo <kokhlo@users.noreply.github.com>
Review finding: dashboard_client_last_seen() discarded the marker once it
was >= 45s old, so after the client disconnected the gateway fell back to
its own (much older) _last_inbound_at and suspended ~45-75s later, not
idle_timeout later as the PR claimed. The staging release leg had in fact
shown 46s.
The marker mtime is a timestamp of real inbound; is_idle already judges
recency. Drop the staleness cutoff entirely and always fold the raw mtime
into the inbound clock (max with _last_inbound_at). An old marker is
harmless: it is outside idle_timeout just like an old _last_inbound_at.
Also:
- touch the marker immediately after ws.accept(), before the ready/skin
setup, so a client waiting on a slow ready frame is still visible
- make the newer-message-wins test discriminating (timeout 10s, chat 5s,
marker 40s: choosing the marker would read idle)
- add < idle_timeout / >= idle_timeout / ancient-marker boundary tests
Re-validated live on hermes-agent-stg-test-6698: last client frame
03:18:11Z -> going dormant 03:20:17Z (126s = 120s timeout + watcher tick)
while the gateway's own inbound clock was 368s stale. Mutation check vs
origin/main files: run.py hunk reverted -> 4 fail, ws.py reverted -> 3
fail, "prefer marker over max" -> 1 fail.
* fix(linux): install Lanczos-resized panel icons, not a PNG in scalable
Cinnamon's panel is ~24px. v2026.8.31 dropped the 1024px asset into
hicolor/scalable (SVG-only), so the Mint panel nearest-neighbor scaled
it into a mangled blob. Decode the PNG and write 24/32/48/256 rasters;
undecodable bytes still copy into one indexed dir. Drop the leftover
scalable file.
* test(linux): cover resized hicolor panel icons and stale scalable cleanup
Pin that a decodeable PNG lands as 24×24/256×256 rasters (not scalable),
a leftover scalable copy from v2026.8.31 is deleted, and truncated
PNGs still fall back to an indexed copy.
`/pr-triage [[ … [412 lines] … ]]` dispatched the LABEL: the paste
expansion was computed but only consumed by /queue, so every other
command received "[412 lines]" as its argument and the agent
faithfully reported the paste as truncated.
prepareSlashSubmission names the split the branch actually needs — the
transcript keeps the collapsed label, the dispatch carries the full
text. Image tokens stay as labels, since the gateway already holds
those files in attached_images.
parseSlashCommand split the whole line on `\s+` and rejoined with a
single space, so `/pr-triage <pasted diff>` reached the skill as one
run-on line. Only the separator between the command name and its
argument belongs to the parser; everything after it is the user text
and now survives verbatim.
Every consumer already re-splits the arg it receives, so subcommand
parsing (`/cron add`, `/model x --global`) is unchanged.
The gateway's idle predicate only stamped _last_inbound_at for messaging
inbound, so a scale-to-zero instance suspended under an open desktop app /
web dashboard / TUI. The client's reconnect loop then re-poked the
Fly-proxied hostname, autostart resumed the box, and the instance flapped
suspend -> proxy-wake every ~60s (13 of 72 active opted-in prod instances
on 2026-09-02, with [PC05] connect timeouts visible to the user).
The dashboard runs in a separate process on hosted instances, so the
signal crosses over as a marker file under HERMES_HOME/state:
- tui_gateway/ws.py touches state/dashboard_clients.heartbeat on every
/api/ws connect and inbound frame (clients gateway.ping every 15s),
throttled to one write per 5s per process.
- gateway/scale_to_zero.py: dashboard_client_last_seen() reads the mtime;
a marker older than 45s (the clients' heartbeat deadline) is "client
gone", a missing marker is "no client" (not fail-awake, or nothing
would ever sleep), an unreadable marker fails awake.
- gateway/run.py folds that into seconds_since_last_inbound, so an
attached client gets exactly the same idle_timeout grace after it
disconnects as a chat message does. No new conjunct, _last_inbound_at
itself is not mutated.
Validated as a hot patch on hermes-agent-stg-test-6698 with a real
/api/ws client pinging every 15s: machine held awake for 6.5 min of zero
proxied traffic (was 2 min), then suspended within ~50s of the client
exiting. Tests exercise the real seams (temp HERMES_HOME, the runner's
_scale_to_zero_is_idle composition, tui_gateway.ws.handle_ws) and were
mutation-checked: reverting either half or dropping the staleness check
fails 2-3 of them.
MattMaximo