Selecting a self-generated pet in the Bot avatar picker always failed with
"Could not load that pet — try another", and its tile showed only a name.
Locally hatched pets have no petdex manifest entry, so pet.gallery reports an
empty spritesheetUrl; the picker cropped frame 0 client-side from that URL and
bailed on the empty string. The same raw CDN fetch also lacked a User-Agent,
which the petdex CDN rejects with 403 (#90465), so manifest pets could fail on
the same path.
Route tile rendering and selection through the gateway's pet.thumb RPC, which
already backs the Settings pet picker: it crops frame 0 server-side from the
installed sheet on disk (or the host-validated CDN URL for uninstalled pets)
and returns a same-origin PNG data URI. The client cache is keyed by slug,
evicts failures so a blip never poisons a tile, and races a 15s deadline so a
hung RPC cannot park a pending promise forever.
Based on analysis from PR #90931, whose target (plugin.js) has since been
decomposed into pet.tsx.
Co-authored-by: m1k3s0 <44042865+m1k3s0@users.noreply.github.com>
Switching to GPT-Live meant Settings → Voice → Voice Chat Mode, which is not
where you are when you want to talk. The composer now offers the choice where
the voice button is:
- folded layout (HUD / narrow): a "Voice chat engine" radio group in the
existing voice menu
- unfolded layout: a small chevron beside the start-voice button opening the
same rows; the button tooltip names the engine that will mount
Rows are hidden until the backend reports a mode (older gateway = no switch
that would 4002); GPT-Live is disabled with the backend's reason when no
OpenAI key resolves. Selecting writes `voice.voice_chat_mode` through
`config.set` on the LIVE gateway (local, SSH, cloud alike) and re-reads the
resolved status; it applies to the next conversation and never touches a
running one.
Backend: `voice.voice_chat_mode` joins the `config.set` word setters
(chained|gpt-live).
Live: headed desktop, chained → menu → GPT-Live → start = RTCPeerConnection
connected; menu → chained → start = no peer connection, chained controls.
`voice.voice_chat_mode: gpt-live` swaps the desktop's chained STT → turn → TTS
loop for OpenAI's gpt-live-1: one voice model that listens while it speaks and
has no tools of its own. Every real request it hears becomes a normal Hermes
turn on the open chat — any model/provider the session selected, full toolset,
memory, approvals — and the voice paraphrases the reply aloud.
Backend
- tools/voice_live.py: mode/credential/persona resolution and the one server-side
step the API needs — POST /v1/live/sessions exchanging the renderer's SDP offer,
pinned to client delegation; the OpenAI key never reaches the renderer. Voice
persona follows the vendor prompting guide (role, style, labelled delegation
policy describing Hermes as the backend). VOICE_LIVE_TURN_NOTE is the per-turn
model-input note (transcript in, speakable prose out).
- REST: GET /api/audio/voice-live/status (mode + readiness, non-secret),
POST /api/audio/voice-live/session (SDP exchange). The offer is passed
byte-exact: a stripped trailing CRLF is a vendor 400 "unmarshal SDP: EOF".
- prompt.submit accepts surface=voice-live (+ voice_context) beside hud; the
note rides the model input via the existing _prepend_note seam, the persisted
user row stays the user's words, the system prompt stays byte-stable.
- config_defaults: voice.voice_chat_mode (chained|gpt-live), voice.gpt_live.*.
Desktop
- lib/voice-live.ts: RTCPeerConnection + oai-events data channel owner, transcript
accumulation, session.commentary/thinking/instructions appends (500-token
chunking), mute, graceful close waiting for session.closed.
- hooks/use-voice-live-conversation.ts: same public shape as useVoiceConversation;
delegation → prompt.submit(surface=voice-live); tool activity → quiet thinking
appends; reply streamed back per sentence; spoken stop phrase ends the chat;
a newer delegation interrupts an in-flight turn.
- use-composer-voice mounts both engines and latches one at conversation start
from the backend-resolved status; gpt-live without a key falls back to chained
with a notice. Settings → Voice gets the mode dropdown, voice picker, persona.
Live-verified on the worktree desktop build (headless Electron, CDP, synthetic
mic): "what is 17 times 23 and which model are you on" → delegation → Hermes
(Claude Sonnet 4.5 via OpenRouter) → spoken "391 … Claude Sonnet 4.5 through
OpenRouter"; follow-up "double that" resolved from the spoken context → 782;
"run uname -r" ran the terminal tool with "Hermes is working: terminal" fed as
quiet context → spoken kernel version; "stop" closed the session
(reason=close_requested). Chained mode creates no RTCPeerConnection.
Reverts the in-tree org skill-marketplace: hermes_wisdom package, three
model tools, CLI/gateway/desktop/dashboard/Telegram/Slack surfaces.
Later non-Wisdom work on shared files (guest onboarding i18n, dashboard
startup schema, Slack adapter, tui_gateway) is kept; Wisdom-only call
sites and config were stripped from those files.
Stop recommending a system-RAM spill when no curated model fits resident.
Preserve explicit model selection and the existing resident quality/speed
ranking, including the separate unified-memory policy.
Require a recommendation for automatic quickstart, expose Browse when
none exists, and rename Configure to Let me choose. Keep policy copy and
reason keys consistent across the four translated local-model sections.
Cover automatic refusal and explicit spilled setup against one budget,
plus the Browse, Download and Use interactions in the desktop pane.
Same-named defaults on different connections were mislabeled as (you) in
member room-delta prompts. Compare speaker/viewer with connection source
identity so only the true self gets the suffix.
Fixes#106851
Co-authored-by: Cursor <cursoragent@cursor.com>
Drop two PrimaryProfilePin cases that only restate the constructor
defaults and blank-string normalisation, and the wiring-routing test that
froze POOL_LIMITS_SETTINGS_ROUTE to a literal string — a snapshot of the
constant, not a behaviour contract. The two kept pin tests cover the bug
(a live primary keeps answering for its booted profile after the stored
preference moves; teardown releases the pin), and the notifications tests
cover the toast action end-to-end.
`primaryProfileKey()` re-read active-profile.json on every call. The rail's
live workspace switch rewrites that file via `hermes:profile:remember`
WITHOUT re-homing the primary, so after a switch the routing table disagreed
with the running process: a request for the profile the primary actually
booted as (e.g. "default") no longer matched `primaryProfile` in
`resolveProfileBackendRoute`, fell through to the pool, and spawned a second
backend for the same HERMES_HOME.
The duplicate was keepalive-fresh so LRU eviction spared it, it burned a pool
slot, and with the default cap of 3 every further profile queued and failed
with `Local backend start for "<profile>" timed out while waiting for a free
slot` (repro in desktop.log: "default" spawned as a pool backend while the
primary "default" was still running; coder/qwen then timed out for 20+ min).
Snapshot the launch profile in `startHermes()` (PrimaryProfilePin.pin) and
release it in `resetHermesConnection()` so the next start follows the stored
preference again. The pin is a tiny pure module with tests; main.ts only owns
the file read and the two call sites.
Switching the pooled dispatch probe to /api/health (salvaged from #97914)
would 404 on every dispatch against a remote older than 0.19, retire the
tunnel and reconnect forever - the same storm #107997 describes, moved to
old backends. Fall back to /api/status on an explicit 404 exactly the way
the boot readiness probe already does (backend-health.ts). The legacy
fallback idea and its test are taken from #101976 (@edosulai); the rest of
that PR (timeout-tolerance streak, ServerAlive SSH options) is not adopted.
Co-authored-by: Edo Sulaiman <edosulai@icloud.com>
Cold /api/status through a Windows no-mux SSH forward routinely exceeds
the 2.5s dispatch budget, so Desktop retires a live tunnel and respawns.
Use the cheap /api/health route (5s, same as DEFAULT_HEALTH_PROBE_TIMEOUT_MS).
Background liveness still probes /api/status at 10s.
A 2500ms dispatch probe is shorter than quiet-box hermes serve cold-start (~6-8s), so a just-woken pooled backend always fails and reconnects. Reuse REMOTE_LIVENESS_TIMEOUT_MS (10s) for that probe.
Co-authored-by: Cursor <cursoragent@cursor.com>
useRoster repaints every 5s and hands pullServerAvatars the active-source
rows. Since the multi-source merge (ed20a6f01a) every such row is
sourceScoped, so the avatar sync branch chose requestForBot and dialed each
bot's OWN backend to read a profile-directory PNG: a fresh WebSocket with
one JSON-RPC message, torn down at refcount 0, per bot per tick (#99336's
"ws accepted / ws closed messages=1" every ~5.2s on background profiles),
and for every bot with no running backend a pool spawn that waits out
POOL_SLOT_WAIT_MS (30s) and is re-queued by the next paint, forever, once
the pool is full (#102913's per-bot "waiting for a free local slot ...
timed out" cadence). The loop was self-sustaining because the plugin's own
160px face raster is deliberately not parked in $botMeta, so the empty
image slot re-fetched it on every tick.
Assets are files under the profile directory; the gateway that just
answered profiles.list reads them for any of its profiles. Route the three
avatar RPCs through host.request like the roster query itself, and remember
face-only answers so a row is fetched once, not once per tick.
Not changed: relay.ts (its loops dedupe to one route per registered
connection and return early below two connections, so a single-connection
desktop never issues a relay RPC), and useRoster's own profiles.list, which
already rides the active socket via requestForBot({name}).
The first Bot Mode roster paint after launch ran pullServerAvatars over every
row, and for a source-scoped row (every row on a local-primary desktop once
host.agents annotates the roster) each profiles.get_asset / set_asset went
through requestForBot -> host.requestProfile -> requestGatewayForAgent, i.e. a
(connectionId, profile) secondary that spawns that profile's pooled backend.
With ~60 registered profiles and 3 warm slots this queued 56 background spawns
at boot; each queued dial then rode reconnectSecondary's backoff until the
stall budget parked it (#107969), so the pool never drained and desktop.log
filled with "waiting for a free local slot" (#102978).
The active gateway's own profiles.list already produced these rows by reading
every local profile directory; get_asset/set_asset are the same directory
reads addressed by name. Route both through host.request on the active socket
with the row's backend profile name (route.targetProfile, so managed aliases
still resolve). No secondary socket, no pool slot, no spawn.
Cross-connection (remoteSource) rows never reached this path: pullServerAvatars
is fed activeSourceRoster, which filters them out.
host.warmAgent — the (connection, profile) sibling of warmProfile that
bot-row.tsx fires on pointerEnter for multi-source roster rows — still
dialed openGatewayForAgent directly, so a pointer sweep across a mixed
roster kept spawning at pointer speed past maxBackends on the registry
path even after warmProfile was guarded. Same bug class as #103631,
different door.
prewarmProfileBackend now takes an optional connectionId: the
active-profile no-op, the 60s throttle (keyed by the pool scope key) and
the pool-saturation skip apply unchanged, and the dial picks
openGatewayForAgent for a scoped source. One resolver owns every
speculative warm in the app; the real click still spawns on demand.
Plugin rosters warm profile backends on pointerEnter with no dwell of
their own. warmProfile dialed openGatewayForProfile directly, bypassing
the pool-saturation guard, hover dwell, and per-profile throttle that
prewarmProfileBackend enforces for the built-in rail — so a pointer
sweep across a roster could spawn past maxBackends and leave the next
profile's real spawn queued until the 30s slot timeout, surfacing as a
profile surface that hangs forever while every other profile renders.
Delegate to prewarmProfileBackend so every speculative warm shares one
resolver and one policy, as the design guide requires. The real click
still spawns on demand; only the speculative head start is gated.
Settings, Layout, and HUD default to the right so tabs keep the left
titlebar. Appearance has a Left/Right control for people who want the
previous left cluster.
(cherry picked from commit 7fe3175e475eb0ea81198bb69a0250662f940b17)
Keep panel tabs in the titlebar moved those app actions next to the
sidebar toggle, which ate the tab strip. Put them back on the right
edge. Sidebar toggle stays left. Fixes#107351.
(cherry picked from commit 7f4460a7f6028cf384506733a5bfa52273792d64)
`revealDesktopPane` drove files/review/sessions/terminal through their
store setters only. Those are same-value no-ops when the pane's `$open`
already reads true while the user minimized its zone from the header
chevron, so the `focus_pane` tool reported success over an invisible
pane (#106009; class noted by @worryfreeaa). Route every tree-backed
revealer through `revealTreePane` after its own setter, which clears
`minimized` and fronts the pane.
(cherry picked from commit 690a1a75108ec63ce5cb1a638543969b0877dbaa)
* fix(desktop): centralize guide handoff receipt reads
Resolve the guide receipt key and value together in setup-profile. Use the helper at all four read sites so connection scoping follows one implementation.
* fix(desktop): recover from unreadable handoff receipts
Memoize receipt reads and show Retry only for the error phase. Quarantine corrupt data before retrying, and resolve the guide identity when the failed request did not retain it so a fresh build can start.
Cover preservation of corrupt data and removal from the active receipt key with an invariant test.
* fix(desktop): validate persisted onboarding phases from one list
Derive OnboardingPhase and persisted-value validation from the same phase list so future phases survive relaunch. Verify every persisted phase reloads and an unknown value falls back to idle.
* fix(desktop): share window centering arithmetic
Extract centeredBounds and use it for onboarding boot and window growth. Keep the existing work-area clamps and coordinate rounding unchanged.
* fix(desktop): compute progress steps inline
Remove the ineffective ProgressCard memo because streaming flushes replace the messages array. Keep the same transcript scan and rendered steps.
* fix(desktop): center the free-tier status chip detail
Wrap the model label and sign-in badge in an inline flex span with a shared gap. This centers the badge beside the model text without changing other status-bar details.
* fix(desktop): derive the guide receipt key in one place
The Retry path spelled the key derivation out again because the read helper throws on a corrupt receipt before it can return the key. A separate guideHandoffReceiptKey serves both the reader and the quarantine, so the derivation has one home again.
* fix(desktop): keep the free-tier badge at its intended leading
Badge declares leading-none, but the class merger drops it behind the size variant's font-size class, so the badge inherits a 1.5 leading and renders 16px tall next to an 11px label. That height, not the inline alignment, is what read as a detached badge. Restating leading-none on the chip's badge brings it to 11.6px, inside the label's cap height. The Badge component itself is left alone; every other badge in the app has the same dropped leading and that is a separate decision.
- Multi-range requests (any comma) now fall back to a full 200 instead of silently serving only
the first part (RFC 7233 permits ignoring Range); documented + pinned by a test.
- Open the file first and fstat that handle, then stream from the same FileHandle, so
Content-Length and the bytes delivered come from one open file (no stat/stream race).
Handing the FileHandle (not the raw fd) to createReadStream avoids a double close.
- ENOENT/ENOTDIR return a 404 Response instead of rejecting; covered by a test.
The custom setPermissionCheckHandler only allowed media/audioCapture/
videoCapture, which made Electron deny the 'automatic-fullscreen'
permission consulted during HTML5 video requestFullscreen(). The
request handler's isMediaCapturePermission() also returned false for
'fullscreen'. Result: the native fullscreen button on <video controls>
in chat silently did nothing.
Allow 'fullscreen' + 'automatic-fullscreen' in both handlers.
Verified with a minimal Electron repro using Hermes' exact handlers:
requestFullscreen() failed with 'TypeError: Permissions check failed'
before; works after. User-verified in the packaged desktop app.
Let settled remote sessions continue their first quit. Fence late local starts and join existing local and SSH drains without cancelling managed update recovery.
Co-authored-by: Gille <4317663+helix4u@users.noreply.github.com>
Co-authored-by: ChanPark03 <parkchan0302@gmail.com>
Direct chat already uploads PDFs into the session workspace. Group turns
called pdf.attach instead, which needs pdftoppm and swallowed failures, so
bots saw the filename and no file. Stage PDFs the same way as other files,
and name a failed attach in that member's prompt.
Group PDFs currently only hit pdf.attach, so a member prompt can name the
file while the session workspace never receives it. These tests require the
same file.attach + @file: ref path 1:1 chat uses, and a named failure when
that staging throws.
Queue a forced follow-up when Test overlaps an older enumeration; retain bounded caching for incidental focus events.
Co-authored-by: FalconOrtiz <falcon.ortiz11@gmail.com>
Keep a runtime turn descriptor, feed its roster key into row mood and activity filtering, and release only the completing invocation’s presence. Stop uses the captured owner rather than a name lookup.
Co-authored-by: Tuna Dev <tuancookiez@gmail.com>
Teknium