Teknium
f923faa0b8
feat(voice): GPT-Live voice chat mode — a full-duplex voice frontend that delegates to Hermes (Desktop)
...
`voice.voice_chat_mode: gpt-live` swaps the desktop's chained STT → turn → TTS
loop for OpenAI's gpt-live-1: one voice model that listens while it speaks and
has no tools of its own. Every real request it hears becomes a normal Hermes
turn on the open chat — any model/provider the session selected, full toolset,
memory, approvals — and the voice paraphrases the reply aloud.
Backend
- tools/voice_live.py: mode/credential/persona resolution and the one server-side
step the API needs — POST /v1/live/sessions exchanging the renderer's SDP offer,
pinned to client delegation; the OpenAI key never reaches the renderer. Voice
persona follows the vendor prompting guide (role, style, labelled delegation
policy describing Hermes as the backend). VOICE_LIVE_TURN_NOTE is the per-turn
model-input note (transcript in, speakable prose out).
- REST: GET /api/audio/voice-live/status (mode + readiness, non-secret),
POST /api/audio/voice-live/session (SDP exchange). The offer is passed
byte-exact: a stripped trailing CRLF is a vendor 400 "unmarshal SDP: EOF".
- prompt.submit accepts surface=voice-live (+ voice_context) beside hud; the
note rides the model input via the existing _prepend_note seam, the persisted
user row stays the user's words, the system prompt stays byte-stable.
- config_defaults: voice.voice_chat_mode (chained|gpt-live), voice.gpt_live.*.
Desktop
- lib/voice-live.ts: RTCPeerConnection + oai-events data channel owner, transcript
accumulation, session.commentary/thinking/instructions appends (500-token
chunking), mute, graceful close waiting for session.closed.
- hooks/use-voice-live-conversation.ts: same public shape as useVoiceConversation;
delegation → prompt.submit(surface=voice-live); tool activity → quiet thinking
appends; reply streamed back per sentence; spoken stop phrase ends the chat;
a newer delegation interrupts an in-flight turn.
- use-composer-voice mounts both engines and latches one at conversation start
from the backend-resolved status; gpt-live without a key falls back to chained
with a notice. Settings → Voice gets the mode dropdown, voice picker, persona.
Live-verified on the worktree desktop build (headless Electron, CDP, synthetic
mic): "what is 17 times 23 and which model are you on" → delegation → Hermes
(Claude Sonnet 4.5 via OpenRouter) → spoken "391 … Claude Sonnet 4.5 through
OpenRouter"; follow-up "double that" resolved from the spoken context → 782;
"run uname -r" ran the terminal tool with "Hermes is working: terminal" fed as
quiet context → spoken kernel version; "stop" closed the session
(reason=close_requested). Chained mode creates no RTCPeerConnection.
2026-09-11 19:14:24 -07:00
..
2026-09-09 21:35:08 -04:00
2026-09-11 12:43:21 -04:00
2026-09-09 11:27:36 +10:00
2026-09-03 03:31:03 -07:00
2026-09-11 15:27:23 -07:00
2026-09-11 19:14:24 -07:00
2026-09-11 12:20:08 -07:00
2026-09-03 09:44:26 -07:00
2026-09-03 15:07:39 -07:00
2026-09-07 08:06:48 -07:00
2026-09-03 15:07:39 -07:00
2026-09-03 02:46:19 -07:00
2026-09-07 06:13:16 -07:00
2026-09-11 19:01:47 -07:00
2026-09-11 11:54:49 -07:00
2026-09-03 17:13:22 -07:00
2026-09-02 21:00:34 -07:00
2026-09-11 03:45:33 +05:30
2026-09-11 03:45:33 +05:30
2026-09-11 03:45:33 +05:30
2026-09-11 03:45:33 +05:30
2026-09-03 13:49:57 -07:00
2026-09-02 20:56:23 -07:00
2026-09-03 13:49:57 -07:00
2026-09-05 18:08:00 +05:30
2026-09-02 22:51:30 -07:00
2026-09-09 10:14:58 -07:00
2026-09-11 03:45:31 +05:30
2026-09-11 03:45:33 +05:30
2026-09-11 03:45:33 +05:30
2026-09-03 13:08:40 -07:00
2026-09-03 13:48:49 -07:00
2026-09-11 15:26:46 -07:00
2026-09-09 21:17:35 +05:30
2026-09-03 13:08:40 -07:00
2026-09-02 21:59:38 -07:00
2026-09-03 13:08:40 -07:00
2026-09-02 21:59:38 -07:00
2026-09-11 11:54:49 -07:00
2026-09-02 20:34:28 -07:00
2026-09-11 11:14:56 +05:30
2026-09-03 02:07:33 -07:00
2026-09-10 18:15:54 -07:00
2026-09-02 21:26:16 -07:00
2026-09-08 05:10:57 -07:00
2026-09-02 23:58:31 -07:00
2026-09-02 23:58:31 -07:00
2026-09-08 03:06:30 -07:00
2026-09-04 00:15:16 -07:00
2026-09-03 13:05:57 -07:00
2026-09-11 03:45:33 +05:30
2026-09-02 21:14:52 -07:00
2026-09-10 11:48:01 -07:00
2026-09-11 03:45:33 +05:30
2026-09-11 11:54:49 -07:00
2026-09-05 06:24:06 -07:00
2026-09-10 11:48:01 -07:00
2026-09-09 12:41:07 +05:30
2026-09-02 21:23:44 -07:00
2026-09-07 08:27:07 -07:00
2026-09-11 06:24:54 -07:00
2026-09-07 08:13:01 -07:00
2026-09-09 15:20:13 -07:00
2026-09-08 13:39:17 -07:00
2026-09-08 13:39:17 -07:00
2026-09-10 11:48:01 -07:00
2026-09-03 13:30:31 -07:00
2026-09-02 23:58:31 -07:00
2026-09-07 21:43:54 +05:30
2026-09-03 09:44:26 -07:00
2026-09-02 21:03:13 -07:00
2026-09-02 15:03:42 -07:00
2026-09-11 11:54:49 -07:00
2026-09-11 11:54:49 -07:00
2026-09-11 11:54:49 -07:00
2026-09-03 00:06:54 -07:00
2026-09-09 02:36:00 -07:00
2026-09-11 19:14:24 -07:00
2026-09-07 04:46:37 -07:00
2026-09-09 04:32:13 -07:00
2026-09-10 01:08:18 -07:00
2026-09-10 02:21:16 +05:30
2026-09-11 06:37:27 -07:00
2026-09-03 09:44:26 -07:00
2026-09-02 20:47:14 -07:00
2026-09-03 03:31:03 -07:00
2026-09-03 09:44:26 -07:00
2026-09-11 15:27:23 -07:00
2026-09-03 00:02:56 -07:00
2026-09-05 16:45:59 +05:30
2026-09-03 15:07:39 -07:00
2026-09-03 00:04:40 -07:00
2026-09-09 10:54:34 -07:00
2026-09-03 09:44:26 -07:00
2026-09-03 14:16:54 -07:00
2026-09-10 01:25:45 -07:00
2026-09-03 00:06:30 -07:00
2026-09-04 00:15:16 -07:00
2026-09-04 01:28:31 -07:00
2026-09-03 13:16:47 -07:00
2026-09-03 17:13:22 -07:00
2026-09-03 13:29:35 -07:00
2026-09-03 00:05:41 -07:00
2026-09-11 06:37:27 -07:00
2026-09-05 19:01:22 +05:30
2026-09-04 01:28:31 -07:00
2026-09-04 07:17:00 -07:00
2026-09-10 18:13:16 -07:00
2026-09-06 05:41:46 -07:00
2026-09-02 21:20:42 -07:00
2026-09-03 17:13:22 -07:00
2026-09-06 09:09:41 -07:00
2026-09-06 09:09:41 -07:00
2026-09-11 03:45:33 +05:30
2026-09-02 20:40:51 -07:00
2026-09-11 15:27:23 -07:00
2026-09-03 09:44:26 -07:00
2026-09-11 15:27:23 -07:00
2026-09-10 01:00:19 -07:00
2026-09-11 02:06:32 -07:00
2026-09-07 00:51:41 -07:00
2026-09-11 15:26:09 -07:00
2026-09-03 00:06:30 -07:00
2026-09-06 07:20:24 -07:00
2026-09-02 20:06:17 -07:00
2026-09-03 00:06:30 -07:00
2026-09-02 21:20:42 -07:00
2026-09-03 00:06:30 -07:00
2026-09-03 00:10:28 -07:00
2026-09-11 03:45:32 +05:30
2026-09-03 17:13:22 -07:00
2026-09-03 13:48:14 -07:00
2026-09-07 14:16:57 -07:00
2026-09-08 22:35:39 +05:30
2026-09-07 14:16:57 -07:00
2026-09-07 14:16:57 -07:00
2026-09-03 13:48:14 -07:00
2026-09-09 12:18:58 -07:00
2026-09-07 07:09:59 -07:00
2026-09-03 17:13:22 -07:00
2026-09-07 06:04:06 -07:00
2026-09-07 04:46:54 -07:00
2026-09-09 12:18:58 -07:00
2026-09-07 04:46:54 -07:00
2026-09-07 04:46:54 -07:00
2026-09-03 13:48:14 -07:00
2026-09-09 12:18:58 -07:00
2026-09-03 13:48:14 -07:00
2026-09-09 12:18:58 -07:00
2026-09-02 20:12:36 -07:00
2026-09-07 04:55:17 -07:00
2026-09-02 20:15:37 -07:00
2026-09-03 09:44:26 -07:00
2026-09-03 09:44:26 -07:00
2026-09-03 15:07:39 -07:00
2026-09-07 05:56:50 -07:00
2026-09-09 10:28:52 -07:00
2026-09-05 15:02:53 +02:00
2026-09-03 13:33:47 -07:00
2026-09-11 03:45:31 +05:30
2026-09-09 10:26:06 -07:00
2026-09-03 15:07:39 -07:00
2026-09-11 11:54:49 -07:00
2026-09-02 21:44:59 -07:00
2026-09-05 15:02:53 +02:00
2026-09-10 01:00:19 -07:00
2026-09-07 08:22:35 -07:00
2026-09-04 00:15:16 -07:00
2026-09-03 17:13:22 -07:00
2026-09-10 18:12:39 -07:00
2026-09-02 21:44:59 -07:00
2026-09-02 20:50:08 -07:00
2026-09-03 09:44:26 -07:00
2026-09-03 17:13:22 -07:00
2026-09-02 21:44:59 -07:00
2026-09-03 09:44:26 -07:00
2026-09-07 06:15:43 -07:00
2026-09-02 20:51:30 -07:00
2026-09-02 20:51:30 -07:00
2026-09-03 03:31:03 -07:00
2026-09-10 04:22:38 -07:00
2026-09-02 20:51:30 -07:00
2026-09-02 20:51:30 -07:00
2026-09-02 21:54:56 -07:00
2026-09-02 21:54:56 -07:00
2026-09-03 15:07:39 -07:00
2026-09-07 21:22:49 -07:00
2026-09-11 03:45:33 +05:30
2026-09-11 03:45:32 +05:30
2026-09-11 03:45:33 +05:30
2026-09-11 16:50:20 -07:00
2026-09-10 09:11:00 -07:00
2026-09-07 08:08:04 -07:00
2026-09-03 13:48:49 -07:00
2026-09-03 13:48:49 -07:00
2026-09-11 16:50:20 -07:00
2026-09-11 16:50:20 -07:00
2026-09-11 03:45:33 +05:30
2026-09-11 03:45:31 +05:30
2026-09-10 18:11:25 -07:00
2026-09-04 07:17:00 -07:00
2026-09-03 09:44:26 -07:00
2026-09-04 00:15:16 -07:00
2026-09-10 14:56:55 -04:00
2026-09-03 09:44:26 -07:00
2026-09-02 20:40:15 -07:00
2026-09-02 20:24:29 -07:00
2026-09-02 20:40:15 -07:00
2026-09-03 17:13:22 -07:00
2026-09-02 20:19:57 -07:00
2026-09-02 21:57:39 -07:00
2026-09-09 04:38:01 -07:00
2026-09-04 04:56:11 -07:00
2026-09-03 14:00:59 -07:00
2026-09-09 04:38:01 -07:00
2026-09-09 04:38:01 -07:00
2026-09-09 04:38:01 -07:00
2026-09-10 05:09:15 -07:00
2026-09-03 00:09:39 -07:00
2026-09-07 13:11:36 +05:30
2026-09-06 13:36:12 -07:00
2026-09-09 04:38:01 -07:00
2026-09-09 04:38:01 -07:00
2026-09-03 09:44:26 -07:00
2026-09-11 19:04:58 -07:00
2026-09-11 03:45:31 +05:30
2026-09-03 09:44:26 -07:00
2026-09-09 04:27:24 -07:00
2026-09-06 05:34:01 -07:00
2026-09-10 01:00:19 -07:00
2026-09-11 06:24:11 -07:00
2026-09-03 00:06:57 -07:00
2026-09-03 13:46:50 -07:00
2026-09-02 21:07:16 -07:00
2026-09-02 21:07:16 -07:00
2026-09-02 22:55:15 -07:00
2026-09-10 14:56:55 -04:00
2026-09-02 21:09:53 -07:00
2026-09-02 21:00:55 -07:00
2026-09-07 02:09:34 -07:00
2026-09-04 20:24:24 -04:00
2026-09-09 11:31:10 -07:00
2026-09-02 21:00:55 -07:00
2026-09-02 20:47:24 -07:00
2026-09-02 21:00:55 -07:00
2026-09-02 22:39:49 -07:00
2026-09-10 18:08:16 -07:00
2026-09-11 02:49:36 -04:00
2026-09-10 14:56:55 -04:00
2026-09-02 22:25:24 -07:00
2026-09-03 14:00:59 -07:00
2026-09-03 17:13:22 -07:00
2026-09-02 21:07:16 -07:00
2026-09-02 21:07:16 -07:00
2026-09-02 22:33:08 -07:00
2026-09-03 09:44:26 -07:00
2026-09-11 06:24:54 -07:00
2026-09-11 06:24:54 -07:00
2026-09-11 06:24:54 -07:00
2026-09-11 06:24:54 -07:00
2026-09-03 00:06:57 -07:00
2026-09-11 06:37:27 -07:00
2026-09-02 23:59:27 -07:00
2026-09-11 06:37:27 -07:00
2026-09-09 13:30:28 -07:00
2026-09-11 06:24:54 -07:00
2026-09-11 06:37:27 -07:00
2026-09-02 22:39:06 -07:00
2026-09-03 13:33:10 -07:00
2026-09-03 09:44:26 -07:00
2026-09-11 11:54:49 -07:00
2026-09-04 07:17:00 -07:00
2026-09-03 13:33:10 -07:00
2026-09-03 13:33:10 -07:00
2026-09-03 00:11:01 -07:00
2026-09-09 03:32:58 -07:00
2026-09-07 22:25:12 -07:00
2026-09-02 20:43:25 -07:00
2026-09-03 13:08:40 -07:00
2026-09-07 05:59:43 -07:00
2026-09-03 09:44:26 -07:00
2026-09-08 03:06:30 -07:00
2026-09-03 17:13:22 -07:00
2026-09-02 23:04:20 -07:00
2026-09-02 20:59:41 -07:00
2026-09-03 17:13:22 -07:00
2026-09-03 09:44:26 -07:00
2026-09-11 03:45:31 +05:30
2026-09-11 15:27:23 -07:00
2026-09-02 20:43:25 -07:00
2026-09-02 21:36:24 -07:00
2026-09-02 21:34:25 -07:00
2026-09-03 22:00:01 -07:00
2026-09-02 21:49:33 -07:00
2026-09-02 20:43:25 -07:00
2026-09-11 06:24:54 -07:00
2026-09-02 20:43:25 -07:00
2026-09-08 19:52:26 +05:30
2026-09-03 13:22:51 -07:00
2026-09-03 13:29:35 -07:00
2026-09-03 14:16:54 -07:00
2026-09-03 13:10:38 -07:00
2026-09-10 02:21:16 +05:30
2026-09-02 20:38:18 -07:00
2026-09-02 21:31:17 -07:00
2026-09-09 09:19:36 -07:00
2026-09-03 09:44:26 -07:00
2026-09-02 16:34:55 -07:00
2026-09-03 09:44:26 -07:00
2026-09-03 14:27:29 -07:00
2026-09-07 08:20:46 -07:00
2026-09-03 09:44:26 -07:00
2026-09-05 00:24:40 -07:00
2026-09-03 09:44:26 -07:00
2026-09-03 09:44:26 -07:00
2026-09-03 09:44:26 -07:00
2026-09-11 02:06:32 -07:00
2026-09-02 21:49:26 -07:00
2026-09-06 20:47:52 +05:30
2026-09-02 21:45:19 -07:00
2026-09-06 14:53:33 +05:30
2026-09-02 22:12:58 -07:00
2026-09-02 21:28:23 -07:00
2026-09-10 11:48:01 -07:00
2026-09-02 21:28:23 -07:00
2026-09-02 21:28:23 -07:00
2026-09-03 13:20:22 -07:00
2026-09-03 17:13:22 -07:00
2026-09-03 09:44:26 -07:00
2026-09-11 19:14:24 -07:00
2026-09-09 21:16:40 +05:30
2026-09-04 20:24:24 -04:00
2026-09-10 03:58:37 -07:00
2026-09-11 15:28:37 -07:00
2026-09-09 04:38:01 -07:00
2026-09-07 07:11:36 -07:00
2026-09-03 14:21:32 -07:00
2026-09-11 15:28:37 -07:00
2026-09-11 11:54:49 -07:00
2026-09-03 14:21:32 -07:00
2026-09-03 14:21:32 -07:00
2026-09-03 14:21:32 -07:00
2026-09-11 03:45:33 +05:30
2026-09-07 05:56:33 -07:00
2026-09-11 06:37:27 -07:00
2026-09-11 15:28:37 -07:00
2026-09-04 00:15:16 -07:00
2026-09-04 20:24:24 -04:00
2026-09-02 23:08:44 -07:00
2026-09-02 23:08:44 -07:00
2026-09-03 13:05:57 -07:00
2026-09-03 09:44:26 -07:00
2026-09-02 21:54:36 -07:00
2026-09-09 02:36:00 -07:00