Commit Graph

13 Commits

Author SHA1 Message Date
Teknium f923faa0b8 feat(voice): GPT-Live voice chat mode — a full-duplex voice frontend that delegates to Hermes (Desktop)
`voice.voice_chat_mode: gpt-live` swaps the desktop's chained STT → turn → TTS
loop for OpenAI's gpt-live-1: one voice model that listens while it speaks and
has no tools of its own. Every real request it hears becomes a normal Hermes
turn on the open chat — any model/provider the session selected, full toolset,
memory, approvals — and the voice paraphrases the reply aloud.

Backend
- tools/voice_live.py: mode/credential/persona resolution and the one server-side
  step the API needs — POST /v1/live/sessions exchanging the renderer's SDP offer,
  pinned to client delegation; the OpenAI key never reaches the renderer. Voice
  persona follows the vendor prompting guide (role, style, labelled delegation
  policy describing Hermes as the backend). VOICE_LIVE_TURN_NOTE is the per-turn
  model-input note (transcript in, speakable prose out).
- REST: GET /api/audio/voice-live/status (mode + readiness, non-secret),
  POST /api/audio/voice-live/session (SDP exchange). The offer is passed
  byte-exact: a stripped trailing CRLF is a vendor 400 "unmarshal SDP: EOF".
- prompt.submit accepts surface=voice-live (+ voice_context) beside hud; the
  note rides the model input via the existing _prepend_note seam, the persisted
  user row stays the user's words, the system prompt stays byte-stable.
- config_defaults: voice.voice_chat_mode (chained|gpt-live), voice.gpt_live.*.

Desktop
- lib/voice-live.ts: RTCPeerConnection + oai-events data channel owner, transcript
  accumulation, session.commentary/thinking/instructions appends (500-token
  chunking), mute, graceful close waiting for session.closed.
- hooks/use-voice-live-conversation.ts: same public shape as useVoiceConversation;
  delegation → prompt.submit(surface=voice-live); tool activity → quiet thinking
  appends; reply streamed back per sentence; spoken stop phrase ends the chat;
  a newer delegation interrupts an in-flight turn.
- use-composer-voice mounts both engines and latches one at conversation start
  from the backend-resolved status; gpt-live without a key falls back to chained
  with a notice. Settings → Voice gets the mode dropdown, voice picker, persona.

Live-verified on the worktree desktop build (headless Electron, CDP, synthetic
mic): "what is 17 times 23 and which model are you on" → delegation → Hermes
(Claude Sonnet 4.5 via OpenRouter) → spoken "391 … Claude Sonnet 4.5 through
OpenRouter"; follow-up "double that" resolved from the spoken context → 782;
"run uname -r" ran the terminal tool with "Hermes is working: terminal" fed as
quiet context → spoken kernel version; "stop" closed the session
(reason=close_requested). Chained mode creates no RTCPeerConnection.
2026-09-11 19:14:24 -07:00
Teknium 08bd8aea74 simplify(compat): web_routers — repoint 231 web_server.<name> seams to the owning module (105 direct imports, 113 late()/LateState() with module arg, 12 lazy imports + kanban plugin _ws_auth_ok) 2026-09-03 14:21:43 -07:00
Teknium fc9ed6973e simplify(compat): tts_tool — drop 57 re-exports/aliases, repoint 10 callers + 23 test files 2026-09-03 13:20:22 -07:00
Teknium 7d60d28f41 refactor(hermes_cli/web_routers): drop post-docstring blank lines 2026-09-02 22:20:21 -07:00
Teknium ec346c7521 refactor(hermes_cli/web_routers): AST-neutral packing of literal dict/list spans 2026-09-02 21:56:23 -07:00
Teknium 0cab29d3f4 refactor(hermes_cli/web_routers): AST-neutral statement packing (actions/audio/memory_providers/config_env) 2026-09-02 21:54:20 -07:00
Teknium 9912d34a3f refactor(hermes_cli/web_routers): AST-neutral bracket packing (oauth/memory_providers/config_env/actions/audio) 2026-09-02 21:43:19 -07:00
Teknium a1fe74c903 refactor(hermes_cli/web_routers): audio — shared config-scoped executor hop, quiet unlink, MIME table 2026-09-02 21:30:49 -07:00
Teknium 4fd970c62f refactor(web): absorb single-router web_server helpers into their routers (75 defs); drop dead imports 2026-09-02 13:33:22 -07:00
Teknium ba4f4b8161 refactor(web): collapse try/except-HTTPException/log/raise boilerplate into _common.http_failure; trim unused fastapi imports 2026-09-02 13:32:46 -07:00
Teknium 8ac12223bb refactor(web): late() proxies replace per-handler lazy web_server imports in extracted routers 2026-09-02 13:32:45 -07:00
Teknium ba91166b1c refactor(web): hoist stdlib imports in extracted routers; drop unused web_server imports 2026-09-02 13:32:28 -07:00
Teknium 9de17448dd refactor(web): extract audio/TTS routes into web_routers/audio.py 2026-09-02 13:30:53 -07:00