Ben Barclay 29033a3fd5 fix(relay): restore voice-note STT — wire media[] MIMEs, message_type "voice", and a User-Agent for CDN downloads (#95274)
* fix(relay): map wire media[] → event.media_types; accept message_type voice

A relayed voice note arrived as MessageType.AUDIO with media_types=[] —
the STT gate (_event_media_is_stt_input) excludes AUDIO unconditionally
and its per-attachment MIME rescue was unreachable, so STT never fired
and the agent fell back to the "user sent an audio file attachment"
context note (live-verified on staging 2026-08-26, Discord + Telegram).

Two wire-boundary fixes, both additive within contract_version 1:

- "voice" parses to MessageType.VOICE: the enum already had it — pinned
  by test so a future refactor can't collapse the two.
- media[] is now mapped into event.media_types (positional alignment
  with media_urls; mime-less entries keep their slot as ""). This is
  what run.py's per-attachment classifiers key off, so EVERY relayed
  attachment — image vs document, audio vs voice — now routes like its
  native-adapter equivalent, not just voice notes.

Behaviour pinned: new-connector voice → STT-eligible; legacy
audio-typed events unchanged (no STT); music uploads never STT-eligible
(direct _event_media_is_stt_input assertions on real wire-parsed
events, not mocks).

Pairs with the gateway-gateway PR that puts "voice" on the wire.

* review: pin the STT gate by test; fail safe on media/media_urls mismatch

Addresses independent review of #95274.

1. The PR's acceptance criterion is STT ROUTING, but no committed test
   called _event_media_is_stt_input — it was only asserted ad-hoc. Adds
   TestSttGate: voice→eligible, voice-without-media_types→eligible
   (the new-connector/old-gateway shape), legacy audio-typed voice
   note→not eligible, music→not eligible. Mutation-verified: removing
   the VOICE branch from the gate turns these RED.

2. media_urls and media[] are INDEPENDENT wire fields that consumers
   index by the same i. Mapping MIMEs positionally without checking
   agreement means a disagreeing producer misassociates a MIME with the
   wrong URL and mis-routes that attachment — strictly worse than no
   MIME, which degrades safely to message-level classification.
   _media_types_from_wire() now maps only when the lengths agree, warns
   and returns [] otherwise.

Note for the record: MessageType.VOICE predates this PR and the gate's
VOICE branch ignores media_types, so a NEW connector against an OLD
gateway ALREADY fires STT. That is desirable, but it is not "unchanged"
— the PR body's rollout matrix said otherwise and is corrected.

* fix(relay): send a User-Agent on relay media requests (Discord CDN 403)

Discord's CDN rejects urllib's default "Python-urllib/x.y" User-Agent
with HTTP 403, and RelayMediaClient never set one. Every Discord CDN
pass-through download therefore failed; _localize_inbound_media then
kept the raw URL (its "a public URL still has value" branch), and the
consumer tried to open a URL as a FILE PATH:

  WARNING gateway.relay.media: relay media download failed for
    https://cdn.discordapp.com/...voice-message.ogg: HTTP Error 403
  INFO gateway.run: Voice transcription failed for https://cdn.discord...
    : Audio file not found: https://cdn.discordapp.com/...

This killed ALL Discord relay media inbound — voice notes, images and
documents alike — not just the voice lane. Telegram/WhatsApp were
unaffected because their media is connector-re-hosted (/relay/media/{id},
fetched from our own host) and localizes to real /tmp paths.

Reproduced from a clean shell against a live CDN URL:
  curl (own UA)                -> 200
  urllib, no UA                -> 403 Forbidden
  urllib + descriptive UA      -> 200, 14583 bytes, OggS magic

Fix: a module-level _MEDIA_USER_AGENT sent on both download() and
upload(). upload() only ever targets our own connector so it was not
broken, but a single client should identify itself consistently.

Validated on staging: hot-patched hermes-agent-stg-test-6698, restarted
the gateway service, and Ben's Discord voice note transcribed
successfully — zero new 403s and zero new transcription failures after
the patch (last 403 predates it).

Test is mutation-verified: removing the UA from download() turns it RED
while the other five media tests stay green.

* fix(relay): keep url↔mime pairing through media localization

Addresses a blocking review finding on my own change: mapping media[]
into media_types created a POSITIONAL contract that the rest of the
inbound path then broke.

1. _localize_inbound_media (adapter.py) filtered media_urls without
   filtering media_types. Dropping a dead connector re-host is a NORMAL
   best-effort path, so every surviving attachment inherited its
   neighbour's mime. Reproduced through the real functions:

     before urls  [.../relay/media/dead, .../kept.png]
            types [application/pdf, image/png]
     after  urls  [.../kept.png]
            types [application/pdf, image/png]   <-- PNG reads as PDF
     _event_media_is_image(ev, 0) -> False

   The loop now carries (url, mime) as PAIRS, so a dropped URL drops its
   mime with it.

2. _media_types_from_wire compared LENGTHS only, which is not alignment:
   equal-length-but-reordered wire fields were accepted and paired
   wrongly, and an absent media_urls skipped the check entirely while
   still emitting types. Resolution is now BY URL (url -> mime lookup
   over media_urls); an unmatched URL degrades to "" and falls back to
   message-level classification.

Tests: 4 new cases driving the real chain (wire parse -> localization ->
run.py classifier), incl. the dropped-first-attachment case the existing
localization test could not catch (it builds events without
media_types). The obsolete length-mismatch test now asserts the stronger
by-url guarantee. Both fixes mutation-verified: reinstating the URL-only
filter fails 1 test, reverting to positional resolution fails 3.

Relay suite 258 passed; media/voice/stt selection 685 passed; ruff clean;
cross-repo integration payload re-verified.

* fix(relay): media_types is always one slot per media_url

Self-review after two review rounds flagged this bug class in adjacent
seams: I checked the function I edited, not every consumer of the
parallel arrays I created. Grepping ALL writers found a third instance.

merge_pending_message_event (gateway/platforms/base.py:2725-2735)
EXTENDS media_urls and media_types together when a second media message
merges into a pending one. My mapping could emit a POPULATED media_urls
with an EMPTY media_types (an older connector sends media_urls but no
media[]), so extend() concatenated lists of different lengths:

  A urls [old1.png, old2.png]  types []
  B urls [new.pdf]             types [application/pdf]
  merged urls  [old1.png, old2.png, new.pdf]
         types [application/pdf]
    -> old1.png reads as application/pdf; the real PDF gets ''

Fix: media_types is now ALWAYS len(media_urls), padded with '' — the
url-keyed lookup runs even when media[] is absent, and the localizer
rewrites the list unconditionally (no  short-circuit that
could leave a stale/short list behind).

Tests: 4 new cases — padding with no media[], the merge shift above
driven through the real merge_pending_message_event, localization
preserving the invariant while dropping an entry, and normalization of
a short/empty media_types arriving from a non-wire source. All
mutation-verified: removing the padding fails 4; restoring the
 guard fails 1.

Relay 262 passed; media/voice/stt selection 689 passed; ruff clean;
cross-repo integration payload re-verified.
2026-08-27 15:07:38 +10:00
2026-08-14 13:55:29 -07:00
…
…
…
…
…
…
…

Hermes Agent

Hermes Agent ☤

Hermes Agent | Hermes Desktop

Documentation Discord License: MIT Built by Nous Research 中文 اردو Español

The self-improving AI agent built by Nous Research. It's the only agent with a built-in learning loop — it creates skills from experience, improves them during use, nudges itself to persist knowledge, searches its own past conversations, and builds a deepening model of who you are across sessions. Run it on a $5 VPS, a GPU cluster, or serverless infrastructure that costs nearly nothing when idle. It's not tied to your laptop — talk to it from Telegram while it works on a cloud VM.

Use any model you want — Nous Portal, OpenRouter, OpenAI, your own endpoint, and many others. Switch with hermes model — no code changes, no lock-in.

A real terminal interfaceFull TUI with multiline editing, slash-command autocomplete, conversation history, interrupt-and-redirect, and streaming tool output.
Lives where you doTelegram, Discord, Slack, WhatsApp, Signal, and CLI — all from a single gateway process. Voice memo transcription, cross-platform conversation continuity.
A closed learning loopAgent-curated memory with periodic nudges. Autonomous skill creation after complex tasks. Skills self-improve during use. FTS5 session search with LLM summarization for cross-session recall. Honcho dialectic user modeling. Compatible with the agentskills.io open standard.
Scheduled automationsBuilt-in cron scheduler with delivery to any platform. Daily reports, nightly backups, weekly audits — all in natural language, running unattended.
Delegates and parallelizesSpawn isolated subagents for parallel workstreams. Write Python scripts that call tools via RPC, collapsing multi-step pipelines into zero-context-cost turns.
Runs anywhere, not just your laptopSeven terminal backends — local, Docker, SSH, Singularity, Modal, Daytona, and Vercel Sandbox. Daytona and Modal offer serverless persistence — your agent's environment hibernates when idle and wakes on demand, costing nearly nothing between sessions. Run it on a $5 VPS or a GPU cluster.
Research-readyBatch trajectory generation, trajectory compression for training the next generation of tool-calling models.

Quick Install

Linux, macOS, WSL2, Termux

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

Windows (native, PowerShell)

Heads up: Native Windows runs Hermes without WSL — CLI, gateway, TUI, and tools all work natively. If you'd rather use WSL2, the Linux/macOS one-liner above works there too. Found a bug? Please file issues.

Run this in PowerShell:

iex (irm https://hermes-agent.nousresearch.com/install.ps1)

The installer handles everything: uv, Python 3.11, Node.js, ripgrep, ffmpeg, and a portable Git Bash (MinGit, unpacked to %LOCALAPPDATA%\hermes\git — no admin required, completely isolated from any system Git install). Hermes uses this bundled Git Bash to run shell commands.

If you already have Git installed, the installer detects it and uses that instead. Otherwise a ~45MB MinGit download is all you need — it won't touch or interfere with any system Git.

Android / Termux: The tested manual path is documented in the Termux guide. On Termux, Hermes installs a curated .[termux] extra because the full .[all] extra currently pulls Android-incompatible voice dependencies.

Windows: Native Windows is fully supported — the PowerShell one-liner above installs everything. If you'd rather use WSL2, the Linux command works there too. Native Windows install lives under %LOCALAPPDATA%\hermes; WSL2 installs under ~/.hermes as on Linux.

After installation:

source ~/.bashrc    # reload shell (or: source ~/.zshrc)
hermes              # start chatting!

Troubleshooting

Windows Defender or antivirus flags uv.exe as malware

If your antivirus (Bitdefender, Windows Defender, etc.) quarantines uv.exe from the Hermes bin folder (%LOCALAPPDATA%\hermes\bin\uv.exe), this is a false positive. The file is Astral's uv — the Rust Python package manager Hermes bundles to manage its Python environment. ML-based antivirus engines commonly flag unsigned Rust binaries that download and install packages.

To verify your copy is authentic:

# Install GitHub CLI if needed
winget install --id GitHub.cli

# Login to GitHub
gh auth login

# Run verification
$uv = "$env:LOCALAPPDATA\hermes\bin\uv.exe"
$ver = (& $uv --version).Split(' ')[1]
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$zip = "$env:TEMP\uv.zip"
Invoke-WebRequest "https://github.com/astral-sh/uv/releases/download/$ver/uv-x86_64-pc-windows-msvc.zip" -OutFile $zip -UseBasicParsing
gh attestation verify $zip --repo astral-sh/uv
Expand-Archive $zip "$env:TEMP\uv_x" -Force
(Get-FileHash "$env:TEMP\uv_x\uv.exe").Hash -eq (Get-FileHash $uv).Hash

If attestation says "Verification succeeded" and the last line prints True, you're good.

To whitelist Hermes:

  • Windows Defender: Run PowerShell as Admin → Add-MpPreference -ExclusionPath "$env:LOCALAPPDATA\hermes\bin"
  • Bitdefender: Add an exception in the Bitdefender console (Protection > Antivirus > Settings > Manage Exceptions)
  • Whitelist the folder, not the file hash — Hermes updates uv and the hash changes every version

For more context, see the upstream Astral reports: astral-sh/uv#13553, astral-sh/uv#15011, astral-sh/uv#10079.


Getting Started

hermes              # Interactive CLI — start a conversation
hermes model        # Choose your LLM provider and model
hermes tools        # Configure which tools are enabled
hermes config set   # Set individual config values
hermes config get   # Print individual config values
hermes gateway      # Start the messaging gateway (Telegram, Discord, etc.)
hermes setup        # Run the full setup wizard (configures everything at once)
hermes claw migrate # Migrate from OpenClaw (if coming from OpenClaw)
hermes update       # Update to the latest version
hermes doctor       # Diagnose any issues

📖 Full documentation →


Skip the API-key collection — Nous Portal

Hermes works with whatever provider you want — that's not changing. But if you'd rather not collect five separate API keys for the model, web search, image generation, TTS, and a cloud browser, Nous Portal covers all of them under one subscription:

  • 300+ models — pick any of them with /model <name>
  • Tool Gateway — web search (Firecrawl), image generation (FAL), text-to-speech (OpenAI), cloud browser (Browser Use), all routed through your sub. No extra accounts.

One command from a fresh install:

hermes setup --portal

That logs you in via OAuth, sets Nous as your provider, and turns on the Tool Gateway. Check what's wired up any time with hermes portal info. Full details on the Tool Gateway docs page.

You can still bring your own keys per-tool whenever you want — the gateway is per-backend, not all-or-nothing.


CLI vs Messaging Quick Reference

Hermes has two entry points: start the terminal UI with hermes, or run the gateway and talk to it from Telegram, Discord, Slack, WhatsApp, Signal, or Email. Once you're in a conversation, many slash commands are shared across both interfaces.

Action CLI Messaging platforms
Start chatting hermes Run hermes gateway setup + hermes gateway start, then send the bot a message
Start fresh conversation /new or /reset /new or /reset
Change model /model [provider:model] /model [provider:model]
Set a personality /personality [name] /personality [name]
Retry or undo the last turn /retry, /undo /retry, /undo
Compress context / check usage /compress, /usage, /insights [--days N] /compress, /usage, /insights [days]
Browse skills /skills or /<skill-name> /<skill-name>
Interrupt current work Ctrl+C or send a new message /stop or send a new message
Platform-specific status /platforms /status, /sethome

For the full command lists, see the CLI guide and the Messaging Gateway guide.


Documentation

All documentation lives at hermes-agent.nousresearch.com/docs:

Section What's Covered
Quickstart Install → setup → first conversation in 2 minutes
CLI Usage Commands, keybindings, personalities, sessions
Configuration Config file, providers, models, all options
Messaging Gateway Telegram, Discord, Slack, WhatsApp, Signal, Home Assistant
Security Command approval, DM pairing, container isolation
Tools & Toolsets 40+ tools, toolset system, terminal backends
Skills System Procedural memory, Skills Hub, creating skills
Memory Persistent memory, user profiles, best practices
MCP Integration Connect any MCP server for extended capabilities
Cron Scheduling Scheduled tasks with platform delivery
Context Files Project context that shapes every conversation
Architecture Project structure, agent loop, key classes
Contributing Development setup, PR process, code style
CLI Reference All commands and flags
Environment Variables Complete env var reference

Migrating from OpenClaw

If you're coming from OpenClaw, Hermes can automatically import your settings, memories, skills, and API keys.

During first-time setup: The setup wizard (hermes setup) automatically detects ~/.openclaw and offers to migrate before configuration begins.

Anytime after install:

hermes claw migrate              # Interactive migration (full preset)
hermes claw migrate --dry-run    # Preview what would be migrated
hermes claw migrate --preset user-data   # Migrate without secrets
hermes claw migrate --overwrite  # Overwrite existing conflicts

What gets imported:

  • SOUL.md — persona file
  • Memories — MEMORY.md and USER.md entries
  • Skills — user-created skills → ~/.hermes/skills/openclaw-imports/
  • Command allowlist — approval patterns
  • Messaging settings — platform configs, allowed users, working directory
  • API keys — allowlisted secrets (Telegram, OpenRouter, OpenAI, Anthropic, ElevenLabs)
  • TTS assets — workspace audio files
  • Workspace instructions — AGENTS.md (with --workspace-target)

See hermes claw migrate --help for all options, or use the openclaw-migration skill for an interactive agent-guided migration with dry-run previews.


Contributing

We welcome contributions! See the Contributing Guide for development setup, code style, and PR process.

Quick start for contributors — use the standard installer, then work from the full git checkout it creates at $HERMES_HOME/hermes-agent (usually ~/.hermes/hermes-agent). This matches the layout used by hermes update, the managed venv, lazy dependencies, gateway, and docs tooling.

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
cd "${HERMES_HOME:-$HOME/.hermes}/hermes-agent"
uv pip install -e ".[all,dev]"
scripts/run_tests.sh

Manual clone fallback (for throwaway clones/CI where you intentionally do not want the managed install layout):

Create the venv outside the cloned source tree — a venv inside the directory the agent operates from can be wiped by a relative-path command the agent runs against its own checkout, destroying the running runtime mid-session.

curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv ~/.hermes/venvs/hermes-dev --python 3.11
source ~/.hermes/venvs/hermes-dev/bin/activate
uv pip install -e ".[all,dev]"
scripts/run_tests.sh

Community

  • 💬 Discord
  • 📚 Skills Hub
  • 🐛 Issues
  • 🔌 computer-use-linux — Linux desktop-control MCP server for Hermes and other MCP hosts, with AT-SPI accessibility trees, Wayland/X11 input, screenshots, and compositor window targeting.
  • 🔌 HermesClaw — Community WeChat bridge: Run Hermes Agent and OpenClaw on the same WeChat account.

License

MIT — see LICENSE.

Built by Nous Research.

S
Description
No description provided
Readme MIT 697 MiB
Languages
Python 73.8%
TypeScript 23.2%
JavaScript 0.9%
Shell 0.4%
PowerShell 0.4%
Other 1.2%