Run models locally as a first-class provider. The CLI grows a managed
llama.cpp runtime (engine install, model download, server supervision);
the desktop app grows the full setup and management story on top of it.
GUI surfaces ship behind the desktop --local launch flag (hermes desktop
--local, or the flag on the packaged app); backend routes and the CLI
are always live.
Runtime (hermes_cli/local_runtime/):
- curated GGUF catalog with per-machine variant selection: hardware
probe (VRAM/RAM/UMA), fit planning with spill accounting, quant choice
by context window
- derived recommendation: quality-ranked picks gated by a predicted
decode-speed floor, bandwidth-aware on unified memory; the decision
table is pinned as a test (pick AND reason per memory class), and the
Recommended badge explains its pick in a tooltip fed by the resolver's
actual branch
- engine install + model download with resumable split parts, cumulative
plan-level progress, and staged-model integrity (a split GGUF counts
only when every part is present)
- server supervision: spawn/adopt/stop, router mode with per-model load
progress relayed over SSE, abandoned-request cleanup
Desktop:
- Settings -> Providers -> Local models: one-click quickstart (install
engine, download the recommended model, boot) plus per-model download/
activate/eject, fit-ranked catalog with context pills
- model pickers (composer dropdown + Cmd+K) show staged local models,
in-flight downloads as live progress rows, and load-into-memory bars
- local-setup campaign tip for eligible hardware; System resources
statusbar widget (GPU/VRAM/RAM); in-chat load progress during sends
- friendly dead-server errors, and failed agent builds retry on the next
send instead of wedging the session
Co-developed with NVIDIA field feedback on RTX 5090 and DGX Spark.
Salvaged from PR #84199 by @RickyYii. DirectAlias gains api_key/key_env; the direct-alias override re-resolves credentials against the alias endpoint (host-gated, #28660) and reuses the pre-alias key only on an origin match; oneshot -m <alias> passes the alias key as explicit_api_key; direct-alias branch gains the OLLAMA_API_KEY host gate. Fixes#83612.
Forward normalized custom-provider capabilities on the default gateway path so native compaction does not depend on session rehydration. Document the content trust boundary and cover both lookup and gateway resolution.
Apply explicit per-model prompt_caching capabilities to custom
chat-completions routes, rather than limiting them to recognized providers,
hosts, or model families.
Keep undeclared routes conservative, derive the marker layout from the wire
transport, and leave Responses and Bedrock caching paths unchanged.
Fold the list/mapping parser INSIDE the existing string-typed-value coercion guard (the `not isinstance(_default_value_for_key(key), str)` block from e4ea0a0ed) instead of running it unconditionally, so a genuinely string-typed setting whose value merely starts with '[' or '{' is left untouched while non-string keys get JSON/YAML flow literals parsed to real lists/dicts.
Update website/docs/user-guide/configuring-models.md: the `config set only writes scalar values` note is no longer accurate; document the list/mapping support with a quoted example.
Fixes#40545#50168
Fixes#67278. Since config v12 (see #8776), custom_providers: list is
legacy — the migration converts it to the providers: dict and the
resolver reads the dict first (runtime_provider.py). Docs still taught
the legacy list as primary.
- integrations/providers.md: all 8 YAML examples converted to
providers: dict (field mapping code-verified: api, default_model,
transport), one consolidated legacy-format note
- configuring-models.md, configuration.md, credential-pools.md,
migrate-from-openclaw.md, provider-runtime.md, faq.md: examples and
prose flipped to dict-first; legacy list noted as still-read
- adding-providers.md untouched (sole mention is a literal test
filename)
Fixes the two review defects that kept PR #29923 open, plus docs:
- gateway: exclude --once from the session-store write-through. The
once-override lived only in memory before, but the write-through
persisted it, so a gateway restart before the finally-restore
rehydrated a supposedly one-turn model permanently.
- TUI: skip _sync_agent_model_with_config while a one-turn restore is
pending. The once-model is deliberately not pinned as a session
model_override, so the config sync saw a model mismatch and clobbered
the once-override back to the config model before the turn ran.
- tests: real _handle_model_command drive asserting --once never
touches set_model_override while --session still does; restore-pop
idempotency.
- docs: /model --once in configuring-models.md with an honest
prompt-cache cost note (one-shot switch breaks the cached prefix
twice; wins for short sessions and cheap-to-expensive escalation).
Cross-checked website/docs against the source at main HEAD and corrected
documented commands, env vars, config keys, headers, and default values
that don't match the code. Docs-only; no behavioral changes.
Refs #36048
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
/model switches, primary-model fallback, and credential-pool key
rotation all change the prompt-cache key (model and/or account), so
the next turn re-reads the entire conversation at full input price.
Add cost warnings everywhere docs recommend or describe these paths:
- reference/slash-commands.md: cost note on both /model rows
- user-guide/features/fallback-providers.md: warning admonition
- user-guide/features/credential-pools.md: warning admonition
- user-guide/configuring-models.md: mid-session switch warning
- guides/tips.md: expand cache tip + /model tip
- reference/faq.md: warning on the switch-back-and-forth example
- user-guide/desktop.md: composer picker bullet
- developer-guide/context-compression-and-caching.md: new
cache-aware design pattern (model identity is part of the key)
Adds hermes_cli/context_switch_guard.py mirroring the model_cost_guard
pattern. When a user switches models mid-session (Herm TUI picker, CLI,
or /model on Telegram/Discord), the warning surfaces on the existing
ModelSwitchResult.warning_message path used by the expensive-model
guard if the new model's compression threshold is below the current
session size.
Partial fix for #23767 — addresses only the 'user-facing guardrail
when switching from a high-context provider to a substantially
lower-context provider' slice. The other proposed fixes from that
issue (hard preflight token guard, metadata cache invalidation on
switch, compression safety invariant, oversized tool-output handling)
are out of scope for this PR.
`hermes portal` (no subcommand) now runs the one-shot Nous Portal onboarding
— OAuth login, switch provider to Nous, offer Tool Gateway — identical to
`hermes setup --portal` and the human-readable alias for
`hermes auth add nous --type oauth` (which still works).
The prior status default moves to `hermes portal info`; `status` is kept as a
hidden back-compat alias. `open`/`tools` subcommands are unchanged.
User-facing hints and docs (status.py, conversation_loop 401 guidance,
SystemPage, README, website docs + zh-Hans) now point at `hermes portal` /
`hermes portal info`. `--manual-paste` references keep the explicit auth
command since `hermes portal` does not expose that flag.
'hermes config get <key>' is referenced in three guides but is not a
valid subcommand. The valid subcommands under 'hermes config' are
{show,edit,set,path,env-path,check,migrate}. 'hermes config show' is
already used elsewhere in the docs (including 'hermes config show |
grep <pattern>' in the FAQ), so it's the idiomatic replacement.
- work-with-skills.md: 'View all skill config' now uses
'hermes config show | grep ^skills\.config'
- migrate-from-openclaw.md: session-policy check now reads the value
from 'hermes config show'
- configuring-models.md: 'inspect what the CLI will actually use'
now uses 'hermes config show | grep ^model\.'
Refs #30195
Kimi K2.6 is natively multimodal — flagged by Shengyuan from the Kimi
growth team. Replace the named-vendor example with a model-agnostic
phrasing so the row doesn't go stale as more vendors ship vision.
The locale switcher appeared broken because hardcoded markdown links
(`](/docs/X)`) got double-prefixed by Docusaurus to `/docs/<locale>/docs/X`
(404) in non-English locales, and the MDX hero `<a href>` on the index
page escaped locale routing entirely.
Changes:
- Rewrite 922 `](/docs/X)` -> `](/X)` across 166 docs files (strip trailing
.md too). Docusaurus prepends locale + baseUrl itself.
- docs/index.md -> index.mdx; hero "Get Started" anchor -> Docusaurus
<Link> so it stays inside the active locale.
- Drop `ko` locale entirely from docusaurus.config.ts + delete i18n/ko/
(4 stale auto-translated kanban pages, <2% coverage, misleading).
Verified `npm run build` succeeds for both en and zh-Hans; `build/zh-Hans/
index.html` has no /docs/zh-Hans/docs/... double-prefixed paths.
PR2 will translate the 335 English docs into i18n/zh-Hans/.
Follow-up to #30869. Adds Portal mentions on user-facing pages that
naturally call for an LLM + tool credentials but didn't previously
acknowledge Portal as a one-stop option.
- getting-started/installation.md: tip after the 'after install' block
pointing at 'hermes setup --portal' for users who want everything wired
at once instead of piecewise via 'hermes model' + 'hermes tools'.
- user-guide/configuring-models.md: small tip near the top — the page is
literally about provider/model choice and previously had zero Portal
mention.
- user-guide/features/voice-mode.md: Prerequisites need both an LLM and
TTS — a Portal subscription is the single setup that covers both.
- user-guide/features/batch-processing.md: highlights Portal as a
predictable-cost option for parallel agent runs that hit many APIs.
- user-guide/features/api-server.md: backend needs models + tools; one
Portal sub gives a fully-equipped OpenAI-compatible endpoint.
- user-guide/windows-native.md: early-beta users on Windows benefit most
from skipping per-tool Windows-key-juggling.
- integrations/providers.md: updates the existing Tool Gateway tip and
the Nous Portal section to mention the new commands.
- user-guide/features/fallback-providers.md: Nous row in the provider
table now lists 'hermes setup --portal' as the fresh-install path.
Tone discipline: one Portal mention per page, concrete CLI commands
(no marketing copy), always solving a problem the page itself sets up.
Companion PR to #27590. Sweeps remaining stale references to the
LLM-summary path that landed in main with #27590 but weren't fully
caught in the followup cleanup commit.
Real rewrites:
- user-guide/sessions.md: 'Session Search Tool' section rewritten to
describe the three calling shapes (discovery / scroll / browse) with
worked examples. Adds the 'Optional parameters' subsection covering
sort and role_filter.
- user-guide/features/memory.md: 'Session Search' overview rewritten,
comparison table updated (speed: ms instead of LLM summarization,
added explicit free-cost row, link to sessions.md for details).
Stale-claim sweeps:
- user-guide/configuring-models.md: drop the 'Session Search' row from
the aux-model override table (no aux model anymore), drop session
search from the auxiliary-models list.
- user-guide/features/codex-app-server-runtime.md: drop session_search
from the ChatGPT-subscription cost note, drop the session_search
block from the per-task override config example.
- developer-guide/provider-runtime.md: drop 'session search
summarization' from the auxiliary tasks list.
- developer-guide/agent-loop.md: drop session search from the
auxiliary fallback chain list.
- user-guide/skills/.../autonomous-ai-agents-hermes-agent.md: drop
session_search from the 'auxiliary models not working' debug step.
Untouched (still accurate as tool-name mentions, not behavioral claims):
- features/tools.md, features/honcho.md, features/acp.md
- cli.md, sessions.md (other sections)
- developer-guide/tools-runtime.md, agent-loop.md (line 157)
- acp-internals.md, adding-tools.md, prompt-assembly.md
- reference/toolsets-reference.md, reference/tools-reference.md
* docs: deep audit — fix stale config keys, missing commands, and registry drift
Cross-checked ~80 high-impact docs pages (getting-started, reference, top-level
user-guide, user-guide/features) against the live registries:
hermes_cli/commands.py COMMAND_REGISTRY (slash commands)
hermes_cli/auth.py PROVIDER_REGISTRY (providers)
hermes_cli/config.py DEFAULT_CONFIG (config keys)
toolsets.py TOOLSETS (toolsets)
tools/registry.py get_all_tool_names() (tools)
python -m hermes_cli.main <subcmd> --help (CLI args)
reference/
- cli-commands.md: drop duplicate hermes fallback row + duplicate section,
add stepfun/lmstudio to --provider enum, expand auth/mcp/curator subcommand
lists to match --help output (status/logout/spotify, login, archive/prune/
list-archived).
- slash-commands.md: add missing /sessions and /reload-skills entries +
correct the cross-platform Notes line.
- tools-reference.md: drop bogus '68 tools' headline, drop fictional
'browser-cdp toolset' (these tools live in 'browser' and are runtime-gated),
add missing 'kanban' and 'video' toolset sections, fix MCP example to use
the real mcp_<server>_<tool> prefix.
- toolsets-reference.md: list browser_cdp/browser_dialog inside the 'browser'
row, add missing 'kanban' and 'video' toolset rows, drop the stale
'38 tools' count for hermes-cli.
- profile-commands.md: add missing install/update/info subcommands, document
fish completion.
- environment-variables.md: dedupe GMI_API_KEY/GMI_BASE_URL rows (kept the
one with the correct gmi-serving.com default).
- faq.md: Anthropic/Google/OpenAI examples — direct providers exist (not just
via OpenRouter), refresh the OpenAI model list.
getting-started/
- installation.md: PortableGit (not MinGit) is what the Windows installer
fetches; document the 32-bit MinGit fallback.
- installation.md / termux.md: installer prefers .[termux-all] then falls
back to .[termux].
- nix-setup.md: Python 3.12 (not 3.11), Node.js 22 (not 20); fix invalid
'nix flake update --flake' invocation.
- updating.md: 'hermes backup restore --state pre-update' doesn't exist —
point at the snapshot/quick-snapshot flow; correct config key
'updates.pre_update_backup' (was 'update.backup').
user-guide/
- configuration.md: api_max_retries default 3 (not 2); display.runtime_footer
is the real key (not display.runtime_metadata_footer); checkpoints defaults
enabled=false / max_snapshots=20 (not true / 50).
- configuring-models.md: 'hermes model list' / 'hermes model set ...' don't
exist — hermes model is interactive only.
- tui.md: busy_indicator -> tui_status_indicator with values
kaomoji|emoji|unicode|ascii (not kawaii|minimal|dots|wings|none).
- security.md: SSH backend keys (TERMINAL_SSH_HOST/USER/KEY) live in .env,
not config.yaml.
- windows-wsl-quickstart.md: there is no 'hermes api' subcommand — the
OpenAI-compatible API server runs inside hermes gateway.
user-guide/features/
- computer-use.md: approvals.mode (not security.approval_level); fix broken
./browser-use.md link to ./browser.md.
- fallback-providers.md: top-level fallback_providers (not
model.fallback_providers); the picker is subcommand-based, not modal.
- api-server.md: API_SERVER_* are env vars — write to per-profile .env,
not 'hermes config set' which targets YAML.
- web-search.md: drop web_crawl as a registered tool (it isn't); deep-crawl
modes are exposed through web_extract.
- kanban.md: failure_limit default is 2, not '~5'.
- plugins.md: drop hard-coded '33 providers' count.
- honcho.md: fix unclosed quote in echo HONCHO_API_KEY snippet; document
that 'hermes honcho' subcommand is gated on memory.provider=honcho;
reconcile subcommand list with actual --help output.
- memory-providers.md: legacy 'hermes honcho setup' redirect documented.
Verified via 'npm run build' — site builds cleanly; broken-link count went
from 149 to 146 (no regressions, fixed a few in passing).
* docs: round 2 audit fixes + regenerate skill catalogs
Follow-up to the previous commit on this branch:
Round 2 manual fixes:
- quickstart.md: KIMI_CODING_API_KEY mentioned alongside KIMI_API_KEY;
voice-mode and ACP install commands rewritten — bare 'pip install ...'
doesn't work for curl-installed setups (no pip on PATH, not in repo
dir); replaced with 'cd ~/.hermes/hermes-agent && uv pip install -e
".[voice]"'. ACP already ships in [all] so the curl install includes it.
- cli.md / configuration.md: 'auxiliary.compression.model' shown as
'google/gemini-3-flash-preview' (the doc's own claimed default);
actual default is empty (= use main model). Reworded as 'leave empty
(default) or pin a cheap model'.
- built-in-plugins.md: added the bundled 'kanban/dashboard' plugin row
that was missing from the table.
Regenerated skill catalogs:
- ran website/scripts/generate-skill-docs.py to refresh all 163 per-skill
pages and both reference catalogs (skills-catalog.md,
optional-skills-catalog.md). This adds the entries that were genuinely
missing — productivity/teams-meeting-pipeline (bundled),
optional/finance/* (entire category — 7 skills:
3-statement-model, comps-analysis, dcf-model, excel-author, lbo-model,
merger-model, pptx-author), creative/hyperframes,
creative/kanban-video-orchestrator, devops/watchers,
productivity/shop-app, research/searxng-search,
apple/macos-computer-use — and rewrites every other per-skill page from
the current SKILL.md. Most diffs are tiny (one line of refreshed
metadata).
Validation:
- 'npm run build' succeeded.
- Broken-link count moved 146 -> 155 — the +9 are zh-Hans translation
shells that lag every newly-added skill page (pre-existing pattern).
No regressions on any en/ page.
User-defined model aliases (config.yaml model_aliases: and
model.aliases.*) have worked since early versions but were entirely
undocumented. Add a dedicated 'Custom model aliases' section to
slash-commands.md covering both YAML config formats and the
'hermes config set' shell form, mirror a shorter version into the
configuring-models 'Alternative methods' section, and cross-link from
the two /model table rows.
Flagged by @weehowe on Twitter — he wasn't aware the feature existed.
Dashboard Models page was analytics-only — no way to pick a model as main
for new sessions or override an auxiliary task slot without hand-editing
config.yaml or running a /model slash command inside a chat.
Changes:
- hermes_cli/web_server.py: three REST endpoints (GET /api/model/options,
GET /api/model/auxiliary, POST /api/model/set). Reuses
list_authenticated_providers() from model_switch.py so the REST path
surfaces the same curated model lists as the TUI-gateway model.options
JSON-RPC. POST /api/model/set writes model.provider + model.default for
scope=main, and auxiliary.<task>.{provider,model} for scope=auxiliary
(with task="" meaning 'all 8 slots' and task="__reset__" resetting them
to auto).
- web/src/components/ModelPickerDialog.tsx: accepts an optional loader +
onApply pair so it works without an open chat PTY. ChatSidebar's
gw-WebSocket path still works unchanged (back-compat).
- web/src/pages/ModelsPage.tsx: Model Settings panel at the top showing
main model + collapsible list of 8 auxiliary tasks with per-row Change
buttons and Reset all to auto. Every existing model card gets a
'Use as' dropdown for one-click assignment to main or any aux slot.
Cards badged 'main' or 'aux · <task>' when currently assigned.
- website/docs/user-guide/configuring-models.md: new docs page walking
through both UI paths, aux task override patterns, troubleshooting,
plus REST/CLI alternatives.
- Screenshots under website/static/img/docs/dashboard-models/.
Applies to new sessions only — running sessions keep their model (use
/model slash command to hot-swap a live session). No prompt-cache
invalidation on existing sessions.
emozilla