Free ($0/$0) Nous Portal models sat with a blank discount column and no
sale star (stealth/ox-alpha, upstage/solar-pro4:free), reading as missing
data next to the -20% sale rows. compute_sale_discount now returns a flat
100% for free models; was_* raws pass through only when the gateway served
a pricing.original, so natively-free models render bare '-100%' with no
fabricated 'was ?/?'. CLI picker star follows on_sale automatically;
inventory feed carries discount_percent=100 to Desktop, whose FREE badge
row now renders the amber -100% pill beside it.
GLM-5.3 is live on api.z.ai (coding plan endpoint) but had no entries in
Hermes, so it silently fell back to the generic 202K GLM context —
triggering premature context compression on a 1M-window model.
- model_metadata: 'glm-5.3': 1_048_576 (same base model as 5.2; 1M
context / 128K max output per docs.z.ai/guides/llm/glm-5.3, verified
2026-08-14)
- auth: add glm-5.3 to coding-plan probe lists (global + CN)
- models: add glm-5.3 to picker/model lists (6 sites)
- zai provider: reasoning_effort mapping covers glm-5.3 (accepted live
by the endpoint, HTTP 200)
A keyless provider has no credential to lack, but every auth-gated
surface treated 'no key' as 'not authenticated', so opencode-free was
invisible in /model, provider:model listing, and the desktop model
pickers unless a user had unrelated OpenCode env vars set.
One policy, three gates, all derived from the HermesOverlay keyless
flag (#91358):
- auth.py get_api_key_provider_status: keyless providers report
configured/logged_in=True with key_source 'keyless' — flows through
get_auth_status to every status consumer (hermes status, dashboards,
list_available_providers).
- model_switch.py list_authenticated_providers: keyless overlay rows
get has_creds=True before any env/pool/auth-store checks — this is
the source for /model, the TUI picker, and the desktop
/api/model/options payload.
- inventory.py explicit-only filter (desktop chat pickers): keyless
providers are kept — there is nothing to 'explicitly configure', and
hiding a zero-setup provider defeats its purpose.
E2E (temp HERMES_HOME, all keys stripped): get_auth_status logged_in,
list_available_providers authenticated, picker row with 6 models,
desktop payload default AND explicit_only both include the provider,
and the full switch pipeline (parse free:x-preview-f-free →
switch_model) resolves to the keyless runtime. 4 new tests.
Reworks the salvaged OpenCode Free provider to match the tier's real
auth contract (verified live 2026-08-21): the Zen relay serves free
models ANONYMOUSLY and 401s any unrecognized bearer, so the provider now
declares no credentials at all and routes every model through the shared
keyless machinery from the Ox Alpha fix (empty Authorization default
header overriding the SDK bearer).
On top of the salvaged base:
- auth.py: no api_key_env_vars; drop the keyed-auth special case
- runtime_provider.py: restore the plain fail-closed path (opencode-free
never reaches it — the keyless runtime resolves first)
- models.py: opencode-free joins the opencode family (prefix stripping,
Zen endpoint routing incl. muse->responses); keyless predicate extended
with unsuffixed free slugs (big-pickle); free runtime pins EVERY
opencode-free model keyless; curated catalog replaces the models.dev
cost==0 filter (it lags reality: deepseek-v4-flash-free stayed 'free'
there after its promo ended and the relay began 401ing it — delisted)
- agent_runtime_helpers.py: replace the httpx transport-sharing auth-strip
wrapper with the shared header policy (no proxy-mount loss)
- model_setup_flows.py: skip the API-key prompt for opencode-free
- plugin profile: keyless headers, no env vars
- .env.example + providers.md: keyless docs (no OPENCODE_FREE_API_KEY)
- tests rewritten to the keyless contract, incl. catalog-membership
invariant (every curated model must satisfy the keyless predicate)
E2E: full AIAgent turns with zero keys complete on x-preview-f-free via
provider opencode-free and alias 'free', incl. a real terminal tool
round-trip; muse routes to /v1/responses; picker lists 8 keyless models.
Adds an OpenCode Free provider plugin. Free model discovery uses models.dev
(cost.input == 0 AND status != "deprecated"), matching opencode CLI's exact
filter logic.
The free tier requires a real account API key and throttles third-party
clients by User-Agent:
- With OPENCODE_FREE_API_KEY configured, the key is sent as a Bearer token
and requests identify as "opencode/latest".
- Without a key, the keyless fallback strips the SDK's always-injected empty
Authorization header and still sends the opencode User-Agent.
- The credential resolver no longer blanks OPENCODE_FREE_API_KEY
unconditionally (the stale keyless-tier assumption), and credential-pool
exhaustion no longer surfaces the misleading "Set OPENCODE_FREE_API_KEY"
message.
Co-authored-by: Jean-François <jfm@laposte.net>
Signed-off-by: Rudraksh Chahal <131520192+rudrakshchahal@users.noreply.github.com>
Widens #90327's line_input() to the whole bug class: all 46 bare input()
free-text prompt sites across the setup wizards (model_setup_flows, setup,
config, gateway, auth, auth_commands, plugins_cmd, skills_hub, bundles,
setup_whatsapp_cloud, main) now route through line_input(), and the shared
cli_output.prompt() / setup.prompt() helpers do too — so every CLI wizard
gets cursor editing, not just the custom model prompt.
Redirected stdin and missing prompt_toolkit keep builtin input() behavior.
E2E: real PTY with raw escape bytes through line_input, cli_output.prompt,
and setup.prompt (arrows + Ctrl+A/E edit correctly); redirected-stdin
fallback verified.
The previous except Exception silently fell back to os.getenv if the
_scoped_key_env import ever failed — under multiplex that is exactly the
fail-open this PR removes (secondary profiles would regress to 'No LLM
provider configured' with zero trace). Catch only ImportError, log a
WARNING naming the consequence, and let any other failure propagate.
Also replaces the lambda fallback with a named nested function.
resolve_provider's auto path read provider API keys with bare
os.getenv — under multiplex a secondary profile's keys live only in its
secret scope, so auto-detection found nothing and every secondary
profile with model.provider: auto failed with 'No LLM provider
configured' at agent init (reproduced on a live 7-profile gateway).
Route both env-key reads (the OPENAI/OPENROUTER tier and the
PROVIDER_REGISTRY loop) through _scoped_key_env, the scope-aware helper
auxiliary_client already uses: secret scope wins under multiplex,
UnscopedSecretError falls back to os.environ (default-profile/CLI
paths unchanged). Same bug class as #86905.
Verified in a gateway-accurate simulation (hermes_home_override +
profile scope): resolve_provider('auto') now returns the secondary
profile's own provider (deepseek) instead of erroring.
get_default_hermes_root() resolves HERMES_HOME against the platform
native home (~80us of path resolution) on EVERY call and is called at
31+ sites — every _load_global_auth_store() (per provider row in the
/model picker), kanban, backup, gateway, update. Its result depends
only on (HERMES_HOME, native home), so memoise it keyed on those two
inputs, compared for free on each call (freshness-correct even if a
test or plugin mutates HERMES_HOME mid-process).
_load_global_auth_store() re-read + re-parsed the global auth.json on
every call; read_credential_pool() -> load_pool() runs it once per
provider row in the /model picker even when the profile has entries and
the global fallback never fires. Memoise keyed on the global auth
file's path+mtime (same pattern as _nous_auth_status_cache); the store
only changes when a global-scope auth write touches the file.
Measured (profile mode, 30-provider global store): get_default_hermes_root
81us -> 10us; _load_global_auth_store 128us -> 66us; load_pool 165us ->
137us per call — ~2ms saved per /model picker render (20 provider rows).
Regression tests: hermes_constants memo pin (no path resolution on
repeat calls, HERMES_HOME change forces a fresh resolution); global-store
memo pins (store read once across repeats, mtime bump re-reads once,
absent store stays cheap).
(cherry picked from commit be348f32e5bd7479c26fabb652de549fb9c8a1e1)
On an Anthropic subscription OAuth credential, every request failed with
HTTP 400 "You're out of extra usage. Add more at claude.ai/settings/usage".
That is not a billing condition: Anthropic's server-side content filter rejects
the first sentence of Hermes' own built-in SKILLS_GUIDANCE prompt, and the
rejection is surfaced with a billing-shaped message. Because the message points
at the usage settings page, it reliably sends people to buy quota they do not
need — the reporter lost three debugging sessions to it.
Bisected against the live API with the real 71,721-char assembled prompt: the
first SKILLS_GUIDANCE sentence alone reproduces the 400 and removing it alone
clears it. Size was ruled out (20 KB of unrelated filler returns 200) and so was
the system[0] identity gate (that returns 429, a different failure).
Three changes, all serving the same outcome — a subscription user can no longer
be misdirected by this 400:
- agent/prompt_builder.py: reword the triggering sentence to the phrasing the
reporter verified returns 200. Meaning, the skill_manage reference, and the
## Skill Safety Rule block are all preserved. The reword is empirically
validated rather than understood, so a comment records the bisect and warns
that any rewrite must be re-verified against an OAuth token, not an API key.
- agent/conversation_loop.py: the Anthropic branch of the billing guidance no
longer asserts exhaustion as fact. It hedges the opening line, names the
content-filter alternative, and gives the operator a way to tell the two apart
(if the usage page still shows quota, suspect a content rejection). It also
points at `hermes auth reset anthropic`, because the credential exhaustion
latch replays the stored error for ~60 min without issuing a request — which
makes a real fix look like it did not work.
- hermes_cli/auth.py: document that CLAUDE_CODE_OAUTH_TOKEN is an OAuth token,
not an API key, despite auth_type="api_key". It stays in api_key_env_vars
because that tuple doubles as the credential-discovery list; removing it would
stop Hermes finding a `claude setup-token` credential at all.
Docs updated to match the reworded prompt.
Fixes#82154
Adds hermes_cli/model_selection_guards.py: a single evaluation point that
runs every selection guard (cost + the new data-policy guard) and returns
the warnings that fired. All seven model-selection surfaces (CLI picker,
cli.py TUI modal, gateway typed /model, dashboard web_server, TUI gateway,
Telegram and Discord pickers) now call the registry instead of importing
model_cost_guard directly — so the data-training-tier warning from
PR #81416 fires everywhere at once, and future guards need zero surface
wiring.
Guard modules keep their public APIs; existing mock patch points
(hermes_cli.model_cost_guard.expensive_model_warning) remain valid.
muse-spark-1.2-contributor is heavily discounted BECAUSE Meta uses your
prompts and completions to train future models. Selecting it for the price
without realising the data trade-off is a footgun.
Add hermes_cli/model_data_policy_guard.py (mirrors model_cost_guard):
data_training_warning(model_id, provider, base_url) -> DataTrainingWarning|None,
driven by a vendor-agnostic rule table. The status is not machine-readable on
/v1/models or models.dev, so the v1 rule keys on the documented '-contributor'
model id (fires regardless of provider, so it also covers custom/gateway
routes). Message mirrors Meta's pricing-doc language and figures
(https://dev.meta.ai/docs/pricing-rate-limits/).
Wire it into the CLI model picker's confirm flow (auth.py) as a [y/N]
disclosure, chained after the expensive-model cost guard. Fires only on the
contributor tier; silent on muse-spark-1.1/1.2 and all other models.
Cold CLI time-to-banner was ~1.8s (hermes) / ~2.8s (hermes -w). The banner
path was paying for work the session doesn't need before first input:
- aux availability probes built REAL OpenAI/httpx clients (openai import
~0.3s + SSL context) just to answer check_fns. New aux_probe_mode()
returns a cache-excluded stub; resolution policy unchanged.
- tools/mcp_tool imported the mcp SDK (~260ms, mcp.types pydantic model
construction) at module import even with zero MCP servers configured.
SDK import is now lazy behind _ensure_mcp_sdk(); _MCP_AVAILABLE is a
find_spec probe so every existing gate/test keeps its semantics.
- banner blocked 500ms on the update-check prefetch; now waits 50ms and
defers the warning line to a daemon thread (prints above the prompt).
- banner recomputed get_tool_definitions + skills scan + git state every
launch; now snapshotted to ~/.hermes/cache/banner_snapshot.json keyed on
(config.yaml, .env, checkout rev, toolsets) and replayed on warm launches
with a background refresh. Agent tool list is still computed fresh.
- _resolve_active_context_length probed the Nous portal /models (~200ms
network) per launch; the tool-search gate now prefers the on-disk
context cache when present.
- schema reconciliation re-executed SCHEMA_SQL in a scratch SQLite DB
(~85ms) per SessionDB(); the reference parse is now disk-memoized by
DDL hash (live-DB diffing still runs every startup).
- bundled-skills sync (~120-170ms rglob/hash) moved off the startup path
to a daemon thread; plugin discovery starts in the background and every
synchronous consumer joins via discover_plugins().
- hermes_cli.auth imported httpx eagerly (~30ms); now a lazy proxy that
test monkeypatching still reaches (setattr forwards to the real module).
- fast chat launch: unambiguous 'hermes'/'hermes chat' invocations skip
building all ~40 subcommand parsers (bails to full dispatch on anything
else, incl. container mode).
- -w path: git worktree add runs with checkout.workers=8 (0.6s→0.2s) and
overlaps HermesCLI construction; --skills preload runs in the background
and is folded in at agent init (finalize_preloaded_skills, same
fail-loud contract for fully-unknown skill lists); stale-worktree prune
moved off the banner path.
Warm results (PTY time-to-banner, 5-run): hermes 1.80s → 0.38-0.40s;
hermes -w -s hermes-agent-dev --yolo 2.82s → 0.57-0.69s.
The auth store readers (_load_auth_store, _import_codex_cli_tokens, and the
shared Nous store reader) called Path.read_text() with no encoding, so bytes
were decoded with locale.getpreferredencoding() — cp1252 on Windows. The
stores are *written* as UTF-8 (os.fdopen(..., encoding="utf-8")), so any
non-ASCII byte (a CJK or emoji credential label, an accented display name in
OAuth state) raised UnicodeDecodeError on read.
Worst case: _load_auth_store's broad except then copied the file to .corrupt
and returned an empty store, silently wiping every provider credential on the
next launch. The sibling reader at line 2161 already used
read_text(encoding="utf-8"), confirming the omission was unintentional.
Use utf-8-sig (matching the .env handling in config.py) so a BOM from a
Notepad-edited file is tolerated too.
Adds regression tests covering the UTF-8 round-trip with a non-ASCII label,
BOM tolerance, no-corrupt-on-valid-load, and that the readers pass an explicit
encoding (guard against future regressions). Verified the tests fail when the
fix is reverted.
Closes no issue — found via cross-platform code audit (the bug is not in the
issue tracker).
When the Nous Portal returns paid_service_access.allowed=false with
reason=member_spend_cap_exceeded, Hermes was falling through to the
generic 'no active subscription or usable credits' message — even
though the user has ample purchased credits and the real blocker is
an org-level per-member spend cap.
This adds a dedicated branch that surfaces the actual cause: names the
spend cap, shows the cap/spend amounts, and tells the user to ask their
org admin to raise it. Also adds member_spend_cap_exceeded to the
billing error code set so the error classifier and auth error formatter
route it through the Nous entitlement message path.
Follow-ups on the salvaged bounded-read fix:
- refresh flow: the non-200 branch reads a STREAMED body, which fails
(ReadError/StreamClosed) once the httpx.Client context has exited —
moved inside the context. Repro + regression test use a real socket
server (MockTransport buffers in memory and cannot catch this).
- truncation guard: >limit bodies end with ...[truncated] (mutation-checked
against the is_stream_consumed fallback).
- test mocks now model the streamed-read surface (is_stream_consumed,
iter_bytes, client.send) so non-200 paths exercise the real bounded read.
The as_completed drain + `with` join made the parallel version WORSE than
sequential main in the common case (first endpoint succeeds fast, others
slow/unreachable): main returned at first success, the parallel version
waited for every straggler. Now: after each completion, walk endpoints in
priority order and return as soon as a success is unbeatable (all
higher-priority probes already finished); pool uses shutdown(wait=False) so
losers drain in the background. Mutation-checked: removing the early exit
makes the new timing test fail (8.2s vs <1.5s).
Z.AI has separate billing for general vs coding plans and global vs
China endpoints. On startup, detect_zai_endpoint() probes up to 4
endpoints sequentially with 8s timeout each, taking 8-9 seconds when
the first endpoints return non-200 (rate limited) before a working one
is found.
Replace the sequential loop with concurrent.futures.ThreadPoolExecutor
to probe all 4 endpoints in parallel. Results are returned in
ZAI_ENDPOINTS priority order so the preference chain is preserved.
Benchmark on macOS M4 Max, Python 3.11, Hermes v0.8.0:
Before: 8.8s (sequential: global=0.9s/429, cn=1.6s/429,
coding-global=4.3s/200, coding-cn=2.0s/200)
After: ~4.5s (single round-trip, bounded by slowest endpoint)
Signed-off-by: Merlin <merlin@merlin.me>
Follow-ups on the startup-burst memo:
- Populate the memo on the valid-token fast path as well. The startup
burst usually finds a VALID token, and each check_fn call still paid
two cross-process file locks + state reads to reach that return; the
original memo only engaged after a refresh. The token has at least
refresh_skew_seconds (>=120s) of life at that return, so a 5s memo can
never serve an expired token.
- Clear the module-level memo in test_nous_portal_staging_allowlist's
refresh-capture helper: with the fast-path populate, a token memoized
by an earlier test would otherwise short-circuit the refresh these
tests assert on (3 tests failed without this).
- Add dedicated memo behavior tests (TTL hit, TTL expiry, insecure
bypass) — the original PR shipped none. Mutation-checked: all 3 fail
against main's un-memoized function, pass on this branch.
check_tool_availability runs once per managed-tool check_fn (browser,
image_gen, etc.) during banner render. Each one independently triggers a
~15s blocking Nous Portal token-refresh network call when the stored token
is expired. On a slow/constrained host (e.g. a small monitoring CT) that
serial burst stretched startup to many minutes, appearing 'stalled'.
Add a per-process memo (5s TTL) so the burst collapses into a single network
round-trip. Only successful, non-forced resolutions are cached; force_fresh
and insecure/ca_bundle callers bypass and don't populate the cache, so
normal refresh semantics are unchanged.
Verified: 3 rapid resolve_nous_access_token() calls -> 1 underlying refresh.
/simplify-code review found _poll_for_token has a second caller:
web_server._nous_poller (dashboard/desktop device login), which surfaces
str(e) as the UI error_message — so wrapping only in
_nous_device_code_login left the dashboard showing the bare timeout.
Move the enrichment into _poll_for_token's deadline raise so every
caller inherits the guidance, and drop the now-redundant try/except
wrap in the CLI login. Add a source-level regression test driving the
real poll loop (authorization_pending stub client) to the deadline.
A bare 'Timed out waiting for device authorization' gives the user
nothing to act on. The most common cause is Portal sign-in failing in
the opened browser tab (including the server-side CAPTCHA loop from
issue #20605), so point at the Portal login page and the hermes portal
retry command.
Salvaged from PR #75290 by @HexLab98 (timeout-guidance kernel only).
The URL-rewrite portion of that PR was dropped: the live Portal has no
/device route (verified 404 with a real user_code), so rewriting the
manage-subscription verification URL would break login entirely.
Guidance text reworded to reference only real URLs.
_load_auth_store() treated every exception from reading auth.json as
corruption and returned an empty store. EMFILE under fd exhaustion,
EACCES, EIO and a stalled network mount all reached that branch. This
module does read-modify-write in roughly fifteen places, so the empty
store was one _save_auth_store() away from erasing every stored
credential.
Separate OSError from parse failure: a file that exists but cannot be
read now raises, naming the real cause and leaving the file on disk
untouched. Only a genuine parse failure takes the preserve-and-start-
empty branch, which is unchanged.
The backup was also unreliable in exactly the conditions that triggered
it: shutil.copy2 opens a file, so under EMFILE it failed too, its bare
except swallowed that, and the log still said "Corrupt file preserved
at ..." when nothing had been written. Track whether the copy landed
and say so accurately.
_save_xai_oauth_tokens had the identical self-sealing bug as
_sync_device_code_entry_to_auth_store: key-presence check before
_store_provider_state, which unconditionally creates the key.
Use _load_provider_state_with_source to decide write-through from
the actual grant source, not key presence.
Also update regression test per review: use the real
_write_through_provider_state_to_global_root helper and assert
rotated token pair values in the root store after each refresh
instead of just counting mock calls.
Follow-up widening of the /api/status fix: add get_nous_auth_status_local(),
a refresh-free auth-store snapshot (local invoke-JWT decode only), and use it
on the read-only display surfaces that previously called
get_nous_auth_status() -> resolve_nous_runtime_credentials() -> live OAuth
refresh POST:
- hermes_cli/status.py (hermes status auth-provider panel)
- hermes_cli/doctor.py (hermes doctor auth-provider checks)
- hermes_cli/portal_cli.py (hermes portal status display)
- hermes_cli/web_server.py /api/portal endpoint and the accounts-tab
provider card dispatcher (_resolve_provider_status nous branch)
Action paths (login flows, portal operations needing a live credential)
keep using get_nous_auth_status(). Part of NS-592.
Save side-tool OAuth tokens without promoting xai-oauth via active_provider
or model.provider so hermes setup tts login no longer hijacks inference routing.
Kimi Coding discovers the flagship as wire id `k3`. Picker search used
only that id, so typing "kimi" hid it next to every other kimi-* model.
Add picker-only search aliases without changing the wire id.
Path.read_text() and Path.write_text() without explicit encoding
default to the system locale encoding. On Windows this is typically
cp1252, which causes UnicodeDecodeError for UTF-8 content (JSON
configs, user data, service scripts).
Add encoding="utf-8" to all read_text() and write_text() calls
across 8 CLI files, matching the pattern established in PR #50534
(security_audit_startup.py) and ruff rule PLW1514.
Fixed files:
- main.py: 4 read_text calls
- auth.py: 3 read_text calls
- banner.py: 1 read_text + 1 write_text
- service_manager.py: 1 read_text + 4 write_text
- container_boot.py: 1 read_text + 4 write_text
- doctor.py: 3 read_text calls
- uninstall.py: 2 read_text calls
- gateway.py: 1 write_text call
The Vertex AI provider (added same-day, commit c73e74386) was never added to
either of the two provider registries that agent/auxiliary_client.py and the
MoA slot-resolution chain depend on, breaking Vertex outside the main
conversation loop:
1. hermes_cli/auth.py::PROVIDER_REGISTRY had no "vertex" entry. The
plugin-auto-extend loop that normally fills gaps explicitly skips
non-api_key auth types (`if _pp.auth_type != "api_key": continue`), and
Vertex was never hand-declared like "bedrock" is. Because
resolve_provider_client() in agent/auxiliary_client.py gates everything
on `pconfig = PROVIDER_REGISTRY.get(provider)` and returns (None, None)
immediately when pconfig is None, its `elif pconfig.auth_type == "vertex"`
branch was permanently dead code — every auxiliary Vertex call (vision,
title generation, reflection, context compression, MoA reference/
aggregator slots) failed outright, not just a MoA-specific edge case.
2. hermes_cli/providers.py::HERMES_OVERLAYS also had no "vertex" entry, so
hermes_cli.providers.get_provider("vertex") returned None. This backs
_preserve_provider_with_base_url() in agent/auxiliary_client.py, which a
MoA slot's resolved (base_url, api_key) pair needs to keep its "vertex"
identity instead of silently collapsing to "custom" — losing the
identity _refresh_provider_credentials() needs to re-mint an expired
OAuth2 token (~1h lifetime) on a 401, and permanently breaking every
subsequent call in that MoA preset for the rest of the session.
Fix mirrors the existing "bedrock"/aws_sdk entries in both registries
exactly, plus adds a "vertex" branch to _refresh_provider_credentials() (it
had branches for openai-codex/nous/anthropic/xai-oauth but not vertex,
so a 401 fell through to `return False` without evicting the stale cached
client).
- hermes_cli/auth.py: hand-declared vertex ProviderConfig(auth_type="vertex")
in PROVIDER_REGISTRY, matching bedrock's shape.
- hermes_cli/providers.py: vertex HermesOverlay(auth_type="vertex") in
HERMES_OVERLAYS + "Google Vertex AI" label override.
- agent/auxiliary_client.py: vertex branch in _refresh_provider_credentials
that re-mints the token via get_vertex_config() and evicts the stale
cached client.
- 8 new regression tests across tests/hermes_cli/test_vertex_provider.py and
tests/agent/test_auxiliary_client.py: registry membership, end-to-end
resolve_provider_client("vertex", ...) building a working client (proving
the previously-dead branch is now reachable), and the 401-refresh/cache-
eviction path.
A user who configured a provider only inside a MoA preset (advisor or
aggregator slot) has explicitly opted into that provider — the consent
gate (is_provider_explicitly_configured) now scans moa.reference_models,
moa.aggregator, and all moa.presets.* slots, so Claude Code OAuth pool
seeding and the auxiliary auto-fallback chain treat MoA-only Anthropic
users consistently with model.provider users.
Salvaged from PR #57778 (trimmed): the auxiliary_client fallback half of
the original PR was independently landed on main in ddd3a2d247 and is
dropped here; a secret-scrubber artifact in the gate test fixture is
restored to the real placeholder token.
Two related races in credential-pool cooldown state:
1. Lost update across processes: write_credential_pool merged only
entries missing from the caller's snapshot; for entries present on
both sides the caller's in-memory copy won wholesale. A process
holding a snapshot taken before another process marked a key
exhausted would, on its next persist (e.g. a round-robin rotation),
write the key back as healthy — erasing the cooldown so every
process resumes hammering a rate-limited key. Merge status fields by
last_status_at recency: adopt the on-disk status only when it is
strictly newer AND still binding (DEAD, or EXHAUSTED with an
unexpired cooldown), and never onto re-authed (token-changed)
entries, so legitimate expiry-clears and fresh logins are preserved.
2. Wrong-key quarantine: when mark_exhausted_and_rotate received an
api_key_hint that matched no entry, it fell through to
current()/_select_unlocked() — on a freshly loaded pool that selects
the NEXT healthy key and benches it for the full cooldown TTL,
punishing an innocent credential. When a hint is provided but
unmatched, rotate without marking anything instead of guessing.
Includes regression tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When Codex returns 429 usage_limit_reached, Hermes persists the provider's
reset_at on the pool entry and freezes the credential until it elapses --
which can be days out for weekly windows. But the upstream window can
reopen EARLY: the user redeems a banked rate-limit reset (Codex CLI /
ChatGPT UI), upgrades their plan, or OpenAI resets the window. Hermes
never re-checked, so it kept erroring with 'Codex provider quota
exhausted (429); retry after Ns' until a manual re-auth rewrote the
tokens (issue #43747, externally-reset variant).
- hermes_cli/auth.py: add _probe_codex_quota_restored() -- a throttled
(5 min/token) GET of the Codex /usage endpoint; quota counts as
restored when every reported window is <100% used. Add
clear_codex_pool_quota_cooldowns() to lift 429/quota-shaped cooldowns
from persisted pool entries (DEAD and auth-shaped entries untouched).
- resolve_codex_runtime_credentials(): before surfacing a pool-only
cooldown as 'quota exhausted', probe upstream; on a positive probe
clear the cooldown and return the pool credential.
- agent/credential_pool.py: _available_entries() probes frozen
openai-codex entries (clear_expired path only) and unfreezes them when
upstream confirms the reset.
- agent/account_usage.py: a successful /usage reset redemption now
clears persisted pool cooldowns immediately.
Negative paths preserved: probe 429/exhausted/indeterminate keeps the
cooldown; read-only enumeration never probes; non-JWT tokens never
probe (no network in hermetic tests).
* fix(billing): rename user-facing "terminal billing" copy to Remote Spending
The capability was renamed Remote Spending on the portal (consent CTA:
"Allow Remote Spending"; per-terminal states Granted/Stopped), but the
terminal, desktop, and docs still said "terminal billing" everywhere.
- Feature name: Remote Spending in titles/labels, lowercase mid-sentence.
- Step-up action verb is now "allow", matching the portal consent CTA.
- Kill-switch-off recovery copy points at the actual control ("a billing
admin can turn it on from the portal's Hermes Agent page") instead of
the dead-end "manage it on the portal".
- Per-terminal revoke copy uses the portal vocabulary ("stopped").
- Wire identifiers (cli_billing_enabled, cli_billing_disabled, ...) are
unchanged; copy, comments, docs, and test expectations only.
* fix(billing): correct the post-step-up denial diagnosis + finish the desktop rename
Adversarial review findings: (1) a repeated insufficient_scope after a
successful step-up is a per-terminal authorization failure, but the copy
blamed the org kill-switch and pointed at the wrong recovery control —
now: "Remote Spending still isn't active for this terminal — the
authorization didn't take. Retry, or make this change on the portal."
(2) the desktop step-up flow started in Remote Spending vocabulary but
finished in "billing management access" — renamed both end states.
(3) prettier formatting on the touched files (matches the post-merge
fmt bot).
* fix: detect env-var-configured providers absent from PROVIDER_REGISTRY
is_provider_explicitly_configured() only checked PROVIDER_REGISTRY (a
manually-maintained dict) for env-var names. Providers that exist solely
in the models.dev catalog — e.g. openrouter — were never recognised as
explicitly configured, so they were filtered out of the desktop model
picker even when their API key was set in .env.
Add a fallback to get_provider() (which reads the models.dev catalog)
when PROVIDER_REGISTRY returns None. Both ProviderConfig and ProviderDef
expose .auth_type and .api_key_env_vars with the same shape.
* test: keep OpenRouter provider gate assertion behavioral
* chore(release): map salvaged OpenRouter contributor
---------
Co-authored-by: zzpigpinggai <zzpigpinggai@users.noreply.github.com>
Switch provider and model together after setup-time auth failure. Serialize global auth-store merges under target-specific locks and preserve auth-to-shared lock ordering for profile OAuth refreshes.
hermes-agent is public/OSS; the forensic-logging comment in
_quarantine_nous_oauth_state named 'Fly' (the specific managed-hosting compute
provider) twice. Reword generically ('a hosted agent', 'a managed log drain may
be WARNING-only') — the behaviour is unchanged, only the comment. Follows the
same scrub applied to the boot re-seed helper (#59983) before merge; this one
slipped through in #59976.
PROVIDER_REGISTRY, its alias map, and CANONICAL_PROVIDERS all auto-extend
from registered ProviderProfiles since the provider-modules refactor
(20a4f79ed). Verified with real imports: registry entry, 'solar' alias
resolution via resolve_provider(), and the picker entry are identical
with the manual entries removed. The hermes_cli/providers.py overlay
stays (models.dev has a stale /v1/solar base URL and no UPSTAGE_BASE_URL
var), and the manual OPTIONAL_ENV_VARS entries stay (non-advanced key +
curated prompt text, matching the fireworks convention).
Teknium