Review follow-up (egilewski): the previous commit only hedged the guidance
text; the exact Anthropic 400 was still classified, persisted, and surfaced
as confirmed billing exhaustion. Carry the ambiguity all the way through:
- agent/error_classifier.py: 'out of extra usage' matches on the 400 and
status-less paths now attach error_context {billing_unverified,
possible_content_filter}. Reason stays FailoverReason.billing (rotation +
fallback remain the right recovery either way); ClassifiedError grows a
billing_unverified property.
- agent/credential_pool.py: new FAILURE_REASON_BILLING_UNVERIFIED. An
unverified billing exhaustion gets the short transient cooldown instead of
the one-hour bench, regardless of pool size: a content-filter rejection
leaves the credential healthy and fails identically on every key, and the
hour-long sole-credential latch is what replayed the stored error and made
real fixes look ineffective. A true 402 keeps the full bench. The marker
persists with the entry so a restart cannot upgrade it back to a bench.
- agent/agent_runtime_helpers.py + run_agent.py: recover_with_credential_pool
threads billing_unverified and hands the pool 'billing_unverified' as the
persisted failure_reason.
- agent/conversation_loop.py: the fallback-switch status, max-retries status,
terminal label, and both structured terminal results hedge when the verdict
is unverified. New _billing_terminal_label + _billing_failure_result build
the returned terminal response in one place; the result dict now carries
billing_unverified and the billing_block gains 'unverified': true. The
confirmed-billing path (a real 402 or an API-key credit depletion) keeps
the original assertive wording, so the caveat no longer dilutes it.
Regression tests: classifier marking (400 + status-less + unambiguous-body
negative), pool cooldown TTLs + persistence round-trip, pool failure_reason
plumbing, and the returned terminal response for both unverified and
confirmed verdicts.
Note: tests/agent/test_credential_pool_routing.py::TestFailureAttribution::
test_unmatched_key_does_not_retry_only_pool_entry fails identically on
current main without this change (pre-existing, unrelated).
On an Anthropic subscription OAuth credential, every request failed with
HTTP 400 "You're out of extra usage. Add more at claude.ai/settings/usage".
That is not a billing condition: Anthropic's server-side content filter rejects
the first sentence of Hermes' own built-in SKILLS_GUIDANCE prompt, and the
rejection is surfaced with a billing-shaped message. Because the message points
at the usage settings page, it reliably sends people to buy quota they do not
need — the reporter lost three debugging sessions to it.
Bisected against the live API with the real 71,721-char assembled prompt: the
first SKILLS_GUIDANCE sentence alone reproduces the 400 and removing it alone
clears it. Size was ruled out (20 KB of unrelated filler returns 200) and so was
the system[0] identity gate (that returns 429, a different failure).
Three changes, all serving the same outcome — a subscription user can no longer
be misdirected by this 400:
- agent/prompt_builder.py: reword the triggering sentence to the phrasing the
reporter verified returns 200. Meaning, the skill_manage reference, and the
## Skill Safety Rule block are all preserved. The reword is empirically
validated rather than understood, so a comment records the bisect and warns
that any rewrite must be re-verified against an OAuth token, not an API key.
- agent/conversation_loop.py: the Anthropic branch of the billing guidance no
longer asserts exhaustion as fact. It hedges the opening line, names the
content-filter alternative, and gives the operator a way to tell the two apart
(if the usage page still shows quota, suspect a content rejection). It also
points at `hermes auth reset anthropic`, because the credential exhaustion
latch replays the stored error for ~60 min without issuing a request — which
makes a real fix look like it did not work.
- hermes_cli/auth.py: document that CLAUDE_CODE_OAUTH_TOKEN is an OAuth token,
not an API key, despite auth_type="api_key". It stays in api_key_env_vars
because that tuple doubles as the credential-discovery list; removing it would
stop Hermes finding a `claude setup-token` credential at all.
Docs updated to match the reworded prompt.
Fixes#82154
The gateway rotation guard (#83339) only protects the rotate path, but
in-place compaction commits inside compress_context() via
archive_and_compact — before the gateway can inspect the result. Add the
anti-growth check at the commit site so both paths are covered: a
compression whose rough output exceeds its input is a strict no-op
(original transcript kept durable, session identity untouched).
Covers the observed failure where session hygiene persisted 426 -> 426
messages and ~379K -> ~688K tokens.
SessionDB could leave native SQLite handles open when construction failed
partway through schema/pragma/FTS/repair/lock/interrupt handling. Other
short-lived callers (MCP reads/polling, session search, reactions, trace
upload, insights, shutdown recovery) opened temporary SessionDB handles
without a complete ownership boundary. API-server profile caches and
RetainDB shutdown had similar late-close races. Under sustained load this
exhausted file descriptors (EMFILE).
- Close partially initialized SessionDB connections on every constructor
exception path via a finally block guarded by an initialization-complete
flag.
- Close temporary/cross-profile SessionDB handles in finally blocks across
CLI, MCP, search, trace, reactions, insights, and recovery paths.
- Add API-server per-profile cache ownership and disconnect cleanup.
- Make RetainDB writer-queue shutdown exception-safe: track connections per
thread, close on worker exit, reject new enqueues after shutdown starts,
and sweep any connections left by short-lived threads.
- Add regression coverage for constructor failures, worker-thread readers,
API disconnect failures, shutdown recovery, RetainDB late enqueue, and
foreign-loop async clients.
Salvage notes: the original PR's per-thread WAL-reader ownership changes
were superseded by main's read-connection pool (permits + checkout/return);
its cron timeout-abandon fix is credited separately to #72822's earlier
identical fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
`_moa_prepared_request` is a private handshake between the conversation
loop and MoAChatCompletions.create. It is attached whenever
agent.provider == "moa", on the assumption that agent.client is still the
in-process MoA facade.
Credential rotation, provider fallback and dead-connection cleanup all
rebuild agent.client from _client_kwargs between attempts, and
pending_moa_prepared_request deliberately carries a prepared request
across exactly that boundary. The rebuilt client is a native OpenAI
client while provider stays "moa", so the key reaches an SDK that has
never heard of it:
TypeError: Completions.create() got an unexpected keyword argument
'_moa_prepared_request'
That error is non-retryable, so every remaining turn on the session
fails. Both dispatch paths are affected: the non-streaming one calls
agent.client directly, and _create_request_openai_client returns
agent.client unchanged for provider "moa".
Re-check the live client at the point the key is attached, which covers
both paths at once. When the facade is gone, send the prepared prompt
without the handshake and log the downgrade.
Residual from PR #70910 after #77509 landed the message-list scrub: a
whitespace-only system content block carrying a cache_control marker
still reached the wire and 400'd the whole request ("text content
blocks must contain non-whitespace text"), wedging the session on every
retry. The block cannot be dropped (it carries the cache breakpoint),
so coerce its text to the shared non-whitespace placeholder when
extracting the system param, copying the block so caller message dicts
are never mutated.
Adds SHL0MS's request-level regression suite from #70910; four of its
five cases already pass on main via #77509 — the system-block case
fails without this fix.
Strict OpenAI-compatible providers (onerouter / Qwen, DeepSeek v4) reject
an assistant message carrying tool_calls: [] (or null) with HTTP 400
'Empty tool_calls is not supported in message.'
The pre-API sanitizer in agent_runtime_helpers.sanitize_api_messages already
drops these on the conversation_loop path, but auxiliary / custom-provider
routes that bypass that sanitizer can still reach the wire with an invalid
empty array and abort the whole session (non-retryable 400).
Normalize at the transport layer too: detect an empty-list / null
tool_calls on assistant messages, strip the key on the per-call copy (never
mutate the stored history), and keep real tool_calls untouched. Includes
unit tests covering empty-list, null, real-call preservation, mixed batches,
user-role non-mutation, copy-on-write, and cross-provider parity.
Follow-up to #58755.
The post-write lint runs `npx tsc --noEmit <file>` on a single .ts file with
no `-p tsconfig`. tsc ignores tsconfig.json for explicit file args, so for any
file in a real TS project it floods phantom diagnostics — unresolved path
aliases (@/... -> TS2307) and ambient globals (Window.hermesDesktop -> TS2339)
that the project config defines. The delta filter sees the same phantom errors
pre- and post-edit, finds no NEW ones, and returns the misleading
'pre-existing lint errors ... the file is still broken' on a perfectly correct
one-line edit — wasting the caller's turns chasing nonexistent breakage.
Existing code already skips shell tsc when an LSP server claims the file, but
that only fires with LSP configured+enabled (not the default), leaving the
common LSP-disabled case fully exposed.
Fix: when an ancestor tsconfig.json exists (local host only), skip the per-file
shell tsc for .ts — its verdict carries zero signal for project files. Real
diagnostics still come from the LSP tier or an explicit `tsc -p tsconfig.json`.
Best-effort ancestor walk; remote/sandbox backends fall back to running the
linter as before. (.tsx already returns skipped via the ext-not-in-LINTERS
branch.)
Tests: ancestor-tsconfig .ts -> skipped even with LSP off; standalone .ts with
no ancestor tsconfig -> shell tsc still runs. 7/7 in the LSP-skip suite.
Co-authored-by: Teknium <teknium1@users.noreply.github.com>
Per-provider ssl_ca_cert / ssl_verify reached the httpx chat client and the
auxiliary clients (#56681), but the endpoint discovery and pricing probes did
not. Both probe families resolved TLS from process-wide env vars only:
- the requests-based metadata/pricing probe
(agent/model_metadata.py::_resolve_requests_verify)
- the urllib-based /models catalog probe
(hermes_cli/models.py::probe_api_models)
A custom endpoint whose chain verifies against the provider's configured
bundle, but not the process SSL_CERT_FILE, then logged a spurious
CERTIFICATE_VERIFY_FAILED on every probe even though the chat path worked.
Pointing a global CA env var at the bundle fixes it but changes verification
for every provider, defeating the point of a per-provider setting.
This threads the selected provider's TLS settings into both probe paths,
reusing get_custom_provider_tls_settings so there is no second precedence
chain:
- _resolve_requests_verify(base_url) looks up the provider's ssl_verify /
ssl_ca_cert before falling back to the env vars. Callers with no base_url
keep the exact env-only behavior.
- probe_api_models builds an ssl.SSLContext from the provider settings and
passes it through open_credentialed_url, which gains an ssl_context seam on
the cloned secure opener. Unmatched or public endpoints pass None and keep
urllib's default policy.
Tests: tests/agent/test_custom_provider_ca_probes.py covers both probe
families (provider CA, ssl_verify:false, unmatched, missing file, config
lookup failure) plus end-to-end assertions that the resolved verify value and
SSLContext actually reach the request seam. Verified against the neighboring
metadata, pricing, TLS, and urllib-security suites (266 tests) with no
regressions.
The messaging gateway multiplexes profiles over ONE shared launch-home
state.db, binding the profile per turn via the HERMES_HOME ContextVar
(copy_context into the worker thread). _agent_home derived the home from
db_path unconditionally, so on that lane the launch home stomped the
correctly-bound profile — deterministically inverting the leak #86313
fixed (found by @kshitijk4poor's post-merge probe; @helix4u flagged the
plugin-metadata half).
- _agent_home: bound override wins; session_db home is the unbound-thread
fallback
- _plugin_session_info: profile_name derives from _agent_home too
- full-prompt wiring regression (SOUL + skills + profile line on a bare
thread with the bot's DB) — reverting any call-site wire fails it;
multiplex, bare-thread, and plugin-metadata cases each pinned;
sabotage-verified both new tests fail against the merged behavior
- skills LRU cap 8 -> 32 (key is now per-profile x platform)
load_soul_md resolved the home ambiently, so a build thread that lost the
HERMES_HOME ContextVar read the launch profile's SOUL.md into another
profile's prompt — same class as the skills-index leak fixed in #86313.
load_soul_md and build_context_files_prompt now accept home_override, and
build_system_prompt_parts passes the agent's own home (from session_db)
at both SOUL call sites. Ambient behavior unchanged when no override.
Review feedback: the previous tests mocked get_hermes_home,
get_default_hermes_root and _resolve_active_profile_name, so they
checked template rendering but never exercised the relationship that
causes the defect — _resolve_active_profile_name returns a named profile
only when the active home is already <root>/profiles/<name>, which is
precisely why appending that suffix doubled it.
Replace them with tests that set a real HERMES_HOME under a tmp root and
mock no resolver. They assert the chain first
(_resolve_active_profile_name == "coder", get_hermes_home == the profile
dir, get_default_hermes_root == the root), then the rendered prompt.
Adds the default-profile branch the same way.
This also removes the order-dependent failures noted in the PR
description. Module-attribute monkeypatching stops taking effect once
other files in tests/agent/ have run, which is what made the mocked
tests fail in a full-directory run; the un-mocked tests are immune.
tests/agent/ now reports 87 failures — identical to the baseline on
unmodified main, with none in this file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The named-profile branch of the "Active Hermes profile" hint built its
paths by appending `/profiles/{active_profile}` to `get_hermes_home()`.
But `_resolve_active_profile_name()` returns a non-default name *only*
when `get_hermes_home()` has already resolved under `<root>/profiles/`
— that is how it derives the name in the first place. Both scoping
mechanisms (a `HERMES_HOME=<root>/profiles/<name>` env var and the
multiplexer's `set_hermes_home_override` contextvar) satisfy that, so
the suffix always doubled.
The same branch used `get_hermes_home()` for the *default* profile's
data pointers, where the root was intended — placing them inside the
active profile.
On a real 4-profile install the hint rendered:
reads and writes ~/.hermes/profiles/via/profiles/via/
default profile's data lives at ~/.hermes/profiles/via/skills/
against actual paths of `~/.hermes/profiles/via/` and `~/.hermes/skills/`.
Use the session home directly as the profile home, and
`get_default_hermes_root()` for the root pointers. Every named-profile
session was shipping a prompt that named nonexistent directories and
mislabeled this profile's own skills/plugins/cron/memories as the
default profile's — the exact cross-profile confusion the hint and
`classify_cross_profile_target` exist to prevent.
The default-profile branch is unchanged.
Fixes#72894
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
On a correctly bound profile session get_hermes_home() returns the profile
dir itself, so relative_to(home/'profiles') never matched and every profile
misreported as 'default' (with wrong paths in the profile hint text). Use
get_default_hermes_root() for both the name derivation and the default-data
root string. Adds regression tests for the bound-profile and root-home cases;
sabotage-verified the bound-profile test fails against the old resolution.
A bot profile's system prompt could list the DEFAULT profile's ~80
skills and print 'Active Hermes profile: default' — while the live
skills_list() correctly showed the bot's real (often empty) set. The
agent plans against that index, so a false inventory makes it claim
capabilities it doesn't have, waste context tokens, and lose trust.
Root cause (confirmed empirically): the skills-prompt builder and the
active-profile line resolve the home through get_hermes_home(), which
reads a HERMES_HOME ContextVar. ContextVars do NOT propagate into
threading.Thread, so an agent build running on a thread that didn't
bind the profile's home falls back to the launch (default) home and
builds default's index. A bare no-override thread builds default's
full 7621-char block; the same thread with the fix builds empty.
Fix: resolve the agent's OWN home from its dedicated _session_db.db_path
(ground truth, ContextVar-independent) and pass it explicitly:
- build_skills_system_prompt(skills_dir_override=...) scopes the index,
the disk snapshot, and external-dir resolution to that home
- the active-profile line derives the profile name from the same home
Both fall back to ambient resolution when no db is present, so the CLI
and default-profile paths are unchanged.
Regression tests: an empty bot profile yields an empty skills block on
a bare thread even with ambient HERMES_HOME bound to a skills-rich
default; agent-home resolution from session_db.db_path. 3/3.
Follow-up on the #84795 salvage: the sequential deadline gets its own
resolver key. Unset, it inherits the concurrent batch deadline (same
value, same HERMES_CONCURRENT_TOOL_TIMEOUT_S bridge) so the two executor
paths cannot drift by default; set, it can be tuned or disabled
independently. Documented in cli-config.yaml.example; 5 contract tests.
Deliberately NOT on run_bounded_sync: the executors extend deadlines
dynamically during human approval waits (authorization-gate excluded
seconds) — the shared primitive is fixed-deadline. Noted in the docstring.
Run the expensive-model warning for explicit startup `-m` / `--provider`
overrides before the chat loop starts, and fail closed for non-interactive
invocations that select an expensive or known-confusing model.
Also classify Nous paid-model 404s that say credits are required as billing
exhaustion so they fail fast with billing guidance.
Tested:
- scripts/run_tests.sh tests/hermes_cli/test_cli_startup_model_cost_guard.py tests/hermes_cli/test_model_cost_guard.py tests/agent/test_error_classifier.py -- --tb=short -q
Regression tests salvaged from PR #70522 by @JoaoMarcos44. The Qwen flat
cached_tokens behavior is provided by the shared top-level fallback from
PR #66105 (@mehmetkr-31); the codex cache_write_tokens read landed in the
previous commit.
Salvaged from PR #85702 by @JoaoMarcos44, composed onto the mapping-safe
_usage_get reads (PR #74591 by @RelaxJonh) and the flat cached_tokens /
Anthropic-name fallbacks (PRs #66105, #52571):
- cache-write precedence in the chat_completions branch:
details.cache_write_tokens > details.cache_creation_input_tokens >
usage.cache_creation_input_tokens > usage.cache_write_tokens
- codex_responses branch reads details.cache_write_tokens (GPT-5.6+
documented name) with cache_creation_tokens fallback (from PR #70522)
- _usage_count(): clamp malformed negative counters to 0
- all reads in every branch are mapping-safe via _usage_get
When the Responses API returns usage as a plain dict (e.g. from a
middleware or proxy that deserialises JSON to dict instead of a typed
SDK object), normalize_usage() used getattr() exclusively, which
silently returned 0 for every field on a dict.
Add _usage_get() helper that reads via .get() for dicts and getattr()
for attribute-style objects. All accessor sites in normalize_usage()
now use this helper, so token counts and cost are correct regardless
of the usage object's type.
Regression tests: two new tests feed the same payload as both a dict
and a SimpleNamespace through the codex_responses and
chat_completions branches, asserting identical output and non-zero
values.
Kimi/Moonshot's native API (api.moonshot.cn / .ai) reports context-cache hits
as a top-level ``usage.cached_tokens``. The chat-completions branch of
normalize_usage() walks a fallback chain of
prompt_tokens_details.cached_tokens -> cache_read_input_tokens ->
prompt_cache_hit_tokens; none of those names match, so direct Kimi sessions
normalized to cache_read_tokens=0. The hits were invisible in accounting and
the cached prefix was billed at the full input rate.
Appended as the last link in that chain, so it only fills a genuine zero and
cannot override a provider that reports the nested OpenAI shape or DeepSeek's
prompt_cache_hit_tokens.
Rebuilt on current main rather than rebased — the branch was ~3400 commits
behind. The DeepSeek half of the original branch is dropped: 03c0b00f4
(#65678) landed prompt_cache_hit_tokens on main, so this is Kimi-only as the
review asked. The scripts/release.py addition to the frozen LEGACY_AUTHOR_MAP
is dropped too; contributors/emails/mehmet.kar@std.yildiz.edu.tr already
exists on main.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Azure Foundry's OpenAI-compatible Responses surface rejects the post-tool
follow-up payload with HTTP 400 `invalid_payload` when a replayed encrypted
`reasoning` item is sent alongside `function_call` / `function_call_output`.
The initial function-call request and ordinary multi-turn continuity are both
accepted, so the failure only appears after the first tool executes.
Detect the Foundry endpoint in `ResponsesApiTransport.build_kwargs` and drop
only the encrypted reasoning replay on that follow-up turn, leaving
function_call / function_call_output continuity intact.
Salvage of #59981, rebuilt on current main. Same root cause and fix direction
as the original, which was correct; this version resolves three defects:
- No `chat_completion_helpers.py` change. main already forwards `provider`
and `base_url` to the Responses transport, so the original's re-added
arguments produced `SyntaxError: keyword argument repeated: provider` on
merge. Dropping the hunk removed the syntax error and the conflict.
- Host matching uses `utils.base_url_host_matches`, not a substring test.
`".services.ai.azure.com" in base_url` also matches URLs carrying the
domain in a path or query segment, which would silently disable reasoning
replay on an unrelated provider.
- The post-tool predicate tests the trailing messages, not the whole history.
Scanning for any tool call plus any tool result made it sticky: one tool
call early in a conversation suppressed reasoning on every later turn.
- Tool calls pair on `call_id` as well as `id`. Responses histories carry the
function call id in `call_id` while `id` holds the response item id
(`fc_...`). Identity is resolved via the converter's own
`_split_responses_tool_id`, covering composite `"call_x|fc_y"` ids and bare
`fc_` ids on both sides of the pairing.
Tests: 27 cases across the transport and the live `build_api_kwargs` bridge,
including six parametrized tool-call id shapes, non-Foundry host lookalikes,
the sticky-history guard, parallel tool results, and an unpaired tool result.
Each guard was confirmed to catch its defect by reverting the fix.
Verified with `scripts/run_tests.sh tests/agent/ tests/run_agent/`:
532 files, 5602 tests passed, 0 failed.
Not verified against a live Azure Foundry endpoint — no credentials. The
original HTTP 400 reproduction and post-fix Foundry Project / Azure Container
Apps harness runs are @AshuJoshi's, from #59981. This change is verified at
the payload-construction layer only.
Closes#59981.
Co-authored-by: Ashu Joshi <AshuJoshi@users.noreply.github.com>
- Cold force_refresh (fresh CLI process, e.g. hermes config refresh)
now hydrates the memory cache from disk before fetching, so the
conditional GET actually fires on the flow the feature was built for
instead of silently re-downloading the full ~2 MB registry
(empirically probed: If-None-Match sent, 304 serves disk data).
- Conditional-GET decision is passed in explicitly
(_fetch_models_dev_from_network(conditional=...)) by callers holding
the fetch lock, removing the hidden read of module globals inside
the fetch; the background worker now fetches INSIDE the lock,
symmetric with foreground (true singleflight — no concurrent
double-download, no fetching against mid-commit etag state).
- Corrupt disk cache is QUARANTINED (renamed to .json.corrupt) rather
than left in place: rejection becomes a one-time event instead of a
re-read + re-parse + warning + unlink on every hot-path call while
offline (probed: 1 warning across 5 calls, was 5).
- Dropped the dead _DEFAULT_MODELS_DEV_URL constant; module and
function docstrings updated to match the servable-cache conditional
semantics.
- Conditional GET now requires a servable in-memory registry: an
If-None-Match sent while holding no cache invited a 304 against
nothing, permanently serving {} with a blocking foreground fetch on
every call (the exact #35838 class this PR fixes). Empirically
repro'd and verified fixed (corrupt cache + stale sidecar: was 3
calls -> {} forever; now 1 unconditional fetch -> real data).
- ETag persists atomically WITH the cache body via
_commit_registry -> _save_disk_cache(data, etag), wiring up the
previously-dead etag param; the sidecar can no longer get ahead of
the registry it vouches for. _save_etag now uses
utils.atomic_write_text (unique tempnames + fsync) instead of a
hand-rolled fixed-name .tmp replace.
- Corrupt/unreadable disk cache clears the ETag sidecar so the
refetch is unconditional; _confirm_cache_not_modified keeps a
defense-in-depth guard (clear sidecar + arm backoff) should a 304
ever land on an empty registry.
- allow_network=True paths use the zero-arg fetch_models_dev() call
shape at all sites (was 1 of 5) — ~46 test sites monkeypatch it
with zero-arg lambdas; the unconditional kwarg broke
test_xiaomi_provider (verified fail->pass).
- _get_models_dev_url falls back to the MODELS_DEV_URL module global
(not the constant) so existing patch sites keep working.
- Tests: replaced two mock-riddled corrupt-cache tests with real
tmp_path file tests; added regression tests for the 304/empty-cache
loop, sidecar clearing, and conditional-GET gating.
Harden the models.dev catalog refresh path (#35838) with three missing
pieces:
1. ETag conditional GET — every network request sends If-None-Match
with the last-known ETag (persisted alongside the cache file). A 304
Not Modified re-confirms the existing cache without re-downloading
the full ~2 MB registry. This makes the 4-hour TTL effectively free
to maintain.
2. No-network-on-hot-paths invariant — allow_network=False is now the
default for every query function called on the conversation hot path:
get_model_capabilities, get_model_info, lookup_models_dev_context,
_get_provider_models. These are called during vision routing, image
routing, cost-guard checks, and context-length resolution on every
turn — they must never block on the network. Interactive flows
(model picker, model switch) explicitly pass allow_network=True.
3. Mirror URL override — models_dev.url in config.yaml lets deployments
point at a self-hosted mirror without code changes. Follows the same
pattern as model_catalog.url.
Additional hardening:
- Cache TTL bumped from 1h to 4h (ETag makes refresh cheap)
- Corrupt/empty disk cache is rejected with a warning instead of being
served as {} and silently breaking provider/model resolution
- _validate_registry() guards against non-dict and empty-dict payloads
Fixes#35838
- Single-source the included note as _INCLUDED_NOTE and attach it at
BOTH status='included' sites (the zero-amount pricing-entry branch
previously returned the same status with no note).
- Docstring/comment precision on format_cost_label: the fallback
triggers on 4dp ROUNDING to 0.0000 (banker's rounding includes the
exact $0.00005 boundary), not truncation; note why the rendered-label
guard beats a naive Decimal threshold.
- Tests: replaced a dead assertion with the exact-boundary case
($0.00005), fixed an overclaiming comment, aligned the terminal
cost column.
- Insights formatters now route aggregate estimated cost through the
shared format_cost_label() instead of hardcoded 2dp — a sub-cent
aggregate (one cheap DeepSeek session, ~$0.0046) no longer renders
'Estimated: ~$0.00', the exact bug class this PR fixes (#79220).
- format_cost_label: positive amounts below $0.00005 render '~$<0.0001'
instead of the zero-looking '~$0.0000' 4dp truncation artifact.
- Renamed _format_cost_label -> format_cost_label (now a cross-module
shared helper).
- Tests: renamed test_gateway_format_hides_cost ->
test_gateway_format_hides_cache_details and
test_no_cost_section_when_all_zero ->
test_unknown_bucket_shown_for_costless_session (names contradicted
behavior); restored a real assertion in the custom-models test that
had been weakened to a comment; added sub-cent-aggregate and 4dp-floor
contract tests (mutation-checked).
Three cost-display honesty fixes:
1. Sub-cent cost label rendering (#79220) — _format_cost_label() scales
precision to magnitude: zero renders as '$0.00', sub-cent (< $0.01)
renders at 4 decimal places (e.g. '~$0.0046'), normal costs keep 2dp.
This fixes the bug where DeepSeek per-turn costs of $0.004640 rendered
as '~$0.00' despite amount_usd carrying full Decimal precision.
2. Cost bucket surfacing (#77223) — insights format_terminal and
format_gateway now display three cost buckets: estimated (with dollar
figure), included (session count, labeled 'subscription — no provider
invoice'), and unknown (session count, labeled 'no pricing data').
Previously, included and unknown sessions silently collapsed to $0 in
the aggregate view, hiding 315 of 473 sessions in the reporter's DB.
3. Subscription-included cost notes — estimate_usage_cost now attaches a
'subscription-included; no provider invoice for usage' note to
CostResult for subscription-included routes (openai-codex), so
consumers can distinguish 'free because subscription' from 'free
because $0 pricing'.
Fixes#79220Fixes#77223
- Deleted the id(cfg)-keyed _OVERRIDE_CACHE layer: id() is unique only
among live objects, so a config reload could serve stale overrides
forever when CPython reuses the freed dict's address. The upstream
load_config_readonly is already (mtime,size)-cached (~1 stat/hit), so
the local layer was redundant state with a correctness risk.
- _override_to_catalog_shape returns (patch, vision) instead of
smuggling an in-band _vision_override sentinel key through the merged
dict; removed the two dead call-site pops.
- _find_model_entry gains the :cloud/-cloud suffix fallback that
lookup_models_dev_context already had, so 'catalog hit' means the
same thing to every consumer — a suffix-keyed model (kimi-k2.6:cloud)
now counts as KNOWN and keeps its catalog capabilities instead of
being displaced by a fill-gap _default (mutation-checked contract
test added).
- get_model_info's unknown-model override path seeds the same safe
defaults as get_model_capabilities (200K ctx, tools on, 8192 out),
so a partial override no longer yields ctx=0/tools-off on that path
(contract test added); the DEFAULT_CONFIG defaults claim is now true
for both paths.
- Activated the previously-dead _MODELS_DEV_TO_PROVIDER reverse map
(lazily built, many-to-one aware) and used it in
_provider_override_section instead of a per-call linear scan.
Review follow-ups on the model_overrides feature:
- ONE canonical override schema everywhere. get_model_info previously
merged the override dict raw into the models.dev catalog shape
({**raw, **override}), so the documented context_window/supports_*
keys silently did nothing on that path (cost guard, inventory) while
working in capabilities/context paths — same config key, two
incompatible schemas. Overrides are now translated into the catalog
shape at the get_model_info boundary (_override_to_catalog_shape),
and sub-dicts (limit, modalities) are MERGED, not clobbered — an
override setting only context_window no longer wipes the catalog's
limit.output.
- _default is now a FILL-GAP default, not an override: it applies only
to models the catalog does not know (the #8731/#84482 self-unblock
path) and never displaces catalog data. A
_default: {context_window: 128000} can no longer clamp every model
of a provider. Explicit per-provider+model entries keep their
win-over-catalog semantics.
- Early-chain _override_context_window (model_metadata step 0b) is
explicit-only, so a _default can never preempt custom_providers
per-model settings or live probes; fill-gap defaults apply at the
lookup_models_dev_context catalog-miss boundary (step 5f) instead.
This fixes the precedence inversion where a provider/global _default
silently overrode an explicit per-endpoint per-model context_length.
- Provider keys accept BOTH id spaces (Hermes id and models.dev id:
copilot/github-copilot both work) and model ids match
case-insensitively, mirroring catalog lookup.
- Malformed override values (context_window: '512k') log a one-shot
warning instead of being silently swallowed.
- DEFAULT_CONFIG comment: removed the false family/dated-snapshot
inheritance claim, documented the recognized field list, fill-gap
semantics, and the id-space rule.
- Tests: rewritten for the new contracts (fill-gap invariants,
dual-id-space keys, sub-dict merge preservation, one-shot warning);
added a real-config-yaml e2e plumbing test (mutation-checked: fails
when the config key wiring is broken).
Add a unified model_overrides config section that lets users manually
declare context_window, max_output_tokens, capabilities, cost, and
family for any provider+model — winning over models.dev, OpenRouter, and
hardcoded defaults.
Resolution order (first hit wins):
1. model_overrides.<provider>.<model_id> (per-provider+model)
2. model_overrides.<provider>._default (per-provider default)
3. model_overrides._default (global default)
4. Normal catalog resolution
Key subtlety: an unknown model id (not in the
catalog) derives base metadata from sensible defaults before patching,
so overriding a model the catalog doesn't know yet is the supported
self-unblock path. This is exactly the #84482 scenario (Upstage
solar-pro4/syn-pro wrong context) and the #8731 scenario (custom/local
models with manual capability declaration).
Wired into:
- get_model_capabilities() — patches capability fields; unknown models
get safe defaults (tools on, vision/reasoning off) before patching
- lookup_models_dev_context() — context_window override, checked before
catalog lookup so it works even for providers not in PROVIDER_TO_MODELS_DEV
- get_model_info() — merges override dict onto catalog entry (shallow
merge); for unknown models, the override is the sole source of metadata
- get_model_context_length() — step 0b in the resolution pipeline,
before custom_providers (0c) and before any network probe
Config example:
model_overrides:
upstage:
solar-pro4:
context_window: 524288
syn-pro:
context_window: 65536
custom:my-local-vllm:
my-llava-model:
context_window: 8192
supports_vision: true
supports_reasoning: false
supports_tools: true
_default:
context_window: 128000
Fixes#8731Fixes#84482
Refs #47247
Unknown hosts (e.g. gateway.example.com) no longer get /anthropic→/v1;
use a real dual-surface MiniMax base for the rewrite assertions and add
a case proving Anthropic-only gateways keep their path on the OpenAI wire.
Substring matching over the whole URL let a path containing
'api.minimax' false-positive an Anthropic-only gateway into the
/anthropic→/v1 rewrite. Parse the host and match exact-domain /
subdomain suffixes (plus the api.minimax.* prefix family) instead.
`_try_anthropic()` applies the configured `model.base_url` only when
`_is_anthropic_compatible_host()` trusts it, but that check accepted only the
literal `api.anthropic.com` host. Anthropic-compatible gateways that expose the
native Messages protocol under a `/anthropic` path suffix (MiniMax, Zhipu GLM,
LiteLLM-style relays, self-hosted proxies) were rejected, so every auxiliary
call (title generation, memory extraction, vision, reflection) and the
`provider: anthropic` fallback chain discarded the configured base_url and fell
back to `https://api.anthropic.com`. That diverges from the primary path, which
already trusts the `/anthropic` suffix via
`runtime_provider._detect_api_mode_for_url`, and fails outright when the gateway
(not Anthropic) holds the credentials.
Accept `/anthropic` and `/anthropic/v1` suffixed URLs in
`_is_anthropic_compatible_host()`, matching the primary-path convention and
`_wrap_if_needed`. A bare non-Anthropic base_url (e.g. `openrouter.ai/api/v1`
left on `provider: anthropic`) still returns False, preserving the #52608 guard.
save_context_length() and _invalidate_cached_context_length() did an
unguarded read-modify-write into $HERMES_HOME/context_length_cache.yaml.
The plain `open(path, "w")` truncates the file before the dump runs. If
the process is killed mid-dump, the file is left empty or partial. The
next _load_context_cache() swallows the YAML error and returns {} —
silently wiping every persisted context length. A concurrent process
reading between truncate and dump-complete also sees a torn file.
After the cache is lost, every model re-probes the network, and when a
probe fails it falls back to the generic 256K default — so a user on a
1M-window model ends up with a wrong, short context window.
Hermes routinely runs several processes against one shared $HERMES_HOME
(a cron agent plus an interactive session, multiple gateway sessions),
so this is hit in normal use.
Switch both writers to the existing utils.atomic_yaml_write helper
(temp file + fsync + os.replace, symlink- and mode-preserving). The real
file is only ever swapped from a fully written temp file, so an
interrupted write leaves the previous cache intact and readers never see
a partial file. Matches the atomic-write pattern already used for
auth.json, config.yaml, and other persisted state.
Makes the persistent model context-length cache write crash-safe. The
old non-atomic write could truncate or wipe the entire cache on an
interrupted or concurrent write, which then forces models onto the wrong
fallback context window. The fix routes both cache writers through the
repo's atomic temp-file + os.replace helper.
N/A
- [x] 🐛 Bug fix (non-breaking change that fixes an issue)
- [ ] ✨ New feature (non-breaking change that adds functionality)
- [ ] 🔒 Security fix
- [ ] 📝 Documentation update
- [ ] ✅ Tests (adding or improving test coverage)
- [ ] ♻️ Refactor (no behavior change)
- [ ] 🎯 New skill (bundled or hub)
- `agent/model_metadata.py`: `save_context_length()` and
`_invalidate_cached_context_length()` now write via
`utils.atomic_yaml_write` instead of a truncating `open(path, "w")`.
Added the `atomic_yaml_write` import.
- `tests/agent/test_model_metadata.py`: added
`test_write_failure_leaves_existing_cache_intact` — simulates a crash
during the atomic swap and asserts the existing cache survives
byte-for-byte with no stray temp file.
1. `pytest tests/agent/test_model_metadata.py -q` — 98 pass, including
the new crash-safety test.
2. The new test seeds a valid cache, forces the swap step to raise, and
confirms the file is not truncated and no `.cache_*.tmp` is left.
3. `ruff check agent/model_metadata.py` passes.
- [x] I've read the Contributing Guide
- [x] My commit messages follow Conventional Commits (`fix(scope):`, etc.)
- [x] I searched for existing PRs to make sure this isn't a duplicate
- [x] My PR contains **only** changes related to this fix
- [x] I've run the affected tests (`pytest tests/agent/test_model_metadata.py -q`) and they pass
- [x] I've added tests for my changes
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)
- [x] I've updated relevant documentation (README, `docs/`, docstrings) — or N/A
- [x] I've updated `cli-config.yaml.example` if I added/changed config keys — or N/A
- [x] I've updated `CONTRIBUTING.md` or `AGENTS.md` if I changed architecture or workflows — or N/A
- [x] I've considered cross-platform impact (Windows, macOS) — the helper uses os.replace, which is atomic on both
- [x] I've updated tool descriptions/schemas if I changed tool behavior — or N/A
An empty/blank model id reaching get_model_context_length() can't be
meaningfully resolved — and it's worse than a miss: the endpoint
metadata fuzzy matcher ('model in key or key in model') is vacuously
true for "", so it matches an ARBITRARY catalog entry from the live
/v1/models response and returns whatever context length that entry
happens to have, persisting it under a junk '@<base_url>' cache key.
This started failing CI on main when the Nous portal catalog changed:
tests/run_agent/test_primary_runtime_restore.py constructs agents with
model='' against the live portal URL, the arbitrary match now lands on
a 32K entry, and init_agent raises the 64K-floor ValueError
(test_allowed_for_nous_anthropic_messages, red on every PR's slice).
Guard early: a blank model id falls back to DEFAULT_FALLBACK_CONTEXT
immediately, before any cache write or network probe.
Salvaged from #65515 by @whirmill (rebased onto current main; the
guard now sits after the malformed-base_url normalization added since,
and carries an explanatory comment for the fuzzy-match footgun).
Fixes the red slice on #85444, #85452 and every other open PR.
Co-authored-by: whirmill <5079591+whirmill@users.noreply.github.com>
1. Use open_credentialed_url() instead of bare urlopen() in
templates.py apply_template() and probe_existing_customization().
Both send Authorization: Bearer headers; bare urlopen forwards
credentials on cross-origin redirects. The codebase has
open_credentialed_url() in hermes_cli/urllib_security.py that
strips credentials on cross-origin redirects — used by 4 other
modules.
2. Guard unavailable_reason() with the dedup set check before
calling it. The gateway builds a fresh AIAgent per message, so
without this guard unavailable_reason() (which calls _load_config()
→ stat + file read + JSON parse, and _check_local_runtime() →
importlib probes) runs on every gateway turn for an unavailable
provider, even though the warning is deduped after the first.
3. Move INDICATOR_GLYPH from Hindsight's eye emoji to a generic
brain (🧠) in core (agent/memory_provider.py). Hindsight overrides
with its own _HINDSIGHT_GLYPH (👁️) in recall_status() and
_emit_saving_indicator(). Other memory providers no longer inherit
Hindsight's brand mark as the default glyph.
Jack Lau