Self-review follow-up: cover call_converse_stream's max_tokens=None path
(same builder, previously unpinned) and document why the shim reads the
caller cap with truthiness rather than 'is None' (parity with the
Anthropic shim's reading).
The Bedrock Converse shim hardcoded 'else 4096' when the caller passed no
max_tokens, so auxiliary vision descriptions stayed capped at 4096 tokens
on the Bedrock wire even after #75253 removed the vision call sites' own
caps (#10809 was only partially fixed there).
Converse's inferenceConfig.maxTokens is optional; when omitted, Bedrock
defaults to the model's maximum allowed output. Thread an explicit
max_tokens=None through build_converse_kwargs/call_converse to omit the
field, and drop an all-empty inferenceConfig from the wire request
entirely. The 4096 default is unchanged for every existing caller (main
transport passes params.get('max_tokens', 4096) explicitly), so only
no-cap aux calls opt in.
Surfaced during review of #75253.
Legacy compaction mode (compression.in_place: false) rotates the physical
session_id mid-conversation. The prompt-cache scope introduced in #79161 was
derived from that physical id, so every rotation moved the same conversation
into a fresh cache bucket - the prompt cache went cold at every rotation
boundary (#79017).
Fix: resolve a rotation-stable logical scope - the compression-lineage ROOT
of the current session (SessionDB.get_compression_lineage, fork-aware
post-#79193) - once per turn, memoized per transcript segment, and prefer it
over the physical session_id at every prompt_cache_key derivation site:
- agent/prompt_cache_scope.py (new): resolve_prompt_cache_scope(agent) -
lineage-root walk with per-segment memo; falls back to the physical id
when no DB is attached or the walk fails, degrading to pre-fix behavior.
- transports/codex.py: build_kwargs accepts cache_scope_id and prefers it
for the body prompt_cache_key, the xAI x-grok-conv-id header, and the
Codex x-client-request-id routing header. The Codex session_id header
keeps the raw physical id (transcript identity, #57012 contract).
- transports/chat_completions.py: _add_prompt_cache_key accepts
cache_scope_id with the same precedence.
- chat_completion_helpers.py: build_api_kwargs threads the resolved scope
into all three build_kwargs call sites (codex, profile, legacy).
- auxiliary_client.py: set_runtime_main carries cache_scope; the aux
Responses cache-key site prefers it over the physical session_id.
- turn_context.py: resolves the scope once per turn and threads it through
set_runtime_main (no DB walk on the per-API-call hot path).
Scope semantics preserved from #79161: /new starts a fresh scope (new
lineage), /branch children, delegate subagents, and tool children stay
isolated (explicit-fork exclusion in get_compression_lineage), unrelated
sessions keep distinct buckets, and cron per-fire timestamps still
normalize via _cache_scope_from_session_id.
Default installs compact in place (session_id never rotates), so they hit
the memo and produce byte-identical keys to before.
Fixes#79017
_to_openai_base_url() matched ZAI (open.bigmodel.cn, api.z.ai, bare
"bigmodel") and Kimi (api.kimi.com) via `substring in url`, so any custom
gateway whose base_url happened to contain one of those strings as a path
segment (e.g. a reverse-proxy prefix like /proxy/bigmodel-fallback/) was
silently misrouted to the wrong OpenAI-wire endpoint shape.
This is the same false-positive class 6f33f510e8 just fixed for the
MiniMax branch in the same function by switching to base_url_host_matches()
(hostname-anchored). Apply the same fix to the ZAI and Kimi branches, which
that commit didn't touch. Drops the bare "bigmodel" substring check since
open.bigmodel.cn is the only canonical bigmodel-family host referenced
anywhere else in the codebase (agent/model_metadata.py, hermes_cli/auth.py).
Added regression tests mirroring the MiniMax marker-in-path tests added in
the same commit.
SessionDB could leave native SQLite handles open when construction failed
partway through schema/pragma/FTS/repair/lock/interrupt handling. Other
short-lived callers (MCP reads/polling, session search, reactions, trace
upload, insights, shutdown recovery) opened temporary SessionDB handles
without a complete ownership boundary. API-server profile caches and
RetainDB shutdown had similar late-close races. Under sustained load this
exhausted file descriptors (EMFILE).
- Close partially initialized SessionDB connections on every constructor
exception path via a finally block guarded by an initialization-complete
flag.
- Close temporary/cross-profile SessionDB handles in finally blocks across
CLI, MCP, search, trace, reactions, insights, and recovery paths.
- Add API-server per-profile cache ownership and disconnect cleanup.
- Make RetainDB writer-queue shutdown exception-safe: track connections per
thread, close on worker exit, reject new enqueues after shutdown starts,
and sweep any connections left by short-lived threads.
- Add regression coverage for constructor failures, worker-thread readers,
API disconnect failures, shutdown recovery, RetainDB late enqueue, and
foreign-loop async clients.
Salvage notes: the original PR's per-thread WAL-reader ownership changes
were superseded by main's read-connection pool (permits + checkout/return);
its cron timeout-abandon fix is credited separately to #72822's earlier
identical fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A captured native-compaction checkpoint lives in the persisted
codex_reasoning_items sidecar, but the wire restructure that follows it
(prune_pre_checkpoint_items) ran unconditionally: the native gate only
decided whether context_management went into the request, and no signal
from it ever reached _chat_messages_to_responses_input.
So a single checkpoint kept deleting every pre-checkpoint item from all
later requests — after a mid-session swap out of the gpt-5.6 family,
after compression.enabled: false, after the rejection kill switch, and
after a session resume that reloads the sidecar from state.db. The model
receiving the opaque blob was no longer the one able to decode it, and
nothing was logged.
Thread a single native_compaction_eligible boolean, derived from the same
value that gates the context_management field, into the converter. When
ineligible: do not replay type: "compaction" items and do not prune. Safe
because native compaction never truncates Hermes' local history, so the
fallback still carries the full conversation.
All Responses call sites are covered: build_kwargs and convert_messages
derive the flag via _native_compaction_active, the auxiliary/compression
client is explicitly ineligible, and the converter defaults to False
(pre-feature wire) so future call sites are safe by construction.
Fixes#85914
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Substring matching over the whole URL let a path containing
'api.minimax' false-positive an Anthropic-only gateway into the
/anthropic→/v1 rewrite. Parse the host and match exact-domain /
subdomain suffixes (plus the api.minimax.* prefix family) instead.
`_try_anthropic()` applies the configured `model.base_url` only when
`_is_anthropic_compatible_host()` trusts it, but that check accepted only the
literal `api.anthropic.com` host. Anthropic-compatible gateways that expose the
native Messages protocol under a `/anthropic` path suffix (MiniMax, Zhipu GLM,
LiteLLM-style relays, self-hosted proxies) were rejected, so every auxiliary
call (title generation, memory extraction, vision, reflection) and the
`provider: anthropic` fallback chain discarded the configured base_url and fell
back to `https://api.anthropic.com`. That diverges from the primary path, which
already trusts the `/anthropic` suffix via
`runtime_provider._detect_api_mode_for_url`, and fails outright when the gateway
(not Anthropic) holds the credentials.
Accept `/anthropic` and `/anthropic/v1` suffixed URLs in
`_is_anthropic_compatible_host()`, matching the primary-path convention and
`_wrap_if_needed`. A bare non-Anthropic base_url (e.g. `openrouter.ai/api/v1`
left on `provider: anthropic`) still returns False, preserving the #52608 guard.
The custom + explicit_base_url branch of resolve_provider_client()
unconditionally rewrote a trailing /anthropic to /v1 via
_to_openai_base_url(), even when api_mode was anthropic_messages. The
Anthropic wrapper then never saw the real /anthropic path, so auxiliary
tasks (title generation, compression, vision, web_extract,
session_search) hit .../v1/chat/completions on a Messages-only endpoint
and failed.
Guard the wrap base on api_mode: for anthropic_messages, pass the raw
/anthropic base to _wrap_if_needed (which builds the Anthropic wrapper),
while the plain OpenAI client keeps the /v1-rewritten base so the
OpenAI-wire fallback (used when the anthropic SDK is unavailable) never
lands on /anthropic/chat/completions.
Refs #16254
Follow-ups on the salvaged #84782 (webtecnica):
1. conversation_loop.py: the empty-response fallback site sits directly
in the OUTER iteration loop, not the retry loop. The salvaged commit's
`break` there exited the conversation loop and ended the turn without
ever calling the just-activated fallback (caught by CI:
test_empty_response_triggers_fallback_provider). Restored `continue`
(which already re-runs the pre-API preflight at the top of the next
outer iteration) while keeping the `_preflight_compression_blocked`
reset. The other 9 sites are inside the retry loop, where `break` to
the restart_with_rebuilt_messages handler is correct.
2. test_prompt_cache_ttl_propagation.py: made the AST guard loop-aware —
retry-loop sites must break, outer-loop sites must continue (the old
assertion pinned the bug in (1)). Mutation-checked both directions.
3. test_failover_identity.py: added `model` to the SimpleNamespace agent
fixture — _redecorate_prompt_cache_for_provider now reads agent.model
for the per-destination TTL clamp (2 CI failures).
4. prompt_caching.py / agent_runtime_helpers.py: single source of truth
for the alibaba-family provider set — ALIBABA_FAMILY_PROVIDERS lives
in prompt_caching and anthropic_prompt_cache_policy imports it, so the
cache-policy opt-in and the TTL clamp can never desync.
5. auxiliary_client.py: threaded the configured tier into
_replan_synchronous_cache_sections via new configured_cache_ttl()
(no live agent on that path) — the aux half of #84733's report also
stopped regressing 1h to 5m. Guarded by
TestAuxFallbackReplanThreadsTtl (mutation-checked).
6. Dropped the redundant `or "5m"` at the two threaded call sites —
effective_cache_ttl already resolves None to "5m", and the `or`
masked the cache-disabled (None) semantics.
Cold CLI time-to-banner was ~1.8s (hermes) / ~2.8s (hermes -w). The banner
path was paying for work the session doesn't need before first input:
- aux availability probes built REAL OpenAI/httpx clients (openai import
~0.3s + SSL context) just to answer check_fns. New aux_probe_mode()
returns a cache-excluded stub; resolution policy unchanged.
- tools/mcp_tool imported the mcp SDK (~260ms, mcp.types pydantic model
construction) at module import even with zero MCP servers configured.
SDK import is now lazy behind _ensure_mcp_sdk(); _MCP_AVAILABLE is a
find_spec probe so every existing gate/test keeps its semantics.
- banner blocked 500ms on the update-check prefetch; now waits 50ms and
defers the warning line to a daemon thread (prints above the prompt).
- banner recomputed get_tool_definitions + skills scan + git state every
launch; now snapshotted to ~/.hermes/cache/banner_snapshot.json keyed on
(config.yaml, .env, checkout rev, toolsets) and replayed on warm launches
with a background refresh. Agent tool list is still computed fresh.
- _resolve_active_context_length probed the Nous portal /models (~200ms
network) per launch; the tool-search gate now prefers the on-disk
context cache when present.
- schema reconciliation re-executed SCHEMA_SQL in a scratch SQLite DB
(~85ms) per SessionDB(); the reference parse is now disk-memoized by
DDL hash (live-DB diffing still runs every startup).
- bundled-skills sync (~120-170ms rglob/hash) moved off the startup path
to a daemon thread; plugin discovery starts in the background and every
synchronous consumer joins via discover_plugins().
- hermes_cli.auth imported httpx eagerly (~30ms); now a lazy proxy that
test monkeypatching still reaches (setattr forwards to the real module).
- fast chat launch: unambiguous 'hermes'/'hermes chat' invocations skip
building all ~40 subcommand parsers (bails to full dispatch on anything
else, incl. container mode).
- -w path: git worktree add runs with checkout.workers=8 (0.6s→0.2s) and
overlaps HermesCLI construction; --skills preload runs in the background
and is folded in at agent init (finalize_preloaded_skills, same
fail-loud contract for fully-unknown skill lists); stale-worktree prune
moved off the banner path.
Warm results (PTY time-to-banner, 5-run): hermes 1.80s → 0.38-0.40s;
hermes -w -s hermes-agent-dev --yolo 2.82s → 0.57-0.69s.
Two lookalike gaps found auditing the titler.
_MACHINE_PREFIXES missed the compressor's legacy summary opener and the
"[System note:" injections, so a compacted or resumed session could be
named after the note that carried it. Take the summary prefix from the
compressor that emits it rather than keeping a fourth local copy.
The fast-model exclude list covered embedders but not the other non-chat
siblings a provider names after its chat model — "gpt-4o-mini-tts"
satisfies the "-mini" rung and cannot answer a prompt.
The fast-model picker reads /v1/models to find the small model a provider
currently serves, and it asked anonymously. Most of those endpoints need a
key, so the fetch 401'd and the empty result read as "this provider has no
small model" — the picker fell back to its curated list and never noticed.
Worse, a failed fetch cached its empty result forever, so one bad moment
during startup disabled live model discovery for the life of the process,
and the processes that read this run for weeks. Give the failure an expiry
and pass the provider's credentials.
The bare family rungs (-mini, -flash, haiku) also picked whichever id
sorted first, which is the oldest generation a provider still serves:
gpt-3.5-mini over gpt-5.4-mini, claude-3-haiku over claude-haiku-4.5.
Compare the digit runs as numbers so the rung meant to keep us current
does.
Titling ran on the user's main chat model, so a five-word title was billed
to a frontier reasoning model and inherited its latency. Pinning a cheap
model id instead just moves the problem: the hardcoded default was already
dead upstream and every call paid a 404 before the retry net caught it.
Match model FAMILIES against the provider's live /v1/models catalog,
preferring rolling '-latest' aliases where a provider publishes them, and
order the families by measured latency. Nothing to bump when a provider
ships a new mini/flash/haiku. Opt-in per task, so compression, vision, and
search keep 'auto means my chat model'.
Follow-up to the auth.json UTF-8 read fix in this PR. A repo-wide scan for
the same bug class found three more callers that read ~/.hermes/auth.json
via Path.read_text() with no encoding — same Windows cp1252 hazard:
- agent/auxiliary_client.py _read_nous_auth: a non-ASCII byte raised
UnicodeDecodeError, the broad except swallowed it, and Nous silently
stopped being available as the auxiliary (vision/summarization) provider.
- tools/xai_http.py has_xai_credentials: same failure mode — xAI OAuth
silently looked absent on Windows.
- hermes_cli/main.py is_setup_complete: same; has a config.yaml fallback so
the impact is milder, but the read is still wrong.
All three now use read_text(encoding="utf-8-sig"), matching _save_auth_store's
write encoding. A repo-wide grep confirms there are no remaining
json.loads(...read_text()) reads of auth.json without an explicit encoding.
Tests: rewrote the Windows-encoding regression tests to actually exercise the
bug on POSIX too — a new windows_default_encoding fixture forces a no-encoding
read_text() to decode as cp1252 (the Windows default), and _write_utf8 now
emits real non-ASCII UTF-8 bytes (ensure_ascii=False) so the bytes actually
trip cp1252. Verified each test fails when its fix is reverted (including
the two new sibling-reader tests).
Thread request_provider through the sibling callsites that the
original PR left on resolved_provider: _set_relay_auxiliary_route
(observability metadata), the 'using X' log line, _is_anthropic_compat_endpoint
(Anthropic image conversion), _provider_requires_stream (streaming
detection), and the initial _relay_sync/async_completion calls.
These are NOT regressions (they passed 'auto' before), but widening
them ensures auto-routed calls to MiniMax get correct image conversion,
streaming-only providers are detected, and observability metadata
records the concrete backend instead of 'auto'.
Cherry-picked from PR #78959 by @JoaoMarcos44 with authorship preserved.
Follow-up: hoist _cache_scope_from_session_id(session_id) to a local in
build_kwargs so it's computed once instead of 4 times per call.
Closes#78941. Closes#79012. Closes#79013. Closes#79014. Closes#79015.
Co-authored-by: JoaoMarcos44 <joaomarcosdias444@gmail.com>
The PR's concurrency wrapper splits call_llm into a semaphore-guarded
entry + _call_llm_impl; main added extra_headers to call_llm's
signature after the PR's base, so the split has to forward it too
(dropped silently otherwise — Azure Foundry and custom-endpoint
callers set it).
Makes interrupt-protected context compression cancellable by an explicit
user or lifecycle stop, without weakening protection against ordinary
incoming messages, voice interjections, or active-turn redirects.
Separates explicit hard cancellation from ordinary interrupt/redirect
state with a dedicated threading.Event; introduces
AuxiliaryExplicitCancellation as an attempt-local frozen-cause signal;
isolates the synchronous provider callback in a bounded daemon worker
during protected compression; atomically linearizes Codex timeout
cleanup against explicit cancellation; propagates hard cancellation
through child agents and explicit stop surfaces; serializes hard-cancel
admission against compression commit admission with
CompressionCommitFence; aborts before session rotation or late DB commit,
restores in-place transcript mutations and compressor state, and releases
the heartbeat and compression lease.
Based on #74449 by @suparious. Resolved merge conflicts in
agent/context_compressor.py (feasibility check + try/except) and
tui_gateway/methods_session.py.
- Merge _aux_free_only() + _aux_openrouter_model() into single
_aux_openrouter_settings() that reads config once via
load_config_readonly (avoids double deepcopy).
- Remove 15-line block comment and 5-line inline comment that
restated what the code already says.
- Trim module docstring from 10 lines to 3.
- Update test patches to target load_config_readonly.
_call_fallback_candidate_sync replans messages/tools for each resolved
destination, but its async mirror still shipped the caller's decorated
sections verbatim — the primary destination's markers (including a
direct-native tool marker) leaked to fallback candidates with different
cache contracts, and the relay saw the display label instead of the
resolved provider/api_mode. Mirror the sync path: resolve the
destination, replan both sections, thread provider/api_mode through
_relay_async_completion, and replan again for the auth-refresh retry
client. Mutation-checked: the new parity test fails on the verbatim
pass-through shape.
Follow-up to #76032 (#20880).
Three copies of the same logic landed with #76032:
- MoA's _call_prepared_aggregator and auxiliary_client's
_replan_synchronous_cache_sections both implemented stub → policy →
strip → plan for a resolved destination. Extract
plan_cache_sections_for_destination() into agent_runtime_helpers (which
already owns the policy functions) and route both through it. Also
removes a redundant full-transcript deepcopy+strip per request (the
caller pre-stripped what build_prompt_cache_plan strips again).
- The fallback_chain[N] label regex + chain-entry lookup lived in
_fallback_entry_timeout AND _fallback_destination. Extract
_fallback_chain_entry() and reuse.
MoA's cache-plan failure log is promoted debug → warning: the call-block
site skips MoA, so this block is the aggregator's only decoration path —
a silent failure ships an undecorated request (the 0%-cache MoA bug class).
Behavior-preserving; 195 targeted tests green.
Follow-up to #76032 (#20880).
resolve_entry_api_key() and the duplicated _fallback_entry_api_key()
read key_env via a raw os.getenv(), bypassing per-profile secret
scoping in the multiplexed gateway. Under multiplexing this can hand
a fallback request another profile's credential. Both now resolve
through agent.secret_scope.get_secret(), which reads the active
profile scope when multiplexing is on and falls back to os.environ
unchanged when it's off, so single-profile behavior is preserved.
Closes#74311
Narrow the moa_aggregator Relay bypass to CodexAuxiliaryClient and add
coverage that call_llm(stream=True) returns the provider's direct
create() result for Responses-shim clients.
Salvaged from PR #74903 by @liusencomic-cyber.
Convert a completed MoA aggregator response into one valid Chat
Completions delta chunk at the MoA facade boundary, normalize completed
message.tool_calls into indexed stream deltas, and classify these local
MoA adapter-shape errors as non-fallback format errors so a local
compatibility bug cannot silently drift the user's MoA route to a
single model (#55933 follow-up).
Salvaged from PR #74903 by @liusencomic-cyber.
Salvaged from #56085 (@Stoltemberg), rebased onto current main: sites
main had already converted (credential_pool, auxiliary_client MoA
paths, model_metadata, moa_loop, agent_runtime_helpers) resolve to
main's versions; the remaining ~29 read-only sites across 16 agent/
files swap to the no-deepcopy readonly loader (~135us saved per call).
Full per-site mutation audit performed (every enclosing function read,
escapes traced): 23 SAFE, 5 ESCAPES with read-only consumers, 1 UNSAFE
path (init_agent -> get_compatible_custom_providers -> normalizer
in-place alias writes) fixed by the preceding no-mutate commits, which
make the normalizer copy-safe for ALL callers.
gemini-2.5-flash shuts down Oct 16 2026 (Google deprecation schedule)
and gemini-3-flash-preview is superseded. Update every hardcoded
default to the current GA flash model:
- gemini_native_adapter: probe_gemini_tier + _create_chat_completion
default params; free-tier guidance de-pinned from a specific model's
RPD number so it doesn't stale again
- auxiliary_client: gemini/kilocode fallback aux models,
_OPENROUTER_MODEL, _NOUS_MODEL -> google/gemini-3.6-flash
(verified live on both OpenRouter and Nous portal /models)
- provider plugins: kilocode + vertex default_aux_model
- hindsight memory plugin: gemini provider default
- setup wizard gemini list: 3-flash-preview -> 3.6-flash (matches the
curated picker catalog)
- tests: aux-client assertions that pinned the old default literal now
reference the _NOUS_MODEL constant, so the next default bump can't
break them (change-detector cleanup)
Fixes#32360.
Direct tool HTTP calls already identified as Hermes-Agent, but the main
OpenAI-SDK chat path still sent OpenAI/Python. Set Hermes-Agent/<ver> for
api.x.ai clients (xai + xai-oauth) so normal text traffic is attributed correctly.
kshitij