Commit Graph

407 Commits

Author SHA1 Message Date
kshitijk4poor b6bcb3e791 docs(credits): document naming-convention trust in aux free-SKU detector
Mirror the credits_tracker caveat in _is_free_model (a paid stealth/
model would bypass the free_only gate and paid-lane warning) and fix
the stale _warn_paid_lane_once docstring.
2026-08-22 04:20:48 +05:30
kshitijk4poor 8a949659c3 refactor(credits): fold review findings for stealth free-tier fix
- credits_tracker: trim inline comment block (duplicated docstring) and
  correct its safety claim - a paid model under stealth/ would fail
  closed (suppressed banner), not open; state the trade-off honestly.
- run_agent: update stale call-site comment to mention stealth/ prefix.
- auxiliary_client: widen sibling free-SKU detector _is_free_model to
  recognize stealth/ prefix (same bug class as #91843: free_only=true
  wrongly skipped the OpenRouter fallback and the paid-lane warning
  fired spuriously for stealth models).
- tests: bind the new sibling behavior (stealth/ox-alpha free,
  my-stealth/model not).
2026-08-22 04:20:48 +05:30
Vinay Shah 41ca67c5b1 fix(bedrock): align auxiliary region resolution with runtime + document Mantle route
Address review feedback on #65076:

- Add resolve_bedrock_runtime_region() to agent/bedrock_adapter.py: the
  config-first region resolution (bedrock.region in config.yaml, then
  AWS_REGION/AWS_DEFAULT_REGION/botocore profile/us-east-1) that the main
  runtime resolver uses, exposed as a shared helper.
- Switch auxiliary client resolution (agent/auxiliary_client.py aws_sdk
  branch) to the new helper. Previously it derived its region with bare
  resolve_bedrock_region() (env-first), so when config.yaml pinned
  bedrock.region to a different region than the ambient AWS env, auxiliary
  calls (compression, memory, vision) left the primary runtime's region.
  Both the AnthropicBedrock/Converse path and the new Mantle OpenAI
  Responses path now resolve identically to the main runtime.
- Add regression tests covering the bedrock.region-vs-AWS_REGION mismatch
  for both the Claude auxiliary path and the Mantle auxiliary path.
- Update website/docs/guides/aws-bedrock.md: the guide claimed Hermes never
  uses the OpenAI-compatible endpoint, which the Mantle route made stale.
  Document the triple routing (AnthropicBedrock / Mantle OpenAI Responses /
  Converse), the Mantle auth model (bearer token or SigV4), and add the
  GPT-5.5/5.6 model IDs to the models table.
2026-08-21 15:02:29 -07:00
Nathaniel Branscum e5b96fcb10 feat(bedrock): support OpenAI Responses models
Route Bedrock-hosted OpenAI GPT-5.5 through the Bedrock Mantle OpenAI Responses endpoint with SigV4 request signing. Keep native Bedrock Converse and Claude Bedrock routing unchanged, and add picker/runtime regression coverage.
2026-08-21 15:02:29 -07:00
unsupportedpastels ac8dff4fbc fix(compression): auto-raise Daybreak Codex threshold 2026-08-21 13:19:07 -07:00
Teknium 1017a56274 fix: OpenCode Zen free-tier models (Ox Alpha) work keyless — any bearer 401s them
The Zen relay serves *-free models (x-preview-f-free / Ox Alpha) ONLY
anonymously: any Authorization bearer it doesn't recognize is a 401
'Invalid API key' — including our no-key-required placeholder and valid
OpenCode GO subscription keys. The Go relay doesn't serve the free tier
at all ('Model x is not supported'). So the free model failed for every
Hermes user: keyless setups got the placeholder bearer, and OpenCode
subscribers sent a Go key to a relay that rejects it.

Fix (class-wide for all 8 current *-free Zen slugs, not just Ox Alpha):
- hermes_cli/models.py: is_opencode_zen_free_model / opencode_zen_free_runtime
  / opencode_zen_free_headers — one shared policy: free slugs pin to the
  Zen relay with a keyless placeholder and an empty Authorization header
  that overrides the OpenAI SDK's 'Bearer <key>'.
- runtime_provider.py: free slugs route through the keyless runtime before
  the credential-pool/explicit/api_key paths (no key required; Go
  selections heal to Zen). Paid models still fail closed without a key.
- agent_init.py + auxiliary_client.py: the placeholder key swaps in the
  empty-Authorization headers at both client-build chokepoints.

Verified live (2026-08-21): anonymous chat/completions 200 incl. tools,
streaming, parallel; bad bearer 401; full E2E AIAgent turn with a real
terminal tool round-trip completes keyless under both opencode-zen and
opencode-go providers. Sabotage run: routing tests fail without the fix.
2026-08-20 23:05:57 -07:00
Teknium d0573880a8 fix: Codex Responses effort vocabulary is now per-model — gpt-5.5 no longer 400s on 'max' (#68365 confirmed live)
Live probes against api.openai.com/v1/responses (Aug 2026):
- gpt-5.6: accepts none/low/medium/high/xhigh/max; rejects minimal, ultra
- gpt-5.5: accepts none/low/medium/high/xhigh; rejects max ('Unsupported
  value'), minimal, ultra

So #68365's premise was half right: 'max' does 400 — but only on pre-5.6
models; blanket-clamping max->xhigh on gpt-5.6 (its fix) would have capped
the one model that supports max. The declared-vocabulary design absorbs
this as data: codex_supported_efforts(model) picks CODEX_GPT56_EFFORTS or
CODEX_LEGACY_EFFORTS, and the shared clamp does the rest. Both the main
Codex transport and the auxiliary client's Responses path use it.

Wire outcomes: ultra -> max on gpt-5.6, ultra/max -> xhigh on gpt-5.5/o5,
minimal -> low everywhere.
2026-08-19 22:37:46 -07:00
Teknium f7d90c9410 refactor: single canonical reasoning-effort vocabulary ends the per-vendor clamp drift
The #89503/#70058/#74295/#87279 bug class kept regenerating because every
transport and provider profile hand-rolled its own effort translation map
(9 sites, 4 distinct policies). New agent/reasoning_effort.py is the single
source of truth:

- EFFORT_LADDER: canonical low->high ordering (superset check against
  VALID_REASONING_EFFORTS pinned by test)
- clamp_effort(): one policy — supported passes verbatim, otherwise nearest
  WEAKER supported level (never escalate, never invert the ladder), floor
  when nothing weaker, 'none' never a degradation target, declared
  vendor-documented overrides win, bespoke names pass through
- declared wire vocabularies as data: OpenAI-compat, Codex Responses,
  xAI (4.6/legacy), Actual relays, Kimi K3/K2, TokenHub, GLM-5.2,
  DeepSeek V4, Ollama Cloud, Meta, Solar

Converted sites (all behavior-preserving except noted):
- chat_completions chokepoint, Kimi + TokenHub paths
- codex transport (backend branches now pick a declared set)
- auxiliary_client Responses path
- hermes_cli.models clamp_reasoning_effort_to_supported -> thin wrapper
- plugins: kimi-coding, zai, opencode-zen, deepseek, ollama-cloud,
  meta-ai, upstage, custom (copilot already routes via the wrapper)

Behavior fixes the shared policy surfaces:
- ollama-cloud/opencode-go 'minimal' now degrades to 'low' instead of
  being dropped (drop left the server default = MORE thinking than asked)

New tests: ladder contract (every configurable level is clamped by every
declared wire set; monotonicity across the full ladder for every set).
2026-08-19 19:29:10 -07:00
Teknium 9ec5750aca fix: widen reasoning-effort wire translation to sibling sites (#89503 class)
The chat_completions chokepoint fix (ultra->max for every model,
cherry-picked from #89509) has siblings with the same bug shape:

- codex.py: ultra->max was gated on gpt-5.6 only; now baseline for all
  Responses-API models (backend-specific branches still override).
- Kimi top-level reasoning_effort: K3 accepts low/high/max only —
  'medium' and upper-ladder levels were dropped to the medium default
  (400s on K3, ladder inversion on K2). Full ladder mapped per family,
  mirroring the kimi-coding plugin's K3 map.
- TokenHub: 'minimal' fell through to the 'high' default (asked least,
  got most); full ladder now mapped onto low/medium/high.
- auxiliary_client Responses path: ultra->max alongside the existing
  minimal->low clamp.
- custom provider plugin: ultra capped at max instead of forwarded
  verbatim to GLM/vLLM/SGLang backends that reject it.
- copilot plugin: ad-hoc downgrade rules replaced with the shared
  clamp_reasoning_effort_to_supported ladder walk so ultra/max resolve
  to the strongest supported level instead of medium (#74295).

Sabotage-verified: new sibling-site tests fail 6/10 without the fixes.
2026-08-19 16:04:22 -07:00
ethernet 3c675019f1 fix(aux): retry once without response_format when a provider rejects it
Some providers reject the structured-output request field with a hard
400. The error classifier marks a 400 as non-retryable, so one rejected
field failed the whole auxiliary call. Session titles stayed derived
forever (#82816), and no fallback fired.

Three rejection shapes are covered, from live reports:
- vLLM gateways translate response_format into guided_grammar and fail
  when the grammar backend is absent (compile_grammar_error: No module
  named 'xgrammar').
- Some OpenAI-compatible endpoints answer "This response_format type
  is unavailable now".
- Anthropic-compatible gateways that predate structured outputs reject
  the translated field: "output_config: Extra inputs are not
  permitted". The documented case is the bedrock-mantle Messages
  endpoint.

The fix is reactive, the same pattern as the temperature and
max_tokens rungs: when the provider rejects the field, retry once
without it. Callers tolerate an unconstrained reply — the title prompt
demands bare JSON and _extract_title_text has a loose-JSON fallback —
so the call succeeds with prompt compliance instead of failing. The
retry only fires when the request carried the field, and both the sync
and async paths get the same rung.

Closes #82816
2026-08-18 20:34:46 -04:00
ethernet 8f2d61e3de fix(aux): translate top-level response_format kwarg on the Anthropic adapter
The adapter builds the Messages body from a fixed allow-list of kwargs.
A caller that passes response_format as a top-level kwarg (the OpenAI
SDK call shape) got it dropped on the floor. The request succeeded, but
the schema contract silently became prompt compliance. No in-tree
caller uses this shape today. The pin-test makes sure that a future
refactor cannot open this leak again.

The top-level kwarg gets the same output_config.format translation as
the extra_body shape. When a caller sends both shapes, the extra_body
value wins because every in-tree caller uses that shape.

Pin-test pattern from PR #85626 review follow-up.

Co-authored-by: Matt McClean <mmcclean@amazon.com>
2026-08-18 20:34:46 -04:00
Soju06 f709bd8445 fix(aux): translate response_format to output_config.format for anthropic transport
Plugin structured completions (plugin_llm.complete_structured) build an
OpenAI Chat Completions response_format payload in extra_body. The
anthropic_messages transport forwarded it verbatim, and strict
Anthropic-compatible gateways reject it with HTTP 400:

  response_format: OpenAI Chat Completions structured-output shape is
  not supported. Use output_config.format = {"type": "json_schema", ...}

Observed live: every discord-thread-autotitle structured call failed
for 2+ days (1,600+ logged errors) once the main provider became an
anthropic_messages gateway.

Fix: _translate_anthropic_response_format converts
- json_schema  -> output_config.format = {type: json_schema, schema: S}
- json_object  -> permissive object schema (SDK 0.87.0 has no
  schema-less JSON mode)

merging into any existing output_config (adaptive-thinking effort
coexists) and excluding response_format from the raw extra_body
passthrough alongside the existing reasoning exclusion. The async
adapter delegates to the sync adapter via asyncio.to_thread and is
covered by a test. Non-Anthropic transports are unchanged.
2026-08-18 20:34:46 -04:00
LordMelkor 6efab28726 feat(auth): add key_cmd credential source for custom providers
Custom providers could only authenticate from a static credential (inline
api_key or a key_env env var). Enterprise gateways -- SSO/OIDC brokers, cloud
IAM, internal auth proxies -- issue short-lived bearers instead, so a value
copied into .env is stale within the hour: long sessions start returning 401s
and the user has to restart or run an external cron that rewrites .env.

The existing `secrets.command` source does not cover this: it runs once per
process at startup (subsequent calls are no-ops by design), so it cannot
re-mint a credential mid-session.

Add providers.<name>.key_cmd: a command that prints a token, wrapped at
resolution in a zero-argument callable. Both wire clients already accept a
callable api_key and invoke it per request (the Entra ID path established
this), so chat_completions, codex_responses and anthropic_messages all work
unchanged and always send a fresh credential. The callable also routes the
Anthropic client through its per-request Authorization hook, which is what
OAuth-gated gateway routes require -- so no per-vendor auth wiring is needed
anywhere in core.

- cached until shortly before the advertised expiry (60s leeway), so the
  helper runs about once per token lifetime rather than once per request
- expiry is read from the OAuth 2.0 relative `expires_in` when present, and
  otherwise from an absolute ISO 8601 deadline (`expiry`, `expiresOn`), which
  is what CLI token helpers commonly print. Reading only `expires_in` treated
  those helpers as advertising no TTL at all, cached their token for the life
  of the process, and returned 401 on every request once the real deadline
  passed. ISO parsing reuses hermes_cli.auth._parse_iso_timestamp rather than
  adding another datetime parser.
- no synthetic expiry: when no TTL is advertised, or the advertised one is
  unparseable or already past, the token is used and refreshed on 401 instead
  of re-minted on an invented schedule
- stdout contract matches OAuth 2.0 token endpoints and existing agent
  helpers (bare token or JSON access_token/expires_in); multi-line output is
  rejected rather than guessed at, so a misconfigured helper surfaces as a
  clear error instead of a corrupt-credential 401
- precedence: explicit --api-key still wins; otherwise key_cmd beats a
  static api_key/key_env on the same entry
- failures never include the helper's output (may hold a partial token) or
  the command string (may embed a client secret)

Resolution happens on two paths. agent/auxiliary_client.py resolves named
custom providers itself rather than calling _resolve_named_custom_runtime, so
key_cmd is honoured in both: wiring only the runtime resolver leaves the main
agent turn working while every auxiliary call (title generation, compression,
vision, embedding) falls back to the no-key-required placeholder and 401s.
Precedence is identical on both paths, so one config entry cannot yield two
different credentials depending on which resolver the caller reached.

Closes #84162

Signed-off-by: LordMelkor <kray@block.xyz>
2026-08-15 03:16:21 -07:00
kshitij 8b58f9f68f test(bedrock): pin stream-path cap omission; document truthiness edge
Self-review follow-up: cover call_converse_stream's max_tokens=None path
(same builder, previously unpinned) and document why the shim reads the
caller cap with truthiness rather than 'is None' (parity with the
Anthropic shim's reading).
2026-08-15 12:47:27 +05:30
kshitij 5ef52273cd fix(bedrock): let aux calls omit the Converse maxTokens cap
The Bedrock Converse shim hardcoded 'else 4096' when the caller passed no
max_tokens, so auxiliary vision descriptions stayed capped at 4096 tokens
on the Bedrock wire even after #75253 removed the vision call sites' own
caps (#10809 was only partially fixed there).

Converse's inferenceConfig.maxTokens is optional; when omitted, Bedrock
defaults to the model's maximum allowed output. Thread an explicit
max_tokens=None through build_converse_kwargs/call_converse to omit the
field, and drop an all-empty inferenceConfig from the wire request
entirely. The 4096 default is unchanged for every existing caller (main
transport passes params.get('max_tokens', 4096) explicitly), so only
no-cap aux calls opt in.

Surfaced during review of #75253.
2026-08-15 12:47:27 +05:30
kshitij cee2446222 fix(cache): keep prompt_cache_key warm across compression session rotation
Legacy compaction mode (compression.in_place: false) rotates the physical
session_id mid-conversation. The prompt-cache scope introduced in #79161 was
derived from that physical id, so every rotation moved the same conversation
into a fresh cache bucket - the prompt cache went cold at every rotation
boundary (#79017).

Fix: resolve a rotation-stable logical scope - the compression-lineage ROOT
of the current session (SessionDB.get_compression_lineage, fork-aware
post-#79193) - once per turn, memoized per transcript segment, and prefer it
over the physical session_id at every prompt_cache_key derivation site:

- agent/prompt_cache_scope.py (new): resolve_prompt_cache_scope(agent) -
  lineage-root walk with per-segment memo; falls back to the physical id
  when no DB is attached or the walk fails, degrading to pre-fix behavior.
- transports/codex.py: build_kwargs accepts cache_scope_id and prefers it
  for the body prompt_cache_key, the xAI x-grok-conv-id header, and the
  Codex x-client-request-id routing header. The Codex session_id header
  keeps the raw physical id (transcript identity, #57012 contract).
- transports/chat_completions.py: _add_prompt_cache_key accepts
  cache_scope_id with the same precedence.
- chat_completion_helpers.py: build_api_kwargs threads the resolved scope
  into all three build_kwargs call sites (codex, profile, legacy).
- auxiliary_client.py: set_runtime_main carries cache_scope; the aux
  Responses cache-key site prefers it over the physical session_id.
- turn_context.py: resolves the scope once per turn and threads it through
  set_runtime_main (no DB walk on the per-API-call hot path).

Scope semantics preserved from #79161: /new starts a fresh scope (new
lineage), /branch children, delegate subagents, and tool children stay
isolated (explicit-fork exclusion in get_compression_lineage), unrelated
sessions keep distinct buckets, and cron per-fire timestamps still
normalize via _cache_scope_from_session_id.

Default installs compact in place (session_id never rotates), so they hit
the memo and produce byte-identical keys to before.

Fixes #79017
2026-08-15 11:09:56 +05:30
pierrenode 2d9f116351 fix(agent): anchor ZAI/Kimi base_url host matching to avoid substring false positives
_to_openai_base_url() matched ZAI (open.bigmodel.cn, api.z.ai, bare
"bigmodel") and Kimi (api.kimi.com) via `substring in url`, so any custom
gateway whose base_url happened to contain one of those strings as a path
segment (e.g. a reverse-proxy prefix like /proxy/bigmodel-fallback/) was
silently misrouted to the wrong OpenAI-wire endpoint shape.

This is the same false-positive class 6f33f510e8 just fixed for the
MiniMax branch in the same function by switching to base_url_host_matches()
(hostname-anchored). Apply the same fix to the ZAI and Kimi branches, which
that commit didn't touch. Drops the bare "bigmodel" substring check since
open.bigmodel.cn is the only canonical bigmodel-family host referenced
anywhere else in the codebase (agent/model_metadata.py, hermes_cli/auth.py).

Added regression tests mirroring the MiniMax marker-in-path tests added in
the same commit.
2026-08-14 22:04:16 -07:00
joaomarcos 39e480c051 fix(state): close leaked SessionDB connections on exception paths (#83226)
SessionDB could leave native SQLite handles open when construction failed
partway through schema/pragma/FTS/repair/lock/interrupt handling. Other
short-lived callers (MCP reads/polling, session search, reactions, trace
upload, insights, shutdown recovery) opened temporary SessionDB handles
without a complete ownership boundary. API-server profile caches and
RetainDB shutdown had similar late-close races. Under sustained load this
exhausted file descriptors (EMFILE).

- Close partially initialized SessionDB connections on every constructor
  exception path via a finally block guarded by an initialization-complete
  flag.
- Close temporary/cross-profile SessionDB handles in finally blocks across
  CLI, MCP, search, trace, reactions, insights, and recovery paths.
- Add API-server per-profile cache ownership and disconnect cleanup.
- Make RetainDB writer-queue shutdown exception-safe: track connections per
  thread, close on worker exit, reject new enqueues after shutdown starts,
  and sweep any connections left by short-lived threads.
- Add regression coverage for constructor failures, worker-thread readers,
  API disconnect failures, shutdown recovery, RetainDB late enqueue, and
  foreign-loop async clients.

Salvage notes: the original PR's per-thread WAL-reader ownership changes
were superseded by main's read-connection pool (permits + checkout/return);
its cron timeout-abandon fix is credited separately to #72822's earlier
identical fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-14 21:41:26 -07:00
joaomarcos 11c5aae104 fix(compaction): gate checkpoint replay/prune on current request eligibility
A captured native-compaction checkpoint lives in the persisted
codex_reasoning_items sidecar, but the wire restructure that follows it
(prune_pre_checkpoint_items) ran unconditionally: the native gate only
decided whether context_management went into the request, and no signal
from it ever reached _chat_messages_to_responses_input.

So a single checkpoint kept deleting every pre-checkpoint item from all
later requests — after a mid-session swap out of the gpt-5.6 family,
after compression.enabled: false, after the rejection kill switch, and
after a session resume that reloads the sidecar from state.db. The model
receiving the opaque blob was no longer the one able to decode it, and
nothing was logged.

Thread a single native_compaction_eligible boolean, derived from the same
value that gates the context_management field, into the converter. When
ineligible: do not replay type: "compaction" items and do not prune. Safe
because native compaction never truncates Hermes' local history, so the
fallback still carries the full conversation.

All Responses call sites are covered: build_kwargs and convert_messages
derive the flag via _native_compaction_active, the auxiliary/compression
client is explicitly ineligible, and the converter defaults to False
(pre-feature wire) so future call sites are safe by construction.

Fixes #85914

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 10:39:25 -07:00
Xie ddbef9cd79 fix(auxiliary): keep ZAI Coding Plan routing 2026-08-13 12:30:29 -07:00
Teknium 6f33f510e8 fix(agent): anchor dual-surface marker matching to the URL host
Substring matching over the whole URL let a path containing
'api.minimax' false-positive an Anthropic-only gateway into the
/anthropic→/v1 rewrite. Parse the host and match exact-domain /
subdomain suffixes (plus the api.minimax.* prefix family) instead.
2026-08-13 11:45:27 -07:00
686f6c61 0d24c4f413 fix(agent): only rewrite /anthropic→/v1 for dual-surface hosts
Unconditionally rewriting any */anthropic base_url to /v1 broke
Anthropic-only custom gateways (e.g. Alibaba Bailian Token Plan) used by
auxiliary compression/vision under provider=auto. Keep MiniMax dual-surface
hosts rewriting; leave pure Anthropic paths intact.

Fixes #83642
2026-08-13 11:45:27 -07:00
iso2kx cd344a280f fix(auxiliary): honor /anthropic-suffixed gateway base_url on aux + fallback calls
`_try_anthropic()` applies the configured `model.base_url` only when
`_is_anthropic_compatible_host()` trusts it, but that check accepted only the
literal `api.anthropic.com` host. Anthropic-compatible gateways that expose the
native Messages protocol under a `/anthropic` path suffix (MiniMax, Zhipu GLM,
LiteLLM-style relays, self-hosted proxies) were rejected, so every auxiliary
call (title generation, memory extraction, vision, reflection) and the
`provider: anthropic` fallback chain discarded the configured base_url and fell
back to `https://api.anthropic.com`. That diverges from the primary path, which
already trusts the `/anthropic` suffix via
`runtime_provider._detect_api_mode_for_url`, and fails outright when the gateway
(not Anthropic) holds the credentials.

Accept `/anthropic` and `/anthropic/v1` suffixed URLs in
`_is_anthropic_compatible_host()`, matching the primary-path convention and
`_wrap_if_needed`. A bare non-Anthropic base_url (e.g. `openrouter.ai/api/v1`
left on `provider: anthropic`) still returns False, preserving the #52608 guard.
2026-08-13 11:41:59 -07:00
Nikita Barkov 3cf8293e44 fix(auxiliary): keep /anthropic base_url for anthropic_messages custom endpoints
The custom + explicit_base_url branch of resolve_provider_client()
unconditionally rewrote a trailing /anthropic to /v1 via
_to_openai_base_url(), even when api_mode was anthropic_messages. The
Anthropic wrapper then never saw the real /anthropic path, so auxiliary
tasks (title generation, compression, vision, web_extract,
session_search) hit .../v1/chat/completions on a Messages-only endpoint
and failed.

Guard the wrap base on api_mode: for anthropic_messages, pass the raw
/anthropic base to _wrap_if_needed (which builds the Anthropic wrapper),
while the plain OpenAI client keeps the /v1-rewritten base so the
OpenAI-wire fallback (used when the anthropic SDK is unavailable) never
lands on /anthropic/chat/completions.

Refs #16254
2026-08-13 10:32:51 -07:00
kshitij d7517d6e73 fix: restore empty-response fallback retry, dedupe alibaba set, thread TTL into aux replan
Follow-ups on the salvaged #84782 (webtecnica):

1. conversation_loop.py: the empty-response fallback site sits directly
   in the OUTER iteration loop, not the retry loop. The salvaged commit's
   `break` there exited the conversation loop and ended the turn without
   ever calling the just-activated fallback (caught by CI:
   test_empty_response_triggers_fallback_provider). Restored `continue`
   (which already re-runs the pre-API preflight at the top of the next
   outer iteration) while keeping the `_preflight_compression_blocked`
   reset. The other 9 sites are inside the retry loop, where `break` to
   the restart_with_rebuilt_messages handler is correct.

2. test_prompt_cache_ttl_propagation.py: made the AST guard loop-aware —
   retry-loop sites must break, outer-loop sites must continue (the old
   assertion pinned the bug in (1)). Mutation-checked both directions.

3. test_failover_identity.py: added `model` to the SimpleNamespace agent
   fixture — _redecorate_prompt_cache_for_provider now reads agent.model
   for the per-destination TTL clamp (2 CI failures).

4. prompt_caching.py / agent_runtime_helpers.py: single source of truth
   for the alibaba-family provider set — ALIBABA_FAMILY_PROVIDERS lives
   in prompt_caching and anthropic_prompt_cache_policy imports it, so the
   cache-policy opt-in and the TTL clamp can never desync.

5. auxiliary_client.py: threaded the configured tier into
   _replan_synchronous_cache_sections via new configured_cache_ttl()
   (no live agent on that path) — the aux half of #84733's report also
   stopped regressing 1h to 5m. Guarded by
   TestAuxFallbackReplanThreadsTtl (mutation-checked).

6. Dropped the redundant `or "5m"` at the two threaded call sites —
   effective_cache_ttl already resolves None to "5m", and the `or`
   masked the cache-disabled (None) semantics.
2026-08-13 15:24:47 +05:30
Magnus Hedemark 79c11aea7a fix(plugins): complete task-routed LLM integration 2026-08-12 16:25:20 -07:00
fangliquanflq 87af576e60 fix(auxiliary): honor main model for title generation (#83636) 2026-08-11 23:36:47 -05:00
Teknium 55f9e472a0 perf(cli): sub-400ms warm startup — probe-mode check_fns, lazy MCP SDK, banner snapshot, parallel worktree add
Cold CLI time-to-banner was ~1.8s (hermes) / ~2.8s (hermes -w). The banner
path was paying for work the session doesn't need before first input:

- aux availability probes built REAL OpenAI/httpx clients (openai import
  ~0.3s + SSL context) just to answer check_fns. New aux_probe_mode()
  returns a cache-excluded stub; resolution policy unchanged.
- tools/mcp_tool imported the mcp SDK (~260ms, mcp.types pydantic model
  construction) at module import even with zero MCP servers configured.
  SDK import is now lazy behind _ensure_mcp_sdk(); _MCP_AVAILABLE is a
  find_spec probe so every existing gate/test keeps its semantics.
- banner blocked 500ms on the update-check prefetch; now waits 50ms and
  defers the warning line to a daemon thread (prints above the prompt).
- banner recomputed get_tool_definitions + skills scan + git state every
  launch; now snapshotted to ~/.hermes/cache/banner_snapshot.json keyed on
  (config.yaml, .env, checkout rev, toolsets) and replayed on warm launches
  with a background refresh. Agent tool list is still computed fresh.
- _resolve_active_context_length probed the Nous portal /models (~200ms
  network) per launch; the tool-search gate now prefers the on-disk
  context cache when present.
- schema reconciliation re-executed SCHEMA_SQL in a scratch SQLite DB
  (~85ms) per SessionDB(); the reference parse is now disk-memoized by
  DDL hash (live-DB diffing still runs every startup).
- bundled-skills sync (~120-170ms rglob/hash) moved off the startup path
  to a daemon thread; plugin discovery starts in the background and every
  synchronous consumer joins via discover_plugins().
- hermes_cli.auth imported httpx eagerly (~30ms); now a lazy proxy that
  test monkeypatching still reaches (setattr forwards to the real module).
- fast chat launch: unambiguous 'hermes'/'hermes chat' invocations skip
  building all ~40 subcommand parsers (bails to full dispatch on anything
  else, incl. container mode).
- -w path: git worktree add runs with checkout.workers=8 (0.6s→0.2s) and
  overlaps HermesCLI construction; --skills preload runs in the background
  and is folded in at agent init (finalize_preloaded_skills, same
  fail-loud contract for fully-unknown skill lists); stale-worktree prune
  moved off the banner path.

Warm results (PTY time-to-banner, 5-run): hermes 1.80s → 0.38-0.40s;
hermes -w -s hermes-agent-dev --yolo 2.82s → 0.57-0.69s.
2026-08-10 10:40:19 -07:00
Brooklyn Nicholson 53a4003208 fix(agent): stop titling a session after our own scaffolding, or a TTS model
Two lookalike gaps found auditing the titler.

_MACHINE_PREFIXES missed the compressor's legacy summary opener and the
"[System note:" injections, so a compacted or resumed session could be
named after the note that carried it. Take the summary prefix from the
compressor that emits it rather than keeping a fourth local copy.

The fast-model exclude list covered embedders but not the other non-chat
siblings a provider names after its chat model — "gpt-4o-mini-tts"
satisfies the "-mini" rung and cannot answer a prompt.
2026-08-09 04:33:58 -05:00
Brooklyn Nicholson 071eab821b fix(models): let the titler actually see a provider's model catalog
The fast-model picker reads /v1/models to find the small model a provider
currently serves, and it asked anonymously. Most of those endpoints need a
key, so the fetch 401'd and the empty result read as "this provider has no
small model" — the picker fell back to its curated list and never noticed.

Worse, a failed fetch cached its empty result forever, so one bad moment
during startup disabled live model discovery for the life of the process,
and the processes that read this run for weeks. Give the failure an expiry
and pass the provider's credentials.

The bare family rungs (-mini, -flash, haiku) also picked whichever id
sorted first, which is the oldest generation a provider still serves:
gpt-3.5-mini over gpt-5.4-mini, claude-3-haiku over claude-haiku-4.5.
Compare the digit runs as numbers so the rung meant to keep us current
does.
2026-08-09 04:33:58 -05:00
Brooklyn Nicholson e358eaf44a perf(sessions): resolve the titling model from the provider's live catalog
Titling ran on the user's main chat model, so a five-word title was billed
to a frontier reasoning model and inherited its latency. Pinning a cheap
model id instead just moves the problem: the hardcoded default was already
dead upstream and every call paid a 404 before the retry net caught it.

Match model FAMILIES against the provider's live /v1/models catalog,
preferring rolling '-latest' aliases where a provider publishes them, and
order the families by measured latency. Nothing to bump when a provider
ships a new mini/flash/haiku. Opt-in per task, so compression, vision, and
search keep 'auto means my chat model'.
2026-08-08 17:07:21 -05:00
Nolan 2fda6a384c fix(auth): cover remaining auth.json readers across modules
Follow-up to the auth.json UTF-8 read fix in this PR. A repo-wide scan for
the same bug class found three more callers that read ~/.hermes/auth.json
via Path.read_text() with no encoding — same Windows cp1252 hazard:

- agent/auxiliary_client.py _read_nous_auth: a non-ASCII byte raised
  UnicodeDecodeError, the broad except swallowed it, and Nous silently
  stopped being available as the auxiliary (vision/summarization) provider.
- tools/xai_http.py has_xai_credentials: same failure mode — xAI OAuth
  silently looked absent on Windows.
- hermes_cli/main.py is_setup_complete: same; has a config.yaml fallback so
  the impact is milder, but the read is still wrong.

All three now use read_text(encoding="utf-8-sig"), matching _save_auth_store's
write encoding. A repo-wide grep confirms there are no remaining
json.loads(...read_text()) reads of auth.json without an explicit encoding.

Tests: rewrote the Windows-encoding regression tests to actually exercise the
bug on POSIX too — a new windows_default_encoding fixture forces a no-encoding
read_text() to decode as cp1252 (the Windows default), and _write_utf8 now
emits real non-ASCII UTF-8 bytes (ensure_ascii=False) so the bytes actually
trip cp1252. Verified each test fails when its fix is reverted (including
the two new sibling-reader tests).
2026-08-08 12:32:23 -07:00
kshitij c95a1b7171 fix(auxiliary): widen effective provider to relay, logging, and endpoint detection
Thread request_provider through the sibling callsites that the
original PR left on resolved_provider: _set_relay_auxiliary_route
(observability metadata), the 'using X' log line, _is_anthropic_compat_endpoint
(Anthropic image conversion), _provider_requires_stream (streaming
detection), and the initial _relay_sync/async_completion calls.

These are NOT regressions (they passed 'auto' before), but widening
them ensures auto-routed calls to MiniMax get correct image conversion,
streaming-only providers are detected, and observability metadata
records the concrete backend instead of 'auto'.
2026-08-07 16:56:37 +05:30
Gille 293e67328c fix(agent): preserve auto-routed provider identity 2026-08-07 16:56:37 +05:30
Justin Bennington a9acb400ba feat(providers): add Actual Computer inference provider 2026-08-05 14:08:32 -07:00
joaomarcos 34c3f06f91 fix(cache): scope prompt_cache_key by session to stop cross-session bucket sharing
Cherry-picked from PR #78959 by @JoaoMarcos44 with authorship preserved.
Follow-up: hoist _cache_scope_from_session_id(session_id) to a local in
build_kwargs so it's computed once instead of 4 times per call.

Closes #78941. Closes #79012. Closes #79013. Closes #79014. Closes #79015.

Co-authored-by: JoaoMarcos44 <joaomarcosdias444@gmail.com>
2026-08-05 12:42:46 +05:30
Jeffrey Quesnelle 5943bab1ec Merge branch 'main' into feat/hermes-relay-model-metrics 2026-08-04 12:07:51 -04:00
kshitijk4poor ddae511ab1 fix: thread extra_headers through the call_llm split
The PR's concurrency wrapper splits call_llm into a semaphore-guarded
entry + _call_llm_impl; main added extra_headers to call_llm's
signature after the PR's base, so the split has to forward it too
(dropped silently otherwise — Azure Foundry and custom-endpoint
callers set it).
2026-08-03 23:01:03 +05:30
konsisumer 23f8ae32c0 fix(agent): cap auxiliary LLM concurrency per task 2026-08-03 23:01:03 +05:30
Alex Fournier a97abcd55a Merge upstream main into model metrics
Signed-off-by: Alex Fournier <afournier@nvidia.com>

# Conflicts:
#	tests/agent/test_auxiliary_relay.py
2026-08-02 20:10:47 -07:00
Teknium ca5ce1110b fix(secrets): route auxiliary-client provider key reads through the profile secret scope
OPENROUTER_API_KEY (client construction, unavailability description,
max-tokens param heuristic), OPENAI_API_KEY (custom runtime + explicit
custom endpoint), named-custom-provider key_env, and auxiliary task-config
key_env now resolve via a shared _scoped_key_env helper (get_secret with
UnscopedSecretError → os.environ fallback for unscoped CLI/startup probes).
2026-08-02 10:02:33 -07:00
Shaun Prince d15b638a88 fix(compression): let explicit interrupts cancel safely
Makes interrupt-protected context compression cancellable by an explicit
user or lifecycle stop, without weakening protection against ordinary
incoming messages, voice interjections, or active-turn redirects.

Separates explicit hard cancellation from ordinary interrupt/redirect
state with a dedicated threading.Event; introduces
AuxiliaryExplicitCancellation as an attempt-local frozen-cause signal;
isolates the synchronous provider callback in a bounded daemon worker
during protected compression; atomically linearizes Codex timeout
cleanup against explicit cancellation; propagates hard cancellation
through child agents and explicit stop surfaces; serializes hard-cancel
admission against compression commit admission with
CompressionCommitFence; aborts before session rotation or late DB commit,
restores in-place transcript mutations and compressor state, and releases
the heartbeat and compression lease.

Based on #74449 by @suparious. Resolved merge conflicts in
agent/context_compressor.py (feasibility check + try/except) and
tui_gateway/methods_session.py.
2026-08-02 22:15:20 +05:30
kshitij 077317a0f6 simplify: collapse aux config reads, trim comments, use load_config_readonly
- Merge _aux_free_only() + _aux_openrouter_model() into single
  _aux_openrouter_settings() that reads config once via
  load_config_readonly (avoids double deepcopy).
- Remove 15-line block comment and 5-line inline comment that
  restated what the code already says.
- Trim module docstring from 10 lines to 3.
- Update test patches to target load_config_readonly.
2026-08-02 12:19:04 +05:30
webtecnica c19c63d9e6 fix(agent): make auxiliary auto-chain fallback configurable and free-only (#75803) 2026-08-02 12:19:04 +05:30
kshitij 536754919d fix(auxiliary): replan cache sections on the async fallback path too
_call_fallback_candidate_sync replans messages/tools for each resolved
destination, but its async mirror still shipped the caller's decorated
sections verbatim — the primary destination's markers (including a
direct-native tool marker) leaked to fallback candidates with different
cache contracts, and the relay saw the display label instead of the
resolved provider/api_mode. Mirror the sync path: resolve the
destination, replan both sections, thread provider/api_mode through
_relay_async_completion, and replan again for the auth-refresh retry
client. Mutation-checked: the new parity test fails on the verbatim
pass-through shape.

Follow-up to #76032 (#20880).
2026-08-01 15:39:58 +05:30
kshitij af06308425 refactor(prompt-caching): collapse triplicated destination-plan and label parsing
Three copies of the same logic landed with #76032:
- MoA's _call_prepared_aggregator and auxiliary_client's
  _replan_synchronous_cache_sections both implemented stub → policy →
  strip → plan for a resolved destination. Extract
  plan_cache_sections_for_destination() into agent_runtime_helpers (which
  already owns the policy functions) and route both through it. Also
  removes a redundant full-transcript deepcopy+strip per request (the
  caller pre-stripped what build_prompt_cache_plan strips again).
- The fallback_chain[N] label regex + chain-entry lookup lived in
  _fallback_entry_timeout AND _fallback_destination. Extract
  _fallback_chain_entry() and reuse.

MoA's cache-plan failure log is promoted debug → warning: the call-block
site skips MoA, so this block is the aggregator's only decoration path —
a silent failure ships an undecorated request (the 0%-cache MoA bug class).

Behavior-preserving; 195 targeted tests green.

Follow-up to #76032 (#20880).
2026-08-01 15:39:58 +05:30
Rod Boev 9fc12bf7a4 perf(prompt-caching): preserve tool-loop cache boundaries (#20880) 2026-08-01 14:27:12 +05:30
joaomarcos d52a1c25e0 fix(auth): resolve fallback api keys through secret_scope, not raw env
resolve_entry_api_key() and the duplicated _fallback_entry_api_key()
read key_env via a raw os.getenv(), bypassing per-profile secret
scoping in the multiplexed gateway. Under multiplexing this can hand
a fallback request another profile's credential. Both now resolve
through agent.secret_scope.get_secret(), which reads the active
profile scope when multiplexing is on and falls back to os.environ
unchanged when it's off, so single-profile behavior is preserved.

Closes #74311
2026-07-31 22:35:26 -07:00
Kyle 48ded07155 fix(codex): handle completed auxiliary responses 2026-07-31 21:52:36 -07:00
knocksen 716d274f58 test(moa): cover direct provider streaming for Codex-shim aggregators
Narrow the moa_aggregator Relay bypass to CodexAuxiliaryClient and add
coverage that call_llm(stream=True) returns the provider's direct
create() result for Responses-shim clients.

Salvaged from PR #74903 by @liusencomic-cyber.
2026-07-31 21:52:35 -07:00