The compression/aux fallback ladder decides "payment error" via
_is_payment_error(), which only knew the spaced "resource exhausted". NVIDIA NIM
and gRPC-style wrappers serialize the same quota signal as ResourceExhausted /
RESOURCE_EXHAUSTED / resource-exhausted, so a 403/429/status-less body carrying
it re-raised instead of walking the configured fallback_chain, and compression
fell to the lossy emergency path. Add the three separator variants to
_PAYMENT_KEYWORDS (same status gate, no broader "exhausted" matching).
evals/auxiliary_resource_exhausted.py drives the real call_llm/async_call_llm
against two local OpenAI-compatible listeners (nvidia profile primary, named
custom fallback) for the before/after check. #85649
Portal serves anthropic/* on two routes. The native /v1/messages wire, which
Hermes has used since 02d5e23085, re-writes the previous turn's prompt cache
on 14-20% of consecutive calls in concurrent tool loops; the chat/completions
route does not. Measured 2026-09-06, 20 concurrent sessions x 6 tool calls on
Fable 5.1, same account, same hour, same first-party pin:
Nous /v1/messages 180 pairs, 25 stuck (13.9%); 3 earlier 40-session
runs 15.0 / 17.3 / 20.2%; unchanged by the pin
Nous /v1/chat/completions 320 pairs (2 runs), 0 stuck
OpenRouter direct, pinned 161 pairs, 0 stuck
"stuck" = cache_read on call k+1 equals cache_read on call k instead of
call k's prompt size: the last write was not visible, the turn (~28K) was
written twice. Every stuck pair had byte-identical system, tools and
per-message shas, so it is not client-side mutation. At $10-20/M for
cache writes that is 15-20% of a fan-out's write bill.
The cause is inside the portal's native route (every element of its
outgoing request reproduces clean from outside; NousResearch/api#227
carries the diagnostics). Until it is fixed, anthropic/* rides
chat/completions. nous.anthropic_wire: native opts back in. Cost of
chat: prior-turn thinking travels as OpenAI-style reasoning fields, and
cache_control scopes are translated by the portal's adapter.
Tests: new test_nous_anthropic_wire_default.py reads the knob through the
real config loader in a temp HERMES_HOME and through
resolve_runtime_provider (both settings); the existing native-wire
contract suite selects native via an autouse fixture so the wire keeps
working for the flip-back. Live: a real AIAgent on this branch,
provider=nous + anthropic/claude-fable-5.1, dispatches to
chat/completions with the session_id intact.
Follow-up to the salvaged contributor commit: the Gemini and Vertex guide model
tables now list both Flash generations the pickers offer, and an invariant test
ties the OpenRouter/Nous curated google/gemini-*-flash entries to (a) a Google
official-docs pricing row on the direct Gemini and Vertex routes and (b)
membership in the direct gemini/vertex picker lists. Red on main
(gemini-3.7-flash -> unknown), green with the contributor commit.
Campaign: https://github.com/NousResearch/hermes-agent/issues/104154
- Generalize Gemini 3 thinking config model prefix match to gemini-3*
- Add pricing snapshot entries for gemini-3.7-flash and gemini-3.8-flash
- Add model identifiers to Google, OpenRouter, and Vertex CLI lists
- Add test coverage for gemini-3.8-flash thinkingLevel configuration
The ChatGPT Codex backend answers a rejected encrypted-reasoning replay with the
same bare 400 {code: invalid_prompt, message: "Request blocked."} it uses for
genuine blocks. classify_api_error bucketed it as format_error, so the one-shot
repair that already exists for invalid_encrypted_content (disable replay, strip
cached codex_reasoning_items, retry once) never ran and the turn aborted.
Classify exactly that envelope from provider openai-codex — SDK 400 body, SSE
``error`` frame, or the ``response.failed`` text — as invalid_encrypted_content
while keeping format_error's abort-and-fallback hints, so the only behavioural
delta is turn_recovery's replay strip, which still requires cached reasoning
items. A block with nothing to strip aborts exactly as before; other providers,
other messages/codes and the #18028 safety refusals are untouched.
evals/codex_masked_replay_review.py drives a real AIAgent (codex_responses)
against a synthetic local Responses server for the before/after check.
Refs #92353, #92357, #94077.
Fold the adapter-level unsigned-preservation case and the name-detector matrices
into one parametrized invariant (thinking family keeps unsigned only; non-thinking
DeepSeek and unrelated models still take the generic third-party strip) plus the
native-endpoint immutability guard. Add evals/anthropic_proxy_thinking_replay.py,
a real-AIAgent probe against a synthetic local Anthropic-compatible relay used for
the before/after wire check.
Third-party Anthropic-compatible relays serving a DeepSeek thinking model
(deepseek-r/v4/pro/flash) follow the same replay contract as DeepSeek's
native /anthropic endpoint: strip signed thinking blocks, preserve
unsigned ones. Previously these sessions hit the generic third-party path
and lost ALL thinking blocks, breaking cross-turn reasoning coherence.
Detection is model-name based (vendor prefixes stripped), gated on the
existing _is_third_party check so direct Anthropic traffic is untouched.
The salvaged helper was appended to the gateway/run.py facade; new behaviour
belongs in a topical sibling. agent/session_activity.py already owns the
activity-snapshot contract the three render sites read from, so the formatter
moves there as format_iteration_progress and the three call sites import it at
module level instead of late-importing the facade.
Tests trimmed to the salvage bar (<= 2 invariant tests per fix): one
parameterized contract on the helper (unbounded/None hide the ceiling, a real
budget keeps N/M) plus the contributor's end-to-end busy-ack test through the
real render path. The per-site heartbeat/timeout tests exercised the same
helper through mocks and are dropped; evals/gateway_status_render/
iteration_ceiling_ab.py drives all three real render sites with a real AIAgent
for before/after evidence (3 sentinel leaks on main -> 0).
Related: #103109 (same fix, same target module; credited in the PR),
#102845 (third render site, closed by its author in favour of #102817).
`provider_routing.models.<model-id>` now takes the same only/ignore/order/sort/
require_parameters/data_collection keys and overlays the flat provider_routing
values whenever the agent is on that model. Resolution lives in the one
chokepoint every request path already uses (_provider_preferences_for_agent),
so CLI, gateway, TUI/Desktop, cron, /model switches, fallback activation and
delegated children on another model all honour it with no per-surface plumbing.
Matching is spelling-tolerant, sharing _canonical_model_variants with
agent.reasoning_overrides.
The OpenRouter profile's speed-tier pin no longer overwrites an explicit user
`only` on the BASE gpt-6-astra slug: the pin exists to keep default routing off
flex/fast, and a user pin is the stronger intent (only: [openai] stays [openai]
instead of becoming [openai, azure, azure/us]). Tier slugs (-fast/-flex) keep
owning `only`.
Live A/B (config only: {gpt-6-astra: [openai], claude-fable-5.1: [anthropic]}):
main sent {"sort":"price"} for fable and OpenRouter served it from Azure; with
this change it sends {"only":["anthropic"],"sort":"price"} and Anthropic serves it.
Schema proposed in #24495 (samplesabotage) and #100711 (Artemonim); this is a
slim chokepoint implementation of that design.
Co-authored-by: samplesabotage <samplesabotage@users.noreply.github.com>
- probe_only is consumed inside __init__ only; no instance attribute.
- _sync_manager is assigned only on the probe branch, after the connection is
up, so a normal SSHEnvironment whose constructor fails still behaves exactly
as before (its __del__ cleanup does not reach the shared socket teardown).
- _remote_home has no reader on the probe path; not assigned.
- prompt_builder passes probe_only=True unconditionally: which backends honor
it is the builder table's decision, not a caller-side env_type check.
- Tests trimmed to the probe_only contract: the cleanup-raises case was green
on main (pre-existing try/except), the socket-name length and double-cleanup
assertions covered pre-existing behaviour.
The prompt-time backend probe built a normal SSHEnvironment just to run a
one-line `uname`. That constructor detects the remote home, creates the
~/.hermes tree, force-uploads every sync file and snapshots a login session;
when the throwaway object was later garbage-collected, __del__ -> cleanup()
ran sync_back() and `ssh -O exit` against the ControlMaster socket the
agent's real environment shares (keyed by user@host:port).
Add an internal probe_only construction path for SSH: an isolated,
same-length ControlMaster socket (keyed by the instance's session id), no
remote dir setup, no FileSyncManager, no session snapshot. The probe now
tears its own connection down explicitly on success, non-zero exit and
exception, without replacing the probe result when cleanup fails. Normal
SSH callers and non-SSH backends are unchanged.
Salvaged from #77933 onto the facade/sibling layout (the probe body moved to
_run_backend_probe, _create_environment to tools/terminal_tool_backends.py).
_find_git_root walked cwd's parents with Path.exists(); on hosts where a parent is
mode 700 for another user that raised PermissionError out of prompt construction.
Treat an unreadable ancestor as 'no .git here'.
Behaviour-preserving cleanup of the salvaged fix:
- reset_statuses goes back to replace(e, **_CLEAR_STATUS, ...) instead of
re-spelling the six status fields; every other clear site uses the
constant, so a seventh field would otherwise drift here. The
count/new_entries/cleared_ids trio collapses to the original stale-list
shape, and entry.failure_reason is read through the dataclass's
__getattr__ like the rest of the module.
- Both bypass sites (write_credential_pool, _update_root_pool_rows) pass
disk_entry=None to _merge_disk_cooldown_state for a cleared id instead of
adding a parallel branch; the helper already returns the entry unchanged
for a missing disk row.
- The cleared-id set in _update_root_pool_rows is built once, not per disk
row (an Iterable argument would also have been drained on the first row).
- persist_pool_entries passes status_cleared_ids plainly; the kwargs splat
existed only for one test fake, which now accepts the kwarg.
- Docstrings trimmed to the WHY.
Same 116 targeted tests green; reverting the two source files to main still
fails the two regression tests and passes the guard test.
Port onto the split layout: persist_pool_entries now forwards
status_cleared_ids to BOTH writers — write_credential_pool (hermes_cli/
auth.py) and the root-store row merge for single-use-refresh providers
(_update_root_pool_rows), which the original fix predates. Both skip the
disk recency merge for deliberately-cleared entries; the kwarg is only
sent when non-empty so upstream fakes with the old signature keep
working. reset_statuses clears failure_reason (lives in extra, replace()
cannot reach it) alongside the status fields.
tests/agent/test_credential_pool.py: 62 passed; refresh-race suite: 3
passed (mutation: the reset/binding tests fail with the fix stashed).
The dashboard-auth-gate / user-providers failures are pre-existing on
pristine upstream/main (reproduced with changes stashed).
- Check text tokens BEFORE building the fenced block: an oversized file no
longer allocates a second MB-scale string just to discard it, and the token
estimate runs once instead of twice.
- Drop the companion warning: the fallback block alone carries the message,
so the model no longer reads "too large to inline safely" twice (Context
Warnings + Attached Context).
- Extract _on_disk_reference_block + _agent_visible_path shared by the binary
and oversized blocks (stat/format_bytes/OSError fallback, container path
translation, nudge text were copy-pasted).
The on-demand subdirectory hint loader (agent/subdirectory_hints.py, #5291) tail-chopped anything past
8,000 chars with a bare marker and no log line. apps/desktop/AGENTS.md (11k) has been arriving cut off in
every Desktop-area session since it was written, and nobody could tell. Compared with the field: Codex
caps its whole instruction chain at 32 KiB (project_doc_max_bytes) and documents it; Claude Code and
Cursor apply no cap to nested files; OpenCode has no nested discovery at all. Our on-demand + cache-safe
+ ancestor-walk design is the strongest of the four; only the constant and the silent cut were wrong.
Now: ceiling 32,000 chars (Codex's number, a guard against a stray huge CLAUDE.md in a vendored tree, not
a target), and truncation goes through prompt_builder._truncate_content — head 70% + tail 20%, a marker
naming the file to read_file for the rest, and a WARNING in the log. Area AGENTS.md files should stay
around 8k anyway: the text lands in a tool result on the first touch of the directory.
Tests: oversized hint keeps head+tail, names the path, and logs; a 12k area file (over the old cap, under
the new) arrives intact.
Review findings on #102117 (independent reviewer + itsflownium):
* hermes_cli.kanban_db.connect / connect_closing pointed at hermes_cli.projects_db (different DB, no
board= parameter). The compat generator ranked candidate homes by path proximity when a name is
defined in several modules. Now it requires shape compatibility with the BASE definition (same
literal for constants, superset of parameter names for defs) and prefers the facade's own
<stem>_* sibling. Same class fixed for tools.tts_tool.DEFAULT_XAI_BASE_URL (-> tts_tool_providers),
and 17 constants/defs that had been pointed at same-named strangers (Matrix MAX_MESSAGE_LENGTH ->
Signal's 8000, tts MAX_TEXT_LENGTH -> BlueBubbles', honcho/retaindb/supermemory *_SCHEMA -> another
plugin's schema, ...) are now restored from BASE verbatim instead.
* send_yuanbao_direct (restored-def): body called adapter._outbound.send_direct, which HEAD moved to
the sender; rewritten to adapter._outbound.sender.send_direct.
* COMPAT_MANIFEST.md states the scope explicitly: public top-level names only; private names and
test monkeypatch seams are not preserved.
* scripts/check_subprocess_stdin.py: _splat_carries_stdin looked 30 lines ahead in the file text
and was satisfied by an unrelated later stdin=; it now finds the splatted name's definition via AST
and requires stdin inside that expression/body.
Tests: tests/test_compat_manifest_targets.py (pointer identity vs the facade's sibling; kanban
connect(board=) opens a Kanban DB, not projects.db; both FAIL on the previous layer),
test_subprocess_stdin_guard gains the false-negative probe, and the MoA -Q quiet-output contract
tests are back (tests/agent/test_moa_quiet_reference_output.py) against build_moa_facade.
Tests did monkeypatch.setattr(<facade module>, name) where name is now defined in a
sibling module and the production path reads the sibling's binding. Where production
reads through BOTH bindings the setattr is duplicated onto the defining module (import
added next to the existing alias import); where only the sibling reads it the target is
repointed. Seams whose production readers go through the facade are left alone.
No runtime consumer read the proxy (terminal_tool/environments call is_interrupted()/set_interrupt()
directly); its only users were tests patching tools.interrupt._interrupt_event, which had no effect on
the code under test. tools/terminal_tool.py's own re-export of the name is owned by another worker.
tools/approval.py no longer re-exports sibling names (approval_context/prompt/floors/detection/
human_wait/smart/gateway_wait); it imports only what it uses. Siblings reference sibling-defined
names directly (module-attribute reads on tools.approval_context so patching the defining module
still works); only facade-owned state (_lock, _gateway_queues, _permanent_approved, _denied,
_denial_breaker_addendum, _gateway_notify_cb) is still read back through tools.approval.
approval_detection calls its own _command_detection_variants instead of late-binding through the facade.
hermes_state.py: delete every '# noqa: F401 (re-exported...)' import block (hermes_state_common/errors/guard/
readpool/sessions/fts/dbfile/wal/repair/registry + agent.context_compressor _DB_PERSISTED_MARKER_KEY); keep
only the names hermes_state.py itself uses, without noqa.
hermes_state_registry.py: drop get_shared_session_db/release_shared_session_db/close_shared_session_dbs
aliases; every caller (gateway/, tools/, tui_gateway/, cron/, mcp_serve, run_agent, tests) now imports
acquire/release/close_all/release_or_close from hermes_state_registry.
hermes_state_titles.py: drop set_auto_title_if_empty shim (title_generator keeps its getattr fallback).
Re-remove shim-only names restored by 34abf954bd: latest_user_message_row_id (tests call
latest_message_row_id(key, role='user'); role-targeting assertions kept) and get_session_activity (tests
build the snapshot via agent.session_activity.build_activity_snapshot over db.get_session(sid)).
hermes_state_wal._log_once resolves its dedupe sets as module globals instead of via hermes_state;
hermes_state_repair helpers call module globals directly (tests patch hermes_state_repair.<name>).
Frozen updater surface untouched (update_cmd_maint imports only SessionDB from hermes_state).
tools/terminal_tool.py: drop 30 pure re-export names (lifecycle/config/backends/
sudo/guards/result/interrupt/utils/_DockerEnvironment/is_managed_tool_gateway_ready)
and the noqa-F401 comments on the 25 names the facade itself uses. Sibling modules
(terminal_tool_backends/_result/_sudo/_lifecycle, environments/base, process_registry)
that read removed names through the facade now import from the defining module.
tools/environments/base.py: drop 11 re-exports (base_output/base_session_env/
path_utils) and the BaseEnvironment.stop() compat alias (no in-tree caller; the
lifecycle hasattr(env, 'stop') fallback stays for third-party envs).
tools/environments/docker.py: drop 1 re-export + the re-export comment.
Callers/tests repointed to tools.terminal_tool_{lifecycle,backends,sudo,config,
guards,result}, tools.interrupt, tools.environments.{base_output,base_session_env,
path_utils}.
run_agent.py: delete the `# noqa: F401` re-export block (agent.process_bootstrap
OpenAI/_SafeWriter/_get_proxy_*, model_tools get_tool_definitions/
handle_function_call/check_toolset_requirements, FailoverReason,
_qwen_portal_headers/_routermint_headers, session_persistence names,
estimate_request_tokens_rough, ContextCompressor + friends, jittered_backoff,
prompt_builder names, message_sanitization names, tool_dispatch_helpers
names) — 41 names run_agent never used itself — and the `_STREAM_DIAG_HEADERS`
back-compat class alias (no in-tree reader). run_agent now imports only what
it uses (get_toolset_for_tool, is_local_endpoint, coalesce/uniquify tool-call
ids, cleanup_vm/get_active_env from terminal_tool_lifecycle).
agent/*: `_ra().X` late-binds that only reached a re-export now import the
defining module directly (agent_runtime_helpers -> process_bootstrap.OpenAI,
model_tools.handle_function_call, session_persistence._safe_session_filename_component;
agent_init -> model_tools.get_tool_definitions/check_toolset_requirements,
_lazy_headers("agent.client_lifecycle", ...) for qwen/routermint;
system_prompt -> agent.prompt_builder / model_tools directly, dropping its
own _ra() shim and the `_r` parameter threading). `_ra()` stays for
run_agent-resident names (logger, AIAgent, _hermes_home, _set_interrupt, ...).
toolsets.py: remove resolve_multiple_toolsets (shim-only, restored by
34abf954bd); tests/test_toolsets.py pins the same union behavior via
resolve_toolset over each name.
providers/__init__.py: drop the OMIT_TEMPERATURE re-export (no callers via the
package); ProviderProfile stays because __init__ uses it for annotations —
2 tests repointed to providers.base.
agent/iteration_budget.py: drop the "run_agent re-exports the class"
docstring pointer; 4 tests import IterationBudget from its home.
model_tools.py (arg_coercion names), agent/tool_executor.py, and
hermes_cli/cli_session_mixin.py repoints landed via a sibling commit on this
shared worktree.
Callers repointed: gateway/run.py, hermes_cli/cli_chat_turn_mixin.py,
hermes_cli/cli_tui_mixin.py, tui_gateway/session_workdir.py,
agent/transports/codex.py (one-line imports) + comment pointers in
tools/file_state.py, tools/schema_sanitizer.py, scripts/tool_search_livetest.py.
Tests: patch("run_agent.X") / monkeypatch.setattr(run_agent, "X") /
`from run_agent import X` -> defining module across 99 test files.
BASE's _compress_codex_app_server_session was the single cache-peek site that read
_agent_cache without a lock when _agent_cache_lock was None (tests/gateway/
test_codex_hygiene_compaction.py::_slash_host relies on it). Every other BASE caller returned
None without a lock. Model that as an explicit lockless_fallback=True at that one call site.
Adolanium review §3 (Medium/Low). BASE shape at every steer/redirect/persist/agent-cache
lock site was: lock = getattr(obj, "_x_lock", None); if lock is not None: with lock: <direct
attribute read>; else: <getattr fallback for object.__new__ test stubs>. The refactor rewrote
several of these as 'with getattr(...) or nullcontext(): getattr(slot, None)', which (a) turned a
missing slot under the lock from a loud AttributeError into silent None and (b) in the gateway
peek helper read the cache without any lock when the lock attribute was absent.
Restored BASE semantics at:
- agent/agent_runtime_helpers.py::_requeue_pending_steer
- agent/interrupt_control.py: steer, redirect, clear_interrupt, _has_pending_redirect,
_drain_pending_redirect, _drain_pending_steer (new _ic_slot helper: direct read under lock)
- agent/session_persistence.py::_persist_lock (explicit None check + BASE rationale)
- gateway/slash_commands.py::_cached_agent_for (BASE callers read ONLY under the lock; no lock -> None)
- gateway/run_agent_cache.py::_evict_cached_agent (BASE: self._agent_cache direct under lock)
- agent/client_lifecycle.py::_is_openai_client_closed: BASE body + docstring verbatim (outer
is_closed first; inner _client.is_closed only when _client exists; else False)
- agent/stream_delivery.py::_ensure_stream_writer_state: restore BASE rationale that the lock is
created unconditionally in agent_init (_STREAM_STATE) and the lazy path is stub-only
A/B (/tmp/rf/rev/ab_lock_fallbacks.py, ab_client_closed.py) is byte-identical BASE vs HEAD on
the reviewer's inputs + edge cases. tests/agent/test_lock_fallback_base_semantics.py pins it
(7 of 25 cases fail on the pre-fix tree).
On BASE the #95514 stream-recovery rebound final_response inline, before
the tail-closing / persist-override / micro-compaction / _persist_session
calls inside the guarded persist try-block, so a raise in any of them left
the caller with the streamed text (cleanup_errors reported the failure).
HEAD's _close_transcript_tail returned the recovered value only on normal
completion; an exception in the same block dropped it and the user-visible
answer regressed to the pre-recovery '' (then rewritten by the explainer).
Split the helper into _drop_transcript_scaffolding / _recover_final_from_stream
/ _close_transcript_tail and rebind final_response in the guarded step as soon
as it is computed, restoring BASE ordering. Adds a regression test.
Teknium