The memory_tool schema advertises new_text as an alias for content, and
memory_tool resolves it when content is None. But the table-driven inline
executor's arg_specs (agent/inline_tool_executors.py) did not list new_text,
so _call_tool's allowlist silently dropped it: a replace call using the
documented alias reached memory_tool with both fields None and failed with
"content is required for 'replace' action." — even though the caller
supplied the value. Forward new_text alongside content/old_text so the
documented alias fires and content still wins when both are set, matching
what the batch path (op.get("content") or op.get("new_text")) already
accepts.
Regression coverage from PR #104214. Live inherited-compress probe reproduces the same failure on main; serial unit runner lock is busy.
Co-authored-by: fangliquanflq <fangliquan@qq.com>
Slim salvage of #104688: place project context before workspace state and
keep cwd outside the stable prefix. Put runtime hints behind a final
renderer-owned boundary so quoted operator, memory, plugin and embedder
examples cannot override the persisted runtime cwd or identity fields.
Retain legacy unmarked prompt validation, add two invariant tests and a
credential-free real-AIAgent/git-worktree replay harness. No provider
cache-hit or billing measurements are claimed.
Co-authored-by: joaomarcos <joaomarcosdias444@gmail.com>
Co-authored-by: HexLab98 <liruixinch@outlook.com>
Track raw task identities across an agent's turns and match them against
process owner_task_id during close. Session IDs and shared terminal keys
are not process ownership, so the old bulk cleanup missed delegated work.
Preserve parent/sibling processes and consume teardown notifications.
Move task-resource cleanup into the lifecycle mixin, add real-process
isolation regressions, and document background process lifetime.
Use request-local stream silence for the waiting notice, preserving quiet
activity heartbeats and all existing watchdog policies. Distinguish a stream
that stopped from a request with no response, and clear this request's notice
on the next poll when events resume. Existing fresh first-event retry phases
also reset the display; recovery deadlines explicitly use total call elapsed.
Add two invariant tests (eight cases), proven red on main, plus EN/ZH docs.
Local SDK SSE through classic CLI callbacks in a PTY verifies active reasoning,
true silence, and an already-visible warning clearing on resumed reasoning.
Related: #92657 addresses repeated waiting notices; its phase deduplication
still labels active streams as no response and is not incorporated here.
The image-part membership test in _payload_chars() ran on every dict in
the request, so a tool JSON Schema with a parameter named "type" (its
value is the sub-schema dict) or a multi-type value like
["string", "null"] raised TypeError: unhashable type before the
request reached the provider. Guard the membership test with
isinstance(str): structured "type" values are payload data, not
content parts, and keep the legacy walk for them.
estimate_request_context_tokens (the stale-stream / non-stream watchdog and Codex TTFB floor
estimator) did len(str(payload)) // 4 over the wire payload, so one native screenshot read as
~100K tokens and selected the 1200s giant-conversation floor while the provider's real prompt
was ~2K (#63871, #76411). Image content parts on both wire shapes (Chat image_url / Responses
input_image) now cost the per-image price learned from provider usage (agent/image_token_cost,
bound per turn; worker threads inherit it via copy_context); base64-looking STRINGS stay text
and tool schemas mentioning image are not images.
A/B, one 300KB screenshot: main estimate 100,545 -> codex floor 1200s / stale 300s; now 2,025
-> floor 0s / stale 180s.
Design and boundary (structured parts only, tools/instructions opaque, plain data-URL strings
stay text) from #76471 by @crdesign8; re-authored to reuse the learned cost instead of a second
fixed constant, and trimmed to 2 invariant tests.
The Chat Completions schema has no `name` field on `role: tool` (only on
the long-removed `role: function`), but Hermes carries the tool name over
onto the result message. Permissive providers ignore it; strict ones
(aki.io) reject the whole payload with `contains item with unknown key
name`, which breaks every tool call in the session.
Follow-up to the review on #51365:
- The strip now goes through the copy-on-write `mutable_msg()` path in
`convert_messages()`, preserving the identity/copy-on-write contract
instead of mutating `msg` in place.
- `handle_max_iterations()` hand-builds its summary payload and calls
`chat.completions.create()` directly, bypassing the transport, so it
leaked `name` even with the transport fixed. It now mirrors the same
role-qualified removal, next to the existing tool_name/codex_*/timestamp
strips.
The removal is role-qualified: `name` stays on user/assistant messages,
where it is schema-valid.
Regression coverage on both paths; both tests fail without the fix.
A flat per-image constant (1500 in the trigger estimator, 1600 in the tail-budget walk) is wrong in
both directions: a screenshot costs ~1,100 tokens on one provider and 4,000+ on a local mmproj
model. In a GUI loop on a 64K window the estimate sat at ~20K while the real prompt passed 80K,
so compaction never fired and the provider rejected every request (#70328).
The provider prices every image exactly on the request that carries it, so the cost is
observable from usage alone, with no vendor formula: with a fresh usage anchor, the residual
between the next real prompt_tokens and anchor + text-only delta is the price of the N images
that delta introduced.
- agent/image_token_cost.py: calibrate_from_usage() runs in record_response_usage before the new
anchor is captured; the learned value (EMA, plausibility-banded) is kept per model@host in
~/.hermes/cache/image_token_costs.json and bound per turn through a ContextVar.
- estimate_messages_tokens_rough, _content_length_for_budget (tail walk) and gateway hygiene all
read the same bound value, so trigger and walk agree; the per-message memo now caches text
tokens and image COUNT so a recalibration re-prices cached rows.
- One flat default (1500) remains only until the first vision turn; the duplicate 1600 is gone.
evals/token_accounting/ab_image_cost_calibration.py (real AIAgent, fake provider pricing images
at 4,000, one screenshot per turn, 64K window): main learns nothing (1500) and the tail walk
under-prices its own protected tail by 56.5%; this branch learns 4,374 after one vision turn
and the walk's error is +8.5%.
Reporter and first-fix credit: @JonthanaHanh (#70328, #70463).
Diagnosing the 1,393-agent run's cache misses took a state.db join and a live
probe because none of the three were on the one line we log per call:
write=<n> cache_creation tokens; a write costs 50x a read, so this is the
money, and "read stuck, write large" on consecutive lines is a
routing miss visible without a probe
id=<id> the provider's response id (Anthropic msg_..., OpenRouter/Nous
gen-...): what a provider needs to look a request up
upstream=<n> who actually served, when the route reports it (OpenRouter's
`provider`); how we learned GMI was not serving
Fields are appended to the existing line and omitted when absent, so every
existing parser (evals/postmortem, the two cache_prefix probes) keeps
matching; the forensics parser reads them when present.
The streamed chat path fabricated `id="stream-<uuid>"` and dropped the
chunks' id/provider on the floor; it now keeps the first chunk's id and
provider, falling back to the fabricated id only when the stream never sent
one. Nothing depended on the prefix.
Live (Nous, Fable 5.1, both wires): chat -> `write=3759
id=gen-1788727882-... upstream=Anthropic`; native -> `id=gen-1788727891-...`
(the Anthropic Message object's id was already real). Tests: fields present
and omitted, prefix unchanged; forensics parser reads new and old lines.
The prefetch spill is the fix; the is_builtin registration flag and duplicate-
instance rejection defended against a provider naming itself "builtin", which no
live path can do (only a test fixture does). Restore the name-based check and the
four test files that only changed for the new kwarg; keep two spill invariants.
Codex Responses reasoning and compaction items carry ciphertext the provider prices by its own
token count, never by bytes; a single native compaction checkpoint is ~5M chars, which the
bytes/4 estimator turned into ~1.29M "tokens" against a 204K threshold (#100611). #104192 deferred
that decision for one request; this removes the mis-pricing at the source so the preflight
estimator and the tail-budget walk agree (a mismatched size class protects blob-heavy rows as
"small" and compaction re-fires). Only real usage prices these items, and the usage anchor carries
that price forward.
evals/native_compaction/ab_checkpoint_preflight.py: preflight estimate after checkpoint
1,292,413 -> 58 rough tokens; the over-threshold negative arm still compresses.
Two parallel "real usage" mechanisms fought each other: the usage anchor (real + delta) and the
compressor's rough/real projection (should_defer_preflight_to_real_usage with
last_rough_tokens_when_real_prompt_fit / _pending_request_rough_tokens / note_request_rough_estimate
baselines). The projection stored an anchored, real-scale figure as its "rough" baseline, so a
rewind that invalidated the anchor produced phantom growth and a spurious compaction (#103391).
Now there is one authority:
- Post-tool gate (turn_preflight.compress_after_tool_results): anchored figure first (the raw
last_prompt_tokens ignored the tool results just appended), then real, then rough.
- Gateway hygiene (run_turn._hmwa_hygiene_plan): real session count, else the anchor persisted on
the session row, else rough.
- Preflight / pre-API gates: an anchored figure is never deferred. A whole-context rough estimate
over threshold waits ONE request for the provider's real count instead of compressing on a guess
(first request, rewind/edit-resend, reloaded history without a persisted anchor).
- The wait is one request, never a disable: a provider that omits usage
(note_usage_less_response, #2153 class), a real reading already over threshold, a rough figure
past the whole window, and provider-proven overflow all compress immediately; the post-compaction
latch (#36718 / #104192) is unchanged.
- Projection baselines and their bookkeeping deleted (-101 LOC in context_compressor); the fixtures
that scripted whole-history estimates now state the fact they relied on (provider omits usage).
Fixes#103391 (closes#103397 by construction — the baseline it repaired no longer exists).
The usage anchor (real usage.prompt_tokens + delta estimate of what was appended since)
identified the priced transcript by id() of the last message, so it was None on EVERY
gateway turn (history is re-read from the DB each turn) and in every fresh process
(--resume, desktop per-turn serve). Those are exactly the surfaces where the bytes/4
estimate then fired local compression against payloads the provider priced far under
threshold (#99421, #104462).
- agent/usage_anchor.py owns the anchor: content fingerprint instead of id(), persisted on
the session row (model_config._usage_anchor) via set_usage_anchor(), restored on the first
resumed turn while the durable transcript still matches, cleared with the row on
compaction / codex-native rewrite / session reset.
- Callers repointed from model_metadata (the compat table follows).
Design and persistence slot from #99585 by @686f6c61; re-authored against the Sep 2026
layout (the branch predates the model_metadata / agent_init split).
The codex_app_server runtime bypasses the conversation loop, so the
usage-anchored context accounting captured there never ran: agent._usage_anchor
stayed None forever. Every preflight estimate therefore fell back to the rough
mirror-transcript heuristic, which is deliberately never compacted on this
runtime (_record_codex_app_server_compaction preserves the mirror), so it grows
monotonically while the real thread may be tiny or freshly compacted. With
compression.codex_app_server_auto=hermes that estimate alone tripped the
threshold and fired thread/compact/start on nearly every turn of a long-lived
session (#100381).
Mirror the main loop's post-response capture: _record_codex_app_server_usage
now snapshots the reported thread usage into agent._usage_anchor (base prompt
+ completion exactly as the provider counted them, with estimation confined to
messages appended since). A usage-less turn keeps the previous anchor, and the
post-compaction invalidation site is unchanged.
Older desktop builds appended a frozen "## Messaging other agents" section (roster
included) to SOUL.md. Since the server started injecting the live section into Bot Chat
sessions, that copy did two wrong things: every CLI/TUI/messenger session paid ~600 tok
for a bot-only protocol, and in Bot Chat itself the probe went silent when SOUL carried
the heading, so bots saw the stale roster instead of the live one.
- load_soul_md strips the legacy section at read time (covers un-migrated profiles and
the ambient-home edge cases the same way the SOUL isolation fix does)
- bot_mode_probe drops the SOUL-carries-heading suppression; a SOUL-era stored Bot Chat
prompt now counts as legacy and is upgraded once (stamped, so it cannot loop)
- config migration v41 rewrites SOUL.md across the default + every profile once
The alias check re-implemented most of _is_same_auth_store (#101356) inline, but
that helper answers False on a samefile() OSError — on the strip path that would
fail OPEN. Resolve identity positively (same resolved path, or samefile says so)
and treat any error as "refuse"; three lines instead of the inline stat dance.
The `preserve_symlinks=False` save path (raw os.replace) bypassed atomic_replace's
EXDEV/Windows fallback to defend against a path swap no concurrent writer performs
on a directory this process just created — removed with its mock-injected test.
Tests kept: shared-store invariant [symlink|hardlink] and two fail-closed variants.
With a request-local watchdog state on every codex request, agent._codex_stream_last_event_ts / _last_progress_ts had no remaining reader; the state-None snapshot only runs for non-codex requests where no watchdog consults it.
fix(nous): adopt a same-account fresh key before expiry and start the keepalive in every process (620 hourly 401s → 0; review-found account takeover closed)
fix(delegate): nested orchestrators get their workers' results back — delegate_task exempt from the 420 s tool deadline; summary budget uses current prompt, not the session sum
The credentials test no longer has to strip trailing text before json.loads;
the new file keeps the single behaviour contract (truncated output round-trips
through json.loads and carries the next offset) — the existing TestSearchHints
cases already cover offset arithmetic.
Follow-up to the pure-JSON search_files fix: the credential-filter test
in tests/agent/ also asserted the old appended '[Hint: ...]' text — the
local run covered tests/tools/ only, so CI slice 7 caught it. The _hint
field is present and correct in the CI failure output itself; only the
assertion syntax was stale.
Portal will serve anthropic/* from more than one upstream (OpenRouter
passthrough today; GMI/Vertex once it is back online). The native Messages
wire is the better transport but is only safe where the upstream keeps
prompt-cache routing sticky: measured false on the OpenRouter path (14-20% of
consecutive calls re-write the previous turn; #104284 moved the default to
chat), untested on GMI. Hermes cannot see the upstream in the request, only
in the response: OpenRouter stamps `provider` (chat wire) and mints
`gen-<unix>-<rand>` ids; GMI/Vertex returns Anthropic-native `msg_...` ids
and no provider.
`auto` therefore starts every session on chat (correct on both upstreams),
classifies the first response, and switches that session to native only
when the upstream is GMI AND `agent/nous_wire.py::GMI_NATIVE_WIRE_CLEARED`
is True. The switch is scheduled at response time and applied at the start
of the next iteration (turn_iteration_prep), so nothing is rebuilt while a
response is being consumed; it goes through switch_model so the client,
cache policy and _primary_runtime stay consistent. One decision per session,
call 1 only; unknown upstream never switches; a failed switch logs and stays.
GMI_NATIVE_WIRE_CLEARED is False: until the 20x6 concurrency probe
(evals/postmortem/live_ab) is clean on a GMI-served anthropic/* id on the
native wire, `auto` behaves exactly like `chat`. Flipping it is the whole
rollout once GMI is measured. Default stays `chat`.
Tests (17): classifier on real Portal response shapes from both wires and
both upstreams; chat for openrouter/unknown, GMI gated on the flag; one
decision per session, call 1 only, explicit chat/native never auto-switch,
other providers/models untouched, switch failure swallowed and final;
record_response_usage on a real AIAgent invokes the hook once.
Live (auto, real Portal, Fable 5.1): arm A, real classification
(OpenRouter today) - stays on chat through a tool loop and a second turn,
cache 97-99%. Arm B, classifier forced to gmi with the flag on - call 1 on
chat, switch applied before call 2, calls 2-3 on the native wire in the
same session, tool result and both turns correct, cache 97-99%. An earlier
shape that switched inside the response path broke call 1 (SimpleNamespace
has no .content); the scheduled apply is why.
A native Responses compaction checkpoint is opaque ciphertext; the rough
preflight estimator counts it as text (5.17M chars -> ~1.29M tokens against
a 204K trigger) and fires local compression on a request whose real prompt is
~116K. Arm the existing one-response real-usage latch when a replayable
checkpoint is captured (build_assistant_message) or restored into a fresh
agent (_hydrate_from_history), honor it in the post-tool gate and idle
compaction, and require non-empty encrypted_content for a checkpoint.
Squash of the author's source commits from #100642 (0e3c234ea0, 771e1b3365,
bb1505a119) plus the fdf140c81d test refresh, re-based onto current main by
patch application. Source delta is byte-identical to the PR head d6ce3e236d.
Fixes#100611
The compression/aux fallback ladder decides "payment error" via
_is_payment_error(), which only knew the spaced "resource exhausted". NVIDIA NIM
and gRPC-style wrappers serialize the same quota signal as ResourceExhausted /
RESOURCE_EXHAUSTED / resource-exhausted, so a 403/429/status-less body carrying
it re-raised instead of walking the configured fallback_chain, and compression
fell to the lossy emergency path. Add the three separator variants to
_PAYMENT_KEYWORDS (same status gate, no broader "exhausted" matching).
evals/auxiliary_resource_exhausted.py drives the real call_llm/async_call_llm
against two local OpenAI-compatible listeners (nvidia profile primary, named
custom fallback) for the before/after check. #85649
Portal serves anthropic/* on two routes. The native /v1/messages wire, which
Hermes has used since 02d5e23085, re-writes the previous turn's prompt cache
on 14-20% of consecutive calls in concurrent tool loops; the chat/completions
route does not. Measured 2026-09-06, 20 concurrent sessions x 6 tool calls on
Fable 5.1, same account, same hour, same first-party pin:
Nous /v1/messages 180 pairs, 25 stuck (13.9%); 3 earlier 40-session
runs 15.0 / 17.3 / 20.2%; unchanged by the pin
Nous /v1/chat/completions 320 pairs (2 runs), 0 stuck
OpenRouter direct, pinned 161 pairs, 0 stuck
"stuck" = cache_read on call k+1 equals cache_read on call k instead of
call k's prompt size: the last write was not visible, the turn (~28K) was
written twice. Every stuck pair had byte-identical system, tools and
per-message shas, so it is not client-side mutation. At $10-20/M for
cache writes that is 15-20% of a fan-out's write bill.
The cause is inside the portal's native route (every element of its
outgoing request reproduces clean from outside; NousResearch/api#227
carries the diagnostics). Until it is fixed, anthropic/* rides
chat/completions. nous.anthropic_wire: native opts back in. Cost of
chat: prior-turn thinking travels as OpenAI-style reasoning fields, and
cache_control scopes are translated by the portal's adapter.
Tests: new test_nous_anthropic_wire_default.py reads the knob through the
real config loader in a temp HERMES_HOME and through
resolve_runtime_provider (both settings); the existing native-wire
contract suite selects native via an autouse fixture so the wire keeps
working for the flip-back. Live: a real AIAgent on this branch,
provider=nous + anthropic/claude-fable-5.1, dispatches to
chat/completions with the session_id intact.
Follow-up to the salvaged contributor commit: the Gemini and Vertex guide model
tables now list both Flash generations the pickers offer, and an invariant test
ties the OpenRouter/Nous curated google/gemini-*-flash entries to (a) a Google
official-docs pricing row on the direct Gemini and Vertex routes and (b)
membership in the direct gemini/vertex picker lists. Red on main
(gemini-3.7-flash -> unknown), green with the contributor commit.
Campaign: https://github.com/NousResearch/hermes-agent/issues/104154
- Generalize Gemini 3 thinking config model prefix match to gemini-3*
- Add pricing snapshot entries for gemini-3.7-flash and gemini-3.8-flash
- Add model identifiers to Google, OpenRouter, and Vertex CLI lists
- Add test coverage for gemini-3.8-flash thinkingLevel configuration
The ChatGPT Codex backend answers a rejected encrypted-reasoning replay with the
same bare 400 {code: invalid_prompt, message: "Request blocked."} it uses for
genuine blocks. classify_api_error bucketed it as format_error, so the one-shot
repair that already exists for invalid_encrypted_content (disable replay, strip
cached codex_reasoning_items, retry once) never ran and the turn aborted.
Classify exactly that envelope from provider openai-codex — SDK 400 body, SSE
``error`` frame, or the ``response.failed`` text — as invalid_encrypted_content
while keeping format_error's abort-and-fallback hints, so the only behavioural
delta is turn_recovery's replay strip, which still requires cached reasoning
items. A block with nothing to strip aborts exactly as before; other providers,
other messages/codes and the #18028 safety refusals are untouched.
evals/codex_masked_replay_review.py drives a real AIAgent (codex_responses)
against a synthetic local Responses server for the before/after check.
Refs #92353, #92357, #94077.
Fold the adapter-level unsigned-preservation case and the name-detector matrices
into one parametrized invariant (thinking family keeps unsigned only; non-thinking
DeepSeek and unrelated models still take the generic third-party strip) plus the
native-endpoint immutability guard. Add evals/anthropic_proxy_thinking_replay.py,
a real-AIAgent probe against a synthetic local Anthropic-compatible relay used for
the before/after wire check.
Third-party Anthropic-compatible relays serving a DeepSeek thinking model
(deepseek-r/v4/pro/flash) follow the same replay contract as DeepSeek's
native /anthropic endpoint: strip signed thinking blocks, preserve
unsigned ones. Previously these sessions hit the generic third-party path
and lost ALL thinking blocks, breaking cross-turn reasoning coherence.
Detection is model-name based (vendor prefixes stripped), gated on the
existing _is_third_party check so direct Anthropic traffic is untouched.
Teknium