Use successful applied result records rather than requested operations, and keep staged writes silent. Include legacy delete/write messages.
Fixes#104506
Co-authored-by: Konstantin Khlopkov <konstantin.khlopkov93@gmail.com>
The corrupt-cause recovery guidance hardcoded `~/.hermes/backups/` while
every other path in the same message follows the active HERMES_HOME
(`{db_path}` is already interpolated). A custom-home or named-profile
deployment was told to restore from a directory that may not exist at all,
mid data-loss incident. Both sites (turn-completion explainer and gateway
startup broadcast) now interpolate `<hermes_root>/backups` via
get_default_hermes_root(), matching hermes_cli/backup.py's real backup
location.
Fixes#104250
The Chat Completions schema has no `name` field on `role: tool` (only on
the long-removed `role: function`), but Hermes carries the tool name over
onto the result message. Permissive providers ignore it; strict ones
(aki.io) reject the whole payload with `contains item with unknown key
name`, which breaks every tool call in the session.
Follow-up to the review on #51365:
- The strip now goes through the copy-on-write `mutable_msg()` path in
`convert_messages()`, preserving the identity/copy-on-write contract
instead of mutating `msg` in place.
- `handle_max_iterations()` hand-builds its summary payload and calls
`chat.completions.create()` directly, bypassing the transport, so it
leaked `name` even with the transport fixed. It now mirrors the same
role-qualified removal, next to the existing tool_name/codex_*/timestamp
strips.
The removal is role-qualified: `name` stays on user/assistant messages,
where it is schema-valid.
Regression coverage on both paths; both tests fail without the fix.
Two parallel "real usage" mechanisms fought each other: the usage anchor (real + delta) and the
compressor's rough/real projection (should_defer_preflight_to_real_usage with
last_rough_tokens_when_real_prompt_fit / _pending_request_rough_tokens / note_request_rough_estimate
baselines). The projection stored an anchored, real-scale figure as its "rough" baseline, so a
rewind that invalidated the anchor produced phantom growth and a spurious compaction (#103391).
Now there is one authority:
- Post-tool gate (turn_preflight.compress_after_tool_results): anchored figure first (the raw
last_prompt_tokens ignored the tool results just appended), then real, then rough.
- Gateway hygiene (run_turn._hmwa_hygiene_plan): real session count, else the anchor persisted on
the session row, else rough.
- Preflight / pre-API gates: an anchored figure is never deferred. A whole-context rough estimate
over threshold waits ONE request for the provider's real count instead of compressing on a guess
(first request, rewind/edit-resend, reloaded history without a persisted anchor).
- The wait is one request, never a disable: a provider that omits usage
(note_usage_less_response, #2153 class), a real reading already over threshold, a rough figure
past the whole window, and provider-proven overflow all compress immediately; the post-compaction
latch (#36718 / #104192) is unchanged.
- Projection baselines and their bookkeeping deleted (-101 LOC in context_compressor); the fixtures
that scripted whole-history estimates now state the fact they relied on (provider omits usage).
Fixes#103391 (closes#103397 by construction — the baseline it repaired no longer exists).
A native Responses compaction checkpoint is opaque ciphertext; the rough
preflight estimator counts it as text (5.17M chars -> ~1.29M tokens against
a 204K trigger) and fires local compression on a request whose real prompt is
~116K. Arm the existing one-response real-usage latch when a replayable
checkpoint is captured (build_assistant_message) or restored into a fresh
agent (_hydrate_from_history), honor it in the post-tool gate and idle
compaction, and require non-empty encrypted_content for a checkpoint.
Squash of the author's source commits from #100642 (0e3c234ea0, 771e1b3365,
bb1505a119) plus the fdf140c81d test refresh, re-based onto current main by
patch application. Source delta is byte-identical to the PR head d6ce3e236d.
Fixes#100611
The simplify reviewers flagged that every cooldown parametrize row was
> 60s, so the emit branch was never asserted against its buffered
complement. _drive_once now records which surface the status went
through, and a 30s row asserts short cooldowns keep the buffered line.
Mutation-checked: flipping the threshold fails the four emit rows.
/simplify-code pass on the Retry-After salvage stack:
- compute_error_backoff now decides "no usable cooldown" exactly once:
a parsed 0.0 (retry-after: 0, or an HTTP-date in the past, which the
shared parser clamps to 0) is treated as absent instead of falling
through an accidental falsy check — prevents a hot-loop retry and
keeps the sentinel semantics uniform (is None / is not None at all
four sites).
- Comment corrected: the sibling unwrap lives in
extract_api_error_context, not _extract_rate_limit_context.
- Test driver deduped: one _retryable_error factory + one _drive_once
shared by the 429 and 524 tests; added the over-cap (3600 → 600) and
no-cooldown fallback rows requested in the #103722 review.
Mutation checks: nested-unwrap neutralized → nested row red; cap
removed → over-cap row red; restored → all 298 green.
Salvage follow-ups on #88236 (krunkosaurus):
- Some providers nest the cooldown as body["error"]["retry_after"] (the
same unwrap _extract_rate_limit_context already uses); only the top-level
shape was read, so those errors silently fell back to jittered backoff.
- A 5xx Retry-After can reach the 600s cap; that wait was buffered
(replayed only on terminal failure), leaving the user silent for
minutes. Long provider cooldowns now emit immediately, mirroring the
zai_coding_overload_long path. Jittered waits keep the old buffering.
Test widened with the nested-body parametrize row; proven red when the
unwrap is neutralized.
Retryable server errors can carry provider cooldowns just like 429 responses. Parse Retry-After from response headers or structured error bodies before falling back to jittered backoff, and cover HTTP 524 behavior with runtime regression tests.
test_nous_anthropic_fallback_uses_the_messages_wire and
test_nous_child_rederives_api_mode_from_model describe the native wire, which
is now opt-in; select it in the test the same way the wire-contract suite
does, so both keep guarding the flip-back.
The chat_completions finalizer now reads collector-observed chunks, not the
consumer loop, so the test feeds the captured on_chunk before calling it —
the same ordering Relay guarantees.
The skill review fork, the combined memory+skill review, and the curator's
consolidation pass all described references/ as the place for
"session-specific detail", and the curator's demote step said to move a
sibling's file under the umbrella. Followed literally over months that
produced one dev skill with a 100k SKILL.md dense in PR numbers and 443
one-per-session reference files, plus five sibling skills restating the
same rules and the repo's AGENTS.md.
The three prompts now share one shape contract: an entry is an imperative
rule plus one clause of why, stated once; no PR/issue numbers, dates, or
quoted chat as content; references/ is a small topical set extended in
place, never a per-session file; skills do not restate always-loaded
context. Consolidation is defined as distilling, and copying a sibling
verbatim under references/ is named as the failure. skill_manage's schema
carries the one-sentence version.
Two advisory linter rules make the shape visible in the tool result the
moment it starts to drift: incident-log-shape (PR/issue-number density in
prose) and references-sprawl (>60 reference files), the latter also run
on references/ writes. On the real before/after: the old skill trips both,
the consolidated one trips neither.
No runtime consumer read the proxy (terminal_tool/environments call is_interrupted()/set_interrupt()
directly); its only users were tests patching tools.interrupt._interrupt_event, which had no effect on
the code under test. tools/terminal_tool.py's own re-export of the name is owned by another worker.
tools/approval.py no longer re-exports sibling names (approval_context/prompt/floors/detection/
human_wait/smart/gateway_wait); it imports only what it uses. Siblings reference sibling-defined
names directly (module-attribute reads on tools.approval_context so patching the defining module
still works); only facade-owned state (_lock, _gateway_queues, _permanent_approved, _denied,
_denial_breaker_addendum, _gateway_notify_cb) is still read back through tools.approval.
approval_detection calls its own _command_detection_variants instead of late-binding through the facade.
hermes_state.py: delete every '# noqa: F401 (re-exported...)' import block (hermes_state_common/errors/guard/
readpool/sessions/fts/dbfile/wal/repair/registry + agent.context_compressor _DB_PERSISTED_MARKER_KEY); keep
only the names hermes_state.py itself uses, without noqa.
hermes_state_registry.py: drop get_shared_session_db/release_shared_session_db/close_shared_session_dbs
aliases; every caller (gateway/, tools/, tui_gateway/, cron/, mcp_serve, run_agent, tests) now imports
acquire/release/close_all/release_or_close from hermes_state_registry.
hermes_state_titles.py: drop set_auto_title_if_empty shim (title_generator keeps its getattr fallback).
Re-remove shim-only names restored by 34abf954bd: latest_user_message_row_id (tests call
latest_message_row_id(key, role='user'); role-targeting assertions kept) and get_session_activity (tests
build the snapshot via agent.session_activity.build_activity_snapshot over db.get_session(sid)).
hermes_state_wal._log_once resolves its dedupe sets as module globals instead of via hermes_state;
hermes_state_repair helpers call module globals directly (tests patch hermes_state_repair.<name>).
Frozen updater surface untouched (update_cmd_maint imports only SessionDB from hermes_state).
cron/scheduler.py no longer re-exports the split modules (scheduler_delivery /
_script / _prompt / _preflight); it imports only the 19 names it calls itself
(bottom-of-file, E402 kept for the import cycle). Dropped the shim-only
`import shutil` and the F401 note on windows_hide_flags (still used by
scheduler.py). Split modules now call same-module helpers directly, reach
sibling split modules via late-bound module refs (_delivery/_script/_preflight)
next to _sched, and import windows_hide_flags themselves; origin-resident names
(load_config, Path, _SCRIPT_TIMEOUT, heartbeat_run_claim, ...) still go through
_sched. Callers/tests import + patch the defining module.
run_agent.py: delete the `# noqa: F401` re-export block (agent.process_bootstrap
OpenAI/_SafeWriter/_get_proxy_*, model_tools get_tool_definitions/
handle_function_call/check_toolset_requirements, FailoverReason,
_qwen_portal_headers/_routermint_headers, session_persistence names,
estimate_request_tokens_rough, ContextCompressor + friends, jittered_backoff,
prompt_builder names, message_sanitization names, tool_dispatch_helpers
names) — 41 names run_agent never used itself — and the `_STREAM_DIAG_HEADERS`
back-compat class alias (no in-tree reader). run_agent now imports only what
it uses (get_toolset_for_tool, is_local_endpoint, coalesce/uniquify tool-call
ids, cleanup_vm/get_active_env from terminal_tool_lifecycle).
agent/*: `_ra().X` late-binds that only reached a re-export now import the
defining module directly (agent_runtime_helpers -> process_bootstrap.OpenAI,
model_tools.handle_function_call, session_persistence._safe_session_filename_component;
agent_init -> model_tools.get_tool_definitions/check_toolset_requirements,
_lazy_headers("agent.client_lifecycle", ...) for qwen/routermint;
system_prompt -> agent.prompt_builder / model_tools directly, dropping its
own _ra() shim and the `_r` parameter threading). `_ra()` stays for
run_agent-resident names (logger, AIAgent, _hermes_home, _set_interrupt, ...).
toolsets.py: remove resolve_multiple_toolsets (shim-only, restored by
34abf954bd); tests/test_toolsets.py pins the same union behavior via
resolve_toolset over each name.
providers/__init__.py: drop the OMIT_TEMPERATURE re-export (no callers via the
package); ProviderProfile stays because __init__ uses it for annotations —
2 tests repointed to providers.base.
agent/iteration_budget.py: drop the "run_agent re-exports the class"
docstring pointer; 4 tests import IterationBudget from its home.
model_tools.py (arg_coercion names), agent/tool_executor.py, and
hermes_cli/cli_session_mixin.py repoints landed via a sibling commit on this
shared worktree.
Callers repointed: gateway/run.py, hermes_cli/cli_chat_turn_mixin.py,
hermes_cli/cli_tui_mixin.py, tui_gateway/session_workdir.py,
agent/transports/codex.py (one-line imports) + comment pointers in
tools/file_state.py, tools/schema_sanitizer.py, scripts/tool_search_livetest.py.
Tests: patch("run_agent.X") / monkeypatch.setattr(run_agent, "X") /
`from run_agent import X` -> defining module across 99 test files.
The retry must land on the same provider cache key as every prior request
in the session. Discard only the one-shot continuation disable and send
agent.reasoning_config verbatim; a config that is itself a disable is
omitted (that session never sent anything else, so nothing warm is lost).
Live: user effort=high, ephemeral disable → 400 → retry carries
{enabled: true, effort: high}.
Reasoning-mandatory routes answer reasoning: {enabled: false} with HTTP 400
"Reasoning is mandatory for this endpoint and cannot be disabled". Hermes
sends that disable for /reasoning none, agent.reasoning_effort: none, and the
one-shot thinking-exhaustion continuation override (which GLM-5.3-flash
triggers on its own). The Nous profile's catalog guard swallows the disable
only when its per-process capability cache already says mandatory; a gateway
that warmed the cache before the route flipped kept sending it, and the 400
was classified as a non-retryable format_error that aborted the turn.
- error_classifier: new reasoning_mandatory reason (retryable, no fallback,
no compression), matched before the request-validation branch.
- conversation_loop: one-shot recovery — set agent._reasoning_disable_rejected,
queue a catalog refresh for the provider, retry.
- chat_completion_helpers: _reasoning_config_for_wire drops every disable
(configured or ephemeral) once the route has rejected one.
- hermes_cli/models: refresh_reasoning_caps_async(provider) forces a
background re-fetch of the Nous/OpenRouter catalog.
- openrouter profile: omit a disable when the catalog marks the route
mandatory (parity with the Nous profile).
Live: z-ai/glm-5.3-flash on the Portal with a poisoned mandatory:false cache.
Before: turn aborted with the 400. After: one retry, thinking stays on, turn
completes.
GLM-style models serialize tool calls as XML in the text channel; when the
stream drops mid-serialization with finish_reason=stop, the orphan
<arg_key>/<arg_value> fragment (or a bare unclosed <tool_call> opener)
matched neither the complete-block stripper nor the partial-stream guard
and was stored and displayed as ordinary assistant content.
strip_think_blocks (storage boundary) and the CLI display copy now strip
an unterminated block-boundary tool-call opener, or any line carrying
stray argument markup, to end of text. The response then reads as empty
and flows through the existing empty-retry path. Complete blocks and
inline prose mentions are unchanged.
Drop the loop-side '(empty)' rewrite (the turn-completion explainer already
owns that at delivery, and gateway/desktop match on the sentinel) and the
extra token-count persistence. Keeps: usage-absent empty streaks arm the
deterministic fast-fail after two attempts with no content or reasoning,
and every completed API call logs even when the provider omits usage
(#101898).
Follow-up to the salvaged #101894 (@jwilson411). The over-threshold
"reclamation did not run" warning is deduped on (reason, rearm mark), and
the key was only cleared when a prune committed. Every other path that
zeroes the rearm mark — compress(), on_session_reset/on_session_end,
bind_session_state, update_model — left the key in place, so a lockout
that warned at rearm=0, then a full compaction, then the same lockout
again was silent, contradicting the helper's own "warns again" contract
(and leaking the key across sessions on a rebound compressor).
- ContextCompressor._reset_proactive_prune_rearm(): one helper for the
five rearm-to-zero sites; clears the dedup key alongside the mark.
- _warn_reclamation_no_op(): dropping back under threshold releases the
key (mirrors _clear_context_overflow_warn semantics on the agent side).
- test_proactive_prune_loop_wiring: the attempts_exhausted fixture now
models the only state the real engine can produce for that branch
(should_compress() is should_compress_info()[0]) — budget spent
(max_compression_attempts=0) with the engine saying RUN, instead of
should_compress=False paired with (True, None).
- Two guards: lockout warns again after a rearm reset; dropping under
threshold releases the key. Both fail with the clears removed.
Teknium