/heartbeat is parsed by the slash-worker process, which starts its own
heartbeat watchdog thread there — but that process never runs turns, so
the queued prompt is armed-but-dead (countdown froze at ~0s, no turn ever
injected). Mirror the existing /loop driver: poll for due heartbeats in
the per-session notification poller and re-enter the live session through
_run_prompt_submit as a plain user turn, preserving alternation and prompt
caching.
(cherry picked from commit 23ae971075f2d8156642f4c268dd010510a7af41)
Publish-UiEvent only enters the delivery-wait loop under $script:UiServer,
and Show-ProgressWindow returns before the WinForms card is built whenever
the browser shim is up, so $script:Ui is always null there. The DoEvents
call could never run; the 50ms sleep loop is the whole wait.
A delayed browser could miss the 900ms terminal event and spin forever after the updater exited. Retain terminal delivery until the page acknowledges it, bound unavailable-client teardown and failed requests, and preserve a truthful final display.
Fixes#103747. Builds on OutThisLife and Teknium detached handoff work in #83634 and the #75895 quiet-window design. Continues Axl Ibiza Windows update investigation (#60233, #94107, #100763), including source/review contributions carried by merged #93353 and #85170. Existing #102373, #103140, #95719, #97299 and #103632 retain their separate scopes.
When a sash drag folds a tool zone to its rail, the store commit re-renders
only the wrappers whose style prop changed; the flex sibling the gesture
had pinned to `flex: 0 1 <px>` kept that inline preview and could not grow
into the freed space. Restore every captured style attribute before the
commit, on every release path.
A sash drag used to stop the moment its seam partner hit its min size, so a
row of panes behaved like unrelated boxes: growing the Browser tile could not
take space from Chat once the pane between them was at its floor. The drag
now plans the whole run from the pointerdown sizes — the partner donates
first, then its next visible sibling, and so on — and commits once on
release (fixed zones get px overrides, the flex run gets weights).
A reverse drag after a cascade stays local to the seam. Release only folds a
tool zone that THIS gesture took to its floor, so an unrelated rail already
resting there (a minimized Terminal) no longer cancels the Files/Chat commit.
Salvaged from #103166 (Jerry Gooch), trimmed to the sash-drag change; the
width/height lock feature, zone-body context menu and the restored-tool CSS
floor are held as separate decisions.
A background delegate_task call used to be ONE async unit: the runner joined on
every child and a single consolidated message re-entered the conversation when
the SLOWEST finished. Fifteen independent PR reviews therefore waited on the
fifteenth before the parent could act on the first.
Each task now carries an optional `group`. `_units_of` partitions the call's
children into units — one per distinct group, one per ungrouped task — and each
unit is dispatched to the async registry on its own, so its results re-enter the
conversation as soon as THAT unit is done. Tasks that must be compared or merged
share a group and still return together.
Capacity is unchanged: every unit of one call joins the first unit's pool slot
(`slot_key` in `async_delegation._dispatch`), so splitting never consumes more of
`delegation.max_concurrent_children` than the call did. Unit ids suffix the
call's id (`deleg_xxxx-1`, `-2`, …) so live transcripts stay under one dir; the
completion block names the group and notes that sibling units report separately;
`active_task_count` counts a unit's own tasks.
test_nous_anthropic_fallback_uses_the_messages_wire and
test_nous_child_rederives_api_mode_from_model describe the native wire, which
is now opt-in; select it in the test the same way the wire-contract suite
does, so both keep guarding the flip-back.
The compression/aux fallback ladder decides "payment error" via
_is_payment_error(), which only knew the spaced "resource exhausted". NVIDIA NIM
and gRPC-style wrappers serialize the same quota signal as ResourceExhausted /
RESOURCE_EXHAUSTED / resource-exhausted, so a 403/429/status-less body carrying
it re-raised instead of walking the configured fallback_chain, and compression
fell to the lossy emergency path. Add the three separator variants to
_PAYMENT_KEYWORDS (same status gate, no broader "exhausted" matching).
evals/auxiliary_resource_exhausted.py drives the real call_llm/async_call_llm
against two local OpenAI-compatible listeners (nvidia profile primary, named
custom fallback) for the before/after check. #85649
Portal serves anthropic/* on two routes. The native /v1/messages wire, which
Hermes has used since 02d5e23085, re-writes the previous turn's prompt cache
on 14-20% of consecutive calls in concurrent tool loops; the chat/completions
route does not. Measured 2026-09-06, 20 concurrent sessions x 6 tool calls on
Fable 5.1, same account, same hour, same first-party pin:
Nous /v1/messages 180 pairs, 25 stuck (13.9%); 3 earlier 40-session
runs 15.0 / 17.3 / 20.2%; unchanged by the pin
Nous /v1/chat/completions 320 pairs (2 runs), 0 stuck
OpenRouter direct, pinned 161 pairs, 0 stuck
"stuck" = cache_read on call k+1 equals cache_read on call k instead of
call k's prompt size: the last write was not visible, the turn (~28K) was
written twice. Every stuck pair had byte-identical system, tools and
per-message shas, so it is not client-side mutation. At $10-20/M for
cache writes that is 15-20% of a fan-out's write bill.
The cause is inside the portal's native route (every element of its
outgoing request reproduces clean from outside; NousResearch/api#227
carries the diagnostics). Until it is fixed, anthropic/* rides
chat/completions. nous.anthropic_wire: native opts back in. Cost of
chat: prior-turn thinking travels as OpenAI-style reasoning fields, and
cache_control scopes are translated by the portal's adapter.
Tests: new test_nous_anthropic_wire_default.py reads the knob through the
real config loader in a temp HERMES_HOME and through
resolve_runtime_provider (both settings); the existing native-wire
contract suite selects native via an autouse fixture so the wire keeps
working for the flip-back. Live: a real AIAgent on this branch,
provider=nous + anthropic/claude-fable-5.1, dispatches to
chat/completions with the session_id intact.
Follow-up to the salvaged contributor commit: the Gemini and Vertex guide model
tables now list both Flash generations the pickers offer, and an invariant test
ties the OpenRouter/Nous curated google/gemini-*-flash entries to (a) a Google
official-docs pricing row on the direct Gemini and Vertex routes and (b)
membership in the direct gemini/vertex picker lists. Red on main
(gemini-3.7-flash -> unknown), green with the contributor commit.
Campaign: https://github.com/NousResearch/hermes-agent/issues/104154
- Generalize Gemini 3 thinking config model prefix match to gemini-3*
- Add pricing snapshot entries for gemini-3.7-flash and gemini-3.8-flash
- Add model identifiers to Google, OpenRouter, and Vertex CLI lists
- Add test coverage for gemini-3.8-flash thinkingLevel configuration
Trim the salvaged suite to the behaviour contracts: a picker exception or Ctrl+C leaves
config.yaml's model exactly as snapshotted (parametrized), and an absent active_provider
stays absent through snapshot+restore. The mock-dispatch tests (asserting calls into our own
helpers) are dropped. evals/cli_fallback_add_picker_error.py drives the real
`hermes fallback add` under a Linux PTY into an ordinary picker OSError and checks the
persisted model: stranded on base, restored after.
Two corrections on top of the mention-preservation pick:
- Stripping our own trigger handle from every group message was replaced by never stripping
it in groups, which broke the pre-existing contract that `@bot 2` answers a pending clarify
prompt and `@bot ok` approves (the intercepts match the exact stripped text). Strip only when
no other participant is named; when other bots/users are mentioned the text is kept verbatim
so `@research_bot , @ops_bot …` no longer arrives as `, @ops_bot …`.
- The addressing block carried a per-message fact ("explicitly mentions you: yes/no") inside
`channel_prompt`, which is part of the cached-agent signature: a mention turn followed by a
reply turn rebuilt the AIAgent (prompt-cache miss) every time the shape flipped. Keep only
the username line, byte-identical for the life of the session.
Tests rewritten to pin both contracts (sole-addressee stripping; identical channel_prompt and
agent signature across a mention turn and a reply turn).
Follow-up to the truncation removal: with the full reply text in the prepared message, a
`@file:`/`@url:` reference inside the *quoted* message reached the context-reference expander
and read a local file on the replier's behalf (the old 500-char slice hid this for long
quotes; short quotes were already exposed on main). Expansion now runs on the new message
text only; the reply pointer is prepended afterwards.
The ChatGPT Codex backend answers a rejected encrypted-reasoning replay with the
same bare 400 {code: invalid_prompt, message: "Request blocked."} it uses for
genuine blocks. classify_api_error bucketed it as format_error, so the one-shot
repair that already exists for invalid_encrypted_content (disable replay, strip
cached codex_reasoning_items, retry once) never ran and the turn aborted.
Classify exactly that envelope from provider openai-codex — SDK 400 body, SSE
``error`` frame, or the ``response.failed`` text — as invalid_encrypted_content
while keeping format_error's abort-and-fallback hints, so the only behavioural
delta is turn_recovery's replay strip, which still requires cached reasoning
items. A block with nothing to strip aborts exactly as before; other providers,
other messages/codes and the #18028 safety refusals are untouched.
evals/codex_masked_replay_review.py drives a real AIAgent (codex_responses)
against a synthetic local Responses server for the before/after check.
Refs #92353, #92357, #94077.
Fold the adapter-level unsigned-preservation case and the name-detector matrices
into one parametrized invariant (thinking family keeps unsigned only; non-thinking
DeepSeek and unrelated models still take the generic third-party strip) plus the
native-endpoint immutability guard. Add evals/anthropic_proxy_thinking_replay.py,
a real-AIAgent probe against a synthetic local Anthropic-compatible relay used for
the before/after wire check.
Third-party Anthropic-compatible relays serving a DeepSeek thinking model
(deepseek-r/v4/pro/flash) follow the same replay contract as DeepSeek's
native /anthropic endpoint: strip signed thinking blocks, preserve
unsigned ones. Previously these sessions hit the generic third-party path
and lost ALL thinking blocks, breaking cross-turn reasoning coherence.
Detection is model-name based (vendor prefixes stripped), gated on the
existing _is_third_party check so direct Anthropic traffic is untouched.
Follow-up on the cherry-picked fix: _defer_update_notice no longer takes the Rich console it
can never safely use (the notice always lands after patch_stdout owns stdout), the unit test
asserts the contract at prompt_toolkit's boundary (an ANSI fragment reaches
print_formatted_text, with no raw ESC/markup in the visible text) instead of patching our own
cprint, and evals/cli_deferred_notice.py drives the real CLI under a Linux PTY with a FIFO-gated
update cache to A/B the late-notice cases (garbled on base, clean after).
The deferred update notice ("N commits behind") was rendered via
console.print which writes ANSI escapes directly to stdout. Under
patch_stdout (active in the interactive CLI), the StdoutProxy strips
ESC bytes, leaving visible [1;33m...[0m artifacts.
Fix: render Rich markup to an ANSI string via a captured Console, then
route through cprint (which uses prompt_toolkit ANSI parser) to
bypass the StdoutProxy mangling.
This is the same class of bug as #2262 (fixed by #2448 for agent._print_fn
and display.py), but the _defer_update_notice callsite in banner.py was
not covered by that fix.
Fixes#83969
GitLab (lib/gitlab/web_hooks.rb, app/services/web_hook_service.rb) sends
webhook-id and webhook-timestamp on EVERY delivery and adds webhook-signature
only when a signing token is configured. Selecting the HMAC path on any
webhook-* header would 401 every legacy X-Gitlab-Token install the moment it
upgrades to GitLab 19, so only the signature header selects the path; id and
timestamp then travel with it and the validator still fails closed when either
is missing. svix-* keeps its existing any-header selection.
Tests trimmed to the invariant bar: one parameterized contract (whsec_ and raw
secrets accept; wrong secret, tampered body and stale timestamp reject) plus the
GitLab legacy-token coexistence contract. evals/webhook_auth/standard_webhooks_ab.py
drives a real aiohttp WebhookAdapter on a dedicated loopback port with real
signed requests for before/after evidence.
Related: #47849 (HwangJohn, cherry-picked here), #92024 (earlier salvage of
#47849), #102080 and #103167 (same alias fix, same target).
The salvaged helper was appended to the gateway/run.py facade; new behaviour
belongs in a topical sibling. agent/session_activity.py already owns the
activity-snapshot contract the three render sites read from, so the formatter
moves there as format_iteration_progress and the three call sites import it at
module level instead of late-importing the facade.
Tests trimmed to the salvage bar (<= 2 invariant tests per fix): one
parameterized contract on the helper (unbounded/None hide the ceiling, a real
budget keeps N/M) plus the contributor's end-to-end busy-ack test through the
real render path. The per-site heartbeat/timeout tests exercised the same
helper through mocks and are dropped; evals/gateway_status_render/
iteration_ceiling_ab.py drives all three real render sites with a real AIAgent
for before/after evidence (3 sentinel leaks on main -> 0).
Related: #103109 (same fix, same target module; credited in the PR),
#102845 (third render site, closed by its author in favour of #102817).
Rebase of the existing fix onto current main: the god-file split that
was in flight when this PR opened has landed, moving both original
call sites out of gateway/run.py into gateway/run_busy.py and
gateway/run_turn.py, which is why this PR showed a merge conflict.
Also folds in a third call site (see below) that a competing PR
(#102845, closed by its author in favor of this one) identified after
this PR first opened.
Three user-facing gateway status lines render "iteration N/M" from
AIAgent.get_activity_summary()'s api_call_count / max_iterations pair:
the long-running heartbeat (run_turn.GatewayTurnMixin.
_run_agent_notify_long_running), the busy-session acknowledgment
(run_busy.GatewayBusySessionMixin._compose_busy_ack_message), and the
gateway-timeout diagnostic message shown to the user when a run is
force-timed-out for inactivity (run_turn.GatewayTurnMixin.
_run_agent_timeout_result). AIAgent.max_iterations defaults to
sys.maxsize (unlimited tool-calling iterations for a top-level session
-- see run_agent.py), so all three printed the literal
9223372036854775807 as the denominator, e.g.:
⏳ Working — 3 min — iteration 2/9223372036854775807, receiving stream response
That reads as a bug rather than "unbounded" and is meaningless to a
user.
Add _format_iteration_progress() to gateway/run.py, a small shared
formatting helper: once the configured max_iterations is at or above
sys.maxsize, it renders "iteration N" alone and omits the denominator;
a genuinely finite budget (e.g. a subagent's delegation.
max_iterations: 250) still renders "iteration N/M" as before. All
three call sites now go through it. The gateway-timeout diagnostic
message's own operator-facing logger.error() call keeps the raw
resolved value for diagnostics -- only the two user-facing diag_lines
built from it are reformatted.
Added tests/gateway/test_format_iteration_progress.py (the helper's
own unit tests: unbounded default, above-sentinel values, finite
budgets, and graceful handling of a missing/malformed max_iterations),
tests/gateway/test_gateway_timeout_iteration_progress.py (both
diagnostic-message branches, unbounded and finite, plus the heartbeat
call site exercised end to end through its async polling loop), and a
new regression test in tests/gateway/test_busy_session_ack.py::
TestBusySessionAck::test_status_detail_omits_denominator_for_unbounded_max_iterations
that exercises the busy-ack call site end to end with the real
sys.maxsize default and asserts the sentinel never reaches the
rendered text.
Defect 1 in the original report (a direct_result tool's raw output
occasionally becoming the user-visible reply) is left for a separate
change -- the reporter frames it as an open core-level design question
("a per-turn suppression option... would fix the class for all
plugins"), not a drop-in fix, and it touches reply composition rather
than status-line formatting.
Part of #102806 (defect 2: the iteration-ceiling display; defect 1 is a separate change).
Replays the native task-card rail and the text-stream rail through the real
slack_sdk AsyncWebClient against a local aiohttp receiver that enforces the
documented markdown_text/chunks mutual exclusion. Local contract proof, not
Slack live: records outgoing bodies before/after #87743.
chat.appendStream rejects a request carrying both markdown_text and
chunks with `cannot_provide_both_markdown_text_and_chunks`, so every
native task-card update failed and each turn silently downgraded to the
plain-text fallback message (#87743). The chunks are the native
rendering; the gateway caller keeps its own editable-text fallback rail
for when the native call itself fails, so the markdown_text field was
both rejected and redundant.
Fixes#87743
Parametrized refusal test (bare/fenced/uppercase-fenced truncated objects) asserts
profile.yaml is byte-identical afterwards; one prose test keeps the lenient
fallback contract. Replaces five near-duplicate tests from #104075.
Campaign tracker: https://github.com/NousResearch/hermes-agent/issues/104154
hermes_cli/profile_describer.py's aux-LLM response parser is documented as
"lenient, never raises": when the reply doesn't parse as JSON via
_extract_json_blob, it falls back to treating the WHOLE raw reply as plain
prose and persists it (truncated to 280 chars) as profile.yaml's
description.
That fallback exists for models that ignore the JSON-output instruction and
just answer in prose -- reasonable. But it doesn't distinguish that case
from a reply that DID start out as the requested JSON object and was cut
off mid-object by the aux model or transport. _extract_json_blob requires a
matching closing brace, so a truncated object (no `}` at all, or one cut
off mid-string) returns None just like plain prose does, and the fallback
then persists the raw JSON fragment verbatim: literal leading `{`, `\n`
escapes, a sentence chopped mid-word (#104067).
Fix: when parsing fails, check whether the reply (after the existing
code-fence strip) still looks JSON-shaped (starts with `{`). If so, it's a
malformed/truncated structured reply, not prose -- refuse it and return
ok=False instead of persisting the fragment. Only fall through to the
raw-text-as-prose fallback when the reply never looked like JSON to begin
with, so genuinely prose-only aux replies keep working exactly as before.
Also logs one INFO line on the refusal path, per the issue's note that the
silent fallback made this undiagnosable.
Update (review feedback from @jerrygooch): profile_describer._FENCE_RE was
case-sensitive, so an uppercase ```JSON fence wasn't stripped -- stripping
left "JSON\n{..." which doesn't start with "{", so a truncated response
under that fence variant still fell through to the prose fallback and got
persisted, defeating the fix for that case. Added re.IGNORECASE to
_FENCE_RE, aligning it with kanban_specify._FENCE_RE (which already used
re.IGNORECASE), and added a regression test for the uppercase-fence case.
Added tests/hermes_cli/test_profile_describer.py coverage: a truncated
object cut off mid-sentence (the real-world shape from the issue), one cut
off right after the opening brace, the same truncation under a ```json
fence, the same truncation under an uppercase ```JSON fence, and a
regression guard that a genuine plain-prose reply still hits the existing
lenient fallback unchanged.
Fixes#104067
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LUtGTop5iGKi5vfWnKnwET
The plugin loader execs sibling modules before the package __init__, so a
module-level `from . import _read_mem0_json` in _setup.py failed against the
empty parent shell and the whole module silently dropped out — every later
`hermes memory setup mem0` died with "cannot import name 'post_setup'".
The test loads mem0 through the real discovery path from a cold sys.modules
and asserts the cached _setup module exposes post_setup (red on base).
Also maps the contributor email for #103078 credit.
Campaign tracker: https://github.com/NousResearch/hermes-agent/issues/104154
The plugin loader pre-registers empty shells in sys.modules before
executing any submodule. When _setup.py ran 'from . import _read_mem0_json'
at module level, Python found the empty parent shell (init not yet executed),
so the import failed silently. This left _setup.py partially loaded and
post_setup undefined, causing 'cannot import name post_setup' during
hermes memory setup mem0.
Move the import inside the three functions that use it, matching the
late-import pattern __init__.py already uses for post_setup.
Fixes: hermes memory setup mem0 crashing with ImportError
#76996 raised DEFAULT_READ_LIMIT, the read_file_tool signature and the schema
default from 500 to 2000, but the registry dispatch handler `_handle_read_file`
kept its own literal `args.get("limit", 500)`. Since models omit `limit` on most
reads, every dispatched read still stopped at 500 lines with `truncated: true`,
so the default flip never reached production traffic.
All three sites now read the single DEFAULT_READ_LIMIT constant, so the schema,
the Python default and the dispatch fallback cannot drift again.
Live probe (registry.dispatch on a 1500-line file, no limit): before 501 lines /
truncated=true; after 1501 lines / truncated=false.
`format_batch_tag` handed out `set N` ordinals from one process-wide table,
so every conversation on a shared backend and every child's nested fan-out
advanced the same counter. A user's second wave of 15 lanes rendered as
`[set 20 · 13/15]`, which reads like 20 batches were spawned.
Scope the ordinal table by the parent conversation (`parent_agent.session_id`)
and thread the parent through the three render sites (batch header,
completion lines, child tree-line prefix via the shared session_ref). The
first fan-out in a conversation is `set 1`, the next `set 2`; sibling
conversations and nested child batches no longer inflate it.
Invariant test proven red on origin/main, green with the fix.
Review follow-ups on the #96187 cherry-pick: skip on win32 like the
sibling tests that shell out; reuse the file's `exec` helper; one temp
root so a failing second mkdtemp cannot leak the shim dir; quote the
shim's redirect target; guard the payload-prefix sentinel so a renamed
loop marker can never make the test execute the real spawn payload;
drop the lockMetadata fields the assertions never read. Move the
mutexPath contract into withRemoteUpdateMutex's doc comment instead of
a third inline restatement.
`apps/desktop/'` is not a real path. It is a literal single-quote directory
holding a full absolute path as nested subdirectories:
apps/desktop/'/var/folders/5h/.../hermes-update-mutex-LMF9y5/home/.hermes-update-in-progress.mutex'
Both files are 0 bytes, nothing in the tree references them, and the leading
and trailing `'` are part of the filenames. They are the fingerprint of the
double-quoted mutex path in `withRemoteUpdateMutex()`: the path reaches Python
with its shell quotes still attached, so it is treated as CWD-relative and
`os.makedirs()` materialises the whole absolute path under whatever directory
the process happened to be in. Running
`apps/desktop/electron/remote-lifecycle.test.ts` from `apps/desktop`
reproduces it on the spot.
They were swept in by a `git add .` in 36620578f0, an unrelated menu-label
commit.
This only removes the committed artifact. The generator is a separate concern
already covered by open PRs (#99189, #96187, #96260) and issues (#99133,
#96212, #96188); those repair the quoting but none of them deletes these two
files, so the litter would survive whichever one lands.
expandRemotePath() returns an already-quoted shell fragment
("$HOME"'/path'), but three call sites wrapped its output in shq()
again: the withRemoteUpdateMutex python argv, the reservation/lock/
owner_file assignments in buildSpawnCommand, and the identity values in
buildOwnedStaleTerminationCommand.
The remote shell strips only one quoting layer, so python received a
mutex path with literal quote characters in it (creating a directory
literally named ' in $HOME), and the payload's mkdir "$reservation"
loop spun on a path that can never exist. Every Desktop SSH backend
spawn hung until the connect timeout, retried, and left an orphaned
flock queue behind; stale-owner cleanup always printed REFUSED for the
same reason.
The regression test parses the composed command with a real sh — the
same parse the remote login shell performs — and requires the mutex
path and the payload's reservation paths to come out fully expanded.
Portal decides routing centrally per model and returns HTTP 400 on any
caller-supplied `provider` object (live-confirmed). Drop the five "OpenRouter or
Nous Portal" claims from the provider-routing page and say plainly that the
setting is ignored on Portal now that the profile no longer forwards it.
Halldrix