Commit Graph

3580 Commits

Author SHA1 Message Date
fangliquan 390c27c6db fix(agent): accumulate streamed tool arguments linearly 2026-09-03 11:43:00 +05:30
kshitijk4poor fafd370094 test(compression): register the heartbeat in ROUTINE_COMPRESSION_STATUS_SAMPLES
Completes the constant + samples + noise-regex trio teknium named as the bar
for this heartbeat on #98371; the Telegram noise-filter and TUI retag
parametrised suites now iterate the heartbeat wording too.
2026-09-03 11:29:36 +05:30
kshitijk4poor c41525b290 fix(compression): heartbeat honours the quiet-engine contract and the single start line
Review follow-ups: the heartbeat opened a visible compacting phase even when
the context engine suppressed the routine start status (no terminal edge would
ever close it), and its start() emitted a second start line milliseconds after
the routine one (two chat messages on adapters without send_or_update_status).
Gate the client-visible heartbeat on the start status having been emitted, drop
the start-time emit, and route ticks through agent._emit_status so CLI print
and gateway filtering match every other compaction status.
2026-09-03 11:27:55 +05:30
kshitijk4poor 548e0ec923 fix(compression): route the heartbeat through the shared compaction status contract
The contributor heartbeat called status_callback("compacting", <ad-hoc text>).
Every other compaction status uses the "lifecycle" key: the TUI gateway
re-tags lifecycle statuses to kind="compacting" via
is_compaction_progress_status, Telegram edits one bubble per status key, and
gateway/run.py's chat-platform filter only recognises the registered
templates — so the ad-hoc text leaked to Telegram/Discord on every tick with
compression.progress_notices off (verified: _prepare_gateway_status_message
passed it through). Register COMPACTION_HEARTBEAT_STATUS next to
COMPACTION_STATUS, emit it under "lifecycle", and cover the filter.
2026-09-03 11:27:55 +05:30
Jack 8381d4fadf compression: emit periodic client-visible status during compaction
Context compression can stream for minutes with no deltas, tool events,
or status lines reaching remote transports. Idle-progress watchdogs on
those clients treat the silence as a dead turn and interrupt it — the
Android relay app's 180s turn watchdog fires session.interrupt, killing
a healthy compression mid-flight and rolling back its work. On sessions
near the context ceiling this loops forever: every new prompt retriggers
preflight compression, which dies at exactly +180s again (observed
telemetry: attempts aborted at 180469ms/180233ms/180219ms with
failure_class=explicit_interrupt).

Fix: the existing _CompressionActivityHeartbeat (which today only
refreshes the SessionDB activity tracker) now also emits a 'compacting'
status through agent.status_callback — once at compression start and on
every heartbeat tick. The gateway already routes status_callback to
status.update events, and clients already reset their watchdogs on any
received event, so each heartbeat re-arms them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 6d2e0e860d6fdb44ce549d1fa11a06662b1a4b0b)
2026-09-03 11:27:55 +05:30
kshitijk4poor d6de054bac refactor(compressor): one helper for the last template-visible role
The re-append slot check duplicated the summary-role scan at the head/summary
boundary; both now call _last_template_visible_role.
2026-09-03 11:27:40 +05:30
kshitijk4poor ab73aeff05 fix(compressor): keep pending tool_calls, avoid double anchors, no header stacking
Review follow-ups on the in-flight replay:

- Run _sanitize_tool_pairs BEFORE the re-append. Its trailing-in-flight
  exemption (#79278) walks back from the list end; with the replay user row
  there, a genuinely pending assistant(tool_calls) looked orphaned and had its
  calls stripped, so the executor's late tool result was dropped.
- When the restatement is merged onto a user-pinned summary carrier, flag the
  carrier (_inflight_replay_merged). The carrier's metadata marks it synthetic,
  so conversation_compression._ensure_compressed_has_user_turn inserted a second
  copy of the same request; it now treats the flag as intent-present, and the
  next cycle recognises the carrier as the task instead of losing it.
- Header idempotency: restate the text after the last header so a task that
  survives several compactions carries one header and one copy.
- Exclude metadata-flagged scaffolding (_todo_snapshot_synthetic, recovery
  nudges) from the in-flight scan via the shared _is_real_user_message.

Tests cover all four; each fails with its fix reverted.
2026-09-03 11:27:40 +05:30
kshitijk4poor 1d260f7683 fix(compressor): judge the re-append slot on template-visible roles
The in-flight replay checked only compressed[-1] before choosing between
appending a user row and merging onto the summary carrier. A user-pinned
summary followed by an exempt tool_calls/tool tail therefore gained a second
visible user turn and broke the Mistral-style alternation pre-flight
(tests/agent/test_summary_role_template_alternation.py::test_zero_user_guard_still_forces_user).
Look through the exempt tail to the last template-visible role instead.
2026-09-03 11:27:40 +05:30
Justin Wilson 0ed3f9b45b fix(cron): re-append in-flight job prompt after compaction
Compaction of a single-prompt (cron) session left no user message after
the handoff, so SUMMARY_PREFIX ordered the model to do nothing and the
scheduler recorded success. Re-append the in-flight task after the
summary.

Fixes #100818

(cherry picked from commit c0ea50ac02fd7553b1ef7deb0162483cb9a1b6ac)
2026-09-03 11:27:40 +05:30
Teknium 4dac5f28af fix(execute_code): parallel cells no longer orphan kernel processes; LSP stops treating package dirs as project roots
Session kernels: a kernel mid-spawn (proc=None) read as dead, so every
concurrent cell for one owner replaced the registry entry and the
winner's process leaked outside the registry (110 live kernels, 1.1 GB,
330 threads under one 4-capped process). Reap/evict also tore down
kernels with cells attached, rmtree-ing the staging dir under the
spawner. Kernels now track attached cells: only settled kernels are
reaped/evicted, a kernel dropped while busy is torn down by its last
cell, and in-cell registry pops never remove a replacement.

LSP: a directory holding __init__.py is a package, not a project root.
hermes_cli/setup.py matched the python marker list and gave every
worktree a second pyright rooted at hermes_cli/ (70 of 105 reaped
clients in one session, ~40 servers / 8.7 GB live).
2026-09-02 21:50:00 -07:00
Teknium 139396995a feat(opencode): send x-opencode-session on every OpenCode request for backend affinity
OpenCode pins requests sharing an x-opencode-session value to one upstream
backend, which is what keeps its prompt cache warm across a conversation.
Hermes never sent it, so cache ratios on OpenCode traffic were poor.

- agent/opencode_affinity.py: single owner of the header — target detection
  (built-in zen/go/free, custom opencode-* providers, any opencode.ai URL)
  and the key (affinity scope → conversation root → session id, cron
  timestamp stripped), same resolution as OpenRouter/xAI affinity hints.
- build_api_kwargs: merged once after the per-mode builder, so
  chat_completions, codex_responses and anthropic_messages all carry it.
- auxiliary _build_call_kwargs: same key from the runtime-main session so
  compression/title/vision calls stay on the conversation's backend; the
  aux Codex and Anthropic adapters now forward extra_headers.

Closes #81584, #81832 (deepseek-v4-flash 400 without the header).
2026-09-02 21:46:36 -07:00
Teknium 116ca1db1e fix: sibling Nous 401 recovery adopts a peer's refresh instead of rotating again
The stampede fix added a `stale_access_token` hint to
resolve_nous_runtime_credentials() so a process whose bearer just 401'd
adopts a token a sibling already rotated instead of re-POSTing the shared
grant — but only the credential-pool caller passed it. The main agent's
401 path (run_agent._try_refresh_nous_client_credentials), the auxiliary
client rebuild, and the proxy adapter all called force_refresh=True with
no hint, so `_already_rotated_by_peer` could never fire: N subagents
hitting hourly expiry still issued N serialized refreshes, each one
invalidating the token a sibling had just adopted.

Live 12-process A/B against a fake Portal: 12 refresh POSTs / 9 distinct
final tokens before, 1 POST / 1 token after.
2026-09-02 21:22:07 -07:00
Jeffrey Quesnelle 48c0c3a873 Merge pull request #96633 from afourniernv/chore/relay-0.8.0
fix(relay): upgrade to 0.8.3
2026-09-02 22:03:27 -04:00
Royalaid 906cd5f443 fix(search): prune heavy trees from zero-match probe 2026-09-03 04:56:59 +05:30
Christopher cf86c47624 fix(agent): evict stale screenshot payloads before send
Call the existing keep-newest vision retirement on the per-call
api_messages clone after sanitization so OpenAI-style tool-result
screenshots are not re-uploaded on every later turn.
2026-09-03 04:51:51 +05:30
fangliquanflq 6064668c8f perf(usage): prefer bundled pricing before metadata fetch 2026-09-03 04:00:51 +05:30
kshitijk4poor cfe88a1f7d feat(config): context_file_read_timeout key + narrow reader catch
Expose the read deadline as a top-level config.yaml key beside
context_file_max_chars (same load_config_readonly resolution shape), default
5s, documented in context-files.md. Narrow the reader thread's catch from
BaseException to Exception: control-flow exceptions can't originate inside
read_text on a worker thread, and re-raising one would bypass the sites'
except Exception / except (OSError, UnicodeDecodeError) handlers.
2026-09-03 03:15:31 +05:30
Edder Talmor 972b94bd29 fix(agent): time out slow context file reads
Context files (SOUL.md, .hermes.md/HERMES.md, AGENTS.md, CLAUDE.md,
.cursorrules, .cursor/rules/*.mdc) and subdirectory hint files are read on
the startup / per-turn path. On network-backed filesystems (iCloud Drive,
OneDrive, NFS) a cold read of an evicted file can block indefinitely, which
stalls system-prompt assembly before the first turn.

Read them on a daemon thread with a 5s deadline via _read_text_with_timeout;
a timed-out file is logged at WARNING and skipped so the next context source
still loads. Read errors propagate exactly as before, so each site's
existing try/except handling is unchanged.

Re-cut against current main from PR #10110 (the original diff predates the
context_length plumbing and the AGENTS.md directory-chain loader); helper,
sites and regression tests follow the original.
2026-09-03 03:15:31 +05:30
kshitijk4poor a1d5a976b3 fix(agent): never floor an anchored pressure figure; keep the estimator total on lone surrogates
Follow-ups from review of the two salvaged #87490 commits:

- _pressure_with_real_floor now applies only on the rough fallback branch.
  A valid usage anchor is provider-exact and wins as-is: on MoA turns the
  anchor deliberately uses the pre-fold aggregator usage while
  last_real_prompt_tokens holds the folded figure, so flooring the anchored
  value would re-add fan-out tokens the anchor exists to exclude. Docstring
  rewritten to describe the real path split (anchor since d3a1c46510).
- estimate_tokens_rough: encode with errors="replace". main's estimator
  never raised; text.encode() on a lone surrogate (routine in tool output,
  see message_sanitization) raised UnicodeEncodeError and would abort a
  turn where main produced a slightly-off number.
- Record the cl100k/o200k/Qwen2.5 calibration for the bytes/4 rule.
- tests: accented Latin within +10% of the ASCII rule; mixed Cyrillic/ASCII
  counts ASCII at one byte; lone surrogates don't raise; anchored pressure
  is never floored (wiring shape).
2026-09-03 03:09:06 +05:30
Darafei Praliaskouski f07b6ff426 fix(agent): count non-CJK sparse text by UTF-8 bytes in the rough estimator
The ~4 chars/token rule is calibrated for ASCII; Cyrillic, Greek, Arabic and
similar 2-byte scripts tokenize at ~2-3 chars/token, so chars/4 under-counts
them ~2x and the pre-flight pressure figure trails real usage by tens of
percent on non-English sessions. Counting UTF-8 BYTES at ~4/token uses the
encoding width itself as the corrective: ASCII is unchanged (1 byte/char),
2-byte scripts count at chars/2, and the CJK dense path keeps its explicit
~1 token/char rule with the sparse remainder byte-counted. The ASCII
isascii() O(1) fast path is preserved; the non-ASCII paths add a single
C-level encode over text that was already being regex-scanned.

Complements the last-real-prompt floor: the floor catches sessions that are
already at the ceiling, this keeps the estimate from lagging in the first
place.
2026-09-03 03:09:06 +05:30
Darafei Praliaskouski 73b8ec3cea fix(agent): floor pre-API compaction pressure at the last real prompt size
The chars/4 rough estimate under-counts Cyrillic and other non-ASCII scripts
by up to ~2x, so a session can ride the provider's real context ceiling while
the rough pressure stays under the compaction threshold. On providers that
silently clip over-window prompts (ollama /v1) the reactive overflow handler
never fires either, and the length-continuation retry path re-enters the API
call without passing the post-response gate — reproducing the truncation
death spiral this branch already addresses (observed live after the first
commit: real prompts 64,842 -> 64,995 against a 55,705 threshold, output
room shrinking 694 -> 541 tokens).

Floor the pre-API pressure figure at the provider's last reported
prompt_tokens — authoritative, script-independent — except for the one turn
after a compaction when that value is known-stale (#36718's
awaiting_real_usage_after_compression window).
2026-09-03 03:09:06 +05:30
Enzo Adami 02a6e95228 fix(files): allow one full read after compaction 2026-09-03 03:07:37 +05:30
webtecnica 4a0aae8aae fix(agent): preserve tool-output dedup state across context compaction (#84857) 2026-09-03 03:07:37 +05:30
joaomarcos a89f6aba44 fix(compaction): stop idle compaction re-summarising a transcript that has not grown
Idle compaction (`compression.idle_compact_after_seconds`) skipped work only
when the context was at or below `threshold_tokens × summary_target_ratio`.
That is a *theoretical* post-compaction target: a real pass lands well above
it because the system prompt, the tool schemas and the protected head/tail
are an incompressible floor.

So a session that compacted to well above the target stayed above it forever,
and every later idle resume re-ran a full summarisation over a transcript that
had not grown. In the reported session the 17:01 pass reduced 64,105 -> 44,579
tokens; the 17:36 resume re-fired because 44,579 was still above the 25,502
theoretical floor, blocking the prompt for another 256 s on a ~55 tok/s local
route and reclaiming nothing. Neither pass was followed by a single API call.

`_should_idle_compact` now also honours `last_compression_rough_tokens` — what
the previous pass on this session actually produced, recorded by
`compress_context` with the same `estimate_request_tokens_rough` shape the idle
estimate uses. When it is known, the transcript must accumulate at least one
`floor_tokens` worth of new content on top of it before another pass is worth
its wall clock. A raised floor is a deferral, not an off switch.

`0` — nothing compacted yet, or the counter cleared by a rebind/recalibration
— keeps the original semantics exactly, so the first idle compaction of any
session is unaffected. The call site type-pins the read so compressor doubles
that expose a Mock there fall back to the original floor.

Fixes a root cause behind #97239. Deliberately does not touch the TUI status
surface (#97253), the digest chain / total ceiling (#93241), or turn-settle
scheduling (#96891).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HV3k1v5nx9ag5d5wXSFZ7o
2026-09-03 03:07:33 +05:30
kshitijk4poor 09fa753366 refactor(terminal): share the force_remove-aware env teardown between cleanup_vm and the probe
Extract the signature-checking cleanup dispatch that cleanup_vm already had
into tools.terminal_tool._cleanup_env and call it from the prompt-time
backend probe instead of carrying a second copy. Comment on the probe
trimmed to the non-obvious part (why ssh is skipped).
2026-09-03 02:07:42 +05:30
kshitijk4poor 962b35b4c7 fix(agent): probe teardown uses signature check and skips the shared SSH master
Follow-up to the cherry-picked fix:
- decide force_remove by inspect.signature (the idiom cleanup_vm already
  uses) instead of catching TypeError, which would misread a TypeError
  raised inside cleanup() as a signature mismatch and call cleanup twice
- skip teardown for ssh: SSHEnvironment has no task-scoped sandbox and its
  cleanup() runs sync_back + 'ssh -O exit' on a ControlMaster socket keyed
  by user@host:port, shared with the agent's real environment
2026-09-03 02:07:42 +05:30
Fatmylin 9640f8e1f8 fix(agent): tear down the backend probe's throwaway sandbox
`_probe_remote_backend` spins up an environment with
`task_id="prompt-backend-probe"` purely to run one `uname` for the system
prompt, then drops the reference without cleaning up. Container backends
default to `container_persistent` / `docker_persist_across_processes`, so
that throwaway sandbox stays up for the entire process lifetime, sitting
idle next to the agent's own `default` sandbox.

Observed on a three-profile Docker-backend host: six running containers
where three were doing any work. Every profile permanently paid for one
extra container whose only job had been a single `uname` at startup.

Fix: wrap the probe in `try/finally` and call
`env.cleanup(force_remove=True)` — the explicit-teardown path that
`DockerEnvironment.cleanup` already documents ("No current caller passes
`force_remove=True`; the parameter is here so the explicit-teardown
semantics can be wired up later"). Backends that inherit the kwarg-less
base `cleanup()` fall back to the bare call.

Verified on a live aarch64 Docker backend: the probe container is now
created, used and reclaimed within the same call. Three regression tests
cover the success path, the exception path, and the kwarg-less fallback;
all three fail without the source change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PLbMzWqHVTPNBFisM8UiNB
2026-09-03 02:07:42 +05:30
kshitijk4poor 5f24f291c2 fix(curator): carry .git file pointers too and document the deleted-skill drop
Submodule and worktree checkouts store .git as a file (gitdir: pointer);
the carry-over only looked at directory names, so that form was still lost
on rollback. Handle files with the same guard. Docstring now states the
deliberate limit: an excluded entry whose skill dir the target snapshot
lacks is dropped with staging (no orphan .git) and is not undoable via the
safety snapshot, which excludes these paths as well.
2026-09-03 01:43:22 +05:30
kshitijk4poor 30be83ab72 fix(curator): carry nested excluded subtrees across rollback
Excluding nested .git from snapshots has a side effect on rollback: the
staging move takes the whole live skill dir (including its .git) into
.rollback-staging-*, the extract restores the snapshot without it, and the
staging dir is then deleted — so a skill that is itself a git checkout lost
its .git on any rollback. Reproduced: main preserves it, the exclusion-only
branch did not.

After a successful extract, move excluded subtrees from the staged copy back
under their restored skill dir (mirroring how a top-level .git survives by
never being staged). Regression test included.
2026-09-03 01:43:22 +05:30
kshitijk4poor 3bfa07fc5d docs(curator): carry over the #91449 growth rationale for the .git exclusion
Folds the incident explanation from #91458 (@liuhao1024) into the comment on
_EXCLUDE_TOP_LEVEL so the reason .git is excluded — compounding snapshot
growth, not just rollback safety — survives next to the set.

Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
2026-09-03 01:43:22 +05:30
Jakub Wolniewicz 5b92b9f913 fix(agent): exclude .git and .curator_backups in curator snapshot_skills 2026-09-03 01:43:22 +05:30
Carry00 edac49e473 fix(skills): stop syncing bookkeeping dirs to sandboxes
iter_skills_files() walked the skills tree with a bare rglob("*"), so the
.hub download cache, .archive, curator backups, and any node_modules/.git
under a skill package were uploaded to the sandbox on every sync. The
sandbox never reads them: skill content is resolved host-side.

EXCLUDED_SKILL_DIRS is already the canonical exclusion set, honoured by
discovery and backup. Apply it to the sync path too, across all three
roots iter_skills_files() walks (local, external, project-local), and add
.curator_backups to the set.

Measured on a local install: 900 files / 67.3 MB -> 771 files / 8.4 MB.

This is not just wasted bandwidth on the SSH backend, where the oversized
payload can exceed the 120s _ssh_bulk_upload deadline and surface as the
agent hanging on every tool call.

The filter intentionally does not reuse is_excluded_skill_path(), which
also prunes references/, templates/, assets/ and scripts/ -- those hold
support files and bundled scripts the sandbox does read and execute.
2026-09-03 01:33:18 +05:30
kshitijk4poor c4e394cdf8 refactor(auxiliary): one predicate for the critical-path retry skip
The sync and async retry sites each re-derived the same three-clause
decision (critical task + full-budget timeout + not a no-progress fail) with
their own copy of the rationale — which is exactly how the async site drifted
in the first place. _should_skip_same_provider_retry() now owns the rule and
its carve-out next to _TIMEOUT_NO_RETRY_TASKS; both sites call it.

Behavior-preserving: same clauses, same exception object, same outer guard.
2026-09-03 01:33:00 +05:30
kshitijk4poor 8115ff897a fix(auxiliary): mirror the no-progress carve-out on the async timeout skip
f50b5bb0fa taught the sync retry site to keep the cheap same-provider retry
when a Codex stream dies inside the 60s no-progress window (zero output),
skipping straight to fallback only on a stall or hard-ceiling timeout. The
async site never got that carve-out, so after widening the skip to vision
(#97572) an async vision call on a stillborn stream would have jumped to
fallback where the sync path retries. Both sites now apply the same rule.

Adds the async twin of the vision-skip test and a no-progress-still-retries
guard for the async site.
2026-09-03 01:33:00 +05:30
xmhua 827cf6fa01 fix(auxiliary): skip same-provider retry on a vision full-budget timeout
Issue #54465 established that a same-provider retry after a full-budget
timeout costs a second whole `timeout` window before the fallback chain is
reached, doubling the user-visible stall, and that compression must not pay
it because it sits on a critical path. The guard added for that is spelled
`task == "compression"`, so vision — which sits on the interactive path —
still retries.

The cost is the same and the stall is more visible: the turn holding the
image cannot answer, and because turns are serialised the following user
messages queue behind it. Two sequential full-budget timeouts on an
unhealthy vision provider is a long stall for something the fallback chain
could have served immediately.

Replaces the string comparison at both retry sites (sync `call_llm` and
`async_call_llm`) with `_TIMEOUT_NO_RETRY_TASKS = {"compression", "vision"}`,
so the two paths cannot drift again. Behaviour is unchanged for every other
task: fast blips (a streaming-close or a 5xx) still retry, and only
full-budget timeouts on those two tasks skip straight to fallback.

Tests: vision now falls straight through to fallback with the primary tried
exactly once, and a non-critical task still gets its one same-provider
retry, so the change stays scoped. Reverting the source change fails the
vision test and leaves the scoping test green.

Not the same as #51513, which fixes five separate defects in the vision
fallback chain (capability detection, sync/async client misuse, geo-block
and RemoteProtocolError classification, and chain iteration). This is about
what happens before that chain is reached.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 01:33:00 +05:30
globalvet2025 b36489be37 fix: gate Nous auth-refresh message behind verbose mode
The 'Nous agent key refreshed after 401' message used a bare print(),
making it always visible. The equivalent xAI/Codex, Copilot, and Anthropic
auth-refresh messages all use _buffer_vprint() (verbose-gated). This brings
the Nous path in line with the others so the routine ~15min OAuth key
refresh no longer prints noise on every retry.
2026-09-02 10:55:46 -07:00
cryptoyasenka 3265801900 fix(aux-client): evict expired auxiliary client on 401 by refreshing under the lookup cache key
On the default Nous config, call_llm acquires the auxiliary client via
_get_cached_client(resolved_model=None), so the cache key's model
element is "". On a 401, _refresh_nous_auxiliary_client rebuilt the
client but keyed the new entry on the resolved wire model (final_model,
e.g. "Hermes-4-405B"). The fresh client therefore landed under a
different key than the lookup, and the stale expired-credential client
under "" was never overwritten: every auxiliary call kept hitting the
dead client, 401ing and forcing a credential portal round-trip on each
request instead of self-healing after the first refresh.

The auto-provider dimensions had the same divergence: call_llm and
async_call_llm dropped task at both acquisition sites, and the async
path additionally dropped main_runtime at acquisition and at both of
its refresh sites, so the refreshed client shadowed the stale one under
a divergent (provider, task, model, runtime) key.

Pass the original lookup model (which may be None) into the refresh as
a separate lookup_model argument used only to build the cache key,
while the resolved model is still stored as the entry's usable model
and returned to the caller. Thread task into both acquisition sites and
main_runtime into the async acquisition and both async refresh sites,
so sync and async compute the same cache key on acquire and on refresh.
The stale client is now overwritten in place instead of lingering under
an orphaned key, preserving the per-model cache keying introduced in
on the default config.

Add end-to-end regression tests that drive the real call_llm and
async_call_llm through the real client cache; the existing 401 tests
patch _get_cached_client wholesale and so cannot observe
acquire/refresh key divergence.
2026-09-02 10:55:46 -07:00
Teknium f6234d00c5 fix(security): close GitSpawn RCE class — malicious repo .git/config no longer executes on context gathering (GHSA-7x36-8jrh-v4pw)
Hermes gathers workspace context by running git against the session
directory automatically — the coding-workspace snapshot, gateway
project-tree build, /diff, @diff|@staged context refs, goal-gate
fingerprint, and -w startup worktree add — before any prompt, tool call,
approval, or trust gate. Those probes ran the system git without
stripping the repository's own config, so a repo delivered as files with
its .git directory intact (a shared zip, sync folder, or USB stick;
git clone never transfers .git/config) could set an execution-sink git
setting and get arbitrary host code execution as the user with nothing
on screen.

- core.fsmonitor / core.hooksPath / pager / editor / credential helper:
  neutralized by routing every automatic probe through
  noninteractive_git_env(), which pins those keys to inert values via
  GIT_CONFIG_* and ignores global/system config. bounded_git_probe (the
  reported sink, coding_context._git + tui_gateway.git_probe) now defaults
  to that env; worktree-add, working_diff, web_git, context_references,
  goals, and subagent_worktree route through it too.
- Attribute-scoped [diff "x"] command=/textconv= drivers: the attacker
  names the driver in .gitattributes, so GIT_CONFIG_KEY overrides can't
  enumerate them. Added harden_git_argv(), which inserts
  --no-ext-diff --no-textconv on diff-rendering subcommands (diff/show/
  log/blame) only — status et al reject the flags. Both flags required
  (verified empirically; each alone leaves the other live).

Builds on the noninteractive_git_env config-scrubbing from the
gemini-cli #28792 port. Real-git E2E regression suite arms a malicious
repo and asserts every automatic path neutralizes fsmonitor, hooks,
external-diff, and textconv; a baseline test proves the repo is armed.
2026-09-02 10:33:43 -07:00
Alexander Prendota 1131b22856 feat(providers): let a provider profile supply its own client
``create_openai_client`` was a hardcoded if-ladder: copilot-acp builds an ACP
stdio shim, gemini builds a native client, everything else gets an
``openai.OpenAI``. There was no extension point, so a provider whose wire
protocol is not OpenAI-over-HTTP could only be added by editing this function —
which is exactly why an ACP provider cannot ship outside this tree today, even
though ``providers/__init__.py`` has discovered out-of-tree profiles from
``~/.hermes/plugins/model-providers/`` and pip entry points for a while.

``ProviderProfile.create_client(**client_kwargs)`` closes that gap. It returns
``None`` by default, so every provider that wants the standard client is
unaffected and the existing ladder still runs as the fallback. copilot-acp is
migrated onto it — its hardcoded branch is gone and its profile supplies the
client in three lines, which is the same three lines an external package writes.

Resolution goes by provider name first, then by ``base_url`` prefix, so a
runtime configured only by URL still reaches its profile — matching what the
replaced ``startswith("acp://copilot")`` branch did. A profile that raises is
logged and skipped: a third-party plugin can fail to provide a client, but it
cannot take the turn down.

Also replaces the two ``isinstance`` checks in ``agent/auxiliary_client.py``
that mean "this client is complete, do not wrap it" with capability flags the
client class declares — ``HERMES_SKIP_TRANSPORT_WRAP`` and
``HERMES_SKIP_ASYNC_WRAP``, mirroring ``SUPPORTS_HERMES_TOOL_CALLS`` in
``background_review.py``. Two in-tree consumers (the ACP shim and the Gemini
native client), an out-of-tree client is covered by the same declaration, and
the hot path no longer imports those modules just to type-test.

Co-Authored-By: Junie <junie@jetbrains.com>
2026-09-02 09:57:39 -07:00
Teknium 3312947e14 fix(auth): concurrent Nous 401 recovery adopts a peer's refresh instead of re-rotating the shared grant
N processes sharing one Nous OAuth pool entry hit the hourly expiry
together; each force-refreshed, each rotation invalidated the token a
sibling had just adopted, and processes that lost the auth-store flock
race had their only entry benched ("matched no nous entry ... pool size
0") — ~120 sessions surfaced 401 'out of funds' on Sep 2 2026.

- resolve_nous_runtime_credentials(stale_access_token=): under the store
  lock, skip the refresh POST when the on-disk token differs from the one
  that failed and is usable (a peer already rotated) — adopt instead.
- credential_pool nous path: adopt a peer-rotated key after the pre-sync,
  pass the failed bearer through, and treat a lock TimeoutError as
  'retry later', never as an exhausted credential.
- Live 120-process stampede harness: 41 refreshes/9 unrecovered -> 1
  refresh/0 unrecovered.
2026-09-02 09:33:05 -07:00
unsupportedpastels afc3d9d34c fix(copilot-acp): prefer stable session config for model selection
Use the ACP v1 session config contract advertised by session/new: locate the category=model option and apply the selected value through session/set_config_option. Retain session/set_model only as compatibility fallback for pre-configOptions agents. Reject unknown and policy-disabled values before prompting.

Verified against the installed Copilot ACP server: its model config option advertises the account-authorized choices, session/set_config_option returns the updated state, and live prompts route gpt-5.6-terra to Terra and claude-sonnet-5 to Sonnet 5.
2026-09-02 20:51:07 +05:30
unsupportedpastels a94b68ad40 fix(copilot-acp): stop substituted models impersonating the requested one
Follow-up to the session/set_model wiring, caught in live use: picking an
org-policy-disabled model (claude-fable-5) produced a response claiming to
BE that model while Copilot actually served its default (Claude Sonnet 5).
Two causes:

1. The prompt preamble injected 'Hermes requested model hint: <id>', so
   whatever model actually served the session parroted the requested name
   back as its identity. Remove the line entirely — the model is applied
   for real via session/set_model now, and identity must come from the
   backend, not prompt suggestion.

2. session/new advertises policy-disabled ids alongside enabled ones
   (_meta.copilotEnablement: 'disabled'); selecting one is accepted but
   silently serves the default. Exclude disabled ids from the offered set
   so the degrade-with-warning path handles them.

Verified live: requesting claude-fable-5 logs the does-not-offer warning
listing the 23 genuinely enabled models, serves the default, and the
response truthfully self-identifies as Claude Sonnet 5.
2026-09-02 20:51:07 +05:30
unsupportedpastels 426dab7de0 fix(copilot-acp): apply the picker-selected model via session/set_model
Selecting a model on the copilot-acp provider had no effect: the model id
never left Hermes. _create_chat_completion() dropped the model argument
before _run_prompt(), so the selection survived only as prompt text
('Hermes requested model hint: ...') and Copilot answered with its own
session default — a user picking gpt-5.6-terra visibly got Claude Sonnet 5.

Live-probing 'copilot --acp --stdio' shows the CLI validates but IGNORES
its --model spawn flag in ACP mode, while session/new advertises
models.availableModels and the ACP-native session/set_model call actually
switches the session. Wire that in: forward the model into _run_prompt,
and after session/new send session/set_model when the id is advertised
(or the server reports no list). Unknown ids degrade to the session
default with a warning instead of failing the turn; the provider-level
virtual slug 'copilot-acp' is never forwarded.

Verified live against the real CLI: requesting gpt-5.6-terra answers as
GPT-5.6 Terra and claude-sonnet-5 answers as Claude Sonnet 5.
2026-09-02 20:51:07 +05:30
Teknium 8e4366d358 fix(tools): freeze tools[] across agent-cache eviction; make /reload-mcp the re-probe hatch
Policy: availability-gated tools (check_fn probes — Docker, HASS_TOKEN,
OAuth…) are frozen for the life of a session. tools[] only changes on
/new, /reload-mcp, or compaction. Two doors remained after #100638:

* Gateway agent-cache eviction (LRU/idle sweep/cross-process invalidation)
  rebuilds a fresh AIAgent for the SAME session and agent_init re-derives
  agent.tools from live probes with no predecessor to preserve. Persist
  the session's resolved tool-name order in a new `sessions.tool_names`
  JSON column (declarative reconciliation, SCHEMA_VERSION 28), written
  alongside the system prompt and re-pinned on every published refresh
  (so /reload-mcp and compaction naturally reset it; /new mints a new
  row). On restore-for-existing-session the fresh definitions are folded
  onto the saved order via the SAME `_merge_preserving_prefix` helper —
  a probe-flipped tool is carried forward from the registry schema, a
  deregistered one dropped, new tools appended at the tail.

* /reload-mcp (CLI, gateway, TUI RPC) now also calls
  `reprobe_tool_availability()` — drops the check_fn verdict cache and the
  get_tool_definitions memo — so a user can consciously pick up a
  credential/daemon that appeared mid-session. Docs updated.
2026-09-02 07:22:59 -07:00
joaomarcos 65b0f00002 fix(agent): stop the between-turns tool refresh from forking the cached prefix
The per-turn MCP refresh re-derives `agent.tools` from live availability and
publishes the result wholesale. Two kinds of bytes move as a result:

* a tool whose `check_fn` merely flapped (headless browser probe, expired
  credential, docker blip) disappears from the array, and
* a late-landing MCP tool splices into sorted position, which can be index 0.

Providers that render `tools` ahead of the messages re-prefill the entire
history behind any moved byte, so either case costs a full re-prefill of the
session — the measured 2% cache hit in #100336. The caller's own comment
claimed the refresh "only ever extends a fresh request prefix"; it did not.

`refresh_agent_mcp_tools(..., preserve_prefix=True)` makes that claim true.
The live order becomes authoritative: existing tools keep their slot (fresh
schemas still land), a tool that is still registered but momentarily
unavailable is carried forward, a tool that genuinely left the registry is
still dropped, and new tools are appended at the tail. Explicit `/reload-mcp`
and the compaction boundary keep the plain rebuild.

Refs #100336
2026-09-02 07:22:59 -07:00
Teknium dc7e1b7ab9 fix(webhook): load URL-resolved profile's skills under multiplex
A `/p/<profile>/webhooks/<route>` request resolved the profile from the URL
but ran the route script, prompt render and `skills:` lookup with no
profile scope — the runner only enters `_profile_runtime_scope` later,
around `handle_message` — so routed webhooks loaded the launch (default)
profile's skills and logged "Skill not found" for the routed profile's own.

- gateway/platforms/webhook.py: add `_profile_scope(profile)` (nullcontext
  when no prefix was resolved; `_profile_runtime_scope(get_profile_dir(p))`
  otherwise, same helper the runner uses) and wrap the script / render /
  skill-injection block in it. Bare routes are unchanged.
- agent/skill_commands.py: `scan_skill_commands` scanned the import-time
  `SKILLS_DIR` (frozen to the launch home), so even a correctly scoped call
  listed default's skills; the #88023 home-keyed cache alone could not fix
  that. Use the call-time `_skills_dir()` there and at the two other
  SKILLS_DIR-relative sites in the module.
- agent/skill_utils.py: `normalize_skill_lookup_name` used the same frozen
  root, so a routed profile's absolute skill_dir was rejected by
  `skill_view` ("must be a relative path within the skills directory").
  Resolve against `_skills_dir()` — the root `skill_view` itself enforces.

Fixes #67277

Co-authored-by: Juani Lezcano <tky.juani@gmail.com>
Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
2026-09-02 07:00:13 -07:00
Teknium 98c27aa25c feat(webhooks): stamp the emitting profile on outbound webhook payloads
Under a multiplexed gateway every profile's outbound webhooks share one
delivery worker, so receivers could not tell which profile fired an
event. Add a top-level `profile` field to the payload, resolved at fire
time from the bound Hermes home via get_active_profile_name() ("default"
outside profiles). Documents the field in the wire-format section.

Reported by @vszgdcn8cj-ctrl.

Fixes #92674
2026-09-02 07:00:13 -07:00
chelsealong 600b9c7e7e fix(gateway): scope force-reload hook re-registration to its own home
re_register_config_hooks() cleared the entire process-global idempotence
set on every force-reload, so a profile-local plugin force-reload dropped
another live profile's ledger key without touching its still-registered
callback — the next registration call for that profile then appended a
duplicate. Scope the clear to the reloading profile's own home, and give
outbound webhooks the same force-reload restoration shell hooks already
had, since unload() wipes both from the shared _hooks dict.
2026-09-02 07:00:13 -07:00
chelsealong 29640e3b5a fix(gateway): register secondary profiles' shell hooks and outbound webhooks
Multiplex gateway startup only ever calls agent.shell_hooks/
outbound_webhooks register_from_config() once, against the
root/default profile's config, before any profile scope exists.
_start_one_profile_adapters() discovers Python plugins per profile
but never registered that profile's own declarative `hooks:` block,
so a secondary profile's shell hooks (e.g. a deny-writes gate) and
outbound webhooks silently never fire.

Load and register each profile's own config inside its
_profile_runtime_scope, and key the module-level idempotence sets in
shell_hooks.py/outbound_webhooks.py by resolved Hermes home so two
profiles configuring an identical hook/webhook both register on
their own plugin manager instead of the second being dropped as a
duplicate of the first.

Fixes #92672
2026-09-02 07:00:13 -07:00
Teknium 782dd635fe feat(tts): speech toggles warm/release plugin and command TTS providers
Extends the TTS lease from #100912 (4e3feb8bbb) beyond built-in local
engines: when the configured tts.provider is user-declared, acquiring
the first lease and releasing the last one now reach it, so a
self-hosted TTS server can preload its model when read-aloud / voice
conversation turns on and unload when it turns off (Discord request).

- agent/tts_provider.py: TTSProvider gains concrete no-op warm() /
  release() (not abstract — existing plugins are unaffected).
- tools/tts_tool.py: _signal_user_tts_provider() forwards the lease
  hook; plugin providers get warm()/release(), command providers run
  optional `warm_command` / `release_command` (config.yaml, under
  tts.providers.<name>) through the existing _run_command_tts helper
  on a daemon thread — best-effort, output discarded, failures at
  debug. warm_tts_provider() and release_tts_provider() call it.
- tests/tools/test_tts_lifecycle_leases.py: fake plugin provider and
  fake command provider observe warm/release through acquire/release
  lease (both fail on main with action == "noop").
- docs: features/tts.md — lease section, command-provider optional
  keys table, plugin optional hooks.
2026-09-02 05:35:14 -07:00