Commit Graph

21003 Commits

Author SHA1 Message Date
kshitij a683ef95d2 feat(stt): pre-upload silence trim for cloud providers
Local faster-whisper gets Silero VAD (bf8004e3a) so silence never
reaches the model. Cloud providers got no such protection: the raw
file uploads untouched, so every second of silence in a voice note is
paid for twice — upload time and per-audio-minute billing — and cloud
Whisper hallucinates junk tokens on silent stretches exactly like
local Whisper did before the VAD hardening. A 13s voice note with two
long pauses is billed as 13s of audio to transcribe ~6s of speech.

Close the gap client-side: before uploading to a built-in cloud
provider (groq/openai/mistral/xai/elevenlabs/deepinfra), collapse long
pauses with ffmpeg's silenceremove filter, keeping
stt.cloud_trim_keep_ms (default 300) of every pause so word boundaries
and natural pacing survive. Uses ffmpeg, already a dependency of this
exact path via _transcode_audio_for_stt — no new dependency.

The trim is strictly best-effort — ALL of these upload the original
untouched, transcription never fails because of the trim:
  - stt.cloud_trim_silence: false
  - ffmpeg/ffprobe missing, trim failure, or timeout
  - trimmed result ~empty (mostly-silence clip: the provider, not a
    client-side dB heuristic, decides whether it contains speech)
  - trim saves <10% (re-encoding for nothing)

Command-type and plugin providers are deliberately NOT trimmed: they
may wrap local CLIs that want the original bytes or run their own VAD.

E2E (real ffmpeg + faster-whisper): 13.2s voice note with 7s pause ->
6.2s upload (-53%); transcript of trimmed audio matches the original
on both utterances. Dense-speech and all-silence WAVs correctly fall
back to the original. 22 unit+E2E tests; STT/voice suite failures
identical to upstream/main baseline (all pre-existing).
2026-08-07 19:26:04 +05:30
kshitij 83a1ca6862 Merge pull request #81046 from kshitijk4poor/fix/vision-stream-download-size-cap
fix(vision): stream image and video downloads with chunk-by-chunk size cap
2026-08-07 19:02:25 +05:30
Austin Pickett 5c6aff1430 fix(desktop): keep the chat in front of the terminal in Focus layout (#81019)
* fix(desktop): hide the terminal overlay when its pane is inactive

One xterm is CSS-overlayed onto whichever `<TerminalSlot />` is active,
positioned with `position: fixed` from the slot's bounding rect. Keep-alive
tab layers stay MOUNTED when inactive — hidden with `visibility: hidden` +
`data-pane-hidden`, deliberately preserving their layout box so scroll state
and xterm survive a tab round-trip.

So an inactive terminal slot still reports a full-size rect identical to the
front tab's, and `rect.width > 0 && rect.height > 0` cannot tell the two
apart. The overlay stayed painted at z-4 over whatever tab the user switched
to, swallowing its clicks.

Sample the hidden state alongside the geometry: `Rect` carries `hidden` from
`isElementInHiddenPane(slot)`, `sameRect` compares it so a tab switch wakes
the tracker, the ancestor MutationObserver watches `PANE_HIDDEN_ATTR`, and
the overlay gates on `!rect.hidden`. `TerminalWorkspace` stays mounted
throughout — PTYs are never torn down, only the surface stops painting.

`opacity: 0` rides alongside `visibility: hidden` because Electron can keep
xterm's WebGL canvas composited after an ancestor goes hidden.

Refs #71407

* fix(desktop): collapse an active tool pane onto the workspace, not a neighbour

`setPaneCollapsed` on the ACTIVE pane of a shared zone that holds the
uncloseable workspace handed the active slot to `group.panes[at - 1]` — the
tab to its left, whichever that happened to be.

The workspace can't minimize (it would strand the app), so tab-switching to a
sibling is the right shape; picking a positional neighbour is not. In the
Focus preset the terminal is a tab in the workspace's own group:

    [workspace, files, review, terminal]

Collapsing the active terminal therefore selected `review`. The user asked for
the terminal to go away and landed on a diff pane they never opened — and with
the overlay still painting (before the previous commit), it read as "the
terminal came back".

Hand the slot to the uncloseable pane itself. That pane is the zone's anchor:
it's the one member guaranteed to be a real destination rather than another
tool the user was not asking for. The positional fallback stays for the
defensive case of collapsing the uncloseable pane itself.

This is deliberately broader than one entry point — every route into
`setPaneCollapsed` for a shared zone gets it: the rail, the tab toggle, and
⌃`. Pure tool-only zones are untouched and still fold as a unit.

* fix(desktop): front the workspace when a fresh chat starts

`startFreshSessionDraft` resets the whole view — messages, usage, timers,
route intent, cwd — but left `$terminalTakeover` set. That atom is not a
cosmetic flag: `controller.tsx` binds it as the terminal's toggle store via
`bindToolPaneCollapse`, so while it stays true the terminal keeps the pane
fronted and ⌘N appeared to bounce straight back into the shell.

Clear it, then `revealTreePane('workspace')`. The reveal is not redundant
with the clear: takeover can already be false while the terminal is simply
the active tab (the flag stays true behind a stacked sibling, and tile flows
never touch it), so the state the user sees and the state the flag describes
drift apart. Clearing homes the common case; revealing states the intent
outright — a new chat shows the chat.

The terminal is not torn down. Tool panels collapse to a rail and keep their
PTYs; re-opening finds the same shell.

The `+` / ⌘T tile path needs no takeover clear — it fronts its new tile
through `revealTreePane` and relies on the hidden-pane-aware overlay.

* fix(desktop): reveal the workspace without closing the terminal

The fresh-session commit cleared `$terminalTakeover` on the way to fronting
the workspace. That atom is not a Focus-only fronting flag — it is the
terminal's open/closed state in every layout, and clearing it is wrong twice
over.

Only the Focus preset stacks the terminal with the workspace. Default,
Terminal deck, and Quad each give it a zone of its own, where it sits beside
the chat and obscures nothing — and there ⌘N minimized a terminal the user
had deliberately open.

The flag is also persisted, so the damage outlived the session. On the next
boot the Focus terminal tab is still in the strip and its zone is not
minimized, so clicking it only calls `activateTreePane`; `PersistentTerminal`
mounts its workspace solely while takeover is true, so the tab fronted empty.

`revealTreePane('workspace')` already carries the whole intent. Behind another
tab the terminal is HIDDEN, not closed: it keeps its PTYs, and the overlay
stops painting on the pane-hidden marker from the first commit in this branch
— which is what was actually covering the chat. Removing the clear costs
nothing and keeps the toggle store truthful.

Two regression tests, both verified to fail when the clear is reinstated: a
terminal in its own zone stays open and visible across a fresh chat, and a
Focus terminal tab still mounts after a restart.

Reported by Copilot review on #81019.

---------

Co-authored-by: izumi0uu <izumi0uu@gmail.com>
Co-authored-by: Ritesh Patel <60716910+DECRUX9812@users.noreply.github.com>
2026-08-07 09:23:03 -04:00
kshitij b7eb97a835 fix(vision): stream image and video downloads with chunk-by-chunk size cap
_download_image() and _download_video() both used client.get() +
response.content, buffering the entire media body into memory before
checking the size cap. A server that omits Content-Length could send
an arbitrarily large payload, causing OOM.

Extract _stream_download_to_file() shared helper: streams via
client.stream() + aiter_bytes(), writes chunks to a temp file, enforces
the running byte count against the cap after each chunk, and atomically
replaces onto the destination on success. Cleans up the temp file on
failure. Uses utils.atomic_replace() for cross-device/symlink safety.

Malformed Content-Length values are now caught and ignored instead of
crashing with ValueError; the streaming cap is the authoritative guard.

Approach adapted from PR #10440 by @WuKongAI-CMU (closed as stale —
14923 commits behind, reverted 32 commits of vision_tools.py evolution
including SSRF-safe client, retry classification, and lazy imports).

Closes #10440
2026-08-07 18:50:39 +05:30
Chen Jin 23dce021a5 perf(fts): drain trash tables with a high-water marker instead of re-scanning
_fts_teardown_trash_step deleted rows via 'WHERE key IN (SELECT key
LIMIT N)' — each chunk's subquery re-scanned from the start of the
table, so chunk k skipped past (k-1)xN already-deleted rows: O(n²)
total row visits. On a v22 shadow table with ~230K rows that is on the
order of 10^8 row visits, turning optimize-storage teardown into a
multi-hour grind on slow disks, with a write lock held per chunk.

Single-column INTEGER-PK trash tables now drain via a fts_teardown_<tbl>_progress
high-water marker mirroring fts_rebuild_step: each chunk claims rows
past the marker (SELECT ... WHERE key > ? ORDER BY key LIMIT N), deletes
the claimed range, and publishes the new marker in the same transaction.
Per-chunk work is bounded → O(n) total.

TEXT-PK tables (the FTS config shadow table, pk like 'version') and
compound-key tables fall back to the legacy chunked delete — those are
small by construction.

Fixes #79324
2026-08-07 18:40:47 +05:30
kshitij f3ec2f36f8 perf(slack): parallelize conversations.info lookups with asyncio.gather
The dedup fix in #80679 reduced ~1.6k sequential API calls to ~30,
but those ~30 calls were still sequential. Extract the per-base-ID
resolution into an async helper and run all lookups concurrently
via asyncio.gather. Turns a 6-15s serial block into <1s.
2026-08-07 18:40:33 +05:30
kyssta-exe 025fc7e74b fix(slack): dedupe thread-qualified channel lookups (#80668) 2026-08-07 18:40:33 +05:30
kshitij e5e96e8bb5 fix: harden _await_disconnect_step against outer cancellation + add claim keys
Follow-up to #80700:

1. _await_disconnect_step was missing the try/except CancelledError around
   asyncio.wait() that _await_adapter_cleanup_with_timeout already has.
   When the outer fatal-handler timeout cancels disconnect() mid-step,
   asyncio.wait does NOT cancel its inner task — the task was orphaned
   with no observer. Add the same cancel+detach+re-raise pattern.

2. _queue_retryable_fatal_platform omitted credential_claim/listener_claim
   keys that all 3 startup-path queue sites include. These are consumed by
   the multiplex reservation logic to prevent secondary profiles from
   taking the endpoint while a primary is queued. Pre-existing latent bug
   — now fixed since the extraction makes it trivial.
2026-08-07 18:32:45 +05:30
HexLab98 95e78556f4 test(gateway): cover fatal-handler queue-before-disconnect (#80598) 2026-08-07 18:32:45 +05:30
HexLab98 7141a6dc3a fix(gateway): queue reconnect before fatal disconnect wedges (#80598)
After a network outage the Telegram fatal handler could hang inside
disconnect() and never populate _failed_platforms, so the reconnect
watcher had nothing to retry and the process stayed permanently deaf.
Queue retryable platforms before any disconnect await, bound the fatal
handler with an outer detach deadline, and release the Telegram token
lock / PTB close steps with detach-on-timeout so recovery cannot stall.
2026-08-07 18:32:45 +05:30
kshitij 6d1f9f8ed4 Merge pull request #81035 from kshitijk4poor/chore/author-map-dombejar-toprakeker
chore: add contributor email mappings for dombejar + toprakeker
2026-08-07 18:32:11 +05:30
kshitij dfa0de92c5 chore: add contributor email mappings for dombejar + toprakeker
PR #80588 (gateway workload isolation + active-turn recovery) uses
bare noreply emails that need explicit mappings.
2026-08-07 18:28:35 +05:30
kshitij 2a0d0bc698 refactor(gateway): drop dead degraded token field; de-churn salvage diff
Follow-up to the #80376 salvage:

- TurnLeaseToken.degraded is dead code since acquire() started raising
  TurnLeaseTimeoutError: the only constructor site passes the default and
  repo-wide there are zero external readers. Remove the field, its ctor
  param, the repr segment, and the always-False guards in rebind()/release();
  the class docstring's 'retained for compatibility' claim described
  consumers that do not exist.
- Revert pure black-rewrap churn in test_config_env_bridge_authority.py and
  test_turn_lease.py (hunks on functions this change does not touch), keeping
  the functional Windows-env/encoding additions.
- Turn the default-value test into an invariant: config default must equal
  gateway.turn_lease.DEFAULT_LEASE_WAIT instead of pinning the 1800 literal.
- Rewrap the dangling 'Released' comment line in gateway/run.py.

Verified: 20/20 targeted gateway tests, ruff clean; mutation check — with
gateway/turn_lease.py reverted to pre-fix main the module fails, restored it
passes.
2026-08-07 18:23:57 +05:30
poisdahl 3a3aed3c1f fix(gateway): keep pending turn lease acquires registered 2026-08-07 18:23:57 +05:30
texasich b3e9e91709 fix(gateway): configure turn lease timeout via yaml 2026-08-07 18:23:57 +05:30
texasich b2b681fefd fix(gateway): harden turn-lease timeout rejection 2026-08-07 18:23:57 +05:30
texasich 29af112cd4 fix(gateway): fail closed when session turn lease times out 2026-08-07 18:23:57 +05:30
HexLab98 2ef294f020 docs: spell out the rewind contract on prompt.submit
External hosts speak this protocol directly, so the parameter that
rewrites a session's stored transcript should not be folklore. Document
what each truncation field means, that an ordinal without
confirm_truncate is refused, and that a client must never hold the
ordinal in state across ordinary submits.
2026-08-07 18:20:19 +05:30
HexLab98 c24ff38c51 fix(gateway): make a history-dropping submit prove it meant to
prompt.submit honored truncate_before_user_ordinal on every request. A
client that carried a leftover ordinal into an ordinary send therefore
issued something the gateway could not tell apart from a real rewind —
same method, same shape, an in-range target — and the cut was applied
with replace_messages(), which DELETEs the durable rows. One report lost
244 messages (296 -> 52) with no prompt and nothing to restore from.

The existing guard only covered ordinal 0, where the cut empties the
transcript; a mid-session ordinal sailed straight through. Only the
client knows whether a submit is a rewind, an edit, or a regenerate, so
require it to say so: an ordinal without confirm_truncate is refused on
4029 and neither memory nor the DB is touched. Desktop sends the flag
from the one place that builds these params, so every rewind path is
covered and a stale build fails closed with an actionable error instead
of quietly deleting a conversation.
2026-08-07 18:20:19 +05:30
kshitij 87fd0ed252 Merge pull request #81024 from kshitijk4poor/chore/author-map-texasich
chore: map texasich commit email to GitHub login
2026-08-07 18:14:45 +05:30
kshitij bf2e193a0a fix(cron): review follow-ups for the fail-closed cwd-lock timeout
- Share one HERMES_CRON_TIMEOUT parser (_cron_inactivity_seconds) between
  run_job's inactivity monitor and the cwd-lock bound so the two sites
  cannot drift - the bound must stay >= the inactivity limit or waiters
  would fail while a healthy holder runs.
- Bind the timeout once per run_job; the raise reports the value that was
  actually used for the wait.
- Timeout message covers the writer-blocked-by-readers path instead of
  always blaming a workdir job.
- The finally's TERMINAL_CWD restore is now gated on _cwd_lock_acquired:
  a fail-closed timeout raised before the env-set, so restoring there
  replayed a pre-wait snapshot over the ACTIVE holder's live override
  (pre-existing microsecond race from the #80912 snapshot placement).
- Comment accuracy: the bound is measured from the waiter's arrival; a
  late-wedging or pre-agent-hung holder can outlive it.
2026-08-07 18:11:41 +05:30
kshitij 30679b876c test(cron): pin fail-closed TERMINAL_CWD lock timeout behavior
- reader/writer run_job timeout paths fail loudly (writer additionally
  proven to never mutate the active holder's TERMINAL_CWD override -
  the fail-open design clobbered it)
- waiter whose holder finishes inside the bound still proceeds
- bound derivation from HERMES_CRON_TIMEOUT (floor, margin, 0/garbage)

The run_job fail-fast test shape follows @necoweb3's #63959.

Co-authored-by: dsad <sswdarius@gmail.com>
2026-08-07 18:11:41 +05:30
dsad 5fcca432f5 test(cron): cover bounded TERMINAL_CWD lock acquisition
Lock-primitive timeout tests from #63959, applied onto the timeout API
that landed via #80912.
2026-08-07 18:11:41 +05:30
kshitij 11ce6419c3 fix(cron): fail closed when the TERMINAL_CWD lock times out (#79768)
The 120s bound added in #80912 proceeded WITHOUT the lock on timeout
(fail-open). That degraded mode fires on every overlap with a HEALTHY
long-running workdir job - the write lock is legitimately held for the
holder's entire agent run - and a workdir-less job that proceeds unlocked
executes its shell/file/code commands with the holder's process-global
TERMINAL_CWD override visible: silent wrong-directory execution, the
exact corruption _ReadWriteLock exists to prevent (see
test_reader_never_observes_writer_override). A degraded WRITER was worse:
it clobbered the active holder's override mid-run and later restored a
pre-wait snapshot over the holder's value.

Fail closed instead: on timeout the job errors loudly with an actionable
message (stagger the holder's schedule / drop its workdir) and is retried
on its next tick. A failed job is visible and recoverable; a job that ran
in the wrong directory is neither.

The bound is now derived from the cron inactivity limit
(HERMES_CRON_TIMEOUT, default 600s) + 60s margin instead of a flat 120s:
a wedged holder stops touching its activity clock, so the inactivity
monitor reaps it and releases the lock within that limit - waiters only
fail when even the monitor could not clear the holder. Healthy workdir
jobs shorter than the inactivity limit can no longer fail their waiters.

Design follows @necoweb3's #63959 (fail-closed semantic); its 30s flat
bound would have failed every waiter overlapping a healthy >30s workdir
run, which is why the bound is derived instead.
2026-08-07 18:11:41 +05:30
kshitij 69cf06a824 chore: map texasich commit email to GitHub login
Bare-noreply author email (no NNN+ prefix) on the PR #80376 salvage is not
auto-skipped by check-attribution; add the mapping file ahead of the salvage
PR.
2026-08-07 18:11:32 +05:30
kshitij 458ce7b2b4 fix(streaming): close the same mid-tool-call drop gap on the Anthropic path
Sibling of the chat_completions zero-byte-args fix (previous commits):
a clean SSE close after content_block_start(tool_use) but before any
input_json_delta / message_delta yields an SDK final-message snapshot
whose content is NON-empty (the tool_use block is present, input={})
and whose stop_reason is None. That shape sailed past both
empty-stream guards (they only fire on empty content) and executed the
tool with empty input — no retry, no error: the same silent-data-loss
class as #80498, one provider transport over.

A legitimate completion always carries a stop_reason, so a
tool_use-bearing message without one is a mid-tool-call stream drop.
Raise EmptyStreamError for it, riding the same bounded stream-retry
(HERMES_STREAM_RETRIES) the eventless-stream case already uses.

Gate checked on both return paths (raw SDK snapshot and
accumulator-modified message). Regression tests cover the dropped
shape (mutation-verified: disabling the gate fails exactly that test),
the legitimate tool_use completion, and the text-only no-stop_reason
shape (pre-existing behavior preserved).
2026-08-07 18:07:19 +05:30
joaomarcos e6f31b07cb test(streaming): cover mixed tool-call and retry-exhaustion paths for #80498
Locks in two gaps left by 015a114a2 (#80623): a mixed response where one
tool call completes validly while a sibling has zero argument bytes still
gets discarded whole via the shared partial-stream-stub path, and the
zero-byte trigger now has an end-to-end test through run_conversation's
retry loop, not just at the chat_completion_helpers unit level.
2026-08-07 18:07:19 +05:30
joaomarcos f734578033 fix(streaming): flag empty tool-call args on clean stream end (#80498)
When the stream closes right after a tool call's name arrives but
before any argument bytes are delivered, has_truncated_tool_args
was never set (the existing check required a non-empty, whitespace-
stripped arguments buffer). The call fell through to a normal "stop"
finish_reason, later coerced to "{}" at dispatch and executed
silently with no arguments and no retry.

Route this case through the same dropped-mid-tool-call stub/retry
path already used for partially-truncated JSON.
2026-08-07 18:07:19 +05:30
kshitij 72b7305263 polish: document newline residual, reuse span local, cheap check first
Review follow-ups on the guard: state the accepted \n-residual in the
comment, reuse the span local in the next condition instead of
re-slicing, and short-circuit the substring checks before the regex.
2026-08-07 17:50:03 +05:30
kshitij 9377c5a539 fix(redact): narrow control-split join guard to line-crossing spans
Post-merge review of aecb9ca89 found the join guard over-broad: skipping
the join whenever ANY fragment self-matches _PREFIX_RE reopened a leak
for non-newline splits — sk-<15 chars>ESC<25 chars> masked only the
self-matching head and left the 25-char tail in cleartext (fully masked
before the guard; main never masked this shape at all, so the merged
state was still >= main, but the salvage's own coverage regressed).

Skip the join only when the span crosses a line boundary (\n / \r) —
that is the shape where adjacent legitimate text gets swallowed
(ghp_<token>-then-'button [ref=e3]' annotation bug). ESC/zero-width
controls never legitimately separate a token from prose, so joining
there is safe and restores full-tail masking.

Both legs mutation-checked: reverting to the unconditional skip fails
the new tail-mask test; removing the guard fails the annotation test.
2026-08-07 17:50:03 +05:30
kshitij afb46fdab4 refactor(cron): polish registration partial-failure surfaces
Follow-up to the salvaged registration contract:
- share one _raise_if_cron_registration_error() helper for the two
  byte-identical dashboard 424 except-blocks (web_server + cron router,
  via the existing late() seam)
- add endpoint-level 424 coverage for /api/cron/blueprints/instantiate
  (previously only the sync worker was tested)
- give chat/CLI surfaces a human-facing user_message() (job name, no
  exception class name) and add a recovery hint (pause/resume or update
  re-registers via provider reconcile) to the model/REST message
- consolidate five inline provider test doubles into one ABC-subclassing
  make_cron_provider conftest factory; the web_server test double now
  subclasses CronScheduler so an ABC rename fails loudly
- narrow the wrapper facade to keyword-only (**kwargs) and route the
  tool's partial-failure return through tool_error()
2026-08-07 17:45:06 +05:30
Gille f346458f29 fix(cron): surface initial scheduler registration failures 2026-08-07 17:45:06 +05:30
kshitij 261aef5268 perf(cron): stat-stamp fast path for the shrink-merge; no caller-list mutation
Follow-up hardening on the salvaged #80687 shrink-merge guard, folding in
the best part of the competing #80703 (credit: @JoaoMarcos44):

- Stat-stamp fast path: load_jobs() inside a _jobs_lock() section records
  jobs.json's (mtime_ns, size, ino) BEFORE reading; the save-path merge
  and the post-stage verify skip their full read+parse when the stamp
  still matches. The healthy no-race save (every mark_job_run /
  claim_dispatch / heartbeat / advance_next_runs tick persist) now costs
  one stat() instead of up to two full JSON parses.
- Fail-safe stamp discipline: the stamp is captured pre-read (a sibling
  racing the load leaves it older than disk -> mismatch -> merge runs),
  includes st_ino (mkstemp+rename always allocates a new inode, so
  coarse-mtime filesystems cannot false-match), resets on section
  entry/exit, and is INVALIDATED - never refreshed - after any write in
  the section (a refresh would let a nested create_job be clobbered by
  an outer caller's stale payload; probe-verified both directions).
- _merge_unexpected_disk_jobs no longer mutates the caller's list in
  place - it returns a new list when anything was recovered, and logs the
  recovered ids.
- The tolerant read cascade (utf-8-sig + strict=False fallback) is
  factored into one shared _parse_jobs_file used by both load_jobs and
  _peek_jobs_unlocked, so future encoding/shape fixes land once. The
  peek's repair-free re-entrancy contract is now documented - a repairing
  read on the save path would recurse through _save_jobs_unlocked (the
  exact defect the stamp-reconcile approach in #80703 had).

4 new regression tests (fast path, no-mutation, corrupt-file save,
nested-create-vs-stale-outer-save), each verified to fail against the
implementation it guards.

Co-authored-by: joaomarcos <joaomarcosdias444@gmail.com>
2026-08-07 17:29:49 +05:30
HexLab98 5511ec623b test(cron): cover jobs.json shrink-merge against concurrent creates
Regress the #80624 no_agent watchdog clobber: a stale empty save must
not wipe a concurrent create, while intentional remove/replace still work.
2026-08-07 17:29:49 +05:30
HexLab98 4d84aa2a63 fix(cron): preserve concurrent creates when saving jobs.json
A stale or smaller in-memory snapshot could overwrite jobs.json and drop
CLI/tool-created jobs (including no_agent watchdogs) while the gateway
ticker was running. Merge unexpected on-disk ids back on save unless the
caller marks them in removed_ids.
2026-08-07 17:29:49 +05:30
kshitij aecb9ca894 fix(redact): don't join across controls when a fragment already matches
CI slice 1/12 caught a regression in _mask_control_split_tokens: a
COMPLETE prefix token at end-of-line followed by ordinary text (browser
accessibility annotations: 'ghp_<tok>\nbutton [ref=e3]: Copy') was
joined across the newline into one stripped-copy match, and the mask
swallowed the adjacent line ('button' disappeared).

Join only when no fragment inside the span matches _PREFIX_RE on its
own — a self-matching fragment is already handled by the ordinary
prefix pass, so joining can only cause damage. All smuggling shapes
(ESC/ZWSP/newline splits with under-length fragments) still mask;
regression test added and mutation-checked (fails without the guard).
2026-08-07 17:01:23 +05:30
Soheil Fakour 8969ebac1c fix(secrets): redact command in process checkpoint file (#77484)
_write_checkpoint persisted s.command verbatim to ~/.hermes/processes.json.
Recovery only uses command for display/logging (the process is already
running; adoption re-validates PID + start time, never re-runs the
command), so masking is lossless.
2026-08-07 17:01:23 +05:30
Soheil Fakour e9d1551e65 fix(redact): strip control chars from mask_secret display (#55319, #55321)
A masked secret's visible head/tail could carry control bytes (newline,
NUL, DEL, C1 0x80-0x9F, zero-width) into config/status/dump output.
Strip every control incl. \n/\t (display differs from redact_sensitive_text,
which preserves \n/\t as line structure) before slicing; all-control values
return the configured empty fallback.

Consolidates the previously-closed #58079 approach (strip controls before
masking) - supersedes it.
2026-08-07 17:01:23 +05:30
Soheil Fakour 5444f6853b test(redact): harden new #77484 tests - assert fragments, opaque values (review) 2026-08-07 17:01:23 +05:30
Soheil Fakour 8563fe3435 fix(redact): close emission gaps - env suffix keys, control-char splits, process(list) (#77484) 2026-08-07 17:01:23 +05:30
kshitij 15d7103aa7 fix: harden .env-read detection — review follow-ups for #61352
- Import file_safety._BLOCKED_PROJECT_ENV_BASENAMES instead of copying
  it (comment-enforced parallel lists drift); lookup is now
  case-insensitive to match file_safety's .lower() semantics (cat .ENV
  on macOS/Windows case-insensitive filesystems reads the same secrets).
- Strip shell quotes plain split() leaves attached (cat ".env").
- Drop the dead _ENV_FILE_EXCLUDE_SUFFIXES logic (exact-basename
  membership already excludes templates) and the stray blank-line noise.
- Document the defense-in-depth limits (sudo/full-path/substitution
  readers) mirroring is_env_dump_command's precedent, and correct the
  docstring overclaim about name-independence.
- Annotate command as str | None (tests pass None).
2026-08-07 16:58:25 +05:30
Peter cf755f5c42 fix: redact .env terminal output via detection instead of known-env-var list
Terminal output from file-read commands (cat, head, tail, ...) uses
code_file=True, which skips the generic ENV-assignment redaction pass.
Reading a .env file through the terminal therefore leaked any key whose
value has no recognized vendor prefix (Mistral, Gemini AQ.*, tvly-dev-,
bu_, Spotify client secrets).

Detect file-read commands targeting .env-style basenames (mirroring
agent/file_safety's blocked list) and route them to code_file=False so
the existing ENV pass masks opaque values. Templates (.env.example,
.env.sample, ...) are excluded.

Salvaged from #61352 (145 commits of drift; conflict with the test-prune
wave resolved by NOT resurrecting pruned tests). Authored by @ShaoRou459.

Closes #61352
2026-08-07 16:58:25 +05:30
kshitij 83902620c8 chore: map soheil.fakour@gmail.com -> thatssoheil for attribution 2026-08-07 16:57:59 +05:30
kshitij 1a02e8a793 fix(agent): preserve destroyed tool-call argument bytes in the WARNING log
Review follow-up (W1): the pre-send transcript sanitizer
(agent_runtime_helpers.sanitize_tool_call_arguments) runs on the
PERSISTED messages list before every api_messages build and rewrites any
json.loads-failing argument string to "{}" in the transcript, prepending
a corruption marker to the paired tool result. That in-transcript repair
is deliberate (the stored turn must be replayable next call), but it
destroys the model's original bytes — for a truncated write_file call
those bytes are the user's streamed file content (#80498), and they
previously survived only as an 80-char log preview.

Until a sidecar-preservation design exists, make the bytes recoverable:
both destruction sites (the transcript sanitizer's WARNING and
_repair_tool_call_arguments' unrepairable-path WARNING) now log the full
original argument string bounded at 100KB instead of 80 chars. Corrupted
calls are rare; an oversized WARNING is a fair price for the only copy
of real user content.
2026-08-07 16:57:11 +05:30
kshitij c18e19c3c7 fix(agent): make the send-path copy structural — close the write-through class
The api_messages build used a shallow msg.copy(), decoupling only
top-level fields. Every nested container (tool_calls entries and their
function dicts, multimodal content-part lists, reasoning_details) stayed
aliased to the persisted history, so ANY in-place transform on the send
copy silently rewrote the stored transcript.

Probed every send-path transform against that aliasing shape on main:

  content strip loop                       safe (top-level reassign)
  _canonicalize_api_tool_calls (repair)    LEAKED  <- #80616's fix
  _sanitize_messages_surrogates            LEAKED  (multimodal parts,
                                                    tc ids/args, reasoning)
  _sanitize_messages_non_ascii             LEAKED  (multimodal parts)
  _sanitize_api_messages                   safe
  _drop_thinking_only_and_merge_users      safe

The retry loop already believed the copies were independent - it
sanitizes messages AND api_messages separately (~L3555) - so the
aliasing was accidental everywhere.

Fix at the chokepoint: _clone_message_for_send clones every container
(dict/list) recursively while sharing immutable leaves, so every
downstream in-place transform - current and future - is safe by
construction. Cost is container-count, not string-bytes: 100KB argument
strings and base64 payloads are shared (measured ~0.5ms vs ~0.1ms per
1500-message build; noise next to one json round-trip). Same clone
applied to the prefill-message insert (same class, same pipeline).

The class-wide invariant test runs the full send-path transform
pipeline over an adversarial fixture (malformed args, surrogates,
non-ASCII, multimodal parts, reasoning fields) and asserts the history
stays byte-identical; an AST contract pins the build-site wiring so the
shallow copy can't quietly return. Both mutation-verified: reverting
the clone to shallow fails 4 isolation tests, unwiring the build site
fails the AST contract.

0xGr1mm's branch fix (previous commit) remains as defense in depth at
the exact site the #80498 incident hit; his regression tests and the
class-wide invariant give layered coverage.
2026-08-07 16:57:11 +05:30
kshitij cd152d9daf chore: map ahmetsonersancak@anadolu.edu.tr -> 0xGr1mm for attribution 2026-08-07 16:57:11 +05:30
Gr1mmJ4w e60ca1c6ca fix(agent): stop the send-path repair from rewriting persisted history
`_canonicalize_api_tool_calls` promises copy-on-write in its own docstring
— "the persisted history is untouched" — and the call site repeats it:
"Operates on api_messages (the API copy) so the original conversation
history in `messages` is untouched."

The canonicalize branch keeps that promise (`tc = {**tc, "function": {...}}`).
The repair branch does not:

    except Exception:
        tc["function"]["arguments"] = _repair_tool_call_arguments(...)

`api_messages` is built with `msg.copy()` — a SHALLOW per-message copy — so
every `tool_calls` entry is the same dict object the persisted history
holds. Assigning into `tc["function"]` therefore writes through to the
stored turn. The sibling loop two lines above only touches `am["content"]`,
one level deep, which is why the aliasing never showed up there.

On the unrepairable path `_repair_tool_call_arguments` returns "{}", so
that write replaces the model's real arguments with an empty object in the
transcript. A stream that dies mid `write_file` loses the file content it
had already streamed — the reported symptom in #80498, where a chapter
draft was silently reduced to `{}` and only a WARNING remained:

    Unrepairable tool_call arguments for write_file — replaced with empty
    object (was: {"content": "# 骨架-第25章\n> 承接...)

Mirror the canonicalize branch: build a new tool-call dict instead of
assigning into the shared one. The API copy still carries "{}" — the
repair's whole purpose is to never ship broken JSON — but the history keeps
what the model actually sent, so the transcript, session persistence and
any later retry still have it.

The in-place write was not an oversight in isolation: it predates the memo
refactor, which preserved it deliberately for byte-parity. The existing
`test_history_not_mutated` asserts exactly this invariant but restricts
itself to valid arguments, and its docstring records the gap — "(Malformed
args take the in-place repair path — pre-existing behavior)". That is why
a test file whose header already claims "the persisted history is never
mutated (copy-on-write preserved)" stayed green through the bug.

Four tests close it: history keeps the original bytes, the send copy is
still repaired, a broken call does not disturb its siblings, and repeated
sends stay lossless. On unpatched main three of them fail; the parity and
complexity tests are unaffected because the difference is only observable
when the history list is separate from the send copy — which is the shape
production uses.

Refs #80498

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 16:57:11 +05:30
kshitij 8b6dd27cdb test(auxiliary): update _resolve_auto patch to _resolve_auto_route
The PR renamed _resolve_auto to _resolve_auto_route (3-tuple return).
This test patches the resolver to mock the auto-detection chain, but was
still patching the old _resolve_auto name. The patch never fired, so the
test fell through to the real auto-detection (which finds no providers
in CI's hermetic env). Update patch target to _resolve_auto_route with
the 3-tuple return value.
2026-08-07 16:56:37 +05:30
kshitij c95a1b7171 fix(auxiliary): widen effective provider to relay, logging, and endpoint detection
Thread request_provider through the sibling callsites that the
original PR left on resolved_provider: _set_relay_auxiliary_route
(observability metadata), the 'using X' log line, _is_anthropic_compat_endpoint
(Anthropic image conversion), _provider_requires_stream (streaming
detection), and the initial _relay_sync/async_completion calls.

These are NOT regressions (they passed 'auto' before), but widening
them ensures auto-routed calls to MiniMax get correct image conversion,
streaming-only providers are detected, and observability metadata
records the concrete backend instead of 'auto'.
2026-08-07 16:56:37 +05:30
Gille 293e67328c fix(agent): preserve auto-routed provider identity 2026-08-07 16:56:37 +05:30