write_file confirmed only SIZE (wc -c) after writing — never content.
Models compensated by re-reading files immediately after writing them
(154 verify-reads in a 400k-msg production window), and a corrupted
write (truncated pipe, backend FS oddity) could silently pass.
The write path now compares the on-disk sha256 against the intended
content (one shell call). Three outcomes:
- match -> result carries verified: true; the schema tells the model
an explicit contract: do NOT re-read to check the write landed.
- mismatch -> hard error ('The write did not persist correctly'),
mirroring patch_replace's existing post-write verification.
- backend can't hash (no sha256sum) -> flag omitted, write unaffected.
Hashes the shim-adjusted content (after CRLF/BOM preservation) so
Windows-line-ending and BOM round-trips verify correctly; surrogatepass
encoding matches the rest of the codebase's hashing of model text.
When old_string survives all 9 fuzzy strategies without a match but
the closest candidate line matches after stripping whitespace, the
failure is whitespace-shaped (tabs vs spaces, indent depth). The
did-you-mean hint now appends a two-line diagnosis with leading
whitespace made visible:
Whitespace difference detected (→ = tab, · = space):
file has: →def start(self):
you sent: ····def start(self):
Use the exact whitespace shown in 'file has'.
Pattern ported from crush's diagnoseMismatch (agent-codebase survey) —
it converts the residual dead-end error into a one-turn fix. Only the
leading run is visualized (interior spacing stays readable); content-
shaped misses and raw-exact candidates are unchanged.
Follow-up to the recovery-recipe commit on this branch, per review:
instead of only TELLING the model to re-author the payload via
write_file (2 turns), materialize the blocked command to
~/.hermes/cache/blocked-scripts/blocked-*.sh and point the recovery
at it directly: 'saved to <path> - review it, then run
terminal(command="bash <path>")' (1 turn).
Safety posture is unchanged or better:
- Nothing is executed here; the file is only written.
- The bash <path> follow-up goes through the normal execution
pipeline, including the referenced-script content guard, which
inspects script files named in commands - the payload is MORE
visible to policy than it was inline.
- Genuine hardline blocks (destructive ops) never save anything
(test-asserted).
- Save failures fall back to the previous manual write_file recipe.
- 7-day opportunistic cleanup of saved payloads.
Two block classes from production mining (250k-call window) that
models answered with blind rephrase-retries:
1. Parser-limit / malformed-payload hardline blocks (198x): these fire
on oversized inline payloads (heredocs, giant one-liners), not on a
forbidden operation - but the message read like a permanent ban.
The block now appends: 'RECOVERY: ... write the script to a file
with write_file, then run bash /path/script.sh - do not retry
inline.' Genuine hardline blocks (destructive filesystem
operations) are unchanged.
2. Backgrounding-wrapper blocks (200x): the guidance now spells out
the exact corrected call shape - 're-send WITHOUT the wrapper as
terminal(command="<cmd>", background=true,
notify_on_complete=true)' - instead of describing the feature
abstractly.
process(action='wait') hitting its window returned status='timeout'
with a terse note — models read it as an error and re-issued identical
waits (process is the #1 exact-duplicate tool call in production: 511
dupes in a 400k-msg window; wait is 57% of all process actions).
The timeout result now carries:
- process_running: true — machine-readable 'this is a status, not a
failure'
- an explicit note: 'Wait window of Ns elapsed — the process is still
running. This is not an error. Uptime: Ms.' plus the right next step:
when notify_on_complete is set, 'you will be notified on exit — do
more work instead of waiting again'; otherwise a pointer to
notify_on_complete for next time.
- the clamp note (requested > max) now composes with the status note
instead of replacing it.
Exited/interrupted results are unchanged.
Third zero-match steering tier: when a content search finds nothing in
visible files, probe once with rg --hidden --no-ignore --count-matches.
If the pattern exists only in dotdirs or gitignored files, the result
says so and tells the model to search the hidden path explicitly.
Found live by the benchmark battery: a task with a match inside
.hidden/ returned a bare 0 and the model missed the file entirely in
2/3 baseline runs.
Two dead-turn classes from production mining (state.db, recent window):
1. 13.9% of 19.6k content searches return 0 matches with no steering.
Now a 0-match content search runs one cheap rg -i --count-matches
probe (plus an rg -F probe when the pattern has regex metachars) and
attaches what it found: '0 exact matches, but N case-insensitive
matches — casing may be wrong' / 'N literal matches — metacharacters
need escaping'. True zero-match results stay clean (no noise).
2. 122 'Path not found' failures came from models passing several paths
in ONE path string ('dir1 dir2 dir3', comma lists). Instead of
failing wholesale, split the string, search every path that exists,
merge results, and report skipped parts in a warning. Single-path
misses keep the existing Similar-paths hint; all-missing multi-path
strings still error.
Both probes are bounded (count-only rg, 30s timeout, max 2 invocations)
and wrapped so a probe failure can never break the search result.
Production mining (state.db, 400k-msg window): 60.2% of 104k terminal
calls carry a defensive 'cd X && ' prefix (~925k tokens of pure prefix)
and 2,462 failed calls led with cd — the model cannot see cwd state, so
it re-asserts it on every call and runs pwd/ls diagnostics after
directory changes.
The result dict now includes a 'cwd' field whenever the session cwd
after the command differs from the cwd it started in (cd, pushd,
chained cd). Stable-cwd commands are unchanged (no field, no noise).
Per-command workdir overrides stay transient by contract and never
echo. Schema note added so models learn to trust session cwd instead
of prefixing. Pattern borrowed from crush's <cwd> injection.
realpath comparison avoids false echoes through symlinks; the echo is
wrapped defensively so a backend without .cwd can never break the
result path.
The #1 patch failure class in production (state.db mining, 250k-window)
is a re-send of an edit that already landed: 'old_string and new_string
are identical' (299 occurrences) plus a share of hunk-not-found errors
where the new text is already in the file. These errored, sending
models into re-read/re-patch loops.
New tools/fuzzy_match.is_already_applied(content, old, new) — a
conservative check requiring (1) non-trivial new_string (>=8 chars),
(2) EXACT presence of new_string, (3) old_string gone (unless
identical). Wired into three sites:
- patch_replace (replace mode): returns success + no_change: true +
an explicit note instead of the identical-strings / no-match error.
- V4A validation phase: an already-applied hunk validates as a no-op
so multi-hunk patches no longer fail wholesale when one hunk landed
in a prior call.
- V4A apply phase: mirrors the same skip so the two phases agree.
Genuine no-matches (new text absent) and half-applied renames (old
text still present) keep their error behavior — covered by tests.
Production mining (state.db, 28.5k read_file calls in the recent
window) shows 74.3% of reads truncated — nearly all by the 500-line
default, not the char budget (22,443 line-limit vs 13 char-budget
truncations). That churn produced 12,229 redundant re-reads, and after
a truncated result the most common next move was fleeing to terminal
cat/sed (4,608 times) — the pagination contract was not trusted.
Median truncated file is 2,422 total lines, so a 2000-line default
makes 44% of today's truncated first-reads complete in one call while
the unchanged ~100K-char budget still caps worst-case result size
(same ceiling as before: 500 lines x 2000-char line cap = the same
100K). Schema max was already 2000.
Touchpoints: DEFAULT_READ_LIMIT + both read_file signatures
(file_operations.py), read_file_tool + schema text (file_tools.py),
execute_code sandbox stub docs (code_execution_tool.py), 3 tests
pinning the old default.
When a command exits non-zero, scan the first 4KB of output for
well-known failure shapes and attach one short, actionable recovery
hint to the tool result ('hint' field):
- gh 'Unknown JSON field' (9.2k occurrences in a 250k-call window)
- git merge conflicts (1.2k) — stop verbatim retries
- command not found (1.0k), incl. python->python3 and pip->pip3
- ModuleNotFoundError (739) — venv activation guidance
- 'already exists' (633), gh rate limits (133), permission denied
- exit-code-only tier: 124 timeout, 126 not-executable, 137 SIGKILL
Hints are suppressed when the existing exit_code_meaning tier already
explains the code (grep=1 etc). Pattern order = production frequency
from state.db mining; first match wins; pure function, no I/O.
Replace the process-global HERMES_CRON_SESSION env var with a per-session
ContextVar so a cron tick in the gateway process cannot leak into unrelated
live gateway/API/TUI turns. The cron scheduler now sets the ContextVar
inside the job's try/finally scope and resets it on cleanup. Gateway, API
server, ACP adapter, and TUI gateway all pass cron_session='' to explicitly
mark their sessions as non-cron, masking any stale process env.
Co-authored-by: hinablue <hinablue@gmail.com>
Closes#37968
Simplify-pass advisory on the #25387 salvage: _read_tracker_lock is one
global lock guarding every task's read/search bookkeeping (15 sites). A
hung stat on a dead network mount inside it would stall all of them.
Check-then-recheck matches the dedup mtime pattern 30 lines below: read
the entry under the lock, stat outside, reacquire only to evict.
Review follow-ups on the #25387 salvage:
1. CRITICAL: a cached miss survived out-of-band file creation (terminal
command, external process) for the full 60s TTL — breaking the common
agent pattern 'check for file -> create it -> read it' (live-repro'd).
Serve-side existence guard: one ~free stat before serving a cached
miss; if the path now exists the entry is evicted and the real read
runs. Also fixes the search-root variant (write under a cached-missing
directory). Both mutation-checked.
2. notify_other_tool_call now clears the task's not_found entries too
(belt: the dispatcher calls it for every non-read tool).
3. Tracking parity: the record sites no longer early-return. On upstream,
error results flow through consecutive-loop detection and dedup
bookkeeping; short-circuiting skipped that and broke
TestDedupInvalidationTaskResolution when preceded by
TestSilentFileMisplacementE2E (bisected: the early return at the
read record site was the trigger). Recording is now
side-effect-identical to upstream; serving from the cache remains the
optimization. Also reuse the already-computed _resolved instead of
resolving a second time.
When read_file or search hits a non-existent path, ShellFileOperations
spawns a subprocess to stat the path and another to walk the parent
directory for "did you mean..." suggestions. A typo'd path retried 13
times (observed in the wild) costs 26 subprocess invocations + 13 ls
walks for a result we already know.
Add a per-task negative-result cache keyed by (op, resolved_path) with
a 60s TTL and a hard cap of 500 entries. On hit, return the cached
error JSON immediately and skip the subprocess + suggestion walk.
The cache is namespaced by operation ("read" vs "search") because the
two callers return different error JSON shapes ("File not found:" vs
"Path not found:"). Eviction:
* TTL (60s) — short, so a path that appears later isn't masked.
* write_file / patch on the same path — _invalidate_dedup_for_path
now also drops the negative-cache entry so a freshly-written file
is read from disk on the next call instead of returning a stale
"not found" stub.
Tests in tests/tools/test_file_tools.py cover:
* read cache hit skips the subprocess on retry
* cache is per-task (no cross-task pollution)
* successful reads do not poison the cache
* search cache hit skips the subprocess on retry
* read and search caches are namespaced (different error shapes)
* write_file invalidates the read negative cache
* TTL expiry evicts stale entries
NOTION_API_KEY/LINEAR_API_KEY defaults, DEEPINFRA_API_KEY model gate, FAL_KEY via the file's own _scoped_credential helper, XAI/VERCEL/DAYTONA presence checks, GITHUB_TOKEN/GH_TOKEN + GitHub App creds in tirith/skills_hub, and the masked HERMES_API_KEY display in tui_gateway config.show all route through get_secret.
Review follow-up: the adapter-level resolver alone left three paths
reading per-profile QQ_* values from raw os.getenv, so a secondary
multiplex profile's scoped opt-in or credentials were ignored (or the
primary's environ values leaked in):
- gateway/authz_mixin.py: route the per-platform allow-all flag and the
per-platform/group allowlist + allow-bots reads through the
scope-aware gateway.config._getenv. Deployment-global GATEWAY_* reads
intentionally stay on os.getenv. This makes the same fix effective
for every own-policy platform, not just QQ; unscoped behavior is
byte-identical to os.getenv.
- gateway/run.py (_own_policy_open_startup_violation): resolve the
per-platform dm/group policy and allow-all opt-in via _getenv; the
secondary-profile caller already runs inside _profile_runtime_scope.
- tools/send_message_tool.py (_send_qqbot): the QQ_APP_ID /
QQ_CLIENT_SECRET fallbacks now honor the active profile scope.
Tests: tests/gateway/test_qqbot_scope_paths.py covers all three paths
end-to-end (scope wins, no environ inheritance for non-opted profiles,
single-profile environ fallback unchanged); the STT suite now asserts
QQ_STT_BASE_URL and QQ_STT_MODEL scoping alongside the API key. All
five scoped-behavior tests fail on the previous commit and pass here.
Salvaged from #56982 (@rayjun): the live piece of the PR. The
hermes_cli/config.py get_env_value scope-honoring change and its
test_env_load_cache.py tests were already merged via ed1170cd8b
(#76462) and are dropped here.
tools/xai_http.py::get_env_value wrapped the scope-aware
hermes_cli.config.get_env_value in except Exception + a raw os.environ
fallback — swallowing UnscopedSecretError and borrowing the process
env, so a multiplexed xAI credential read could silently pick up
another profile's XAI_API_KEY. Narrow the except to ImportError (the
only legitimate degraded case) so get_secret's verdict propagates: an
unscoped multiplexed read fails closed, and a scoped miss returns the
default instead of the foreign environ value.
Co-authored-by: rayjun <rayjun0412@gmail.com>
Makes interrupt-protected context compression cancellable by an explicit
user or lifecycle stop, without weakening protection against ordinary
incoming messages, voice interjections, or active-turn redirects.
Separates explicit hard cancellation from ordinary interrupt/redirect
state with a dedicated threading.Event; introduces
AuxiliaryExplicitCancellation as an attempt-local frozen-cause signal;
isolates the synchronous provider callback in a bounded daemon worker
during protected compression; atomically linearizes Codex timeout
cleanup against explicit cancellation; propagates hard cancellation
through child agents and explicit stop surfaces; serializes hard-cancel
admission against compression commit admission with
CompressionCommitFence; aborts before session rotation or late DB commit,
restores in-place transcript mutations and compressor state, and releases
the heartbeat and compression lease.
Based on #74449 by @suparious. Resolved merge conflicts in
agent/context_compressor.py (feasibility check + try/except) and
tui_gateway/methods_session.py.
Review follow-up on the #76194 salvage: the readonly swap makes this
function leak the live config-cache 'approvals' sub-dict to callers.
All current callers are read-only (audited); the docstring now carries
the do-not-mutate contract for future ones.
The terminal-command guard path loaded config 2-3x per invocation via
load_config(), which pays a defensive deepcopy of the entire config on
every call (~356us of the ~376us warm-cache cost measured on a real
config.yaml). All six swapped call sites were audited read-only — every
caller takes scalar reads or iterates the returned structures; none
mutate (the save path at save_permanent_allowlist keeps load_config) —
so they now use load_config_readonly(), the API built for exactly this
(precedent: #74211, #74322; the one unsafe-site lesson from #56085's
salvage is covered by the mutation audit and a cache-integrity test).
Measured (real config.yaml, warm cache): load_config 376.0us ->
load_config_readonly 19.9us (18.9x); full guard pass
check_all_command_guards('ls -la','local') 930.7us -> 241.8us (3.85x).
Tests: new test_approval_config_readonly.py drives the real functions
against a temp HERMES_HOME — readonly call counts per function, a
no-deepcopy pin for the full guard pass, and cache-identity/integrity
checks. Existing test mocks retargeted from load_config to
load_config_readonly (same injection intent). Note: 6
test_approval_mode_parity failures are pre-existing ordering flakes —
identical with the change stashed on clean main.
kill_started_since duplicated kill_all's collect-under-lock/kill-outside-lock
loop line for line; it is now a thin delegate through new kill_all kwargs
(exclude_ids, source, consume_output). Public signatures unchanged — existing
callers and test monkeypatch seams keep working. kill_process's docstring now
names the deliberate consume_output=True exception for abandoned-turn reaping
so the deviation isn't 'fixed' later.
An agent turn can spawn a long-running background subprocess (e.g.
`next build`) and later be abandoned via inactivity timeout, /stop,
/new, or a client disconnect. Before this fix the gateway interrupted
the agent loop but never touched the subprocess: it kept running
inside the gateway's cgroup, unbounded, until memory pressure starved
the event loop and made every platform/cron look hung (#76115).
The process registry already knew how to kill a process tree — the
missing piece was per-turn ownership: nothing distinguished a process
that predates the turn (must survive), a process the turn started and
finished successfully (must survive), and a process an abandoned turn
left running (must be reaped).
- tools/process_registry.py: snapshot_running_ids() captures a turn's
starting baseline; kill_started_since() reaps only IDs created after
it, scoped to one task_id.
- gateway/turn_context.py: TurnContext carries process_task_id +
process_baseline so the timeout/interrupt paths can reach them.
- gateway/run.py: baseline is snapshotted right before the turn's
executor task starts; the inactivity-timeout path and the explicit
/stop|/new|disconnect interrupt path both reap via the same helper.
A daemon-thread watchdog backs up the asyncio-based timeout poll,
since a starved event loop is exactly the failure mode this bug
causes. The turn's own worker clears its ownership markers the
instant it finishes, closing a race where a /stop landing right
after normal completion could reap a background process the turn
deliberately left running.
Related but insufficient on their own: #37454 (cgroup ExecStopPost
reaper only fires on service restart) and #68915 (orphaned-pipe
grandchild detection, a registry bug not a turn-lifecycle gap).
Neither ties process cleanup to turn abandonment.
- logger.warning when the 6h ceiling stops the heartbeat (matches the
delegate_task stale-stop precedent) so the eventual watchdog reap is
explainable from logs instead of silent
- new mutation-checked test: past the ceiling the heartbeat stops while
the job still completes
- clearer assertion messages (surface res on failure)
- heartbeat loop continues past a raising activity callback instead of
silently stopping (matches delegate_task / touch_activity_if_due
swallow-and-continue semantics) — one transient error must not drop
watchdog protection for the rest of a long job
- hard 6h elapsed ceiling so a wedged job under HERMES_CRON_TIMEOUT=0
(unlimited child watchdog) cannot mask the gateway watchdog forever
- public get_activity_callback() accessor in tools/environments/base.py
instead of importing the private _get_activity_callback cross-module
- tests: deterministic heartbeat test (event-gated, no timing sleep),
no-callback test now asserts the thread is truly never created, new
exception-survival guard; dead started event removed
- fix comment: delegate_task heartbeat cadence is 30s, not 10s
/simplify-code findings on the salvage stack:
- extract _category_skill_dirs() as the single category detector; the
install guard and hermes_cli._existing_categories() now share it
(third copy of the heuristic eliminated)
- filter rglob hits through is_excluded_skill_path so vendored /
support-dir SKILL.md files (node_modules, references/pkg) no longer
misclassify a plain directory as a category and block install
- fix inaccurate WHY comment (lock-file check only guards hub-installed
skills, not hand-authored dirs), drop underscore prefixes on locals,
fold the file-collision guard under the single exists() check
Follow-up to the salvaged #76000 guard:
- refuse installing a skill INTO an existing skill directory (hybrid
skill-plus-category dirs whose later update/uninstall rmtree would
destroy the nested skill — sibling case of #75983)
- refuse a stray regular file at the install path with the caller's
ValueError contract instead of an uncaught NotADirectoryError
- regression tests: nested-only category (skills at depth >= 2),
category-inside-skill, file collision
Replaces the synchronous per-sentence streamer.stream() loop in
stream_tts_to_speaker with a per-sentence prefetch pipeline. Each
sentence gets its own background thread that fires the HTTP request
immediately, buffering PCM chunks into a per-segment queue (capped at
3 concurrent via semaphore). A single playback worker drains segments
in FIFO order with PortAudio error recovery (reinit up to 3 attempts,
then temp-file fallback). Also fixes PCM chunk alignment for odd-byte
HTTP chunks and increases the worker join timeout from 30s to 300s.
Salvage of PR #71084 onto current main.
Closes#71084
After cleaning up an expired cloud browser session, re-check under
_cleanup_lock whether another thread has already created a replacement.
If so, return the live replacement instead of falling through to create
yet another session (which would be orphaned, or worse, a second
concurrent cleanup could destroy the replacement).
Follow-up to helix4u's expired-session renewal fix in PR #76356.
Hermes installs runtimes for itself — `uv` at `$HERMES_HOME/bin/uv`, Node
at `$HERMES_HOME/node` — and neither directory is on an arbitrary
process's PATH. Every `shutil.which("node"/"npm"/"npx"/"uv")` in Hermes's
own code therefore has two failure modes: the managed runtime is invisible,
so the caller reports "not installed" or degrades to a slower tier on a
machine that has exactly what it needed; and when a system copy also
exists, the one Hermes does not own wins.
Routed the Hermes-owned call sites through managed-aware resolvers:
- `agent/lsp/install.py`, `hermes_cli/dep_ensure.py`, `hermes_cli/main.py`
(`_make_tui_argv`), `hermes_cli/tools_config.py` (`_run_post_setup`) now
use `find_node_executable()`.
- `hermes_cli/tools_config.py::_pip_install` and `hermes_cli/setup.py`'s
vercel install use `ensure_uv()` (installing uv is in scope during setup,
and the Windows installer's `uv venv` does not seed pip, so the fallback
tier is "No module named pip"). `tools/lazy_deps.py` uses `resolve_uv()`
— a lookup, not a bootstrap, because it runs mid-turn for an optional
dependency and downloading a runtime as a side effect exceeds what the
caller asked for.
- `hermes_cli/gateway.py`: extracted `_append_node_dir_for_service()`,
shared by the systemd unit and launchd plist generators, which appends
the managed dirs before the PATH-resolved one. A service definition is
written once and survives reboots, so resolving a system Node that
happens to lead the installing shell's PATH bakes the wrong interpreter
in permanently. Managed dirs are profile-scoped, so each profile's unit
still names its own Node; the existing symlink-parent rule (don't
`.resolve()`) is preserved verbatim.
- `tools/environments/local.py`: the terminal tool's subshell PATH gains
the managed dirs, appended alongside the sane entries rather than
prepended — a tool the user deliberately put on their own PATH still
wins, and the managed one only fills a gap. This is also what makes the
bare `which("uv")` in `tools/env_probe.py` correct: that probe reports
the environment the *model* sees, and the model can only run what is on
that subshell's PATH.
`scripts/install.ps1`: the persisted User PATH update becomes
`Set-ManagedNodeFirstOnUserPath`, a move-to-front rather than an
add-if-missing. Installs made by an older install.ps1 already have the
managed dir in User PATH — at the tail, behind a system Node — and an
add-if-missing check sees it present and leaves that ordering in place
forever, so the users the bug hurt would never be repaired. Unrelated
entries keep their relative order (empty segments included; a trailing
`;` is legal and the installer's other PATH code preserves them),
duplicates collapse, and it writes only when the string actually changes.
Tests:
- `tests/test_managed_runtime_resolution.py` — AST guard that fails any
new bare `which()` for a managed runtime, with a short justified
allow-list and a companion test that fails when an allow-list entry goes
stale. Reading source is banned by AGENTS.md and this is the documented
exception: the property is "no call site anywhere spells it this way",
which no runtime seam can observe.
- `scripts/ci/test_install_ps1_path_migration.ps1` — behavioral, not a
source regex: it lifts the real `Set-ManagedNodeFirstOnUserPath` out of
install.ps1's AST and rewrites only the two registry calls into an
in-memory store, so the shipped split/dedupe/prepend/change-detection
logic executes for real. Not in the default lane (Linux runners have no
PowerShell host); runs under `pwsh`. 13/13 assertions pass.
Three lower-severity core-tool robustness fixes from a targeted audit, each
reproduced live:
1. terminal_tool did not validate non-positive timeouts. 'timeout or default'
silently coerced 0 to the config default (0 can't mean 'no timeout'), and a
negative value is truthy so it flowed into 'deadline = now + timeout' and
fired an immediate '-Ns' timeout. Reject timeout <= 0 with a clear message.
2. fuzzy_find_and_replace accepted a whitespace-only old_string, which matches
trivially (blank line / run of spaces) and mass-replaces under replace_all
or raises an opaque ambiguity error. Reject it alongside the empty check.
3. The '/private/var/' sensitive-path prefix over-blocked ALL macOS temp-file
writes: , /tmp, and /var/folders realpath into /private/var/folders
on macOS (and paths are resolved through symlinks), and /private/var/tmp is
a normal temp dir. Narrowed to the genuinely-sensitive subtrees
(/private/var/db, /private/var/root); /etc and /private/etc stay blocked.
All verified with sabotage-checked regression tests. 85 terminal/fuzzy/file
tests pass; normal timeouts, legit replacements, and /var + /boot + /etc
blocking are unaffected.
Two V4A parse/validate bugs found in a core-tools audit, reproduced live:
1. CRLF patch body injected stray carriage returns. parse_v4a_patch split
on '\n' only, so a CRLF-encoded patch kept '\r' inside every HunkLine
and wrote mixed line endings into an LF file; the anchored Begin/End
markers could also fail to match because of the trailing '\r'. Strip a
trailing '\r' from each line at split time.
2. Move-then-Update of the same file was rejected. _validate_operations read
the UPDATE target from disk before the MOVE ran, so 'Move a->b' + 'Update
b' failed validation with 'b: file not found'. Added a small pending-move
overlay so UPDATE/DELETE/MOVE reads during validation see prior ops'
effects (moved-in destinations resolve, moved-away sources read as gone),
while a genuine 'destination already exists' conflict is still caught.
Both verified with sabotage-checked regression tests. 113 patch/fuzzy/file
tests pass.
Strategy 9 (context_aware, the last-resort fuzzy strategy used by
patch_replace, V4A UPDATE hunks, and skill_manage) had two serious flaws,
both reproduced live against current main:
1. CORRECTNESS: it accepted a block when >=50% of its lines were >=0.80
similar. A 2-line pattern with one real line and one garbage line matched,
silently deleting the non-matching line and persisting a wrong edit as
success. Now requires the first AND last lines to anchor-match and EVERY
non-blank pattern line to be >=0.80 similar — one garbage line disqualifies
the block.
2. PERFORMANCE: it scored every content window with per-line SequenceMatcher,
so every failed match paid O(file_lines x pattern_lines) — measured ~5.5s
for a single 40-line no-match on a 10k-line file, per hunk. The first/last
line anchor pre-filter skips non-candidate windows: same case now ~160ms
(34x faster).
Also gate replace_all: a similarity-based strategy (block_anchor,
context_aware) with multiple matches under replace_all would overwrite every
approximate block, not just exact ones. Now refused with a clear error
directing the caller to precise text.
All verified with sabotage-checked regression tests (fail against the old
50% logic). 158 file/patch/fuzzy tests pass; legit fuzzy edits (indent drift,
unique near-match) unaffected.
Two DATA-LOSS bugs in ShellFileOperations found in a core-tools audit,
each reproduced live against current main:
1. Non-UTF-8 file content silently corrupted on read->write. The terminal
env decodes stdout with errors='replace', so a latin-1/8859 file's bytes
arrive as U+FFFD before _is_likely_binary inspects them. U+FFFD is
'printable', so the >30%-non-printable check never flagged it, and the
agent would read the mojibake and write it back, permanently replacing the
original bytes. Fix: treat a sample containing U+FFFD as binary (read-only).
2. Writing through a symlink destroyed the link and orphaned the target. The
atomic temp-file + 'mv -f' swap replaced the symlink itself with a plain
file; the real target was never updated. Fix: resolve the link with
readlink -f/realpath first and recompute the temp dir from the resolved
target so the mv stays same-filesystem atomic. Broken links fall back to
the original path (no regression).
Both verified with sabotage-checked regression tests (fail without the fix).
Proper UTF-8 text (incl. non-ASCII) and plain-file writes are unaffected.
Two independent HIGH-severity correctness bugs found in a core-tools audit,
each reproduced live against current main:
1. Read-dedup was never evicted after a write on non-default tasks.
_invalidate_dedup_for_path looked up the read-tracker under the correct
task_id but resolved the path with _resolve_path(filepath) — which
DEFAULTS task_id='default'. The dedup cache is keyed by the task-resolved
absolute path, so for any task whose workspace cwd differs from the process
cwd (every -w worktree / Desktop / ACP session using relative paths) the
computed key never matched and the stale entry was never removed. A
read_file after a write_file/patch could then return the OLD content stub
when mtime coincided. Fix: pass task_id through.
2. A per-command workdir override permanently hijacked the session cwd.
The post-command dual-write unconditionally recorded env.cwd (stamped to
the transient workdir) into the durable session-cwd store, so every later
command that omitted workdir inherited the one-off directory — contradicting
the documented 'Working directory for this command' contract. Fix: skip the
session-cwd record when workdir was explicitly supplied.
Both verified with sabotage-checked regression tests (fail without the fix).
patch_tool resolved V4A header paths against the task workspace for
locking, staleness, and reporting, but handed the original (often
relative) patch text to file_ops.patch_v4a — which re-resolved headers
against the backend env's own cwd. When the two diverge (the git-worktree
cwd bug), a relative header landed in a different directory than
everything the tool locked and reported: a silent wrong-file write.
Rewrite Update/Add/Delete/Move File headers to the resolved absolute
paths before apply, only for host-filesystem backends (container/remote
namespaces keep their own paths). Header patterns mirror patch_parser
(no-space ***Update File: form) and cover Move File: src -> dst.
Salvage of #53176 by @necoweb3, reimplemented onto current main (the
original branch predates the sensitive-path/Move-header extraction and
per-path locking now in patch_tool).
Co-authored-by: necoweb3 <sswdarius@gmail.com>
The boundary scan in parse_v4a_patch used substring matching, so a
content line mentioning "*** End Patch" (docs about the patch format,
nested patch text) truncated the patch, and "*** Begin Patch" in
content reset the start boundary — silently dropping already-parsed
operations while reporting success. Match only whole-line markers at
column 0, preserving the no-space "***Begin Patch" tolerance.
Teknium