Commit Graph

2746 Commits

Author SHA1 Message Date
kshitij d81f2f49ea refactor(compression): fold simplify findings — dedup floor constant, drift-guard test
- Wire the Pass-1 dedup floor (len < 200) to the shared _PRUNE_MIN_CHARS
  constant it was already documented as matching, and use the constant in
  the remaining test literal.
- Restructure the clarify 'resolved' computation (is_answer_shaped +
  sentinel check) instead of compute-then-flip.
- Add a live producer->recognizer drift guard: the REAL oneshot no-user
  callback's output must be recognized as a sentinel, so producer wording
  drift fails a test instead of silently reintroducing false attribution.
- Document the any()-poisoning semantic for multi-select sentinel lists.
2026-08-08 14:27:31 +05:30
kshitij 39056e8de4 fix(compression): filter clarify non-response sentinels; share prune floor constant
Follow-up to the salvaged #81244 commits:

- Timeout/no-user clarify callbacks (CLI timeout, gateway timeout and
  delivery failure, oneshot no-user) embed sentinel prose as
  user_response; quoting those as '[clarify] user responded: ...' would
  be false attribution. Route them to the generic summary path.
- Extract the shared _PRUNE_MIN_CHARS = 200 floor (prune default +
  proactive clamp) and cap the clarify summary at _PRUNE_MIN_CHARS - 1,
  removing the knife-edge equality the summary's survival depended on
  and keeping it out of the >=200-char dedup pass.
- Tests: 4 sentinel shapes + multi-select sentinel; mutation-checked.
2026-08-08 14:27:31 +05:30
Crypto Intern 3090e9e871 fix: reject forged clarify summaries 2026-08-08 14:27:31 +05:30
Crypto Intern 6433d5723f fix(compression): make clarify summaries UTF-8 safe 2026-08-08 14:27:31 +05:30
Crypto Intern d6511aecb6 fix(compression): preserve clarify responses 2026-08-08 14:27:31 +05:30
kshitij 98e96e1a60 refactor(agent): drop the run_agent classify_persistence_error delegating wrapper
Post-merge simplify finding on #81613: the wrapper's docstring claimed it
existed for 'existing callers', but every caller was introduced by the same
PR - there was never a pre-existing import path to preserve. All callers
(conversation_loop, tool_executor, run_agent's own flush handler, tests)
now import the canonical hermes_state.classify_persistence_error directly,
matching how is_disk_full_error is consumed. No behavior change; imports
stay lazy inside the exception handlers.
2026-08-08 14:25:12 +05:30
PRATHAMESH75 aed114a69b fix(agent): treat max-iteration nudge as synthetic during compaction
handle_max_iterations() appends its runtime summary request as a plain
role="user" row, which SessionDB persists verbatim. On later compaction the
synthetic-turn filters only recognized compaction summaries, continuation
rows, and todo snapshots, so the nudge could be selected as the latest
actionable user turn — becoming the task snapshot / auto-focus input and
getting summarized as "User asked: ...", demoting the real human task.

Metadata flags do not survive SessionDB projection (the reason the existing
markers are content-based), so recognition must key off stable content.
Extract the nudge into a shared MAX_ITERATIONS_SUMMARY_REQUEST constant and
teach _is_synthetic_compression_user_turn() to recognize it, mirroring the
continuation/todo markers. Every _is_actionable_user_turn call site already
pairs the synthetic guard, so the single recognizer change covers anchor
selection, auto-focus, and real-user-turn detection.

Fixes #78580
2026-08-08 14:24:39 +05:30
Victor Kyriazakos 2a9f5b3476 fix(agent): classify session-persistence failures so lock contention is not misdiagnosed as disk-full
An enterprise deployment hit sustained SQLite write-lock contention on a
shared multi-gigabyte state.db (gateway + CLI processes writing
concurrently). Turns correctly failed closed with
session_persistence_failed, but the only user-facing wording claimed the
disk was full and the gateway rendered a generic failure.

The fast-fail semantics are deliberate and unchanged. This adds a pure
classifier (locked / disk / unknown) applied where the SQLite error is
still visible, threads the cause through the turn-completion explainer,
and stamps a machine-readable failure_reason
(session_persistence_failed:<cause>) plus a guaranteed non-empty error on
the result for downstream surfaces. The cron scheduler's explainer-text
suppression now matches every cause variant so refined wording cannot
leak into scheduled-job deliveries.
2026-08-08 14:18:26 +05:30
Ken Weiner 367dda813c fix: print transform_llm_output appended content after CLI streaming
When the CLI streams a response token-by-token, it marks the response as
already displayed and skips re-printing after the tool loop. This means any
content appended by a transform_llm_output plugin fires after streaming — the
appended text is in the final response and stored in history, but never shown
to the user.

Fix by tracking the pre-transform response in finalize_turn() and including it
in the result dict as pre_transform_response. The CLI then checks whether the
response was transformed and, if so, prints only the appended suffix.

Previously the already_streamed branch was a no-op pass. Now it detects
post-stream plugin additions and outputs them without re-printing the streamed
body.
2026-08-08 14:06:30 +05:30
kshitij a4af262638 fix: revert except ImportError back to except Exception
load_config_readonly() can raise FileNotFoundError (deleted profile),
RuntimeError (managed mode), and PermissionError before any inner
try/except protects the caller. The except ImportError narrowing would
crash build_system_prompt_parts() — aborting agent startup — instead of
falling back to the base Telegram hint. The isinstance guards from the
second commit handle the TypeError case; the broad except Exception
handles the remaining failure surface.
2026-08-08 13:57:44 +05:30
bex 95520b812f fix(agent): fail open on malformed telegram extra config
Guard both extra lookups with isinstance(dict) before merging, so a
truthy non-mapping `extra` value (e.g. `extra: "true"`) degrades to the
base Telegram hint instead of raising TypeError and aborting
system-prompt construction. Keep the narrowed except ImportError.

Add an integration test exercising the real config path (HERMES_HOME +
gateway.platforms.telegram.extra.rich_messages) and a regression test
for the malformed-extra fail-open path. The integration test fails on
main and passes with the fix.
2026-08-08 13:57:44 +05:30
bex 9f582aca1d fix(agent): read Telegram rich_messages config from correct path
Commit b45a217e0 gated the TELEGRAM_RICH_MESSAGES_HINT extension behind
a config read at the top-level ``platforms.telegram.extra.rich_messages``
key, but the Telegram adapter reads the same setting from the canonical
``gateway.platforms.telegram.extra.rich_messages`` path.  When users set
the setting in the canonical location (the only one documented), the
lookup returned None and the extension never fired — the model degraded
pipe tables to bullet lists, task lists to plain dashes, and never
produced <details> blocks or block math.

Fix: merge both ``gateway.platforms.telegram.extra`` and the top-level
``platforms.telegram.extra`` with the same precedence the adapter uses
(top-level leaf wins), so config-wizard writes and dashboard-setup keys
are visible alongside the canonical gateway location.  Narrow the
except-guard to ImportError so real config-stack failures surface.
2026-08-08 13:57:44 +05:30
0xarkstar eaeba6474f feat(agent): add skip_background_review flag to AIAgent constructor
Phase 8 of the Hermes Agent token leak mitigation plan
(ralplan-hermes-token-leaks.md §3.9). Adds a boolean kwarg
`skip_background_review` (default False) to AIAgent.__init__ that
suppresses the end-of-turn _spawn_background_review fork.

Each background review fork instantiates a new AIAgent with its own
~15K input tokens + up to 8 LLM iterations, accumulating ~30K tokens
per event in the worst case. On cron sessions there is no
human-in-the-loop benefit from the review (no skill-creation pressure,
nobody curating MEMORY.md), so the cost is pure waste.

The end-of-turn guard now reads:

    if (final_response and not interrupted
            and not getattr(self, "skip_background_review", False)
            and (_should_review_memory or _should_review_skills)):

skip_memory=True already disables the memory-review trigger; this
flag is the explicit single-switch off for both review paths.

Defaults to False, so behavior is unchanged for gateway/CLI callers
that omit the kwarg.

Tests: 5 new unit tests in tests/agent/test_skip_background_review.py
covering the default value, flag persistence, the gate short-circuit,
the gate fall-through, and a source-text assertion that the cron
scheduler sets the flag to True (separate commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-08-08 00:07:14 +05:30
Teknium fa1a5c0485 Integrate verify subsystem with the existing verification stack
Rescope: hermes verify fills only the runtime-smoke gap and plugs into
the pieces Hermes already has instead of standing beside them.

- agent/verification_evidence.py: record_verify_run() — explicit ledger
  write for hermes verify results (shared _insert_evidence factored out
  of record_terminal_result). Passing runs mark the workspace passed
  like scripts/run_tests.sh; failures are recorded; --phase/--skip-start
  runs are recorded as targeted scope.
- hermes_cli/verify_cmd.py: record results into the ledger on completion
  (fail-silent, HERMES_SESSION_ID attribution); on the detect path merge
  detect_project_facts verify commands the recipe missed into the
  recipe's test list (never applied to a saved manifest).
- agent/verification_stop.py: recipe-aware nudge — when the workspace
  has a runnable recipe (start command or .hermes/environment.json),
  suggest hermes verify --json as the preferred full check; cheap,
  try/except-guarded detection that can never break the nudge path.
- agent/verify/recipes.py: document layer ownership (coding_context =
  cheap prompt facts; verify/recipes = deep runtime recipe).
- tests/verify/test_ledger_and_nudge_integration.py: 17 tests covering
  ledger pass/fail recording, the closed edit->nudge->verify->satisfied
  loop, recipe-aware nudge wording + fail-silence, and the facts merge.
2026-08-07 10:11:05 -07:00
Teknium cc1acfb229 fix: Windows-safe process-group teardown in verify runner (footgun CI) 2026-08-07 10:11:05 -07:00
Teknium 47a35d63c0 Port from superagent-ai/grok-cli: verify subsystem (run-recipe detection + environment manifest + hermes verify smoke runner)
Scoped port of grok-cli's verify subsystem:
- agent/verify/recipes.py: static run-recipe detection mirroring grok's
  detection order (Node frameworks w/ lockfile-based package-manager
  choice, Django/FastAPI/Flask/generic Python, Go, Rust, Maven/Gradle,
  Makefile targets, docker-compose)
- agent/verify/environment.py: versioned, user-editable manifest at
  <project>/.hermes/environment.json; tolerant loader; manifest wins
  over fresh detection
- agent/verify/runner.py: bootstrap -> build -> test -> background start
  -> HTTP readiness poll -> process-group teardown, structured result
- hermes verify CLI command (--detect-only, --save, --skip-start,
  --phase, --port, --json)

Sources:
https://github.com/superagent-ai/grok-cli/blob/main/src/verify/recipes.ts
https://github.com/superagent-ai/grok-cli/blob/main/src/verify/environment.ts
2026-08-07 10:11:05 -07:00
Teknium 5db1b72b1f feat(cli): global emergency stop — hermes pause / hermes resume
Resumable ESTOP sentinel at $HERMES_HOME/ESTOP that halts NEW work only:

- agent/estop.py: sentinel engage/disengage/is_engaged (single stat, no
  caching), optional reason + timestamp stored as JSON, paused_reply()
  notice, check_paused() log-once-per-engagement helper. Corrupt/empty
  sentinel still pauses (fail safe); a `touch ~/.hermes/ESTOP` works.
- cron/scheduler.py: tick() skips dispatch while engaged (logged once per
  engagement, not per tick). Due jobs simply wait for the next tick after
  resume — in-flight runs are never touched.
- gateway/kanban_watchers.py: dispatcher skips auto-decompose and worker
  spawning while engaged; zombie reaping still runs and running workers
  finish naturally.
- gateway/run.py: new gateway turns (post-auth, non-internal) get a brief
  "Hermes is paused" reply instead of an agent run. Internal events
  (in-flight background completions) bypass the gate.
- hermes_cli/subcommands/pause.py: `hermes pause [--reason]` and
  `hermes resume`, wired into main() and _BUILTIN_SUBCOMMANDS.
- hermes_cli/status.py: `hermes status` shows a PAUSED banner (one stat).
- tests/test_estop.py: 20 tests — sentinel lifecycle, reason surfacing,
  log-once, cron skip + resume, kanban gate, gateway paused reply +
  internal bypass, CLI idempotence, builtin-set parity, status line.

Never kills in-flight work; resumable with no restart. Footprint ladder:
CLI command only, no new model tool, no new env vars.

Ported from: gastownhall/gastown estop.go (MIT); related prior art:
#26778 (/panic — kill/exit semantics, deliberately different: ours is a
resumable pause), #44617 (interrupt in-flight cron — out of scope here).
2026-08-07 08:58:14 -07:00
kshitij 4a3942d948 fix: show explicit member spend cap message instead of 'no credits'
When the Nous Portal returns paid_service_access.allowed=false with
reason=member_spend_cap_exceeded, Hermes was falling through to the
generic 'no active subscription or usable credits' message — even
though the user has ample purchased credits and the real blocker is
an org-level per-member spend cap.

This adds a dedicated branch that surfaces the actual cause: names the
spend cap, shows the cap/spend amounts, and tells the user to ask their
org admin to raise it. Also adds member_spend_cap_exceeded to the
billing error code set so the error classifier and auth error formatter
route it through the Nous entitlement message path.
2026-08-07 19:47:40 +05:30
Yishova be14a4bee3 tui_gateway: close dedicated profile SessionDB handles at teardown too
Follow-up to the review on the session.resume ownership fix. Closing the
pre-transfer early returns left two gaps, both real.

1. The transfer had no owner on the other side. Once ownership moved to the
   agent, teardown ran AIAgent.close() (via _teardown_session on session.close
   and the orphaned-session reaper), which called session_db.end_session() —
   that finalizes the session ROW, not the connection. A successfully resumed
   profile session kept its dedicated handle, its db/-wal/-shm fds and its
   background token-writer thread for the life of the gateway.

   AIAgent now carries an explicit _owns_session_db, defaulting False so the
   SHARED launch handle — which outlives every agent and backs every other live
   session — is still never closed there. Only the dedicated-open sites set it,
   at the point ownership actually changes hands.

2. session.resume was not the only profile-scoped open with no close on its
   failure paths. Covered here with the same flag, via a _transfer_db_to_agent
   helper that refuses the transfer unless the agent really holds that handle:

   - the deferred builder (_start_agent_build), including the session-reaped-
     mid-build case, where the built agent is discarded and never torn down, so
     transferring to it would leak exactly as before;
   - session.branch's branch_db;
   - the compute host's per-profile open;
   - AIAgent's own lazy open in _get_session_db_for_recall, which no other
     object ever references and so was unconditionally abandoned.

Where a handle has already reached a registered session, the drop is
unconditional and the transfer is best-effort on top: a refused transfer leaves
the old leak, which is survivable, whereas closing under a live session is the
permanent "Cannot operate on a closed database" break the original patch exists
to avoid.

Tests: tests/tui_gateway/test_session_db_ownership_teardown.py (new, 14).
11 of the 14 fail without this change; the 3 that pass are the "must NOT close"
guards, which hold in both directions by design.
2026-08-07 19:44:41 +05:30
kshitij fecba5afcc refactor(agent): fold simplify findings — DB picker parity, single scan, canonical strip delegation
Review-pass follow-ups (three parallel reviewers, findings verified):

- hermes_state_search.py list_recent_user_messages now drops legacy
  standalone compaction handoffs in the decode loop (SQL can't see them:
  durable role=user, no display_kind). Closes the /undo N pairing skew
  where the in-memory count (new predicate) and the DB soft-delete pick
  (old predicate) targeted different turns on legacy sessions. Fetches
  with headroom so the requested limit is still honored. 3 new tests,
  mutation-checked (no-op'ing the skip fails 2/3).
- _should_skip_model_call_for_reference_handoff: single drive-check scan
  (was two — once inside the restore helper, once after); the restore
  helper no longer re-scans and its return value now decides the verdict.
- _final_response_from_messages replaced by the _HANDOFF_SKIP_FINAL_RESPONSE
  constant it always returned (parameter was unused).
- _handoff_carries_live_user_content delegates to the canonical
  _strip_context_summary_handoff_message — also fixes the edge where a
  merged-shaped row with an EMPTY preserved prior tail was wrongly
  treated as carrying live content.
- Site-level guard test for rollback.restore with a legacy handoff row
  (predicate-in-context, complements the unit tests).
2026-08-07 19:44:35 +05:30
kshitij 4eabb595f0 fix(agent): finish the #80622 bug class — sibling predicates, refund ordering, prompt carve-out, honest skip response
Follow-ups on top of the salvaged #80696 fix (review findings):

- Sibling sites: rollback.restore, gateway /retry, CLI /retry and /undo N,
  and both CLI resume turn counters now use is_user_originated_turn so
  legacy-persisted standalone handoffs (durable role=user, no display_kind)
  can never be truncation targets or counted as user turns (#80622
  suggested regression 4, dispatcher-wide).
- Site-1 guard: hoist the api_call_count decrement + iteration-budget
  refund above the break so a skipped turn no longer leaks a budget unit
  and finalize_turn logs the true call count (matches the ollama early-exit
  and the site-2 sibling).
- Site-2 guard: run the handoff guard BEFORE reanchoring so a restored
  user ask is what the anchor lands on, not a stale pre-restore index.
- SUMMARY_PREFIX: add the mid-tool-loop carve-out the code-side guard
  already implements, so a literal-minded model doesn't halt an in-flight
  exchange after in-place compaction.
- Skip path returns a short compaction status instead of replaying the
  previous turn's answer (finalize_turn would append it as a fresh
  assistant row — duplicate prose in transcript and delivery).
2026-08-07 19:44:35 +05:30
HexLab98 6d3ff6eda8 fix(agent): stop reference-only compaction handoff from becoming the active turn
After a completed assistant stop, a standalone CONTEXT COMPACTION handoff
could occupy the sole user slot and resume stale Historical Task Snapshot
work with no new human ask. Guard post-compaction continues, hide
standalone handoffs from session dispatch, and harden SUMMARY_PREFIX for
the empty-after-handoff case (#80622).
2026-08-07 19:44:35 +05:30
kshitij 3737bb1adf docs(compression): correct the projection's safety claims (review findings)
Two docstring corrections on top of the salvaged #80997 fix — behavior
unchanged, both verified against the code:

- The 'rough growth over-counts every content class' claim is false for
  Cyrillic/Greek/Thai/Arabic (chars/4 vs ~2-3 chars/token on o200k):
  growth there can under-count up to ~2x (#62605's direction). Document
  the real backstops instead: an at/over-threshold real reading clears
  the baseline (post-response gate fires on real usage within one call)
  and the provider overflow handler compacts reactively.
- Document the two measurement bases (turn-prologue raw messages vs the
  loop's fully assembled request that seeds the baseline) and why the
  prologue's smaller basis can only OVER-defer — the loop's pre-API
  pressure check re-runs the projection with the aligned basis before
  every provider call, so a prologue over-defer never skips a needed
  compaction.
2026-08-07 19:44:29 +05:30
Alan Hsu 6d89b10653 fix(agent): project real usage in preflight defer instead of fixed growth tolerance
The rough preflight estimate intentionally overestimates, but not by a
fixed margin: CJK text is counted at ~1.7x its o200k cost and
Responses-mode reasoning replay blobs at several times their billed
cost. Heavy sessions show rough estimates 2-3x real usage and compact
at 35-55% of the real window, stalling turns for minutes and discarding
detail (churn), because the defer guard only tolerated 5% rough growth
and sessions that never compressed had no baseline at all.

Pair every request's rough estimate (note_request_rough_estimate,
recorded in the conversation loop right after the pressure estimate)
with the provider's real prompt_tokens in update_from_response(), then
defer preflight while projected real usage — last real + rough growth
since that reading — stays under the threshold. Rough growth is itself
an overestimate of real growth, so the projection is an upper bound and
deferring below the threshold is safe; the provider's context-overflow
handler remains the backstop.

The baseline no longer ratchets on defer: it is refreshed by the
response pairing, and advancing it without a matching real reading
would shrink apparent growth and defer on stale data.
2026-08-07 19:44:29 +05:30
kshitij 458ce7b2b4 fix(streaming): close the same mid-tool-call drop gap on the Anthropic path
Sibling of the chat_completions zero-byte-args fix (previous commits):
a clean SSE close after content_block_start(tool_use) but before any
input_json_delta / message_delta yields an SDK final-message snapshot
whose content is NON-empty (the tool_use block is present, input={})
and whose stop_reason is None. That shape sailed past both
empty-stream guards (they only fire on empty content) and executed the
tool with empty input — no retry, no error: the same silent-data-loss
class as #80498, one provider transport over.

A legitimate completion always carries a stop_reason, so a
tool_use-bearing message without one is a mid-tool-call stream drop.
Raise EmptyStreamError for it, riding the same bounded stream-retry
(HERMES_STREAM_RETRIES) the eventless-stream case already uses.

Gate checked on both return paths (raw SDK snapshot and
accumulator-modified message). Regression tests cover the dropped
shape (mutation-verified: disabling the gate fails exactly that test),
the legitimate tool_use completion, and the text-only no-stop_reason
shape (pre-existing behavior preserved).
2026-08-07 18:07:19 +05:30
joaomarcos f734578033 fix(streaming): flag empty tool-call args on clean stream end (#80498)
When the stream closes right after a tool call's name arrives but
before any argument bytes are delivered, has_truncated_tool_args
was never set (the existing check required a non-empty, whitespace-
stripped arguments buffer). The call fell through to a normal "stop"
finish_reason, later coerced to "{}" at dispatch and executed
silently with no arguments and no retry.

Route this case through the same dropped-mid-tool-call stub/retry
path already used for partially-truncated JSON.
2026-08-07 18:07:19 +05:30
kshitij 72b7305263 polish: document newline residual, reuse span local, cheap check first
Review follow-ups on the guard: state the accepted \n-residual in the
comment, reuse the span local in the next condition instead of
re-slicing, and short-circuit the substring checks before the regex.
2026-08-07 17:50:03 +05:30
kshitij 9377c5a539 fix(redact): narrow control-split join guard to line-crossing spans
Post-merge review of aecb9ca89 found the join guard over-broad: skipping
the join whenever ANY fragment self-matches _PREFIX_RE reopened a leak
for non-newline splits — sk-<15 chars>ESC<25 chars> masked only the
self-matching head and left the 25-char tail in cleartext (fully masked
before the guard; main never masked this shape at all, so the merged
state was still >= main, but the salvage's own coverage regressed).

Skip the join only when the span crosses a line boundary (\n / \r) —
that is the shape where adjacent legitimate text gets swallowed
(ghp_<token>-then-'button [ref=e3]' annotation bug). ESC/zero-width
controls never legitimately separate a token from prose, so joining
there is safe and restores full-tail masking.

Both legs mutation-checked: reverting to the unconditional skip fails
the new tail-mask test; removing the guard fails the annotation test.
2026-08-07 17:50:03 +05:30
kshitij aecb9ca894 fix(redact): don't join across controls when a fragment already matches
CI slice 1/12 caught a regression in _mask_control_split_tokens: a
COMPLETE prefix token at end-of-line followed by ordinary text (browser
accessibility annotations: 'ghp_<tok>\nbutton [ref=e3]: Copy') was
joined across the newline into one stripped-copy match, and the mask
swallowed the adjacent line ('button' disappeared).

Join only when no fragment inside the span matches _PREFIX_RE on its
own — a self-matching fragment is already handled by the ordinary
prefix pass, so joining can only cause damage. All smuggling shapes
(ESC/ZWSP/newline splits with under-length fragments) still mask;
regression test added and mutation-checked (fails without the guard).
2026-08-07 17:01:23 +05:30
Soheil Fakour e9d1551e65 fix(redact): strip control chars from mask_secret display (#55319, #55321)
A masked secret's visible head/tail could carry control bytes (newline,
NUL, DEL, C1 0x80-0x9F, zero-width) into config/status/dump output.
Strip every control incl. \n/\t (display differs from redact_sensitive_text,
which preserves \n/\t as line structure) before slicing; all-control values
return the configured empty fallback.

Consolidates the previously-closed #58079 approach (strip controls before
masking) - supersedes it.
2026-08-07 17:01:23 +05:30
Soheil Fakour 8563fe3435 fix(redact): close emission gaps - env suffix keys, control-char splits, process(list) (#77484) 2026-08-07 17:01:23 +05:30
kshitij 15d7103aa7 fix: harden .env-read detection — review follow-ups for #61352
- Import file_safety._BLOCKED_PROJECT_ENV_BASENAMES instead of copying
  it (comment-enforced parallel lists drift); lookup is now
  case-insensitive to match file_safety's .lower() semantics (cat .ENV
  on macOS/Windows case-insensitive filesystems reads the same secrets).
- Strip shell quotes plain split() leaves attached (cat ".env").
- Drop the dead _ENV_FILE_EXCLUDE_SUFFIXES logic (exact-basename
  membership already excludes templates) and the stray blank-line noise.
- Document the defense-in-depth limits (sudo/full-path/substitution
  readers) mirroring is_env_dump_command's precedent, and correct the
  docstring overclaim about name-independence.
- Annotate command as str | None (tests pass None).
2026-08-07 16:58:25 +05:30
Peter cf755f5c42 fix: redact .env terminal output via detection instead of known-env-var list
Terminal output from file-read commands (cat, head, tail, ...) uses
code_file=True, which skips the generic ENV-assignment redaction pass.
Reading a .env file through the terminal therefore leaked any key whose
value has no recognized vendor prefix (Mistral, Gemini AQ.*, tvly-dev-,
bu_, Spotify client secrets).

Detect file-read commands targeting .env-style basenames (mirroring
agent/file_safety's blocked list) and route them to code_file=False so
the existing ENV pass masks opaque values. Templates (.env.example,
.env.sample, ...) are excluded.

Salvaged from #61352 (145 commits of drift; conflict with the test-prune
wave resolved by NOT resurrecting pruned tests). Authored by @ShaoRou459.

Closes #61352
2026-08-07 16:58:25 +05:30
kshitij 1a02e8a793 fix(agent): preserve destroyed tool-call argument bytes in the WARNING log
Review follow-up (W1): the pre-send transcript sanitizer
(agent_runtime_helpers.sanitize_tool_call_arguments) runs on the
PERSISTED messages list before every api_messages build and rewrites any
json.loads-failing argument string to "{}" in the transcript, prepending
a corruption marker to the paired tool result. That in-transcript repair
is deliberate (the stored turn must be replayable next call), but it
destroys the model's original bytes — for a truncated write_file call
those bytes are the user's streamed file content (#80498), and they
previously survived only as an 80-char log preview.

Until a sidecar-preservation design exists, make the bytes recoverable:
both destruction sites (the transcript sanitizer's WARNING and
_repair_tool_call_arguments' unrepairable-path WARNING) now log the full
original argument string bounded at 100KB instead of 80 chars. Corrupted
calls are rare; an oversized WARNING is a fair price for the only copy
of real user content.
2026-08-07 16:57:11 +05:30
kshitij c18e19c3c7 fix(agent): make the send-path copy structural — close the write-through class
The api_messages build used a shallow msg.copy(), decoupling only
top-level fields. Every nested container (tool_calls entries and their
function dicts, multimodal content-part lists, reasoning_details) stayed
aliased to the persisted history, so ANY in-place transform on the send
copy silently rewrote the stored transcript.

Probed every send-path transform against that aliasing shape on main:

  content strip loop                       safe (top-level reassign)
  _canonicalize_api_tool_calls (repair)    LEAKED  <- #80616's fix
  _sanitize_messages_surrogates            LEAKED  (multimodal parts,
                                                    tc ids/args, reasoning)
  _sanitize_messages_non_ascii             LEAKED  (multimodal parts)
  _sanitize_api_messages                   safe
  _drop_thinking_only_and_merge_users      safe

The retry loop already believed the copies were independent - it
sanitizes messages AND api_messages separately (~L3555) - so the
aliasing was accidental everywhere.

Fix at the chokepoint: _clone_message_for_send clones every container
(dict/list) recursively while sharing immutable leaves, so every
downstream in-place transform - current and future - is safe by
construction. Cost is container-count, not string-bytes: 100KB argument
strings and base64 payloads are shared (measured ~0.5ms vs ~0.1ms per
1500-message build; noise next to one json round-trip). Same clone
applied to the prefill-message insert (same class, same pipeline).

The class-wide invariant test runs the full send-path transform
pipeline over an adversarial fixture (malformed args, surrogates,
non-ASCII, multimodal parts, reasoning fields) and asserts the history
stays byte-identical; an AST contract pins the build-site wiring so the
shallow copy can't quietly return. Both mutation-verified: reverting
the clone to shallow fails 4 isolation tests, unwiring the build site
fails the AST contract.

0xGr1mm's branch fix (previous commit) remains as defense in depth at
the exact site the #80498 incident hit; his regression tests and the
class-wide invariant give layered coverage.
2026-08-07 16:57:11 +05:30
Gr1mmJ4w e60ca1c6ca fix(agent): stop the send-path repair from rewriting persisted history
`_canonicalize_api_tool_calls` promises copy-on-write in its own docstring
— "the persisted history is untouched" — and the call site repeats it:
"Operates on api_messages (the API copy) so the original conversation
history in `messages` is untouched."

The canonicalize branch keeps that promise (`tc = {**tc, "function": {...}}`).
The repair branch does not:

    except Exception:
        tc["function"]["arguments"] = _repair_tool_call_arguments(...)

`api_messages` is built with `msg.copy()` — a SHALLOW per-message copy — so
every `tool_calls` entry is the same dict object the persisted history
holds. Assigning into `tc["function"]` therefore writes through to the
stored turn. The sibling loop two lines above only touches `am["content"]`,
one level deep, which is why the aliasing never showed up there.

On the unrepairable path `_repair_tool_call_arguments` returns "{}", so
that write replaces the model's real arguments with an empty object in the
transcript. A stream that dies mid `write_file` loses the file content it
had already streamed — the reported symptom in #80498, where a chapter
draft was silently reduced to `{}` and only a WARNING remained:

    Unrepairable tool_call arguments for write_file — replaced with empty
    object (was: {"content": "# 骨架-第25章\n> 承接...)

Mirror the canonicalize branch: build a new tool-call dict instead of
assigning into the shared one. The API copy still carries "{}" — the
repair's whole purpose is to never ship broken JSON — but the history keeps
what the model actually sent, so the transcript, session persistence and
any later retry still have it.

The in-place write was not an oversight in isolation: it predates the memo
refactor, which preserved it deliberately for byte-parity. The existing
`test_history_not_mutated` asserts exactly this invariant but restricts
itself to valid arguments, and its docstring records the gap — "(Malformed
args take the in-place repair path — pre-existing behavior)". That is why
a test file whose header already claims "the persisted history is never
mutated (copy-on-write preserved)" stayed green through the bug.

Four tests close it: history keeps the original bytes, the send copy is
still repaired, a broken call does not disturb its siblings, and repeated
sends stay lossless. On unpatched main three of them fail; the parity and
complexity tests are unaffected because the difference is only observable
when the history list is separate from the send copy — which is the shape
production uses.

Refs #80498

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 16:57:11 +05:30
kshitij c95a1b7171 fix(auxiliary): widen effective provider to relay, logging, and endpoint detection
Thread request_provider through the sibling callsites that the
original PR left on resolved_provider: _set_relay_auxiliary_route
(observability metadata), the 'using X' log line, _is_anthropic_compat_endpoint
(Anthropic image conversion), _provider_requires_stream (streaming
detection), and the initial _relay_sync/async_completion calls.

These are NOT regressions (they passed 'auto' before), but widening
them ensures auto-routed calls to MiniMax get correct image conversion,
streaming-only providers are detected, and observability metadata
records the concrete backend instead of 'auto'.
2026-08-07 16:56:37 +05:30
Gille 293e67328c fix(agent): preserve auto-routed provider identity 2026-08-07 16:56:37 +05:30
kshitij 1e5b507440 fix(cron): move watchdog state under the request lock; fail closed on resolver errors
Follow-up hardening on top of the salvaged #80809 watchdog, porting the
locked state-machine design from #75301 (credit: @Zeraphim):

- All lifecycle transitions (stale/cancelled/done) now happen under
  request_client_lock. A user or monitor interrupt marks the request
  'cancelled' so a racing stale timer can no longer misclassify the kill
  as provider staleness and feed a false +1 into the #58962 cross-turn
  circuit breaker.
- 'done' is set under the lock on completion, so a late timer callback
  that lost the race to a successful response is inert instead of
  leaving a spurious streak=1 behind the reset.
- Registration race closed: if the budget expires while the client is
  still being constructed, _make_client aborts the freshly-registered
  client and fails the call with a retryable TimeoutError instead of
  opening a brand-new socket after the only watchdog already fired.
- _resolve_direct_stale_timeout now fails closed: a raising resolver
  propagates (same as the worker path) instead of being swallowed into
  an infinite budget that would silently disarm the watchdog and
  reinstate the very hang #80759 is about.

4 new regression tests, each verified to fail against the pre-fix
watchdog implementation.

Co-authored-by: Zeraphim <diamantejc87@gmail.com>
2026-08-07 15:32:37 +05:30
HexLab98 cb066a971b fix(cron): bound the inline non-streaming call with a stale watchdog
Cron turns and delegated children are routed onto direct_api_call, which
ran the request inline with no stale detector. The abort plumbing was
registered but nothing ever invoked it, so a provider that accepted the
request and then went silent — connection held open, zero bytes, no
error — hung the run until an external actor killed it, which also
orphaned the execution row. The httpx read timeout is not a usable bound
(1800s default, and this failure mode never trips it), and the job-level
inactivity monitor was observed not to fire.

Arm a watchdog timer on the same budget the interrupt worker's poll loop
uses, so these turns get exactly the patience every other non-streaming
request already gets. On expiry it only aborts the in-flight sockets
through the already-registered hook — it never issues a request, so the
inline / no-worker property that fixes the nested-pool deadlock is
preserved — bumps the cross-turn stale circuit breaker, and surfaces a
retryable TimeoutError so the outer loop reconnects on a fresh pool.

Fixes #80759
2026-08-07 15:32:37 +05:30
kshitij 1fe53bd1ab docs: comment accuracy — pending-ness is a presumption, not a construction guarantee
Review follow-up: after the walk-back widening, the exempted assistant
is often not the final message, and the partial-batch shape is
byte-identical to a settled-but-malformed orphan — so say 'presumed
pending' and document WHY presuming is safe (sanitize_api_messages
step 2 stubs any genuinely unanswered call pre-API on every path).
2026-08-07 14:13:32 +05:30
kshitij 03beb662e8 fix: cover the partial multi-call batch in the in-flight exemption
Widen #79293's trailing-in-flight guard from 'last message is assistant'
to 'last non-tool message is assistant': a multi-call batch snapshotted
between the executor's per-result appends looks like
[..., assistant(c1,c2,c3), tool(c1)] — c2/c3 are pending, not orphaned,
but the tail-only guard missed that shape and stripped them (same silent
result loss as the original bug, via concurrent /compress or the gateway
hygiene pass).

Preserving is safe on both shapes: the pre-API chokepoint
(sanitize_api_messages step 2) injects stub results for any call that
genuinely never gets an answer, while stripping a live call silently
loses its late result.

test_sanitizer_strips_orphaned_keeps_valid's mixed valid/orphan shape
moves mid-list — at the tail it is byte-identical to a live partial
batch and the sanitizer now correctly presumes in-flight there.

New regression test fails without the walk-back (c2/c3 stripped),
passes with it.
2026-08-07 14:13:32 +05:30
Shotflame 788b8ab497 fix(compress): preserve in-flight tool chain across context compression (#79278)
Tool_executor.py appends role=tool results AFTER running each call. When
context compression fires mid-chain, the trailing assistant(tool_calls)
message is a pending request whose result has not yet been appended.
_sanitize_tool_pairs previously stripped it as an 'orphan', so when the
executor later appended the real result, repair_message_sequence dropped
it as unmatched and the completed side effect (and final synthesis) was
lost. Preserve the trailing in-flight call verbatim; only genuinely
orphaned calls in the discarded region are stripped.

Adds regression tests: three unit tests for _sanitize_tool_pairs plus an
end-to-end test reproducing compression -> side-effect completion ->
result-returned flow. Confirmed failing on pre-fix code, passing with
the fix.
2026-08-07 14:13:32 +05:30
kshitij a0801b878a fix: bind continuation-marker exclusions to the queried parent (fail-open fix)
Adversarial review of the salvaged recovery found a reachable fail-open:
compression continuations inherit the rotated agent's model_config
verbatim (publish_compression_child callers pass
agent._session_init_model_config), so a delegate subagent's continuation
carries _delegate_from=<the delegate's own parent>. The marker-PRESENCE
filters in reopen_orphaned_compression_session and
find_live_compression_child misclassified such a REAL continuation as a
delegate child:

- reopen: parent 'orphaned' -> reopened while a live continuation exists
  -> two live heads in one lineage (verified with a live repro)
- find_live: adoption misses the continuation (fail-closed, masked the
  fork pre-PR; the PR made it active)

Fix: markers only disqualify a child when they point at the queried
parent (shared _NON_CONTINUATION_CHILD_FILTER_SQL fragment, also
resolving the duplicated-SQL drift risk flagged by the reuse reviewer).
Both directions regression-tested: reopen fails closed on an
inherited-marker continuation; find_live adopts it.

Also from review: reopen-failure log raised debug->warning (the failure
hard-fails the turn moments later), commit-semantics hardening comment
on the lease DELETE path, blank-line nit.

The three read-only projection walks (get_compression_tip,
list_sessions_rich chain, resume walk) share the marker-presence shape
but fail closed (skip a continuation -> resume shows the parent), and
the fixed adoption path self-heals that case at turn start; left as-is.
2026-08-07 13:24:56 +05:30
izumi0uu 988f2baaf8 fix(sessions): recover compression parents without continuations 2026-08-07 13:24:56 +05:30
Teknium 32e7fb07a0 feat(/learn): expansive knowledge-base skills for books and large corpora
Inspired by virgiliojr94/book-to-skill (MIT): /learn now picks the skill
shape by the source. Workflows and small sources still get one tight
SKILL.md; books, paper stacks, specs, and large doc corpora get a
knowledge-base layout — a lean always-loaded SKILL.md index plus one
distilled file per chapter/topic under references/, loaded on demand via
skill_view so query cost stays proportional to the answer.

- agent/learn_prompt.py: new _KNOWLEDGE_SKILL_STANDARDS block (index +
  per-chapter references/, structure-not-summary distillation, never
  reproduce source passages, fold-in instead of duplicating) and a
  _SOURCE_HYGIENE block pinning extracted source text as data and
  dropping invisible/bidi Unicode (Trojan Source class). Clarified that
  the ~200-line cap and hub-skill ban apply to SKILL.md itself, not a
  knowledge skill's own references/ files.
- tests: contracts for the knowledge-base layout, the three embedded
  standards blocks, and the source-hygiene coverage.
- docs: skills.md documents the knowledge-base shape.
2026-08-06 22:14:52 -07:00
brooklyn! 55505be152 Merge pull request #80770 from NousResearch/bb/desktop-session-integrity
fix: preserve session history when a turn crashes
2026-08-06 22:12:22 -06:00
Brooklyn Nicholson fc05247be8 fix: preserve session history when a turn crashes 2026-08-06 23:08:23 -05:00
Brooklyn Nicholson 6bb630ef78 fix(codex): split reasoning summary parts on summary_index
The native Responses stream does carry summary_index, so the part boundary is
structured data here rather than something to infer. Break on a change of
index, and leave streams that send no index (plain reasoning_text) untouched.
2026-08-06 22:02:46 -05:00
Brooklyn Nicholson 0f83661808 fix(reasoning): keep gpt-5.x summary parts as separate blocks on the chat wire
Reasoning-summary models emit one reasoning_content delta per completed
summary part, each a self-contained bold heading. The Responses API delimits
those parts with summary_index; the OpenAI chat wire carries no such field —
verified live against Nous Portal, whose reasoning chunks contain nothing but
delta.reasoning_content — so concatenating them glued every part into one
unspaced, half-bold paragraph.

Re-derive the boundary from the signal the wire does carry: a delta opening a
closed bold heading against a mid-line tail. This matches Hermes own Responses
adapter, which already joins its summary parts with a blank line.
2026-08-06 22:02:37 -05:00