Commit Graph

234 Commits

Author SHA1 Message Date
Teknium e293872ac5 review-fix(comments): keep the restored #95681 rationale above the try so the guard-window source test still sees it 2026-09-03 10:52:45 -07:00
Teknium e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium 0071ba9965 Merge origin/main (561b053f79) into simp/forwardport: forward-port 220 main commits into the simplified tree 2026-09-03 03:31:03 -07:00
Teknium 017906fb4f refactor(agent/conversation_compression): flatten feasibility probe call shape 2026-09-02 23:42:15 -07:00
Teknium bcdd78e3c3 refactor(agent/conversation_compression): compact fence/constant docstrings and comments; hug remaining docstring bodies 2026-09-02 23:38:31 -07:00
Teknium 566adb178d refactor(agent/conversation_compression): compact docstrings — summary line hugs its prose body (text unchanged) 2026-09-02 23:30:28 -07:00
Teknium 58d9f24663 refactor(agent/conversation_compression): AST-identical — drop paragraph blanks inside function bodies 2026-09-02 23:26:28 -07:00
Teknium 01ebcde193 refactor(agent/conversation_compression): fold in-place/rotation post-commit writes into their branches; heartbeat stop helper 2026-09-02 23:23:38 -07:00
Teknium 4b041206d0 refactor(agent/conversation_compression): try/except-assign -> suppress with defaults, fold shrink accept gate, flatten warning text assembly 2026-09-02 23:21:35 -07:00
Teknium 3675c9d83a refactor(agent/conversation_compression): collapse guard ladders, dict/payload builders and duplicated temporaries 2026-09-02 23:14:55 -07:00
Teknium 02ef6480de refactor(agent/conversation_compression): AST-identical — pack trailing log args after multi-line format strings 2026-09-02 23:08:53 -07:00
Teknium 8998c13bcd refactor(agent/conversation_compression): reflow prose docstrings to 110 cols (text unchanged) 2026-09-02 23:06:46 -07:00
Teknium 0fa8263bf8 refactor(agent/conversation_compression): drop pre-return lease releases already covered by the finally 2026-09-02 23:03:21 -07:00
kshitijk4poor fafd370094 test(compression): register the heartbeat in ROUTINE_COMPRESSION_STATUS_SAMPLES
Completes the constant + samples + noise-regex trio teknium named as the bar
for this heartbeat on #98371; the Telegram noise-filter and TUI retag
parametrised suites now iterate the heartbeat wording too.
2026-09-03 11:29:36 +05:30
kshitijk4poor c41525b290 fix(compression): heartbeat honours the quiet-engine contract and the single start line
Review follow-ups: the heartbeat opened a visible compacting phase even when
the context engine suppressed the routine start status (no terminal edge would
ever close it), and its start() emitted a second start line milliseconds after
the routine one (two chat messages on adapters without send_or_update_status).
Gate the client-visible heartbeat on the start status having been emitted, drop
the start-time emit, and route ticks through agent._emit_status so CLI print
and gateway filtering match every other compaction status.
2026-09-03 11:27:55 +05:30
kshitijk4poor 548e0ec923 fix(compression): route the heartbeat through the shared compaction status contract
The contributor heartbeat called status_callback("compacting", <ad-hoc text>).
Every other compaction status uses the "lifecycle" key: the TUI gateway
re-tags lifecycle statuses to kind="compacting" via
is_compaction_progress_status, Telegram edits one bubble per status key, and
gateway/run.py's chat-platform filter only recognises the registered
templates — so the ad-hoc text leaked to Telegram/Discord on every tick with
compression.progress_notices off (verified: _prepare_gateway_status_message
passed it through). Register COMPACTION_HEARTBEAT_STATUS next to
COMPACTION_STATUS, emit it under "lifecycle", and cover the filter.
2026-09-03 11:27:55 +05:30
Jack 8381d4fadf compression: emit periodic client-visible status during compaction
Context compression can stream for minutes with no deltas, tool events,
or status lines reaching remote transports. Idle-progress watchdogs on
those clients treat the silence as a dead turn and interrupt it — the
Android relay app's 180s turn watchdog fires session.interrupt, killing
a healthy compression mid-flight and rolling back its work. On sessions
near the context ceiling this loops forever: every new prompt retriggers
preflight compression, which dies at exactly +180s again (observed
telemetry: attempts aborted at 180469ms/180233ms/180219ms with
failure_class=explicit_interrupt).

Fix: the existing _CompressionActivityHeartbeat (which today only
refreshes the SessionDB activity tracker) now also emits a 'compacting'
status through agent.status_callback — once at compression start and on
every heartbeat tick. The gateway already routes status_callback to
status.update events, and clients already reset their watchdogs on any
received event, so each heartbeat re-arms them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 6d2e0e860d6fdb44ce549d1fa11a06662b1a4b0b)
2026-09-03 11:27:55 +05:30
kshitijk4poor ab73aeff05 fix(compressor): keep pending tool_calls, avoid double anchors, no header stacking
Review follow-ups on the in-flight replay:

- Run _sanitize_tool_pairs BEFORE the re-append. Its trailing-in-flight
  exemption (#79278) walks back from the list end; with the replay user row
  there, a genuinely pending assistant(tool_calls) looked orphaned and had its
  calls stripped, so the executor's late tool result was dropped.
- When the restatement is merged onto a user-pinned summary carrier, flag the
  carrier (_inflight_replay_merged). The carrier's metadata marks it synthetic,
  so conversation_compression._ensure_compressed_has_user_turn inserted a second
  copy of the same request; it now treats the flag as intent-present, and the
  next cycle recognises the carrier as the task instead of losing it.
- Header idempotency: restate the text after the last header so a task that
  survives several compactions carries one header and one copy.
- Exclude metadata-flagged scaffolding (_todo_snapshot_synthetic, recovery
  nudges) from the in-flight scan via the shared _is_real_user_message.

Tests cover all four; each fails with its fix reverted.
2026-09-03 11:27:40 +05:30
Teknium e0b3879d6b refactor(agent/conversation_compression): AST-identical layout — hug bracket spans, merge split string literals 2026-09-02 22:51:28 -07:00
Teknium a6dd9dd07a refactor(agent/conversation_compression): collapse redundant guards and defensive layers 2026-09-02 22:46:53 -07:00
Teknium 53800aab9d refactor(agent/conversation_compression): drop duplicated comments 2026-09-02 20:14:49 -07:00
Teknium 6082f7d442 refactor(agent/compression): join remaining fragmented log/warning literals (AST-identical) 2026-09-02 20:13:50 -07:00
Teknium 551fe6ea78 refactor(agent/conversation_compression): fold heartbeat suppress predicate, lease-refresh swallow, cooldown capture ordering 2026-09-02 19:57:49 -07:00
Teknium 3a1e4a7661 refactor(agent/compression): fold boundary bookkeeping locals; pack one-item-per-line call/def argument lists (AST-identical) 2026-09-02 19:51:06 -07:00
Teknium 264cc6bd99 refactor(agent/conversation_compression): thread an _Attempt carrier through compress_context phases 2026-09-02 19:45:08 -07:00
Teknium fa4fc3ea1b refactor(agent/compression): reflow docstrings/comment blocks to 118 cols (word-identical) 2026-09-02 19:39:34 -07:00
Teknium ec0984c62b refactor(agent/conversation_compression): flatten _strip_stale_todo_snapshot list branch 2026-09-02 19:37:58 -07:00
Teknium 0419866f01 refactor(agent/conversation_compression): fold anchor placement, steer markers, orphan reopen, timeout config parsing 2026-09-02 19:37:08 -07:00
Teknium e0d7f18fc4 refactor(agent/compression): drop blank lines after function-local imports (AST-identical) 2026-09-02 19:31:44 -07:00
Teknium 64215c980c refactor(agent/conversation_compression): log-and-swallow try/except -> _swallow() context manager (27 sites) 2026-09-02 19:30:28 -07:00
Teknium 4fc0d74fe3 refactor(agent/compression): try/except-pass -> contextlib.suppress (21 sites) 2026-09-02 19:24:14 -07:00
Teknium 7bd7c1114a refactor(agent/compression): join fragmented adjacent string literals (AST-identical) 2026-09-02 19:19:41 -07:00
Teknium 4b0116025b refactor(agent/conversation_compression): lift durable lock acquire + contention sit-out out of _acquire_compression_lease 2026-09-02 19:13:27 -07:00
Teknium ec16f113a4 refactor(agent/compression): reflow to 120 columns (AST-identical); joins fragmented call/string continuations 2026-09-02 19:08:36 -07:00
Teknium 37b0aaa530 refactor(agent/conversation_compression): collapse duplicated commit rollback branches 2026-09-02 19:07:05 -07:00
Teknium ac7591019e refactor(agent/conversation_compression): extract codex route and start-announcement from compress_context 2026-09-02 19:01:49 -07:00
Teknium 9672db5399 refactor(agent/conversation_compression): extract attempt prologue and lease sit-out helper; fold codex failure flag 2026-09-02 18:59:05 -07:00
Teknium 0f60feec52 refactor(agent/conversation_compression): lift feasibility threshold-lowering warning out of check_compression_model_feasibility 2026-09-02 18:54:31 -07:00
Teknium 0e9d46511a refactor(agent/conversation_compression): lift image-shrink nested helpers, lease lock-API resolution, rotation state carry-over, durable cooldown rollback 2026-09-02 18:49:03 -07:00
Teknium 58c7789e36 refactor(agent/conversation_compression): extract summary phase of compress_context and the timeout wrapper's wait/cancel/unwind helpers 2026-09-02 18:33:49 -07:00
Teknium 719cbb6e55 refactor(agent/conversation_compression): extract SessionDB commit phase; fold automatic gate + codex fence bracket 2026-09-02 18:27:34 -07:00
Teknium 7524e9ec8a refactor(agent/compression): unify session rebind, scoped-twin stamping; lift facade timeout callbacks to module helpers 2026-09-02 18:17:52 -07:00
Enzo Adami 02a6e95228 fix(files): allow one full read after compaction 2026-09-03 03:07:37 +05:30
webtecnica 4a0aae8aae fix(agent): preserve tool-output dedup state across context compaction (#84857) 2026-09-03 03:07:37 +05:30
Teknium 3a2eceabc8 refactor(agent/compression): remove dead memory-containment helpers; fold codex skip-log ladder
- delete _cached_prompt_reflects_builtin_memory + _builtin_memory_prompt_snapshot
  (zero callers since the commit site moved to byte-equality; the only test
  reference asserts their ABSENCE from the commit window)
- delete CompressionExecutorSaturatedError (never raised or caught anywhere)
- _compress_context_via_codex_app_server: three near-identical skip branches
  collapse into one skip_reason + single log line (same message text)
2026-09-02 13:29:37 -07:00
Teknium 7e3cef7d87 refactor(agent/compression): extract candidate rejection ladder from compress_context
_candidate_rejected(): compressor abort, no-progress, empty transcript and
superseded-attempt checks in their original order; caller handles the
shared release + unchanged-return. Aborted-branch try/finally collapsed
(the finally only released the lease, which the caller now does).
2026-09-02 13:29:37 -07:00
Teknium 0ea4c7963a refactor(agent/compression): extract post-commit boundary tail of compress_context
- _finish_compaction_boundary(): parent label clear, context-engine + memory
  provider notifications, compression-count warning, session:compress hook,
  usage re-arm, file/skill dedup reset (zero back-refs; returns rough estimate)
- _warn_summary_or_aux_fallback(): once-only summary/aux-model warnings
compress_context 794 -> 653 lines; bodies moved verbatim.
2026-09-02 13:29:37 -07:00
Teknium 01d16451f7 refactor(agent/compression): extract commit-phase helpers out of compress_context
- _fold_todo_snapshot(): stale-snapshot strip + live todo fold into the tail
- _rebuild_system_prompt_at_boundary(): tool refresh + byte-equal keep-prompt
- _salvage_or_refuse_grown_transcript(): commit-site anti-growth guard
- _publish_rotated_compaction(): parent flush, child publish, id re-point,
  goal/heartbeat/loop/title carry-over
- old_session_id is now an explicit Optional local instead of locals().get()
compress_context 1181 -> 794 lines; bodies moved verbatim.
2026-09-02 13:29:37 -07:00
Teknium cbd709124f refactor(agent/compression): extract pre-summary phases of compress_context
- _adopt_grown_durable_parent(): rotation-only durable-snapshot adoption
- _pre_compress_memory_context(): provider on_pre_compress / checkpoint gate
- _resolve_compress_call(): compress() + signature-filtered kwargs
- _run_summary_dispatch(): progress hook, stream deadline, interrupt guard
compress_context 1359 -> 1181 lines; bodies moved verbatim.
2026-09-02 13:29:37 -07:00
Teknium 42c41dfcff refactor(agent/compression): extract lease acquisition and lifecycle out of compress_context
- _CompactionLifecycle: the one-shot terminal status edge (was a nonlocal closure)
- _CompressionLease: holder/watermark/ttl + refresher, holder-only release,
  fence lock-setup bracket, release() (was 4 nested closures + 9 locals)
- _acquire_compression_lease(): the 190-line lock acquisition ladder
- _adopt_if_parent_rotated(): late-contender adoption check
compress_context 1767 -> 1359 lines; bodies moved verbatim.
2026-09-02 13:29:37 -07:00