Commit Graph

375 Commits

Author SHA1 Message Date
Teknium e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium 0071ba9965 Merge origin/main (561b053f79) into simp/forwardport: forward-port 220 main commits into the simplified tree 2026-09-03 03:31:03 -07:00
kshitijk4poor c06f7cf924 test(compression): pin rearm-reset wiring through a public boundary
Post-review cleanup on the salvage stack:
- The re-warn-after-reset guard now drives on_session_reset() instead of
  the private helper, so a site regressing to a bare rearm-zero fails it.
- One _over_threshold_warnings() helper replaces four inline caplog filters.
- Comment the redundant None check that narrows current_tokens for ty.
2026-09-03 12:26:22 +05:30
kshitijk4poor 58a4a11727 fix(compression): release the reclamation no-op dedup key on every rearm reset
Follow-up to the salvaged #101894 (@jwilson411). The over-threshold
"reclamation did not run" warning is deduped on (reason, rearm mark), and
the key was only cleared when a prune committed. Every other path that
zeroes the rearm mark — compress(), on_session_reset/on_session_end,
bind_session_state, update_model — left the key in place, so a lockout
that warned at rearm=0, then a full compaction, then the same lockout
again was silent, contradicting the helper's own "warns again" contract
(and leaking the key across sessions on a rebound compressor).

- ContextCompressor._reset_proactive_prune_rearm(): one helper for the
  five rearm-to-zero sites; clears the dedup key alongside the mark.
- _warn_reclamation_no_op(): dropping back under threshold releases the
  key (mirrors _clear_context_overflow_warn semantics on the agent side).
- test_proactive_prune_loop_wiring: the attempts_exhausted fixture now
  models the only state the real engine can produce for that branch
  (should_compress() is should_compress_info()[0]) — budget spent
  (max_compression_attempts=0) with the engine saying RUN, instead of
  should_compress=False paired with (True, None).
- Two guards: lockout warns again after a rearm reset; dropping under
  threshold releases the key. Both fail with the clears removed.
2026-09-03 12:23:55 +05:30
Justin Wilson 9f12121206 fix(compression): do not let prune rearm lock out over-threshold sessions
Message-only rearm could sit just above the body estimate while provider
prompt_tokens (system + tool schemas) already exceeded threshold_tokens,
so prune no-oped forever with no log. Bypass that rearm short-circuit on
the billed basis, warn once when over-threshold reclamation no-ops, and
name attempts_exhausted when should_compress_info says run but the loop
skips.

Fixes #101889
2026-09-03 12:23:55 +05:30
Teknium 0d27aaa247 refactor(agent/context_compressor): fold cooldown/pressure/handoff-strip branches; drop lazy-import blank lines 2026-09-02 23:19:04 -07:00
Teknium e43eb87b4e refactor(agent/context_compressor): template-driven tool-result summarizers; compact docstrings/comments (every rule kept) 2026-09-02 23:16:11 -07:00
Teknium 72bbbf8f75 refactor(agent/context_compressor): fold gate/backoff loops, content-shape helpers, echo/bridge index math 2026-09-02 23:09:25 -07:00
Teknium 2539771553 refactor(agent/context_compressor): AST-identical bracket hug pass 2026-09-02 23:01:55 -07:00
Teknium ef34ee566b refactor(agent/context_compressor): extract compress() summary/assembly phases; flatten boundary, handoff-strip, and placement helpers 2026-09-02 22:58:16 -07:00
kshitijk4poor d6de054bac refactor(compressor): one helper for the last template-visible role
The re-append slot check duplicated the summary-role scan at the head/summary
boundary; both now call _last_template_visible_role.
2026-09-03 11:27:40 +05:30
kshitijk4poor ab73aeff05 fix(compressor): keep pending tool_calls, avoid double anchors, no header stacking
Review follow-ups on the in-flight replay:

- Run _sanitize_tool_pairs BEFORE the re-append. Its trailing-in-flight
  exemption (#79278) walks back from the list end; with the replay user row
  there, a genuinely pending assistant(tool_calls) looked orphaned and had its
  calls stripped, so the executor's late tool result was dropped.
- When the restatement is merged onto a user-pinned summary carrier, flag the
  carrier (_inflight_replay_merged). The carrier's metadata marks it synthetic,
  so conversation_compression._ensure_compressed_has_user_turn inserted a second
  copy of the same request; it now treats the flag as intent-present, and the
  next cycle recognises the carrier as the task instead of losing it.
- Header idempotency: restate the text after the last header so a task that
  survives several compactions carries one header and one copy.
- Exclude metadata-flagged scaffolding (_todo_snapshot_synthetic, recovery
  nudges) from the in-flight scan via the shared _is_real_user_message.

Tests cover all four; each fails with its fix reverted.
2026-09-03 11:27:40 +05:30
kshitijk4poor 1d260f7683 fix(compressor): judge the re-append slot on template-visible roles
The in-flight replay checked only compressed[-1] before choosing between
appending a user row and merging onto the summary carrier. A user-pinned
summary followed by an exempt tool_calls/tool tail therefore gained a second
visible user turn and broke the Mistral-style alternation pre-flight
(tests/agent/test_summary_role_template_alternation.py::test_zero_user_guard_still_forces_user).
Look through the exempt tail to the last template-visible role instead.
2026-09-03 11:27:40 +05:30
Justin Wilson 0ed3f9b45b fix(cron): re-append in-flight job prompt after compaction
Compaction of a single-prompt (cron) session left no user message after
the handoff, so SUMMARY_PREFIX ordered the model to do nothing and the
scheduler recorded success. Re-append the in-flight task after the
summary.

Fixes #100818

(cherry picked from commit c0ea50ac02fd7553b1ef7deb0162483cb9a1b6ac)
2026-09-03 11:27:40 +05:30
Teknium 45a4672428 refactor(agent/context_compressor): split _build_summary_prompt into template/temporal helpers (prompt text byte-identical) 2026-09-02 22:35:45 -07:00
Teknium 29aa1bdf3d refactor(agent/context_compressor): share micro-cursor reset, breaker predicate, real-usage verdict; collapse session-state boilerplate 2026-09-02 22:34:33 -07:00
Teknium 081484e259 refactor(agent/context_compressor): unify tool-call field access, JSON-arg parsing, content rewrite; compact module helpers 2026-09-02 22:27:25 -07:00
Teknium c14ae92fea refactor(agent/context_compressor): fold derived-budget cache resets 2026-09-02 20:44:33 -07:00
Teknium bed463381f refactor(agent/context_compressor): class-qualify helper calls in _clear_compression_failure_cooldown (bare-stub binding) 2026-09-02 20:31:31 -07:00
Teknium 71bfa94ff9 refactor(agent/context_compressor): use the failure-kind dataclass directly; compact threshold/arg helpers 2026-09-02 20:23:47 -07:00
Teknium 932f7532ca refactor(agent/context_compressor): third AST-identical reflow pass at 118 cols 2026-09-02 20:19:10 -07:00
Teknium 5f7284e86e refactor(agent/context_compressor): repack historical-prefix and abort-message literals (values byte-identical) 2026-09-02 20:15:48 -07:00
Teknium 684fa87ad4 refactor(agent/context_compressor): fold parent-row rotation reads 2026-09-02 20:10:37 -07:00
Teknium 91cf271093 refactor(agent/context_compressor): second AST-identical reflow pass 2026-09-02 20:08:10 -07:00
Teknium 322f04f8fb refactor(agent/context_compressor): reflow docstrings/comments to width (every word preserved) 2026-09-02 20:04:00 -07:00
Teknium 321e8a79b0 refactor(agent/context_compressor): one newest-first token walk shared by tail-cut and prune boundary 2026-09-02 19:53:45 -07:00
Teknium d3b93c45c9 refactor(agent/context_compressor): flatten historical-media strip, summarizer part/tool-call rendering, orphan sanitizer 2026-09-02 19:49:48 -07:00
Teknium adbac330c7 refactor(agent/context_compressor): repack implicitly-concatenated string literals (values byte-identical) 2026-09-02 19:46:07 -07:00
Teknium 7b8d547ffc refactor(agent/context_compressor): normalize blank lines (ruff E302/E303/E305) 2026-09-02 19:43:18 -07:00
Teknium 939305f186 refactor(agent/context_compressor): pack multi-line def signatures (AST-identical) 2026-09-02 19:42:06 -07:00
Teknium b233420b3d refactor(agent/context_compressor): shared real-user index helper, folded guard ladders, telemetry loop cleanup 2026-09-02 19:39:24 -07:00
Teknium 1d7fb737cd refactor(agent/context_compressor): prompt scaffolding helpers, shared real-usage reset, compact text/length helpers 2026-09-02 19:25:44 -07:00
Teknium 5f716407b9 refactor(agent/context_compressor): move per-section summarizer instructions into a keyed table (text byte-identical) 2026-09-02 19:21:58 -07:00
Teknium a7e616ee5f refactor(agent/context_compressor): flatten _scan_window_handoffs around an early return 2026-09-02 19:17:16 -07:00
Teknium e468577b13 refactor(agent/context_compressor): shared content-part text helpers for handoff projection/unwrapping 2026-09-02 19:13:23 -07:00
Teknium 9a89ce5548 refactor(agent/context_compressor): pack short literal collections and call arguments (AST-identical) 2026-09-02 19:08:33 -07:00
Teknium 34f96ff29c refactor(agent/context_compressor): reflow short multi-line call statements 2026-09-02 19:01:32 -07:00
Teknium 6fe54ec6d7 refactor(agent/context_compressor): AST-identical reflow of short multi-line statements and headers 2026-09-02 19:00:02 -07:00
Teknium 6549a8bfb6 refactor(agent/context_compressor): extract _call_summary_llm and a summary-failure classifier dataclass 2026-09-02 18:51:31 -07:00
Teknium f00024ecc8 refactor(agent/context_compressor): lift static-fallback helpers to module level, extract anchor collection 2026-09-02 18:48:29 -07:00
Teknium 83168ff4e4 refactor(agent/context_compressor): split _prune_old_tool_results into boundary/dedupe/demote/pressure helpers 2026-09-02 18:45:36 -07:00
Teknium 608e8560fe refactor(agent/context_compressor): split compress() into phase helpers (494 -> ~130 LOC) 2026-09-02 18:33:40 -07:00
Teknium 6bbdb45df1 refactor(agent/context_compressor): unify durable read/write/load helpers and cooldown persistence paths 2026-09-02 18:27:34 -07:00
Teknium 806d2c3274 refactor(agent/context_compressor): dedupe image-part stripping, summarizer one-liners into dispatch table, alias coerce helper 2026-09-02 18:24:29 -07:00
Christopher cf86c47624 fix(agent): evict stale screenshot payloads before send
Call the existing keep-newest vision retirement on the per-call
api_messages clone after sanitization so OpenAI-style tool-result
screenshots are not re-uploaded on every later turn.
2026-09-03 04:51:51 +05:30
Teknium b3e41128ff refactor(agent): share predicates, truncation and the timeout ladder
- User-turn predicates reuse _is_context_summary_message (single detection path).
- _serialize_for_summary hoists the shared head/tail truncation.
- __init__ initialises per-session state via _reset_session_compaction_state.
- Timeout cooldown ladder (60/300/900s) is one module helper used by both
  record_timeout_failure and the summary failure path; module-level so a real
  method bound onto a stub compressor still exercises it.
2026-09-02 13:29:38 -07:00
Teknium b2717f4f4c refactor(agent): restore rationale comments lost in the docstring compaction
Hunk-by-hunk audit of the compaction commit found 14 dropped WHY/invariant
statements (route-pin scope, persistence-marker sweep failure mode, stamp
asymmetry, reasoning-replay boundary, size-class parity with preflight,
small-context floor cap, tail-anchor ordering rules, defer-baseline backstops).
Each is restored as a compact 1-3 line comment at its anchor.
2026-09-02 13:29:38 -07:00
Teknium ea090ccb62 refactor(agent): unify durable-state helpers, session reset and tail-cut walks
- Nine near-identical durable-row getters/setters -> _durable_read/_durable_write
  (also used for parent-lineage lookups in on_session_start).
- on_session_reset/on_session_end share _reset_session_compaction_state.
- _find_tail_cut_by_tokens: the two backward token walks become one _walk_back
  (randomized parity vs. the previous version: 1500 transcripts identical).
- _strip_context_summary_handoff_message: one _unwrapped() helper for all paths
  (parity: 1296 content shapes identical).
- ruff SIM102/SIM108/SIM114 collapses.
2026-09-02 13:29:38 -07:00
Teknium 4a64d42f9b refactor(agent): extract micro-compaction into agent/micro_compaction.py
The 15 rolling micro-compaction methods (cursor resolution, exchange
serialization, one-exchange summarization, defrag, splice, DB sync, telemetry)
move verbatim into MicroCompactionMixin, mixed into ContextCompressor
(MRO: ContextCompressor -> MicroCompactionMixin -> ContextEngine). Origin
constants and call_llm are imported lazily so tests patching
agent.context_compressor.X still apply; logger name parity is kept.
2026-09-02 13:29:38 -07:00
Teknium 64576a34f8 refactor(agent): decompose _generate_summary and the compress() abort path
- _generate_summary(): prompt assembly lifted into _build_summary_prompt, the
  exception path into _on_summary_failure; the two identical main-model retry
  branches are merged behind a reason lookup.
- compress(): terminal summary-failure abort classes are table-driven via
  _TERMINAL_SUMMARY_FAILURES (same precedence, log text and telemetry class;
  flag reads stay lazy so __new__-built test doubles keep working).
2026-09-02 13:29:38 -07:00