Commit Graph

4 Commits

Author SHA1 Message Date
Teknium 0913885250 fix(delegation): summary headroom uses the aggregator's own prompt size; unknown usage means the static ceiling, never zero context
Independent review found two holes in the first fix. A parent with no usage
row yet was treated as 0 tokens used, so a 190K/200K prompt received a
384K-char dynamic summary budget instead of ~4K; the budget now returns None
(static ceiling only) when nothing is known. And under MoA the folded usage
includes advisor prompts that are not in the parent's context, over-stating
the prompt size and wrongly truncating summaries; turn_usage now records the
aggregator's pre-fold prompt_tokens as _last_prompt_size_tokens and the
budget reads that first.

Tests (2 new): unknown usage -> None; MoA-folded and unfolded parents with
the same real prompt get the same budget.
2026-09-05 06:18:41 -07:00
Teknium 903b9bf187 fix(delegate): nested orchestrators get their workers' results back — no 420 s deadline on delegate_task, summary budget uses the current prompt not the session sum
Two defects in the same path, both measured on the 1,393-agent refactor run.

1. A nested orchestrator (depth > 0) runs delegate_task synchronously by
   design: it needs its workers' results inside its own turn. But the
   sequential tool runner put every tool call under the generic 420 s
   deadline, and delegate_task was not exempt, so every batch longer than
   seven minutes returned "timed out after 420.0s" while the children kept
   running as orphans. 332 such timeouts in 234 orchestrator sessions; only
   89 nested delegate_task calls in the whole run ever returned a real result.
   The orchestrators then spent 388 h of wall time polling: 1,526 reads of
   the live transcript files, 551 list actions, 242 h of explicit sleep,
   about $4k of API turns. delegate_task is now exempt from the sequential
   deadline (the batch owns its liveness: per-child heartbeats, the stale
   monitor, delegation.child_timeout_seconds).

   Live A/B, depth-1 orchestrator dispatching a 75 s leaf with the deadline
   set to 40 s (glm-5.3-flash via Nous): main -> "Error executing tool
   'delegate_task': timed out after 40.0s", leaf result lost; branch ->
   orchestrator blocked 161 s and returned the leaf's LEAF_DONE_MARKER.

2. _parent_summary_char_budget computed the parent's context headroom from
   session_prompt_tokens, which is the running SUM of prompt tokens over
   every API call in the session. After a few hundred calls it exceeds any
   window, headroom goes negative, and every child summary collapses to the
   2,000-char floor with the full text spilled to disk. All 1,393 child
   summaries in the run were truncated this way; the orchestrator planned
   from stubs. The budget now reads the last call's prompt_tokens from
   _last_turn_usage.

Tests: delegate_task is in the exempt set and the set is narrow; budget for
a long-lived parent equals the budget for a fresh parent with the same
current prompt, and exceeds the floor.
2026-09-05 00:10:26 -07:00
Teknium 39975613b1 test: prune wave 2 + speed fixes — 28,106 → 19,757 test functions, suite wall 315s → 294s
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.

Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
  (DEFAULT_CONFIG smart-approval leaked in) — pinned approval
  mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
  silently probed live provider catalogs (~2s/test) — stubbed
  cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
  shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
  MagicMock agents — interrupt.side_effect now clears _running_agents
  (22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
  (24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
  patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
  5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
  with rationale comment — the watchdog-dump assertion needs the loop
  blocked well past deadline+grace under parallel load (flaked once in
  the 40-worker verification run at a 0.2s margin).

Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
2026-07-29 13:39:40 -07:00
teknium1 35a0803a3b fix(delegation): budget subagent summaries against parent context headroom
Batch delegation returned each subagent's full final_response verbatim
into the parent's context. A fan-out of N children could dump 60k+ tokens
at once, blowing the parent's context window and — on rate-limited
providers — triggering a compression/429 death spiral (429 misread as
context-too-large -> window step-down -> retry loop -> conversation dies).

Cap each summary against the parent's *remaining* context headroom split
across the batch (not a magic char count). When trimming, mirror the
web_extract convention: spill the full text to cache/delegation (mounted
into remote backends via credential_files._CACHE_DIRS) and return a
head+tail window (75/25, line-snapped) plus a footer with the exact
read_file offset to page the omitted middle. Both the subagent's opening
AND its closing (outcomes / files-changed / issues, which live at the end)
survive in-context, and nothing is lost — the parent can read_file the
full version on any backend.

delegation.max_summary_chars (default 24000) is a static ceiling layered
on top as belt-and-suspenders for models that ignore 'be concise'; 0
disables it. Child prompt tightened to lead with outcomes / bullets.

Co-authored-by: rc-int <rcint@klaith.com>
2026-06-30 03:07:40 -07:00