Commit Graph

4 Commits

Author SHA1 Message Date
Teknium 6b81590c55 test: prune low-value tests suite-wide (wave 1) — 46,820 → 28,106 test functions
Systematic prune per AGENTS.md test policy, one pass over every major
test tree (gateway, hermes_cli, tools, agent, run_agent, plugins, cli,
cron, tui_gateway, honcho/openviking, root-level):

- DELETE: source-reading tests (read_text/getsource on prod files),
  change-detector tests (exact catalog counts, model-name snapshots,
  config version literals), mock-echo tests (assert a mock returns what
  it was told), assertion-free/trivial tests, near-duplicate
  parametrizations (boundaries + one representative kept), async/sync
  twin duplicates, cosmetic within-file variations.
- KEEP (mandatory): security/redaction/approval guards, message-role
  alternation invariants, prompt-caching/deterministic-call-id
  invariants, issue-number regression tests (deduped), E2E tests.
- 6 test files deleted outright (script-style/no-assert or fully
  redundant); conftest.py, fakes/, fixtures/ untouched.
- tests/acp/conftest.py added: autouse fixture stubs the live
  models.dev/GitHub/Copilot/Anthropic inventory fetches that ACP server
  tests performed on every session create — test_server.py 147s → 3.4s,
  and the tests are now genuinely hermetic.
- Sleep-based slowness shrunk where safe (codex_ttfb_watchdog,
  compression_concurrent_fork, etc.); no wall-clock assertion tightened.

Verification: full hermetic suite via scripts/run_tests.sh —
2439 files, 31,130 tests passed, 0 failed, 0 flaky retries, 315s wall
(baseline: 583s wall, 13,564s subprocess CPU).
2026-07-29 13:10:23 -07:00
teknium1 5a0f51325c fix(agent): make dropped tool-call nudge pair ephemeral scaffolding
Review follow-up for the dropped tool-call recovery (#69630): the
re-prompt pair was tagged _dropped_toolcall_nudge, but that marker was
not part of the ephemeral-scaffolding contract. _persist_session /
_flush_messages_to_session_db would therefore write the synthetic
'issue the actual tool call now' user message (and the narration-only
interim assistant turn) as real transcript rows — a resumed session
could replay the internal retry instruction as user-authored context
and prompt unsolicited tool use.

- Add _dropped_toolcall_nudge to _EPHEMERAL_SCAFFOLDING_FLAGS
  (run_agent.py) so both SQLite and JSON persistence skip the pair.
- Add it to _SYNTHETIC_USER_FLAGS (conversation_compression.py) so the
  compressor never treats the nudge as human intent.
- Flag the interim assistant half of the pair too, and include the
  marker in the finalization scaffolding pop so a genuine turn end
  strips the pair from the live transcript (mirrors the
  _empty_recovery_synthetic pattern).
- Regression tests: flagged messages classify as ephemeral, the
  returned transcript contains no scaffolding, and the turn tail stays
  on the real assistant answer.
2026-07-24 15:56:39 -07:00
Jash Lee 923704c7c2 fix(agent): move dropped tool-call recovery to the finalization chokepoint
The initial fix guarded the no-tool-calls else branch, but that branch only
SETS final_response — the turn actually finalizes later, in a separate block
after final_msg is built. Runs that reached finalization via that path exited
without the guard ever running (observed live: a scheduled PR reviewer stalled
at tool_turns=1-2 with zero recovery nudges).

Move the recovery to the finalization chokepoint (right after final_msg is
built), so it catches every path that ends a turn. Single guard now:
- increments a consecutive-stall counter and re-prompts (bounded to 3),
- resets on any successful tool round, and
- resets on a genuine (non-mismatch) turn end,
so it guards each stall independently without capping the whole run and
without looping forever.

Verified live: the scheduled reviewer now recovers through the stalls and
submits real reviews — PR 57800 APPROVED, PR 54826 COMMENTED — one PR per run.
2026-07-24 15:56:39 -07:00
Jash Lee 63954d508c fix(agent): recover from dropped tool calls (finish_reason=tool_calls, empty array)
Some providers (observed: claude-opus-4.8 / claude-sonnet-4.5 on GitHub
Copilot, ~2026-07) return finish_reason="tool_calls" while the parsed
tool_calls array is empty — the model signalled it wanted to act but the
payload shipped no call. The conversation loop took the no-tool-calls
else branch, treated the turn's narration as the final answer, and exited
with the task unstarted.

On unattended multi-step jobs this is silent failure: a scheduled PR
reviewer, for example, would narrate "Let me verify the PR..." and stop
at tool_turns=0 every run, never submitting a review, while the job still
reported success.

Fix: in the no-tool-calls branch, detect the provider contract violation
(finish_reason == "tool_calls" with zero tool_calls) and re-prompt the
model to emit the call instead of exiting. Bounded to 3 consecutive
stalls; the budget resets after any successful tool round so it guards
each stall independently rather than capping the whole run. The narration
may live in content or only in the reasoning field (empty content) — the
guard keys on the finish_reason/tool_calls mismatch, so both are covered.
A genuine finish_reason="stop" text turn is unaffected.

Verified live: a scheduled reviewer that died at tool_turns=0 every run
now recovers through 20+ stalls per run and submits real reviews.

Tests: tests/run_agent/test_dropped_tool_call_recovery.py covers the
re-prompt, the empty-content case, the clean-stop control, and the
bounded-loop guard.
2026-07-24 15:56:39 -07:00