fix(agent): stop the between-turns tool refresh from forking the cached prefix

The per-turn MCP refresh re-derives `agent.tools` from live availability and
publishes the result wholesale. Two kinds of bytes move as a result:

* a tool whose `check_fn` merely flapped (headless browser probe, expired
  credential, docker blip) disappears from the array, and
* a late-landing MCP tool splices into sorted position, which can be index 0.

Providers that render `tools` ahead of the messages re-prefill the entire
history behind any moved byte, so either case costs a full re-prefill of the
session — the measured 2% cache hit in #100336. The caller's own comment
claimed the refresh "only ever extends a fresh request prefix"; it did not.

`refresh_agent_mcp_tools(..., preserve_prefix=True)` makes that claim true.
The live order becomes authoritative: existing tools keep their slot (fresh
schemas still land), a tool that is still registered but momentarily
unavailable is carried forward, a tool that genuinely left the registry is
still dropped, and new tools are appended at the tail. Explicit `/reload-mcp`
and the compaction boundary keep the plain rebuild.

Refs #100336
This commit is contained in:
joaomarcos
2026-09-01 16:28:28 -03:00
committed by Teknium
parent 2e25b47210
commit 65b0f00002
3 changed files with 145 additions and 11 deletions
+14 -7
View File
@@ -635,13 +635,18 @@ def build_turn_context(
# Between-turns MCP refresh: an MCP server that finished connecting since
# the previous turn (slow HTTP/OAuth servers routinely take 2-6s on a cold
# connect, missing the bounded startup wait) lands in THIS turn's tool
# snapshot. This is cache-safe by construction: it runs in the per-turn
# snapshot. Timing is cache-safe by construction: it runs in the per-turn
# prologue, before this turn's first API call assembles ``tools=``, so it
# only ever extends a fresh request prefix — it never mutates the cached
# prefix of an in-flight turn. No-op when no MCP servers are registered
# (the common case, gated by the cheap ``has_registered_mcp_tools`` check)
# or when the tool set is unchanged (``refresh_agent_mcp_tools`` diffs by
# name and leaves the snapshot untouched on no-change).
# never mutates the prefix of an in-flight turn. ``preserve_prefix`` makes
# the *content* cache-safe too (#100336): a plain rebuild re-derives the
# array from live availability, so a flapping ``check_fn`` silently drops a
# tool and a late arrival splices into sorted position — either one forks
# the tool block and re-prefills the whole history behind it, every turn it
# happens. With the flag the live order is authoritative and the array
# only ever grows. No-op when no MCP servers are registered (the common
# case, gated by the cheap ``has_registered_mcp_tools`` check) or when the
# tool set is unchanged (``refresh_agent_mcp_tools`` diffs by name and
# leaves the snapshot untouched on no-change).
try:
if not getattr(agent, "_skip_mcp_refresh", False):
# Import-cost gate: ``tools.mcp_tool`` pulls in the whole ``mcp``
@@ -656,7 +661,9 @@ def build_turn_context(
if "tools.mcp_tool" in _sys.modules:
from tools.mcp_tool import has_registered_mcp_tools, refresh_agent_mcp_tools
if has_registered_mcp_tools():
refresh_agent_mcp_tools(agent, quiet_mode=True)
refresh_agent_mcp_tools(
agent, quiet_mode=True, preserve_prefix=True,
)
except Exception:
logger.debug("between-turns MCP tool refresh skipped", exc_info=True)