fix(gateway): merge instead of overwrite agent.request_overrides on reused turns

Preserve initialization-time request overrides (custom-provider extra_body
merged at agent construction) while replacing only the previous turn's
routing overrides during the per-turn agent refresh. This keeps
custom-provider extra_body settings without leaving stale fast-mode
service_tier or speed values on cached agents.

Reimplemented from PR #52432 at the code's current location (the per-turn
refresh moved into the TurnRunner path since the original patch), keeping
the original merge semantics: snapshot this turn's route overrides in
agent._gateway_turn_request_overrides, evict only unchanged previous-turn
keys, then layer the new turn overrides on top.

Salvaged-from: #52432
Co-authored-by: Heng Cai <abtion@outlook.com>
This commit is contained in:
Heng Cai
2026-08-29 18:33:48 -07:00
committed by Teknium
parent d2af990043
commit 2f469d7e1e
+17 -1
View File
@@ -5998,7 +5998,23 @@ class TurnRunner:
agent.event_callback = ctx._event_callback_sync
agent.reasoning_config = reasoning_config
agent.service_tier = self._runner._service_tier
agent.request_overrides = turn_route.get("request_overrides") or {}
# Merge, never overwrite: init-time request overrides (e.g. a custom
# provider's extra_body merged at agent construction) must survive
# every reused-agent turn. Drop only the PREVIOUS turn's routing
# overrides (fast-mode service_tier/speed) before layering this
# turn's route overrides on top, so stale per-turn values never
# linger while construction-time values persist.
request_overrides = dict(getattr(agent, "request_overrides", {}) or {})
previous_turn_overrides = dict(
getattr(agent, "_gateway_turn_request_overrides", {}) or {}
)
for key, value in previous_turn_overrides.items():
if request_overrides.get(key) == value:
request_overrides.pop(key, None)
turn_request_overrides = dict(turn_route.get("request_overrides") or {})
request_overrides.update(turn_request_overrides)
agent.request_overrides = request_overrides
agent._gateway_turn_request_overrides = turn_request_overrides
# Must-deliver notes for THIS turn ride the current user message
# (api_content sidecar), never the system prompt: staged by
# _handle_message_with_agent (auto-reset note, first-contact