fc00e36c6b
A `custom_providers` entry can carry an `extra_body` (e.g.
`chat_template_kwargs` to toggle a local vLLM model's thinking).
`resolve_runtime_provider()` correctly surfaces it as `request_overrides`
on the resolved runtime dict, but the gateway never plumbed it through to
the per-turn agent:
- `_resolve_runtime_agent_kwargs()` rebuilt the runtime dict from a fixed
key whitelist that omitted `request_overrides`.
- `_resolve_turn_agent_config()` rebuilt `runtime` from the same whitelist
and set `route["request_overrides"]` solely from `/fast` service-tier
overrides (`{}` otherwise).
- The per-turn `agent.request_overrides = turn_route.get(...)` assignment
then clobbered the value `_merge_custom_provider_extra_body()` applied at
agent construction.
Net: on the gateway, a custom provider's configured `extra_body` never
reached the model -- only `/fast` overrides survived. The CLI/TUI path
(which does not go through `_resolve_turn_agent_config`) and the auxiliary
client (which sends `extra_body` directly) were unaffected.
Fix: carry `request_overrides` through the runtime resolvers
(`_resolve_runtime_agent_kwargs`, `_try_resolve_fallback_provider`) and
merge the provider overrides into the per-turn route, layering any `/fast`
service-tier overrides on top (top-level keys, no collision with
`extra_body`).
Adds tests/gateway/test_turn_request_overrides.py.
Known follow-up: the mid-session `/model`-switch override path
(`_session_model_overrides` / `ModelSwitchResult`) does not yet carry
`request_overrides`.