Sora-bluesky
d61411d131
Review (Sol xhigh) on the prior commit found a P1: the generic strip-
and-retry fallback ("any non-retryable 400 with image parts present
strips and retries") was too blunt. It couldn't tell an actual
image-corruption 400 apart from an unrelated one — bad tool schema,
unsupported parameter, billing, content policy — that merely happened
to carry image parts in the request. Any of those would silently erase
vision history and retry the still-invalid request, degrading sessions
that were never bricked in the first place. That's worse than the bug
it was meant to fix.
Revert the generic fallback (agent/conversation_loop.py). Keep only
the classifier-routed path: FailoverReason.image_corrupt +
_IMAGE_CORRUPT_PATTERNS, checked before _IMAGE_TOO_LARGE_PATTERNS
because shrinking corrupt bytes can't repair them. Corrupt-image
wordings still route to strip-and-retry; everything else falls through
to normal (non-retryable) handling as before. Add xAI's second wire
wording for the same corruption class ("base64 string of provided
image cannot be decoded", returned on unaligned truncation vs "Invalid
PNG image." on aligned truncation) and a compound-message test pinning
that image_corrupt wins when a body matches both pattern lists.
Drop TurnRetryState.stripped_images_this_turn. It's unnecessary now
that only one branch is left: the branch already only retries when
_strip_images_from_messages reports it removed something, and that
helper strips every image part from the request in one pass — so a
second corrupt-image hit on the retried (now text-only) request has
nothing left to strip and falls through on its own. No separate
one-shot flag needed.
Add a run_conversation integration test at the sequenced-provider
layer: corrupt 400 on attempt 1, strip, retry succeeds on attempt 2,
with explicit before/after assertions on the outgoing image_url part.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: paultaki <paultaki@users.noreply.github.com>
94 lines
4.9 KiB
Python
94 lines
4.9 KiB
Python
"""Per-attempt recovery bookkeeping for the conversation turn loop.
|
|
|
|
The inner retry loop in ``run_conversation`` (``while retry_count <
|
|
max_retries``) makes several distinct recovery attempts on a single model API
|
|
call: a credential-pool 429 retry, a per-provider OAuth refresh (codex,
|
|
anthropic, nous, copilot), a long-context compression restart, a length-
|
|
continuation restart, and a handful of format-recovery branches (thinking-
|
|
signature stripping, multimodal-tool-content stripping, llama.cpp grammar
|
|
fallback, image shrink, invalid-encrypted-content, 1M-beta header).
|
|
|
|
Each of those branches is guarded by a one-shot boolean so it fires at most
|
|
once per attempt. They used to be ~16 bare ``*_attempted`` / ``has_retried_*``
|
|
/ ``restart_with_*`` locals declared inline before the loop and threaded
|
|
through its 2,400-line body. ``TurnRetryState`` collapses them into one object
|
|
the loop mutates in place (``state.codex_auth_retry_attempted = True``), giving
|
|
the recovery bookkeeping a single named, testable home.
|
|
|
|
Loop-control variables (``retry_count``, ``max_retries``,
|
|
``max_compression_attempts``) intentionally stay as plain locals — they are the
|
|
``while`` mechanics, not recovery bookkeeping, and putting them on the object
|
|
would add indirection without clarifying anything.
|
|
|
|
This module is dependency-free so it can be unit-tested in isolation and
|
|
imported by the turn loop without an import cycle.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from dataclasses import dataclass, fields
|
|
|
|
|
|
@dataclass
|
|
class TurnRetryState:
|
|
"""One-shot recovery guards + restart signals for a single API-call attempt.
|
|
|
|
A fresh instance is created for each iteration of the outer turn loop
|
|
(once per ``api_call_count``). Each guard fires its recovery branch at most
|
|
once; the ``restart_with_*`` signals are read by the loop after the attempt
|
|
to decide whether to rebuild the request and retry.
|
|
"""
|
|
|
|
# ── Per-provider OAuth / credential refresh guards ───────────────────
|
|
codex_auth_retry_attempted: bool = False
|
|
anthropic_auth_retry_attempted: bool = False
|
|
nous_auth_retry_attempted: bool = False
|
|
nous_paid_entitlement_refresh_attempted: bool = False
|
|
copilot_auth_retry_attempted: bool = False
|
|
# Copilot surfaces a stale/degraded credential as a 400
|
|
# ``model_not_available_for_integrator`` / ``model_not_supported`` instead
|
|
# of a clean 401 (e.g. a raw OAuth token seeded when the token exchange
|
|
# degraded at startup, routing the request to the restricted
|
|
# ``copilot-language-server`` integrator). Guard a single-shot forced
|
|
# re-exchange + client rebuild for that case, separate from the 401 guard
|
|
# so both can fire within one attempt if needed.
|
|
copilot_stale_cred_retry_attempted: bool = False
|
|
vertex_auth_retry_attempted: bool = False
|
|
|
|
# ── Format / payload recovery guards ─────────────────────────────────
|
|
thinking_sig_retry_attempted: bool = False
|
|
invalid_encrypted_content_retry_attempted: bool = False
|
|
native_compaction_reject_retry_attempted: bool = False
|
|
image_shrink_retry_attempted: bool = False
|
|
multimodal_tool_content_retry_attempted: bool = False
|
|
oauth_1m_beta_retry_attempted: bool = False
|
|
llama_cpp_grammar_retry_attempted: bool = False
|
|
|
|
# ── Transport / rate-limit recovery ──────────────────────────────────
|
|
primary_recovery_attempted: bool = False
|
|
has_retried_429: bool = False
|
|
|
|
# ── Auth-failure provider failover ───────────────────────────────────
|
|
# Set once we've escalated a persistent 401/403 (after the per-provider
|
|
# credential-refresh attempt above failed) to the fallback chain, so we
|
|
# don't loop on the same auth failover within one attempt.
|
|
auth_failover_attempted: bool = False
|
|
|
|
# ── Restart signals (read by the outer loop after the attempt) ───────
|
|
restart_with_compressed_messages: bool = False
|
|
restart_with_length_continuation: bool = False
|
|
# Set when a content-filter stream stall (e.g. MiniMax "new_sensitive")
|
|
# has been escalated to the fallback chain: the partial-stream content
|
|
# was rolled back off ``messages`` and the loop should re-issue the API
|
|
# call against the newly-activated provider (#32421).
|
|
restart_with_rebuilt_messages: bool = False
|
|
# A user correction cancelled the in-flight provider request. The outer
|
|
# loop must append a role-safe checkpoint + user message, rebuild the API
|
|
# payload, and retry the same logical iteration.
|
|
restart_with_redirected_messages: bool = False
|
|
|
|
def __iter__(self):
|
|
# Convenience for debugging / tests: iterate (name, value) pairs.
|
|
for f in fields(self):
|
|
yield f.name, getattr(self, f.name)
|