fix(compression): provider-proven overflow gets one real compaction attempt while the failure cooldown is armed
After one failed/stalled summary attempt arms the 60/300/900s compression- failure cooldown, a provider context_length_exceeded rejection entered the reactive overflow branch in conversation_loop, which called _compress_context without force. Since #97488 the cooldown gate returns the soft "temporarily paused, retry in a moment" deferral instead of exhaustion, so every turn deferred until the cooldown lapsed, and the next failure extended the ladder: long-running sessions wedged with no automatic recovery (#100661, four sessions lost). Thread a narrow `bypass_cooldown` kwarg from the three provider-proven overflow call sites (generic overflow, 413, output-cap recovery) through AIAgent._compress_context -> compress_context -> ContextCompressor.compress -> _generate_summary. It skips ONLY the summary-failure cooldown check at each gate. Unlike force=True it does not clear the cooldown, does not skip the feasibility / anti-thrash breakers, and a failed attempt records its cooldown normally. The attempt is bounded by the existing compression_attempts/max_compression_attempts budget, so there is no retry loop. The preflight threshold gate is unchanged: ordinary over-threshold pressure still honors the cooldown (#11529). Engines whose _automatic_compression_blocked()/compress() predate the kwarg (plugins, test doubles) are called with the legacy signature. Tests: cooldown armed + bypass_cooldown -> summarizer invoked and transcript compacted; ordinary pass still deferred. Docs note the cooldown/overflow contract in the developer guide. Fixes #100661 Closes #97766 (overflow-force idea; the bundled continuation changes were not taken) Co-authored-by: sgtworkman <178342791+sgtworkman@users.noreply.github.com>
This commit is contained in:
@@ -3954,9 +3954,17 @@ class ContextCompressor(ContextEngine):
|
||||
except Exception as exc:
|
||||
logger.debug("compression ineffective-count refresh failed: %s", exc)
|
||||
|
||||
def _automatic_compression_blocked(self) -> bool:
|
||||
"""Return whether automatic compaction is in cooldown or tripped."""
|
||||
if not self._automatic_compression_blocked_locally():
|
||||
def _automatic_compression_blocked(self, *, ignore_cooldown: bool = False) -> bool:
|
||||
"""Return whether automatic compaction is in cooldown or tripped.
|
||||
|
||||
``ignore_cooldown=True`` evaluates only the breakers that are NOT the
|
||||
summary-failure cooldown. Used by provider-proven overflow recovery
|
||||
(#100661): the provider already rejected the request, so waiting out
|
||||
the cooldown just wedges the session — every turn defers and the next
|
||||
failure extends the ladder. The overflow path gets one real attempt;
|
||||
the ineffective/structural breakers still apply.
|
||||
"""
|
||||
if not self._automatic_compression_blocked_locally(ignore_cooldown=ignore_cooldown):
|
||||
return False
|
||||
# Blocked on the in-memory snapshot. Durable guard rows may have
|
||||
# been cleared by another agent since bind_session_state() — a
|
||||
@@ -3966,9 +3974,9 @@ class ContextCompressor(ContextEngine):
|
||||
# local block outlive the durable state that justified it. The
|
||||
# unblocked hot path above never pays for the DB reads.
|
||||
self._refresh_durable_guards()
|
||||
return self._automatic_compression_blocked_locally()
|
||||
return self._automatic_compression_blocked_locally(ignore_cooldown=ignore_cooldown)
|
||||
|
||||
def _automatic_compression_blocked_locally(self) -> bool:
|
||||
def _automatic_compression_blocked_locally(self, *, ignore_cooldown: bool = False) -> bool:
|
||||
"""Evaluate the automatic-compaction gate on in-memory state only."""
|
||||
# Do not trigger compression while the summary LLM is in cooldown.
|
||||
# On a 429/transient failure _generate_summary() sets a cooldown and
|
||||
@@ -3980,7 +3988,7 @@ class ContextCompressor(ContextEngine):
|
||||
# force=True, which clears this cooldown in compress() before running,
|
||||
# so it still retries immediately.
|
||||
_cooldown_remaining = self._summary_failure_cooldown_until - time.monotonic()
|
||||
if _cooldown_remaining > 0:
|
||||
if _cooldown_remaining > 0 and not ignore_cooldown:
|
||||
if not self.quiet_mode:
|
||||
logger.debug(
|
||||
"Compression deferred — summary LLM in cooldown for %.0fs more",
|
||||
@@ -5013,6 +5021,7 @@ Summary generation was unavailable, so this is a best-effort deterministic fallb
|
||||
turns_to_summarize: List[Dict[str, Any]],
|
||||
focus_topic: Optional[str] = None,
|
||||
memory_context: str = "",
|
||||
bypass_cooldown: bool = False,
|
||||
) -> Optional[str]:
|
||||
"""Generate a structured summary of conversation turns.
|
||||
|
||||
@@ -5035,7 +5044,10 @@ Summary generation was unavailable, so this is a best-effort deterministic fallb
|
||||
if self._compression_cancelled():
|
||||
raise AuxiliaryExplicitCancellation()
|
||||
now = prompt_started_at
|
||||
if now < self._summary_failure_cooldown_until:
|
||||
# bypass_cooldown (#100661): provider-proven overflow gets ONE real
|
||||
# summary attempt while the cooldown is armed; a failure below still
|
||||
# records/extends the cooldown normally.
|
||||
if now < self._summary_failure_cooldown_until and not bypass_cooldown:
|
||||
logger.debug(
|
||||
"Skipping context summary during cooldown (%.0fs remaining)",
|
||||
self._summary_failure_cooldown_until - now,
|
||||
@@ -7729,6 +7741,7 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
focus_topic: Optional[str] = None,
|
||||
force: bool = False,
|
||||
memory_context: str = "",
|
||||
bypass_cooldown: bool = False,
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Compress conversation messages by summarizing middle turns.
|
||||
|
||||
@@ -7765,6 +7778,10 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
summary path. Auto-compress callers pass False.
|
||||
memory_context: Optional provider-supplied context to preserve in
|
||||
the summary prompt. Whitespace-only values are ignored.
|
||||
bypass_cooldown: If True, run the summary LLM even while the
|
||||
summary-failure cooldown is armed, WITHOUT clearing it
|
||||
(#100661). Set by provider-proven overflow recovery, which
|
||||
is already bounded by the caller's attempt budget.
|
||||
"""
|
||||
# Reset per-call summary failure state — callers inspect these fields
|
||||
# after compress() returns to decide whether to surface a warning.
|
||||
@@ -8102,6 +8119,7 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
turns_to_summarize,
|
||||
focus_topic=summary_focus_topic,
|
||||
memory_context=memory_context,
|
||||
bypass_cooldown=bypass_cooldown,
|
||||
)
|
||||
except AuxiliaryExplicitCancellation:
|
||||
# Explicit cancellation is a true no-op. Restore state mutated by
|
||||
|
||||
@@ -2012,6 +2012,25 @@ def context_compression_timed_out(agent: Any) -> bool:
|
||||
return getattr(agent, "_last_compression_timed_out", None) is True
|
||||
|
||||
|
||||
def _automatic_gate_blocked(
|
||||
blocked: Any, compressor: Any, bypass_cooldown: bool
|
||||
) -> bool:
|
||||
"""Evaluate the automatic breaker gate, optionally ignoring the cooldown.
|
||||
|
||||
Provider-proven overflow recovery (#100661) passes ``bypass_cooldown``;
|
||||
engines whose gate predates the kwarg (plugins, test doubles) are called
|
||||
with the legacy no-argument shape.
|
||||
"""
|
||||
if bypass_cooldown:
|
||||
try:
|
||||
accepts = "ignore_cooldown" in inspect.signature(blocked).parameters
|
||||
except (TypeError, ValueError):
|
||||
accepts = False
|
||||
if accepts:
|
||||
return bool(blocked(compressor, ignore_cooldown=True))
|
||||
return bool(blocked(compressor))
|
||||
|
||||
|
||||
def compression_blocked_transiently(agent: Any) -> bool:
|
||||
"""Type-pinned read of the transient-block signal (#97488).
|
||||
|
||||
@@ -2248,6 +2267,7 @@ def _supported_compression_kwargs(
|
||||
focus_topic: Optional[str],
|
||||
force: bool,
|
||||
memory_context: str,
|
||||
bypass_cooldown: bool = False,
|
||||
) -> dict:
|
||||
"""Return only compression kwargs accepted by an engine callable.
|
||||
|
||||
@@ -2261,6 +2281,8 @@ def _supported_compression_kwargs(
|
||||
"focus_topic": focus_topic,
|
||||
"force": force,
|
||||
}
|
||||
if bypass_cooldown:
|
||||
candidates["bypass_cooldown"] = True
|
||||
if memory_context:
|
||||
candidates["memory_context"] = memory_context
|
||||
try:
|
||||
@@ -3289,6 +3311,7 @@ def compress_context(
|
||||
task_id: str = "default",
|
||||
focus_topic: Optional[str] = None,
|
||||
force: bool = False,
|
||||
bypass_cooldown: bool = False,
|
||||
defer_context_engine_notification: bool = False,
|
||||
commit_fence: Optional[CompressionCommitFence] = None,
|
||||
) -> Tuple[list, str]:
|
||||
@@ -3308,6 +3331,13 @@ def compress_context(
|
||||
by the manual ``/compress`` slash command so users can retry
|
||||
immediately after an auto-compress abort. Auto-compress
|
||||
callers use the default ``False``.
|
||||
bypass_cooldown: If True, the automatic breaker gates ignore ONLY the
|
||||
summary-failure cooldown for this attempt (#100661). Set by the
|
||||
provider-proven overflow recovery path: the provider already
|
||||
rejected the request, so deferring until the cooldown lapses
|
||||
wedges the session. Unlike ``force`` it does not clear the
|
||||
cooldown, and the ineffective/structural breakers still apply;
|
||||
a failed attempt records its cooldown normally.
|
||||
defer_context_engine_notification: Delay the existing context-engine
|
||||
hook until a manual host commits its outer history transaction.
|
||||
commit_fence: Optional cooperative fence for executor callers that
|
||||
@@ -3425,7 +3455,9 @@ def compress_context(
|
||||
"_automatic_compression_blocked",
|
||||
None,
|
||||
)
|
||||
if callable(blocked) and blocked(agent.context_compressor):
|
||||
if callable(blocked) and _automatic_gate_blocked(
|
||||
blocked, agent.context_compressor, bypass_cooldown
|
||||
):
|
||||
_mark_compression_blocked_transient(agent, agent.context_compressor)
|
||||
existing_prompt = getattr(agent, "_cached_system_prompt", None)
|
||||
if not existing_prompt:
|
||||
@@ -3896,7 +3928,9 @@ def compress_context(
|
||||
"_automatic_compression_blocked",
|
||||
None,
|
||||
)
|
||||
if callable(blocked) and blocked(compressor):
|
||||
if callable(blocked) and _automatic_gate_blocked(
|
||||
blocked, compressor, bypass_cooldown
|
||||
):
|
||||
_mark_compression_blocked_transient(agent, compressor)
|
||||
_release_lock()
|
||||
existing_prompt = getattr(agent, "_cached_system_prompt", None)
|
||||
@@ -4093,6 +4127,7 @@ def compress_context(
|
||||
focus_topic=focus_topic,
|
||||
force=force,
|
||||
memory_context=memory_context,
|
||||
bypass_cooldown=bypass_cooldown,
|
||||
)
|
||||
if memory_context.strip() and "memory_context" not in compress_kwargs:
|
||||
engine_name = getattr(
|
||||
|
||||
@@ -6171,6 +6171,11 @@ def run_conversation(
|
||||
messages, system_message,
|
||||
approx_tokens=estimate_request_tokens_rough(api_messages, tools=agent.tools or None),
|
||||
task_id=effective_task_id,
|
||||
# #100661: the provider proved the request does not fit.
|
||||
# Ignore the summary-failure cooldown for this ONE
|
||||
# attempt (bounded by max_compression_attempts) instead
|
||||
# of deferring every turn until the ladder lapses.
|
||||
bypass_cooldown=True,
|
||||
)
|
||||
if messages is _overflow_input and compression_skipped_due_to_lock(agent):
|
||||
# #69870 lock-skip: the provider proved the request
|
||||
@@ -6347,6 +6352,7 @@ def run_conversation(
|
||||
messages, system_message,
|
||||
approx_tokens=request_input_estimate,
|
||||
task_id=effective_task_id,
|
||||
bypass_cooldown=True, # #100661 provider-proven overflow
|
||||
)
|
||||
if messages is _overflow_input and compression_skipped_due_to_lock(agent):
|
||||
compression_attempts -= 1
|
||||
@@ -6510,6 +6516,11 @@ def run_conversation(
|
||||
messages, system_message,
|
||||
approx_tokens=estimate_request_tokens_rough(api_messages, tools=agent.tools or None),
|
||||
task_id=effective_task_id,
|
||||
# #100661: the provider proved the request does not fit.
|
||||
# Ignore the summary-failure cooldown for this ONE
|
||||
# attempt (bounded by max_compression_attempts) instead
|
||||
# of deferring every turn until the ladder lapses.
|
||||
bypass_cooldown=True,
|
||||
)
|
||||
if messages is _overflow_input and compression_skipped_due_to_lock(agent):
|
||||
# #69870 lock-skip: the provider proved the request
|
||||
|
||||
+5
-1
@@ -8546,6 +8546,7 @@ class AIAgent:
|
||||
task_id: str = "default",
|
||||
focus_topic: str = None,
|
||||
force: bool = False,
|
||||
bypass_cooldown: bool = False,
|
||||
defer_context_engine_notification: bool = False,
|
||||
commit_fence=None,
|
||||
) -> tuple:
|
||||
@@ -8554,7 +8555,9 @@ class AIAgent:
|
||||
``force=True`` is passed by the manual ``/compress`` slash command
|
||||
so users can bypass the summary-failure cooldown after an
|
||||
auto-compress abort. Auto-compress callers use the default
|
||||
``force=False``.
|
||||
``force=False``. ``bypass_cooldown=True`` is passed by the
|
||||
provider-proven overflow recovery path so one real attempt runs while
|
||||
the cooldown is armed (#100661) — without clearing it.
|
||||
"""
|
||||
# Per-attempt signal consumed by turn-start preflight (#98424) and the
|
||||
# in-loop pre-API/overflow consumers. A stalled compression must not
|
||||
@@ -8635,6 +8638,7 @@ class AIAgent:
|
||||
approx_tokens=approx_tokens, task_id=task_id,
|
||||
focus_topic=focus_topic,
|
||||
force=force,
|
||||
bypass_cooldown=bypass_cooldown,
|
||||
defer_context_engine_notification=(
|
||||
defer_context_engine_notification
|
||||
),
|
||||
|
||||
@@ -287,3 +287,73 @@ class TestTransientBlockIsNotExhaustion:
|
||||
mock_agent = MagicMock()
|
||||
# MagicMock auto-attributes are truthy but not str.
|
||||
assert compression_blocked_transiently(mock_agent) is False
|
||||
|
||||
|
||||
def _summary_response(content: str):
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
response = MagicMock()
|
||||
response.choices = [MagicMock()]
|
||||
response.choices[0].message.content = content
|
||||
return response
|
||||
|
||||
|
||||
class TestProviderOverflowBypassesCooldown:
|
||||
"""#100661: a provider-proven overflow must get one REAL summary attempt
|
||||
while the summary-failure cooldown is armed. Before the fix every turn of
|
||||
a wedged session hit the cooldown gate, returned the soft "temporarily
|
||||
paused" deferral, and the next failure extended the ladder — 4 long
|
||||
sessions were lost this way. Ordinary (non-overflow) automatic passes
|
||||
must still defer."""
|
||||
|
||||
def _armed_agent(self, tmp_path: Path, session_id: str):
|
||||
db, agent = _build_agent(tmp_path, session_id)
|
||||
# Realistic arming: a failed/stalled attempt recorded the ladder.
|
||||
agent.context_compressor.record_timeout_failure(
|
||||
"stall", failure_kind="stalled"
|
||||
)
|
||||
assert agent.context_compressor.should_compress_info(500_000)[0] is False
|
||||
return db, agent
|
||||
|
||||
def test_overflow_attempt_invokes_summarizer_while_cooldown_armed(
|
||||
self, tmp_path: Path
|
||||
):
|
||||
db, agent = self._armed_agent(tmp_path, "OVERFLOW_BYPASS")
|
||||
calls = []
|
||||
|
||||
def fake_call_llm(**kwargs):
|
||||
calls.append(kwargs)
|
||||
return _summary_response("## Goal\nRecovered after overflow.")
|
||||
|
||||
# Bulky turns so the compacted transcript is genuinely smaller.
|
||||
live = [
|
||||
{"role": "user" if i % 2 == 0 else "assistant", "content": f"m{i} " * 400}
|
||||
for i in range(20)
|
||||
]
|
||||
with patch("agent.context_compressor.call_llm", fake_call_llm):
|
||||
out, _ = compress_context(
|
||||
agent, live, "sys", approx_tokens=500_000, bypass_cooldown=True
|
||||
)
|
||||
assert len(calls) == 1, (
|
||||
"provider-proven overflow must reach the summary LLM even while "
|
||||
"the failure cooldown is armed (#100661)"
|
||||
)
|
||||
assert compression_blocked_transiently(agent) is False
|
||||
assert len(out) < len(live), "the attempt must actually compact"
|
||||
|
||||
def test_non_overflow_pass_still_deferred_by_cooldown(self, tmp_path: Path):
|
||||
db, agent = self._armed_agent(tmp_path, "OVERFLOW_ORDINARY")
|
||||
calls = []
|
||||
|
||||
def fake_call_llm(**kwargs): # pragma: no cover - must not run
|
||||
calls.append(kwargs)
|
||||
return _summary_response("unexpected")
|
||||
|
||||
live = _messages()
|
||||
before = copy.deepcopy(live)
|
||||
with patch("agent.context_compressor.call_llm", fake_call_llm):
|
||||
out, _ = compress_context(agent, live, "sys", approx_tokens=500_000)
|
||||
assert calls == [] and out == before
|
||||
assert compression_blocked_transiently(agent) is True, (
|
||||
"ordinary threshold pressure keeps honoring the cooldown (#11529)"
|
||||
)
|
||||
|
||||
@@ -73,6 +73,21 @@ Located in `agent/context_compressor.py`. This is the **primary compression
|
||||
system** that runs inside the agent's tool loop with access to accurate,
|
||||
API-reported token counts.
|
||||
|
||||
#### Failure cooldown and provider-proven overflow
|
||||
|
||||
A failed or stalled summary attempt arms a per-session **failure cooldown**
|
||||
(escalating 60s → 300s → 900s, persisted in `state.db`). While it is armed,
|
||||
ordinary threshold-triggered compaction is deferred so a broken summary backend
|
||||
does not re-fire every turn. Two paths run a real attempt anyway:
|
||||
|
||||
- Manual `/compress` (`force=True`) — clears the cooldown and retries.
|
||||
- **Provider-proven overflow** — when the provider itself rejects the request
|
||||
with a context-length error, the recovery pass ignores the cooldown for one
|
||||
bounded attempt (`max_compression_attempts`) without clearing it. Deferring
|
||||
here would wedge the session: every turn would bounce off the provider and
|
||||
the next failure would extend the ladder (#100661). If that attempt fails,
|
||||
the cooldown is recorded normally.
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
|
||||
Reference in New Issue
Block a user