fix(compression): provider-proven overflow gets one real compaction attempt while the failure cooldown is armed

After one failed/stalled summary attempt arms the 60/300/900s compression-
failure cooldown, a provider context_length_exceeded rejection entered the
reactive overflow branch in conversation_loop, which called _compress_context
without force. Since #97488 the cooldown gate returns the soft "temporarily
paused, retry in a moment" deferral instead of exhaustion, so every turn
deferred until the cooldown lapsed, and the next failure extended the ladder:
long-running sessions wedged with no automatic recovery (#100661, four sessions
lost).

Thread a narrow `bypass_cooldown` kwarg from the three provider-proven overflow
call sites (generic overflow, 413, output-cap recovery) through
AIAgent._compress_context -> compress_context -> ContextCompressor.compress ->
_generate_summary. It skips ONLY the summary-failure cooldown check at each gate.
Unlike force=True it does not clear the cooldown, does not skip the feasibility /
anti-thrash breakers, and a failed attempt records its cooldown normally. The
attempt is bounded by the existing compression_attempts/max_compression_attempts
budget, so there is no retry loop. The preflight threshold gate is unchanged:
ordinary over-threshold pressure still honors the cooldown (#11529).

Engines whose _automatic_compression_blocked()/compress() predate the kwarg
(plugins, test doubles) are called with the legacy signature.

Tests: cooldown armed + bypass_cooldown -> summarizer invoked and transcript
compacted; ordinary pass still deferred. Docs note the cooldown/overflow
contract in the developer guide.

Fixes #100661
Closes #97766 (overflow-force idea; the bundled continuation changes were not taken)

Co-authored-by: sgtworkman <178342791+sgtworkman@users.noreply.github.com>
This commit is contained in:
Teknium
2026-09-02 04:51:44 -07:00
parent c7e2e0b779
commit d1efa0d78d
6 changed files with 163 additions and 10 deletions
+25 -7
View File
@@ -3954,9 +3954,17 @@ class ContextCompressor(ContextEngine):
except Exception as exc:
logger.debug("compression ineffective-count refresh failed: %s", exc)
def _automatic_compression_blocked(self) -> bool:
"""Return whether automatic compaction is in cooldown or tripped."""
if not self._automatic_compression_blocked_locally():
def _automatic_compression_blocked(self, *, ignore_cooldown: bool = False) -> bool:
"""Return whether automatic compaction is in cooldown or tripped.
``ignore_cooldown=True`` evaluates only the breakers that are NOT the
summary-failure cooldown. Used by provider-proven overflow recovery
(#100661): the provider already rejected the request, so waiting out
the cooldown just wedges the session — every turn defers and the next
failure extends the ladder. The overflow path gets one real attempt;
the ineffective/structural breakers still apply.
"""
if not self._automatic_compression_blocked_locally(ignore_cooldown=ignore_cooldown):
return False
# Blocked on the in-memory snapshot. Durable guard rows may have
# been cleared by another agent since bind_session_state() — a
@@ -3966,9 +3974,9 @@ class ContextCompressor(ContextEngine):
# local block outlive the durable state that justified it. The
# unblocked hot path above never pays for the DB reads.
self._refresh_durable_guards()
return self._automatic_compression_blocked_locally()
return self._automatic_compression_blocked_locally(ignore_cooldown=ignore_cooldown)
def _automatic_compression_blocked_locally(self) -> bool:
def _automatic_compression_blocked_locally(self, *, ignore_cooldown: bool = False) -> bool:
"""Evaluate the automatic-compaction gate on in-memory state only."""
# Do not trigger compression while the summary LLM is in cooldown.
# On a 429/transient failure _generate_summary() sets a cooldown and
@@ -3980,7 +3988,7 @@ class ContextCompressor(ContextEngine):
# force=True, which clears this cooldown in compress() before running,
# so it still retries immediately.
_cooldown_remaining = self._summary_failure_cooldown_until - time.monotonic()
if _cooldown_remaining > 0:
if _cooldown_remaining > 0 and not ignore_cooldown:
if not self.quiet_mode:
logger.debug(
"Compression deferred — summary LLM in cooldown for %.0fs more",
@@ -5013,6 +5021,7 @@ Summary generation was unavailable, so this is a best-effort deterministic fallb
turns_to_summarize: List[Dict[str, Any]],
focus_topic: Optional[str] = None,
memory_context: str = "",
bypass_cooldown: bool = False,
) -> Optional[str]:
"""Generate a structured summary of conversation turns.
@@ -5035,7 +5044,10 @@ Summary generation was unavailable, so this is a best-effort deterministic fallb
if self._compression_cancelled():
raise AuxiliaryExplicitCancellation()
now = prompt_started_at
if now < self._summary_failure_cooldown_until:
# bypass_cooldown (#100661): provider-proven overflow gets ONE real
# summary attempt while the cooldown is armed; a failure below still
# records/extends the cooldown normally.
if now < self._summary_failure_cooldown_until and not bypass_cooldown:
logger.debug(
"Skipping context summary during cooldown (%.0fs remaining)",
self._summary_failure_cooldown_until - now,
@@ -7729,6 +7741,7 @@ This compaction should PRIORITISE preserving all information related to the focu
focus_topic: Optional[str] = None,
force: bool = False,
memory_context: str = "",
bypass_cooldown: bool = False,
) -> List[Dict[str, Any]]:
"""Compress conversation messages by summarizing middle turns.
@@ -7765,6 +7778,10 @@ This compaction should PRIORITISE preserving all information related to the focu
summary path. Auto-compress callers pass False.
memory_context: Optional provider-supplied context to preserve in
the summary prompt. Whitespace-only values are ignored.
bypass_cooldown: If True, run the summary LLM even while the
summary-failure cooldown is armed, WITHOUT clearing it
(#100661). Set by provider-proven overflow recovery, which
is already bounded by the caller's attempt budget.
"""
# Reset per-call summary failure state — callers inspect these fields
# after compress() returns to decide whether to surface a warning.
@@ -8102,6 +8119,7 @@ This compaction should PRIORITISE preserving all information related to the focu
turns_to_summarize,
focus_topic=summary_focus_topic,
memory_context=memory_context,
bypass_cooldown=bypass_cooldown,
)
except AuxiliaryExplicitCancellation:
# Explicit cancellation is a true no-op. Restore state mutated by
+37 -2
View File
@@ -2012,6 +2012,25 @@ def context_compression_timed_out(agent: Any) -> bool:
return getattr(agent, "_last_compression_timed_out", None) is True
def _automatic_gate_blocked(
blocked: Any, compressor: Any, bypass_cooldown: bool
) -> bool:
"""Evaluate the automatic breaker gate, optionally ignoring the cooldown.
Provider-proven overflow recovery (#100661) passes ``bypass_cooldown``;
engines whose gate predates the kwarg (plugins, test doubles) are called
with the legacy no-argument shape.
"""
if bypass_cooldown:
try:
accepts = "ignore_cooldown" in inspect.signature(blocked).parameters
except (TypeError, ValueError):
accepts = False
if accepts:
return bool(blocked(compressor, ignore_cooldown=True))
return bool(blocked(compressor))
def compression_blocked_transiently(agent: Any) -> bool:
"""Type-pinned read of the transient-block signal (#97488).
@@ -2248,6 +2267,7 @@ def _supported_compression_kwargs(
focus_topic: Optional[str],
force: bool,
memory_context: str,
bypass_cooldown: bool = False,
) -> dict:
"""Return only compression kwargs accepted by an engine callable.
@@ -2261,6 +2281,8 @@ def _supported_compression_kwargs(
"focus_topic": focus_topic,
"force": force,
}
if bypass_cooldown:
candidates["bypass_cooldown"] = True
if memory_context:
candidates["memory_context"] = memory_context
try:
@@ -3289,6 +3311,7 @@ def compress_context(
task_id: str = "default",
focus_topic: Optional[str] = None,
force: bool = False,
bypass_cooldown: bool = False,
defer_context_engine_notification: bool = False,
commit_fence: Optional[CompressionCommitFence] = None,
) -> Tuple[list, str]:
@@ -3308,6 +3331,13 @@ def compress_context(
by the manual ``/compress`` slash command so users can retry
immediately after an auto-compress abort. Auto-compress
callers use the default ``False``.
bypass_cooldown: If True, the automatic breaker gates ignore ONLY the
summary-failure cooldown for this attempt (#100661). Set by the
provider-proven overflow recovery path: the provider already
rejected the request, so deferring until the cooldown lapses
wedges the session. Unlike ``force`` it does not clear the
cooldown, and the ineffective/structural breakers still apply;
a failed attempt records its cooldown normally.
defer_context_engine_notification: Delay the existing context-engine
hook until a manual host commits its outer history transaction.
commit_fence: Optional cooperative fence for executor callers that
@@ -3425,7 +3455,9 @@ def compress_context(
"_automatic_compression_blocked",
None,
)
if callable(blocked) and blocked(agent.context_compressor):
if callable(blocked) and _automatic_gate_blocked(
blocked, agent.context_compressor, bypass_cooldown
):
_mark_compression_blocked_transient(agent, agent.context_compressor)
existing_prompt = getattr(agent, "_cached_system_prompt", None)
if not existing_prompt:
@@ -3896,7 +3928,9 @@ def compress_context(
"_automatic_compression_blocked",
None,
)
if callable(blocked) and blocked(compressor):
if callable(blocked) and _automatic_gate_blocked(
blocked, compressor, bypass_cooldown
):
_mark_compression_blocked_transient(agent, compressor)
_release_lock()
existing_prompt = getattr(agent, "_cached_system_prompt", None)
@@ -4093,6 +4127,7 @@ def compress_context(
focus_topic=focus_topic,
force=force,
memory_context=memory_context,
bypass_cooldown=bypass_cooldown,
)
if memory_context.strip() and "memory_context" not in compress_kwargs:
engine_name = getattr(
+11
View File
@@ -6171,6 +6171,11 @@ def run_conversation(
messages, system_message,
approx_tokens=estimate_request_tokens_rough(api_messages, tools=agent.tools or None),
task_id=effective_task_id,
# #100661: the provider proved the request does not fit.
# Ignore the summary-failure cooldown for this ONE
# attempt (bounded by max_compression_attempts) instead
# of deferring every turn until the ladder lapses.
bypass_cooldown=True,
)
if messages is _overflow_input and compression_skipped_due_to_lock(agent):
# #69870 lock-skip: the provider proved the request
@@ -6347,6 +6352,7 @@ def run_conversation(
messages, system_message,
approx_tokens=request_input_estimate,
task_id=effective_task_id,
bypass_cooldown=True, # #100661 provider-proven overflow
)
if messages is _overflow_input and compression_skipped_due_to_lock(agent):
compression_attempts -= 1
@@ -6510,6 +6516,11 @@ def run_conversation(
messages, system_message,
approx_tokens=estimate_request_tokens_rough(api_messages, tools=agent.tools or None),
task_id=effective_task_id,
# #100661: the provider proved the request does not fit.
# Ignore the summary-failure cooldown for this ONE
# attempt (bounded by max_compression_attempts) instead
# of deferring every turn until the ladder lapses.
bypass_cooldown=True,
)
if messages is _overflow_input and compression_skipped_due_to_lock(agent):
# #69870 lock-skip: the provider proved the request
+5 -1
View File
@@ -8546,6 +8546,7 @@ class AIAgent:
task_id: str = "default",
focus_topic: str = None,
force: bool = False,
bypass_cooldown: bool = False,
defer_context_engine_notification: bool = False,
commit_fence=None,
) -> tuple:
@@ -8554,7 +8555,9 @@ class AIAgent:
``force=True`` is passed by the manual ``/compress`` slash command
so users can bypass the summary-failure cooldown after an
auto-compress abort. Auto-compress callers use the default
``force=False``.
``force=False``. ``bypass_cooldown=True`` is passed by the
provider-proven overflow recovery path so one real attempt runs while
the cooldown is armed (#100661) — without clearing it.
"""
# Per-attempt signal consumed by turn-start preflight (#98424) and the
# in-loop pre-API/overflow consumers. A stalled compression must not
@@ -8635,6 +8638,7 @@ class AIAgent:
approx_tokens=approx_tokens, task_id=task_id,
focus_topic=focus_topic,
force=force,
bypass_cooldown=bypass_cooldown,
defer_context_engine_notification=(
defer_context_engine_notification
),
@@ -287,3 +287,73 @@ class TestTransientBlockIsNotExhaustion:
mock_agent = MagicMock()
# MagicMock auto-attributes are truthy but not str.
assert compression_blocked_transiently(mock_agent) is False
def _summary_response(content: str):
from unittest.mock import MagicMock
response = MagicMock()
response.choices = [MagicMock()]
response.choices[0].message.content = content
return response
class TestProviderOverflowBypassesCooldown:
"""#100661: a provider-proven overflow must get one REAL summary attempt
while the summary-failure cooldown is armed. Before the fix every turn of
a wedged session hit the cooldown gate, returned the soft "temporarily
paused" deferral, and the next failure extended the ladder — 4 long
sessions were lost this way. Ordinary (non-overflow) automatic passes
must still defer."""
def _armed_agent(self, tmp_path: Path, session_id: str):
db, agent = _build_agent(tmp_path, session_id)
# Realistic arming: a failed/stalled attempt recorded the ladder.
agent.context_compressor.record_timeout_failure(
"stall", failure_kind="stalled"
)
assert agent.context_compressor.should_compress_info(500_000)[0] is False
return db, agent
def test_overflow_attempt_invokes_summarizer_while_cooldown_armed(
self, tmp_path: Path
):
db, agent = self._armed_agent(tmp_path, "OVERFLOW_BYPASS")
calls = []
def fake_call_llm(**kwargs):
calls.append(kwargs)
return _summary_response("## Goal\nRecovered after overflow.")
# Bulky turns so the compacted transcript is genuinely smaller.
live = [
{"role": "user" if i % 2 == 0 else "assistant", "content": f"m{i} " * 400}
for i in range(20)
]
with patch("agent.context_compressor.call_llm", fake_call_llm):
out, _ = compress_context(
agent, live, "sys", approx_tokens=500_000, bypass_cooldown=True
)
assert len(calls) == 1, (
"provider-proven overflow must reach the summary LLM even while "
"the failure cooldown is armed (#100661)"
)
assert compression_blocked_transiently(agent) is False
assert len(out) < len(live), "the attempt must actually compact"
def test_non_overflow_pass_still_deferred_by_cooldown(self, tmp_path: Path):
db, agent = self._armed_agent(tmp_path, "OVERFLOW_ORDINARY")
calls = []
def fake_call_llm(**kwargs): # pragma: no cover - must not run
calls.append(kwargs)
return _summary_response("unexpected")
live = _messages()
before = copy.deepcopy(live)
with patch("agent.context_compressor.call_llm", fake_call_llm):
out, _ = compress_context(agent, live, "sys", approx_tokens=500_000)
assert calls == [] and out == before
assert compression_blocked_transiently(agent) is True, (
"ordinary threshold pressure keeps honoring the cooldown (#11529)"
)
@@ -73,6 +73,21 @@ Located in `agent/context_compressor.py`. This is the **primary compression
system** that runs inside the agent's tool loop with access to accurate,
API-reported token counts.
#### Failure cooldown and provider-proven overflow
A failed or stalled summary attempt arms a per-session **failure cooldown**
(escalating 60s → 300s → 900s, persisted in `state.db`). While it is armed,
ordinary threshold-triggered compaction is deferred so a broken summary backend
does not re-fire every turn. Two paths run a real attempt anyway:
- Manual `/compress` (`force=True`) — clears the cooldown and retries.
- **Provider-proven overflow** — when the provider itself rejects the request
with a context-length error, the recovery pass ignores the cooldown for one
bounded attempt (`max_compression_attempts`) without clearing it. Deferring
here would wedge the session: every turn would bounce off the provider and
the next failure would extend the ladder (#100661). If that attempt fails,
the cooldown is recorded normally.
## Configuration