feat(compression): provider-scoped model_thresholds keys ("<provider>:<substr>")

A bare `astra: 0.85` in compression.model_thresholds was written for the Codex
OAuth route, where Astra is capped at 272K and 50% would compact at ~136K. The
key is substring-matched on the model name alone, so it also fired on
openai/gpt-6-astra via OpenRouter and Nous, where the window is 1.1M: the user's
0.5 global threshold was silently replaced by 0.85 and the session sat at 620K
(~59%) without compacting.

Keys may now carry a provider prefix: `"openai-codex:astra": 0.85` applies only
when the session's provider is openai-codex; bare keys keep their route-agnostic
behaviour. Ranking is by model-substring length with scope as the tie-break, so
`astra-900k` still outranks `openai-codex:astra` for the 900K picker. The
provider flows through ContextCompressor (ctor + update_model), the ContextEngine
base class and the TUI hot-reload path, so a /model switch between routes
re-scopes the override.
This commit is contained in:
Teknium
2026-09-11 01:49:02 -07:00
parent 9ce7547faf
commit fc3d60af09
6 changed files with 65 additions and 12 deletions
+25 -7
View File
@@ -1555,13 +1555,31 @@ def _summarize_tool_result_unguarded(tool_name: str, tool_args: str, tool_conten
return f"[{tool_name}]{first_arg} ({content_len:,} chars result)"
def resolve_model_threshold(model: str, model_thresholds: dict[str, float] | None, default: float) -> float:
"""Per-model threshold: longest matching ``model_thresholds`` substring key wins, else ``default``.
Module-level so plugin context engines can reuse it."""
def _model_threshold_key_rank(key: str, model: str, provider: str) -> "tuple[int, int] | None":
"""Match rank for one ``model_thresholds`` key, or None when it does not apply.
``"<provider>:<substr>"`` keys apply only on that provider; bare keys apply on every route.
The same slug means different windows on different routes (Codex caps Astra at 272K; OpenRouter
serves the full window), so a bare ``astra: 0.85`` written for Codex silently leaks everywhere.
Rank = (substring length, scoped): the most specific model match wins, scope breaks ties."""
scope, sep, substr = key.partition(":")
if not sep:
return (len(key), 0) if key in model else None
return (len(substr), 1) if scope.strip().lower() == provider and substr in model else None
def resolve_model_threshold(
model: str, model_thresholds: dict[str, float] | None, default: float, provider: str = "",
) -> float:
"""Per-model threshold: longest matching ``model_thresholds`` key wins, else ``default``.
Keys are substrings of the model name, optionally provider-scoped as ``"<provider>:<substr>"``
(a scoped key outranks a bare one of the same substring). Module-level so plugin context
engines can reuse it."""
if not model_thresholds or not model:
return default
best_key = max((key for key in model_thresholds if key in model), key=len, default="")
return float(model_thresholds[best_key]) if best_key else default
provider = (provider or "").strip().lower()
ranked = ((_model_threshold_key_rank(key, model, provider), key) for key in model_thresholds)
best = max(((rank, key) for rank, key in ranked if rank is not None), default=None)
return float(model_thresholds[best[1]]) if best else default
def _memory_provider_section(memory_context: str) -> str:
@@ -2169,7 +2187,7 @@ class ContextCompressor(SummaryDispatchMixin, MicroCompactionMixin, ContextEngin
self.context_length = context_length
# Re-resolve from the raw config value so a switch away from an overridden model falls back correctly.
_config_pct = getattr(self, "_config_threshold_percent", self.threshold_percent)
self._base_threshold_percent = resolve_model_threshold(model, self.model_thresholds, _config_pct)
self._base_threshold_percent = resolve_model_threshold(model, self.model_thresholds, _config_pct, provider)
self.threshold_percent = self._effective_threshold_percent(context_length, self._base_threshold_percent)
# max_tokens=None means "unspecified": keep the existing output reservation.
# A switch that genuinely changes the output budget passes the new value explicitly. (#43547)
@@ -2295,7 +2313,7 @@ class ContextCompressor(SummaryDispatchMixin, MicroCompactionMixin, ContextEngin
self.model_thresholds = model_thresholds or {}
# Raw config value, before override/floor; fallback when switching to a model with no override.
self._config_threshold_percent = threshold_percent
self._base_threshold_percent = resolve_model_threshold(model, self.model_thresholds, threshold_percent)
self._base_threshold_percent = resolve_model_threshold(model, self.model_thresholds, threshold_percent, provider)
self.threshold_percent = self._base_threshold_percent
# Effective trigger = min(ratio threshold, cap); re-applied in update_model().
self.threshold_tokens_cap = self._coerce_threshold_tokens_cap(threshold_tokens_cap)
+1 -1
View File
@@ -237,6 +237,6 @@ class ContextEngine(ABC):
if not hasattr(self, "_config_threshold_percent"):
self._config_threshold_percent = self.threshold_percent
self._base_threshold_percent = resolve_model_threshold(
model, getattr(self, "model_thresholds", {}), self._config_threshold_percent)
model, getattr(self, "model_thresholds", {}), self._config_threshold_percent, provider)
self.threshold_percent = self._base_threshold_percent
self.threshold_tokens = int(context_length * self.threshold_percent)
+3 -2
View File
@@ -627,8 +627,9 @@ DEFAULT_CONFIG = {
# path.
"in_place": True,
# Per-model threshold overrides: keys substring-match the model name (longest wins), values
# replace the global `threshold`, e.g. {"glm-5.2": 0.40}. The <512K floor (0.75) still
# applies raise-only on top.
# replace the global `threshold`, e.g. {"glm-5.2": 0.40}. Prefix a key with "<provider>:" to
# scope it to one route ({"openai-codex:astra": 0.85} leaves Astra on OpenRouter/Nous at the
# global value). The <512K floor (0.75) still applies raise-only on top.
"model_thresholds": {},
# Opt-in idle compaction (0 = off): a session resuming after this many idle seconds compacts
# up front, before the first reply. Time-based complement to `threshold`; skipped when
@@ -124,3 +124,27 @@ class TestContextEngineModelThresholds:
assert engine.threshold_percent == 0.25
assert engine.threshold_tokens == int(1_000_000 * 0.25)
class TestProviderScopedKeys:
def test_scoped_key_applies_only_on_its_provider(self):
overrides = {"openai-codex:astra": 0.85}
assert resolve_model_threshold("gpt-6-astra", overrides, 0.50, "openai-codex") == 0.85
# Same slug via another route keeps the global value; the bare-key path is unchanged.
assert resolve_model_threshold("openai/gpt-6-astra", overrides, 0.50, "openrouter") == 0.50
assert resolve_model_threshold("openai/gpt-6-astra", {"astra": 0.85}, 0.50, "openrouter") == 0.85
# Specificity is judged on the model substring: the bare 900k key beats the shorter scoped one;
# a scoped key beats the bare key with the identical substring.
both = {"openai-codex:astra": 0.85, "astra-900k": 0.50, "astra": 0.30}
assert resolve_model_threshold("gpt-6-astra-900k", both, 0.50, "openai-codex") == 0.50
assert resolve_model_threshold("gpt-6-astra", both, 0.50, "openai-codex") == 0.85
@patch("agent.context_compressor.get_model_context_length", return_value=1_100_000)
def test_compressor_switch_between_routes_rescopes(self, _mock):
cc = ContextCompressor(
model="gpt-6-astra", threshold_percent=0.50, provider="openai-codex",
model_thresholds={"openai-codex:astra": 0.85}, quiet_mode=True,
)
assert cc.threshold_percent == 0.85
cc.update_model(model="openai/gpt-6-astra", context_length=1_100_000, provider="openrouter")
assert cc.threshold_percent == 0.50
+3 -1
View File
@@ -127,7 +127,9 @@ def _apply_live_compression_config(agent: Any, cfg: dict | None) -> None:
if pct is None:
pct = _derived_default_threshold_percent(agent, compression)
cc._config_threshold_percent = cc._configured_threshold_percent = pct
base = cc._base_threshold_percent = resolve_model_threshold(getattr(agent, "model", "") or "", cc.model_thresholds, pct)
base = cc._base_threshold_percent = resolve_model_threshold(
getattr(agent, "model", "") or "", cc.model_thresholds, pct, getattr(agent, "provider", "") or "",
)
try:
cc.threshold_percent = cc._effective_threshold_percent(cc.context_length, base)
except Exception:
@@ -191,7 +191,7 @@ auxiliary:
| Parameter | Default | Range | Description |
|-----------|---------|-------|-------------|
| `threshold` | `0.50` | 0.0-1.0 | Compression triggers when prompt tokens ≥ `threshold × context_length` |
| `model_thresholds` | `{}` | map | Per-model overrides of `threshold`. Keys are substring-matched against the model name (longest match wins). The small-context floor still applies on top (see below) |
| `model_thresholds` | `{}` | map | Per-model overrides of `threshold`. Keys are substring-matched against the model name (longest match wins); `"<provider>:<substring>"` keys apply only on that provider. The small-context floor still applies on top (see below) |
| `target_ratio` | `0.20` | 0.10-0.80 | Controls tail protection token budget: `threshold_tokens × target_ratio` (legacy mode only — `lean` uses its own clamp) |
| `tail_mode` | `lean` | `lean`, `legacy` | Tail retention policy. `legacy` keeps a `target_ratio`-sized verbatim tail (~100K+ tokens on big-window models). `lean` keeps a clamped tail of `2.5% × context window` (10K floor, 25K cap) and instead carries continuity in the summary: a detailed identifier-preserving session log (produced by the same single summary request — lean compaction makes exactly one auxiliary LLM call per attempt), a mechanically extracted anchor index (PR numbers, SHAs, paths, error strings — regex, never paraphrased), every real user message quoted verbatim (newest-first budget), and a `session_search` recovery pointer so the agent can re-access anything summarized away. Oversized regions are evenly sampled into the summarizer input (with explicit elision markers) rather than triggering extra calls. Result on 500K-token real sessions: ~49K retained vs ~162K, with higher recall when paired with recovery (see `evals/compaction/results/`). Old tool results inside the lean tail are demoted to one-line stubs carrying a recovery pointer |
| `protect_last_n` | `20` | ≥1 | Minimum number of recent messages always preserved |
@@ -241,12 +241,20 @@ compression:
"glm-5.2": 0.40
"glm-5.2-1M": 0.25
"claude-sonnet": 0.35
"openai-codex:astra": 0.85 # only on the Codex OAuth route (272K cap)
```
Resolution rules:
- Keys are **substring-matched** against the model name; the **longest
matching key wins** (`glm-5.2-1M` beats `glm-5.2` for model `glm-5.2-1M`).
- Keys may be **provider-scoped** as `"<provider>:<substring>"` (e.g.
`"openai-codex:astra": 0.85`). A scoped key only matches when the session's
provider is that route, so the same slug served with a different window
elsewhere (OpenRouter, Nous, direct OpenAI) keeps the global `threshold`.
Ranking uses the model substring only, so `"astra-900k"` still beats
`"openai-codex:astra"` for the 900K picker; a scoped key beats a bare key
with the identical substring.
- When no key matches (or the map is empty), the global `threshold` applies.
- The override is re-resolved on every `/model` switch; switching to a model
with no matching key falls back to the global `threshold`.