Commit Graph

13 Commits

Author SHA1 Message Date
teknium1 3bef6b6a5c fix(agent): explicit "retry after N s" wins over "resets in ..." in the shared reset table
Unifying the credential pool's `_RETRY_DELAY_PATTERNS` into `RETRY_DELAY_PATTERNS`
flipped the pool's precedence: "retry after 30s; resets in 4hr" cooled the
credential for 14400 s where the pool used to take 30. A body carrying both
describes a short throttle inside a long quota window; the explicit retry-after
is the wait the provider actually asks for, so it is tried before "resets in".

Review follow-up on #109539.
2026-09-13 05:09:43 -07:00
teknium1 164c5ea142 fix(agent): LLM traffic honours CIDR and wildcard NO_PROXY entries like the platform adapters do
Three answers to "is this host in NO_PROXY": process_bootstrap used the stdlib
proxy_bypass_environment (no CIDR, no `*.`), gateway/platforms/base.py had a
full matcher (should_bypass_proxy) and a second suffix-only one
(is_host_excluded_by_no_proxy, used by Slack). Live-verified: with
NO_PROXY=10.0.0.0/8 Telegram bypassed the proxy while the LLM call to a 10.x
endpoint went through it.

The full matcher moves to the leaf module agent/proxy_bypass.py (stdlib only,
importable at early boot); both base.py functions are one-line forwarders and
process_bootstrap._get_proxy_for_base_url uses it (passing host:port so
port-qualified entries match). The six-key proxy env scan is also shared.
2026-09-13 05:09:43 -07:00
teknium1 398234748f refactor(agent): one Retry-After parser and one reset-grammar table feed every retry wait
Seven sites hand-rolled `float(headers.get("Retry-After"))` (anon_auth,
shared_metrics_sender, gemini_native_adapter, extract_api_error_context,
nous_rate_guard, skills_hub_github, skills_hub_clawhub x2) and silently
dropped RFC 7231 HTTP-date values that the conversation loop already honours
via agent/retry_utils.py::parse_retry_after_seconds. They now call it; per-site
caps/floors stay at the call site.

The free-text "resets in / quotaResetDelay / retry after N s" regexes lived in
two tables (agent_runtime_helpers vs credential_pool) whose "resets in"
grammars diverged: the pool accepted only integer `Nhr Nmin` while the error
context accepted h/hr/hours + m/min/minutes + s/seconds with decimals. One table
(agent/retry_utils.py::RETRY_DELAY_PATTERNS / reset_delay_from_message) using
the wider grammar, so a pooled credential's cooldown and the UI's reset time
now agree.
2026-09-13 05:09:43 -07:00
Teknium 922717d538 refactor(agent/G_small): compact docstrings (keep WHY/invariants), inline constant comments 2026-09-02 19:16:38 -07:00
Teknium 270128294e refactor(agent/G_small): single-line signatures/calls, github effort fallback table, any() slot scan 2026-09-02 19:08:38 -07:00
Teknium d038249b12 refactor(agent/G_small): second code pass — merged guards, table-driven compact display, joined wrapped calls 2026-09-02 18:51:24 -07:00
Teknium 0e1e86bef4 refactor(agent/retry_utils,runtime_cwd,ssl_*): inline single-use helpers, reuse is_truthy_value, compact docs 2026-09-02 18:36:16 -07:00
Teknium 6a88ec4eb3 refactor(agent/adapters): simplify azure identity, response guards, error surface, retry utils (-603 LOC)
Drop dead read_error_body_or_default / LAYER_RUNTIME; inline single-use
_build_default_credential/_safe_close; compact incident narratives to their
invariants in the guards. Byte-cap and deadline semantics of
read_streaming_error_body verified identical.
2026-09-02 13:29:47 -07:00
teknium1 7c198c5e44 refactor: single shared Retry-After parser 2026-07-29 10:13:50 -07:00
kshitijk4poor 45f5a6e659 refactor(retry): single-source Z.AI overload short-attempts + drop change-detector assert
Follow-up on the salvage of #59523. Two low-risk cleanups surfaced by review:

- Extract _ZAI_CODING_OVERLOAD_SHORT_ATTEMPTS as a module constant so
  adaptive_rate_limit_backoff() and zai_coding_overload_retry_ceiling()
  share one source of truth. Previously both hardcoded short_attempts=3
  independently; tuning one without the other would silently desync the
  retry ceiling from the backoff schedule.
- Replace the tautological formula-mirroring assert in
  test_zai_overload_retry_ceiling_exceeds_short_attempts with a behavior
  invariant (ceiling leaves headroom for every long-backoff entry), per the
  repo's contracts-over-snapshots testing rule.
2026-07-07 11:57:01 +05:30
xxxigm 1c702aa73e fix(agent): run Z.AI overload adaptive backoff on the overloaded path
Z.AI Coding Plan GLM-5.2 reports server overload as HTTP 429 code 1305
("temporarily overloaded"). classify_api_error routes that to
FailoverReason.overloaded (so a valid credential pool isn't burned), but
the adaptive Z.AI backoff was gated on is_rate_limited — which excludes
overloaded — so it never ran (policy=default) and the request failed after
a few quick short retries.

Two compounding causes, both fixed here:

1. Detect the Z.AI overload 429 directly and let its adaptive backoff run
   on the overloaded path, not only the rate_limit path.
2. Raise the retry ceiling for this narrow case via
   zai_coding_overload_retry_ceiling(). The long-backoff tier
   (30/60/90/120s) starts after short_attempts (3) retries, but the default
   api_max_retries is also 3, so the loop always gave up before the long
   tier could run — leaving the whole long-backoff schedule as dead code.

Scope is limited to the existing narrow is_zai_coding_overload_error match,
so other providers' 429/503/529 handling is unchanged.
2026-07-07 11:44:58 +05:30
benbenwyb 6f2b2a1f34 fix: handle named custom providers and Z.AI overload retries 2026-06-25 00:17:17 -07:00
zocomputer e1befe5077 feat(agent): add jittered retry backoff
Adds agent/retry_utils.py with jittered_backoff() — exponential backoff
with additive jitter to prevent thundering-herd retry spikes when
multiple gateway sessions hit the same rate-limited provider.

Replaces fixed exponential backoff at 4 call sites:
- run_agent.py: None-choices retry path (5s base, 120s cap)
- run_agent.py: API error retry path (2s base, 60s cap)
- trajectory_compressor.py: sync + async summarization retries

Thread-safe jitter counter with overflow guards ensures unique seeds
across concurrent retries.

Trimmed from original PR to keep only wired-in functionality.

Co-authored-by: martinp09 <martinp09@users.noreply.github.com>
2026-04-08 00:41:36 -07:00