feat: keyless web tier becomes a 5-vendor round-robin ring (adds Tavily, Firecrawl, Keenable)
Fresh installs with zero web credentials now rotate web_search/ web_extract across FIVE vendors' public free tiers — Exa, Parallel, Tavily, Firecrawl, Keenable — instead of a 2-vendor 50/50 split, with next-in-line ring failover on rate limits (multi-hop until a vendor serves or the ring is exhausted; served_by marks the actual vendor). - plugins/web/keenable/: new bundled provider (search via /v1/search, fetch via /v1/fetch; keyed Bearer or keyless with the mandatory X-Keenable-Title app header). Credit: integration proposed by Ilya Gusev (Keenable) in #49758; Free/Paid picker rows included. - keyless_mcp: tavily/firecrawl/keenable keyless search+extract wrappers, _KEYLESS_RING + per-process round-robin cursor (seeded by the random session id, advances per unpinned request), pinned-vendor entry (pin = start there; rotation off), paid-pinned vendors excluded from the ring entirely. - Tavily/Firecrawl providers route keyless traffic through the ring; both are now default-on ring members (no longer selection-gated). - web_tools/registry: keenable in backend sets, auto-detect, availability probes; _keyless_preference() delegates to the ring cursor. - KEENABLE_API_KEY in OPTIONAL_ENV_VARS; docs updated (ring semantics). Live E2E: all 10 vendorXcapability paths (5 search + 5 extract) served real results keyless; rotation cycled all five vendors over 5 dispatch calls; double-throttle failover walked exa->parallel->tavily.
This commit is contained in:
+5
-1
@@ -169,7 +169,7 @@ def _load_web_config() -> dict:
|
||||
# WebSearchProvider. Keep the two sets aligned by hand: if xai ever ships as
|
||||
# a registered provider, drop it here so the registry path takes over.
|
||||
_LEGACY_WEB_BACKENDS = frozenset(
|
||||
{"parallel", "firecrawl", "tavily", "exa", "searxng", "brave-free", "ddgs", "xai"}
|
||||
{"parallel", "firecrawl", "tavily", "exa", "searxng", "brave-free", "ddgs", "xai", "keenable"}
|
||||
)
|
||||
|
||||
|
||||
@@ -262,6 +262,7 @@ def _get_backend() -> str:
|
||||
("tavily", _has_env("TAVILY_API_KEY")),
|
||||
("exa", _has_env("EXA_API_KEY")),
|
||||
("parallel", _has_env("PARALLEL_API_KEY")),
|
||||
("keenable", _has_env("KEENABLE_API_KEY")),
|
||||
("firecrawl", _has_env("FIRECRAWL_API_KEY") or _has_env("FIRECRAWL_API_URL")),
|
||||
("firecrawl", _is_tool_gateway_ready()),
|
||||
("searxng", _has_env("SEARXNG_URL")),
|
||||
@@ -387,6 +388,8 @@ def _is_backend_available(backend: str) -> bool:
|
||||
return _has_env("EXA_API_KEY")
|
||||
if backend == "parallel":
|
||||
return _has_env("PARALLEL_API_KEY")
|
||||
if backend == "keenable":
|
||||
return _has_env("KEENABLE_API_KEY")
|
||||
if backend == "firecrawl":
|
||||
return check_firecrawl_api_key()
|
||||
if backend == "tavily":
|
||||
@@ -450,6 +453,7 @@ def _web_requires_env() -> list[str]:
|
||||
"EXA_API_KEY",
|
||||
"PARALLEL_API_KEY",
|
||||
"TAVILY_API_KEY",
|
||||
"KEENABLE_API_KEY",
|
||||
"FIRECRAWL_API_KEY",
|
||||
"FIRECRAWL_API_URL",
|
||||
"FIRECRAWL_GATEWAY_URL",
|
||||
|
||||
Reference in New Issue
Block a user