fix(skills-hub): include owner in ClawHub source URLs and add retry on 429 (#51236)

Two fixes for the Skills Hub "View source" links on ClawHub skills:

1. Source URL generation was missing the required {owner} segment —
   https://clawhub.ai/skills/{slug} → 404. Correct format is
   https://clawhub.ai/{owner}/skills/{slug}. When the owner handle is
   unavailable, source_url is now "" (card omits the button) instead of
   emitting a broken link.

2. _fetch_owner_handle() previously delegated to _get_json() which
   returned None on any non-200 response with no retry. Under HTTP 429
   rate-limiting the "50 consecutive failures" safety rail in
   enrich_owners() fired immediately — the documented claim "Respects
   HTTP 429 rate-limit responses with exponential backoff" was not
   actually implemented. Now has its own retry loop: 3 attempts, honours
   Retry-After on 429, exponential backoff on 5xx/transport errors, no
   retry on 4xx.

Changes:
- tools/skills_hub.py: _coerce_skill_payload carries owner from top-level
  response; inspect() captures owner from detail API; _fetch_owner_handle()
  added with bounded retry/backoff; enrich_owners() batch method with
  safety rails (30 workers, early termination at 50 consecutive failures).
- website/scripts/extract-skills.py: _source_url() reads extra["owner"]
  for ClawHub.
- scripts/build_skills_index.py: batch enrichment step after crawling.
- tests: 35 URL/enrichment tests + 7 retry tests (42 total).

Signed-off-by: dongjiang <dongjiang1989@126.com>
This commit is contained in:
dongjiang
2026-07-15 15:07:03 +08:00
committed by Teknium
parent 67d4800d67
commit de6a672168
6 changed files with 543 additions and 4 deletions
+8 -2
View File
@@ -221,9 +221,15 @@ def _source_url(source: str, identifier: str, extra: dict) -> str:
return ""
if src == "clawhub":
# identifier is a bare slug (the "clawhub/" prefix is added at install time)
# identifier is a bare slug (the "clawhub/" prefix is added at install time).
# ClawHub URLs require the owner handle: https://clawhub.ai/{owner}/skills/{slug}.
# Without the owner we cannot build a valid URL — return "" rather than
# a broken link (the card will simply omit the "View source" button).
slug = identifier[len("clawhub/"):] if identifier.startswith("clawhub/") else identifier
return f"https://clawhub.ai/skills/{slug}"
owner = extra.get("owner", "") if isinstance(extra, dict) else ""
if owner:
return f"https://clawhub.ai/{owner}/skills/{slug}"
return ""
if src in {"skills.sh", "skills-sh"}:
# "skills-sh/owner/repo/skill" -> the skills.sh detail page