fix(skills-hub): include owner in ClawHub source URLs and add retry on 429 (#51236)

Two fixes for the Skills Hub "View source" links on ClawHub skills:

1. Source URL generation was missing the required {owner} segment —
   https://clawhub.ai/skills/{slug} → 404. Correct format is
   https://clawhub.ai/{owner}/skills/{slug}. When the owner handle is
   unavailable, source_url is now "" (card omits the button) instead of
   emitting a broken link.

2. _fetch_owner_handle() previously delegated to _get_json() which
   returned None on any non-200 response with no retry. Under HTTP 429
   rate-limiting the "50 consecutive failures" safety rail in
   enrich_owners() fired immediately — the documented claim "Respects
   HTTP 429 rate-limit responses with exponential backoff" was not
   actually implemented. Now has its own retry loop: 3 attempts, honours
   Retry-After on 429, exponential backoff on 5xx/transport errors, no
   retry on 4xx.

Changes:
- tools/skills_hub.py: _coerce_skill_payload carries owner from top-level
  response; inspect() captures owner from detail API; _fetch_owner_handle()
  added with bounded retry/backoff; enrich_owners() batch method with
  safety rails (30 workers, early termination at 50 consecutive failures).
- website/scripts/extract-skills.py: _source_url() reads extra["owner"]
  for ClawHub.
- scripts/build_skills_index.py: batch enrichment step after crawling.
- tests: 35 URL/enrichment tests + 7 retry tests (42 total).

Signed-off-by: dongjiang <dongjiang1989@126.com>
This commit is contained in:
dongjiang
2026-07-15 15:07:03 +08:00
committed by Teknium
parent 67d4800d67
commit de6a672168
6 changed files with 543 additions and 4 deletions
+20 -2
View File
@@ -65,14 +65,32 @@ def test_source_url_synthesizes_github_root_when_no_subpath(mod):
def test_source_url_synthesizes_clawhub(mod):
assert mod._source_url("clawhub", "go-music-skill", {}) == "https://clawhub.ai/skills/go-music-skill"
# ClawHub URLs require the owner handle; without it we cannot build a
# valid URL, so the result is "" (better than a broken 404 link).
assert mod._source_url("clawhub", "go-music-skill", {}) == ""
def test_source_url_synthesizes_clawhub_strips_prefix(mod):
# identifier may arrive already prefixed; we must not double-prefix.
assert (
mod._source_url("clawhub", "clawhub/go-music-skill", {})
== "https://clawhub.ai/skills/go-music-skill"
== ""
)
def test_source_url_synthesizes_clawhub_with_owner(mod):
# When the owner handle is available in extra, the URL includes it.
assert (
mod._source_url("clawhub", "go-music-skill", {"owner": "somepublisher"})
== "https://clawhub.ai/somepublisher/skills/go-music-skill"
)
def test_source_url_synthesizes_clawhub_with_owner_strips_prefix(mod):
# Owner + prefixed identifier: prefix is stripped, owner is used.
assert (
mod._source_url("clawhub", "clawhub/go-music-skill", {"owner": "somepublisher"})
== "https://clawhub.ai/somepublisher/skills/go-music-skill"
)