The Skills tab now lists the entire official optional-skills catalog
(optional-skills/ shipped with the repo) below the installed skills.
Each catalog row has an Install button that routes through the standard
hub action pipeline; once the install finishes the row flips into the
installed list with the normal enabled/disabled toggle.
- backend: GET /api/skills/hub/official — OptionalSkillSource.list_local()
scan (no network) + per-profile installed flags from the hub lock
- desktop: catalog section in SkillsView with search/scope integration,
install-state spinners off $hubActions, and an OfficialSkillDetail pane
(hub preview: frontmatter + full SKILL.md + Install)
- CapRow gains an optional action slot (button instead of the Switch)
- electron: route the new endpoint with the skills family (primary backend)
- i18n: officialCatalog/officialPill keys across en/ja/zh/zh-hant
Follow-up reconciling the four cherry-picked contributor fixes with the
full-dir GitHubSource.fetch() that landed in #98246:
- Missing SKILL.md-linked support paths now warn and install without the
file at all three sources (GitHub full-dir, GitHub fallback, UrlSource) —
dangling links are prose over-matches or repo-only dev tools, not install
blockers (#66760/#90081). A referenced path present in the tree as a
SYMLINK stays a hard rejection.
- Extension requirement dropped from the glob/placeholder filter:
references/LICENSE is a legitimate support file (82236's tests pin this).
Truncated prose placeholders (references/type-<name>.md -> 'type-') are
still rejected via the trailing-separator shape.
- percent-quoted Contents-API path (82236) merged with revision pinning
(96336) in _fetch_file_bytes.
- Fixture typo fix: four cherry-picked test strings used '\---' where
'\n---' was meant (DeprecationWarning + frontmatter never parsed).
Validation: 167/167 across tests/tools/{skills_hub,skills_guard,
skill_bundle_provenance} + tests/hermes_cli/test_skills_hub.py; live GitHub
fetches (impeccable 163 files rev-pinned; anthropics frontend-design).
Addresses the review on #96336:
- Every GitHub byte fetch in an install (SKILL.md included) now carries
the resolved tree's SHA as ?ref=, closing the pre-existing TOCTOU
where /contents floated to the default-branch HEAD and bytes could
come from a newer revision than the tree the paths were validated
against. The tree is resolved first (idempotent + cached) so the pin
covers the whole install.
- Same-dir link targets are canonicalized before validation: query/
fragment stripped via urlsplit, percent-decoded, leading ./ removed —
the same normalization the support-dir branch applies.
- A case-variant link to skill.md never ships as a bundle entry, and a
case-folded collision among accepted siblings (A.md + a.md) drops the
pair — both would overwrite/collide on case-insensitive filesystems.
_referenced_support_paths only kept links whose first path segment was
one of the five support directories (references/templates/scripts/
assets/examples), so a SKILL.md linking same-directory siblings —
mattpocock/skills' domain-modeling links ./CONTEXT-FORMAT.md and
ADR-FORMAT.md — installed 'successfully' with those files silently
omitted: the bundle came out semantically incomplete with unresolved
links.
A second pass now collects same-directory markdown-link targets
(](./FILE.ext) or ](FILE.ext)) that name an extension-bearing file,
carry no internal slash, and are not SKILL.md itself; a leading '..'
is rejected fail-closed exactly like the support-dir traversal branch,
external URLs/anchors/mailto/site-absolute targets are left to their own
resolution, and every accepted name still runs the bundle path
validator. Unlinked siblings remain excluded — the fetch-minimization
contract is unchanged; only files the document explicitly links ship.
A SKILL.md that mentions a support-path token which doesn't exist in the
repo (a prose glob like `references/type-*.md`, a placeholder truncated
by the regex, or a repo-only dev script) previously made GitHubSource
and UrlSource fetch fail with 'Could not fetch ... from any source',
even though every real file was present.
- _referenced_support_paths: skip glob/placeholder tokens and bare
prefix matches that can't name an actual file.
- GitHubSource/UrlSource fetch: warn and continue when a referenced
support file is missing or unfetchable, bundling what exists, instead
of aborting the whole install. Non-regular tree entries (e.g. git
symlinks) are still rejected.
Regression tests for both syntax filtering and the fetch loops.
UrlSource.fetch() aborted the whole SKILL.md URL install if any single
referenced support file (references/templates/scripts/assets) 404'd or
was otherwise unreachable, even when the SKILL.md itself and most
companion files were fine. Skip the missing file with a warning and
keep the ones that were fetched successfully.
Fixes#66760
hermes skills install impeccable (and the docs-page install button) now
installs the impeccable frontend-design skill as an official optional-skills
entry. The local optional-skills/creative/impeccable/ dir is a catalog STUB:
its frontmatter declares metadata.hermes.upstream (repo + path), and
OptionalSkillSource.fetch() pulls the real 163-file bundle live from
pbakaus/impeccable:.hermes/skills/impeccable — the Hermes-native bundle
upstream maintains and verifies. Nothing vendored, never stale.
New mechanism (generic, not impeccable-specific):
- OptionalSkillSource._upstream_pointer(): parses/validates the upstream
pointer (owner/name repo, clean relative path, traversal rejected).
- _fetch_from_upstream(): delegates to GitHubSource.fetch(), relabels the
bundle official/<rel> at trust 'trusted' (curated endorsement, but
third-party content — dangerous scan verdicts still block).
- The live-repo fallback path redirects stubs the same way, so stale local
checkouts behave identically.
Three real gaps this surfaced, all fixed:
- GitHubSource.fetch() only downloaded SKILL.md plus paths linked from a
canonical support dir (references/, scripts/, ...). Impeccable keeps its
playbooks under reference/ (singular) and links scripts only from
reference files, so fetch shipped 1 of 163 files. fetch() now downloads
the full skill directory via the git tree (same approach as the
optional-skills live fetch), still rejecting symlinks/hidden/unsafe paths
and still failing on a missing SKILL.md-linked references/ path.
- The five env_exfil_* scanner patterns flagged loopback requests as
critical exfiltration: impeccable's live mode polls
http://localhost:PORT/status?token=TOKEN and scored two CRITICALs.
Scheme-anchored loopback exemption added; evil.com/?u=localhost decoys
still fire (10-case regex matrix in tests).
- unified_search() truncated to limit before ranking, so official catalog
entries got crowded out by skills.sh mirrors and bare-name installs
stalled on an ambiguity table. Results now stable-sort by trust rank
before the cut, and _resolve_short_name prefers a sole official exact
match over community mirrors.
Also fixes pre-existing test pollution: TestInstallPathSafety's fixture
monkeypatched the PEP 562 dynamic SKILLS_DIR, permanently shadowing dynamic
resolution and breaking the served_repo E2E tests in any combined run
(reproducible on main).
Validation: live E2E do_install("impeccable") against real GitHub —
resolves to official/creative/impeccable, verdict SAFE, 163 files on disk,
skill loads, /impeccable slash command registers. 128/128 targeted tests;
full-dir fetch test sabotage-verified. Docs: optional-skills catalog row,
generated skill page, sidebar.
_fetch_owner_handle and the browse catalog walk both inlined the
same owner-handle extraction logic that _owner_from_payload was
extracted to centralize. Replace both with calls to the helper.
ClawHub treated the last path segment as a slug, so a GitHub-style
id like owner/repo/skills/skillopt fetched a different author's
skillopt. Pair metadata and files from the same source so inspect
cannot show one registry's header and another's SKILL.md.
Sweep of open Windows issues affecting day-to-day agent operation
(explicitly excluding install/setup and locale classes):
- hermes_cli/_subprocess_compat.py: new split_command_line() — Windows-
safe command-line tokenizer (posix=False + quote stripping) so
backslash paths survive. POSIX behavior unchanged (plain shlex.split).
- hermes_cli/console_engine.py (#83934): console commands like
'sessions export C:\Users\me\out.jsonl' no longer silently mangle the
path into a relative filename in the cwd.
- agent/shell_hooks.py (#78293): hook commands with backslash paths now
spawn, resolve their script path, and pass hooks doctor instead of
reporting 'not executable'. All three shlex sites routed through the
shared splitter.
- agent/prompt_builder.py (#51755): system prompt now reports
Windows (11) on Windows 11 — platform.release() returns 10 for both;
distinguish via sys.getwindowsversion().build >= 22000.
- hermes_cli/commands.py (#42016): @ autocomplete no longer crashes the
prompt_toolkit event loop when rg emits a path on a different mount
(device paths \.\nul, other drive letters) — relpath ValueError is
skipped per-entry.
- tools/browser_use_cli.py (#83884): screenshot-path detection now
matches Windows drive-letter paths (C:\... and C:/...) in addition to
POSIX; Browser Use screenshots attach on Windows.
- tools/skills_hub.py + tools/skills_guard.py (#62310): the two 'MUST
stay symmetric' skill content hashes actually agree on Windows now.
Bundle keys are normalized to POSIX separators before hashing, and the
disk digest sorts by rel-posix STRING (case-sensitive) instead of Path
objects (case-insensitive on Windows). Fixes permanent false-positive
update_available for every installed skill.
Tests: tests/tools/test_windows_agent_loop_papercuts.py — 16 cases
covering each fix, including a disk-vs-bundle hash symmetry check built
with native Windows separators and a mixed-case filename.
Optional skills merged to main after a user's install was cut were
invisible to 'hermes skills install official/...' until they ran
'hermes update' — the OptionalSkillSource only scanned the local
optional-skills/ checkout.
Now, when an official/<category>/<skill> identifier is not found
locally, OptionalSkillSource resolves it against the live default
branch of NousResearch/hermes-agent: one Trees API call enumerates
optional-skills/*/SKILL.md dirs (cached on disk via the shared index
cache, 1h TTL), then the full skill directory is downloaded byte-exact
(including root-level install scripts, LICENSE, tests/ — files the
generic GitHubSource.fetch path drops). search() and inspect() also
surface remote-only skills so discovery works pre-update too.
Local checkout always wins when present; offline degrades to the old
local-only behavior; traversal and ambiguous bare names are refused;
provenance stays official/builtin.
_normalize_bundle_path rejected absolute paths, .. traversal, and a bare
drive-letter prefix, but permitted a colon inside a later path component.
On NTFS a bundle member named scripts/helper.py:payload writes a hidden
Alternate Data Stream into the visible file scripts/helper.py. The skill
scanner walks with rglob('*'), which does not enumerate streams, so both
operator review and the guard scanner miss the executable bytes.
Reject a colon in any component (the whole class, not just the trailing
one). This subsumes the previous bare drive-letter check, which is folded
into the single colon guard. '/' is the only legal separator once
normalized, so no portable bundle path needs a colon.
Adds an OS-independent quarantine_bundle regression plus a direct
normalizer unit test covering leading/mid/trailing-component colons,
bare/qualified drive letters, and the empty stream name.
Reported-by: JoaoMarcos44 <87440198+JoaoMarcos44@users.noreply.github.com>
NOTION_API_KEY/LINEAR_API_KEY defaults, DEEPINFRA_API_KEY model gate, FAL_KEY via the file's own _scoped_credential helper, XAI/VERCEL/DAYTONA presence checks, GITHUB_TOKEN/GH_TOKEN + GitHub App creds in tirith/skills_hub, and the masked HERMES_API_KEY display in tui_gateway config.show all route through get_secret.
/simplify-code findings on the salvage stack:
- extract _category_skill_dirs() as the single category detector; the
install guard and hermes_cli._existing_categories() now share it
(third copy of the heuristic eliminated)
- filter rglob hits through is_excluded_skill_path so vendored /
support-dir SKILL.md files (node_modules, references/pkg) no longer
misclassify a plain directory as a category and block install
- fix inaccurate WHY comment (lock-file check only guards hub-installed
skills, not hand-authored dirs), drop underscore prefixes on locals,
fold the file-collision guard under the single exists() check
Follow-up to the salvaged #76000 guard:
- refuse installing a skill INTO an existing skill directory (hybrid
skill-plus-category dirs whose later update/uninstall rmtree would
destroy the nested skill — sibling case of #75983)
- refuse a stray regular file at the install path with the caller's
ValueError contract instead of an uncaught NotADirectoryError
- regression tests: nested-only category (skills at depth >= 2),
category-inside-skill, file collision
Two fixes for the Skills Hub "View source" links on ClawHub skills:
1. Source URL generation was missing the required {owner} segment —
https://clawhub.ai/skills/{slug} → 404. Correct format is
https://clawhub.ai/{owner}/skills/{slug}. When the owner handle is
unavailable, source_url is now "" (card omits the button) instead of
emitting a broken link.
2. _fetch_owner_handle() previously delegated to _get_json() which
returned None on any non-200 response with no retry. Under HTTP 429
rate-limiting the "50 consecutive failures" safety rail in
enrich_owners() fired immediately — the documented claim "Respects
HTTP 429 rate-limit responses with exponential backoff" was not
actually implemented. Now has its own retry loop: 3 attempts, honours
Retry-After on 429, exponential backoff on 5xx/transport errors, no
retry on 4xx.
Changes:
- tools/skills_hub.py: _coerce_skill_payload carries owner from top-level
response; inspect() captures owner from detail API; _fetch_owner_handle()
added with bounded retry/backoff; enrich_owners() batch method with
safety rails (30 workers, early termination at 50 consecutive failures).
- website/scripts/extract-skills.py: _source_url() reads extra["owner"]
for ClawHub.
- scripts/build_skills_index.py: batch enrichment step after crawling.
- tests: 35 URL/enrichment tests + 7 retry tests (42 total).
Signed-off-by: dongjiang <dongjiang1989@126.com>
The Skills Hub 'Marketplace' tab showed a single useless entry: Anthropic
changed .claude-plugin/marketplace.json to bundle-shaped plugins whose
source is './', so all plugins collapsed to one identifier pointing at the
repo root, and the second marketplace repo (aiskillstore/marketplace) is
gone (404). Everything in anthropics/skills is already surfaced by the
GitHub tap as the Anthropic tab, making this source fully redundant.
Removes ClaudeMarketplaceSource and all wiring: source router, index
builder (crawl + floors + sort order + rate-limit messaging), extract
labels/install/URL mapping, hub UI tab, web server labels, CLI limits,
docs (en + zh), the legacy index-cache snapshot, and test fixtures.
Stale skills-index entries with source 'claude-marketplace' still install
fine: HermesIndexSource fetches via resolved GitHub paths generically.
`hermes skills update` could silently replace a same-named skill from a
DIFFERENT registry — deleting the user's files and rewriting the lockfile's
recorded provenance. Two defects on main:
- tools/skills_hub.py: check_for_skill_updates fell back to *all* sources
(`... or sources`) when no adapter matched the recorded source, so any
registry with a same-named skill could satisfy the fetch and be reported
as an update — silently reassigning provenance. Now reports the entry as
`unavailable` instead of cross-registry fallback.
- hermes_cli/skills_hub.py: do_update called do_install with a bare, slash-
less identifier and no source constraint, letting _resolve_short_name fuzzy-
match a same-named skill in another registry. do_install now takes an
optional source_id pin that ABORTS (rather than falls back) when no adapter
matches, and do_update forwards the lockfile's recorded source as that pin.
Includes regression tests reproducing the cross-registry hijack.
Salvaged from PR #72216 onto current main; re-attributed to the human
contributor.
Co-authored-by: menhguin <menhguin@users.noreply.github.com>
Path.read_text() without an explicit encoding uses the platform's
default encoding. On Windows this is typically cp1252 or mbcs, which
causes UnicodeDecodeError or silent data corruption when reading
UTF-8 content (JSON files, user text, config with non-ASCII chars).
This is the read-side companion to the write_text() encoding fix.
Fixed the most critical locations that read JSON data, user content,
and config files across 14 files with 31 call sites.
Pattern: .read_text() → .read_text(encoding='utf-8')
json.loads(path.read_text()) → json.loads(path.read_text(encoding='utf-8'))
On Windows with Chinese locale (GBK), subprocess.run(text=True) without
explicit encoding causes UnicodeDecodeError crashes. This fix adds
encoding='utf-8', errors='replace' to all subprocess.run() and
subprocess.Popen() calls that use text=True across 76 non-test Python files.
Fixes#53428 (master tracker for Windows GBK locale crash).
Note: credential_pool.py and electron changes excluded per reviewer request —
those will be submitted as separate focused PRs.
Install connect-time DNS validation for Hermes-owned direct httpx clients so SSRF-sensitive fetch paths dial a vetted IP instead of re-resolving after preflight. This preserves Host/SNI semantics for direct HTTP(S) connections and keeps proxy routing as an explicit trusted egress boundary.
Wire the guarded clients into media cache downloads, vision downloads, Skills Hub direct/raw fetches, and platform attachment fetch paths that already perform SSRF preflight and redirect validation.
Fixes#8033
Co-authored-by: Tom Qiao <zqiao@microsoft.com>
The previous commit fixes the canonical agent/skill_utils.parse_frontmatter.
Six more modules reimplement the '---' fence check locally and had the
same bug:
- tools/skill_manager_tool.py _validate_frontmatter — rejected BOM'd
skill_manage create/edit content outright
- tools/skills_hub.py GitHubSource._parse_frontmatter_quick and
OptionalSkillSource._parse_frontmatter — hub browse/install metadata
- hermes_cli/skills_hub.py — local skill install validation
- gateway/run.py — skill slug discovery for disabled-skill hints
- agent/prompt_builder.py _strip_yaml_frontmatter — BOM'd context files
(AGENTS.md) leaked raw frontmatter into the system prompt
- tools/blueprints.py _split_frontmatter — str.lstrip() does not strip
U+FEFF (not whitespace), so the existing lstrip never covered it
Sibling-surface regression tests added.
Bug class also fixed upstream in cline/cline#12218 (found by the weekly
Cline PR scout).
The Skills Hub 'Browse Hub' landing page and index-backed search render
empty on fresh deployments (e.g. Fly.io VPS agents) with no stale cache.
Root cause: the centralized index at /docs/api/skills-index.json is a
large body (~34MB, tens of MB compressed) served with Content-Encoding:
br. httpx's streaming Brotli decoder — backed by brotlicffi 1.2.0.1,
which is pinned so aiohttp can decode Discord attachments — trips over
its own output_buffer_limit on a payload this size and raises:
DecodingError("brotli: decoder process called with data when
'can_accept_more_data()' is False")
_load_hermes_index() catches that (DecodingError is an httpx.HTTPError
subclass) and silently falls back to the on-disk cache. On a fresh box
that cache never existed, so HermesIndexSource.is_available is False,
the index contributes 0 skills, and the hub landing page — which is
built solely from an empty-query index search — is blank. Existing
installs only appear to work because they serve a (possibly weeks-)stale
cached index instead.
Fix: request 'gzip, deflate' on the index fetch so httpx never
negotiates the broken Brotli path, and retry once with 'identity' if a
DecodingError still occurs (defends against a proxy that ignores the
header). Falls through to the stale cache only when both attempts fail.
Verified on a live staging VPS agent: index_available flips False->True
and the featured landing list repopulates from 0 to 12.
Also un-freezes already-deployed images: skills added after an image was
built (e.g. the 'unbroker' optional skill) become reachable again via
the index, which is the whole point of the centralized catalog.
The Contents-API fallback's directory listing used a raw httpx.get with
no retry, so a 429/403 rate limit aborted the whole skill download even
though file fetches already retry via _github_get. Route the directory
listing through the same helper (429/reset-aware backoff, 5xx retry,
rate-limit flagging).
Salvage of PR #3033's intent — rerouted through the _github_get helper
that landed after the PR was opened, instead of the PR's ad-hoc retry
loops.
Co-authored-by: 0xbyt4 <35742124+0xbyt4@users.noreply.github.com>
Three CLI reliability fixes:
1. Interrupt reliability: chat() only re-queued the user's interrupt
message when the turn result carried interrupted=True. When the agent
thread raced past its last interrupt check (or finished) before the
interrupt landed, the message was silently dropped — and the stale
_interrupt_requested flag left on the agent instantly aborted the
NEXT turn. Un-acknowledged interrupt messages are now re-queued as
the next turn and the stale flag is cleared (only when the agent
thread actually exited). The clarify-race path also parks the message
in _pending_input instead of dropping it.
2. Slow exit (5+ min): stdlib ThreadPoolExecutor workers are non-daemon
and joined unconditionally by concurrent.futures' atexit hook — even
after shutdown(wait=False). One wedged tool worker (abandoned after
interrupt/timeout) held the process open forever. Promoted
async_delegation's daemon executor to a shared tools/daemon_pool
module and adopted it in tool_executor (concurrent tool batches),
memory_manager (background sync), delegate_tool (child timeout wrapper
+ batch fan-out), and skills_hub (source fan-out). Added a 30s exit
watchdog (HERMES_EXIT_WATCHDOG_S) armed at _run_cleanup start as a
backstop for wedged cleanup steps.
3. Exit jank: after prompt_toolkit tears down the input/status bars the
terminal sat silent for the whole cleanup window, looking hung. Print
'Shutting down… (finalizing session)' immediately at exit start.
E2E: live PTY interrupt of a foreground 'sleep 120' terminal tool now
aborts in ~1s and the typed message runs as the next turn; wedged-worker
+ wedged-cleanup subprocess exits in 5.8s (watchdog) instead of hanging.
- tools/skills_hub.py: the per-call resolvers now honor a test-injected real
module attribute (patch.object(hub, 'SKILLS_DIR', ...) / monkeypatch.setattr)
before falling back to dynamic profile resolution. PEP 562 __getattr__ only
fires when no real attribute exists, so an unpatched module resolves the
active profile and a patched one respects the test's value — keeping the
existing skills_hub test seam intact (5 tests had broken).
- tests/test_profile_isolation_runtime.py: real two-profile (no-mock) suite
driving each previously-leaking site under override A then B and asserting
the active profile's path/identity is used: skills_hub paths + derived
constants + default-arg resolution, gateway cache getters (incl. the
monkeypatch-still-wins seam), rich_sent_store path, and thread/executor
context propagation (raw-thread hazard documented; primitive + _run_async
worker proven to preserve the override).
In single-process multi-profile runtimes (desktop tui_gateway), profile
scoping is a context-local ContextVar override, not a process env var. Three
subsystems froze their HERMES_HOME-derived paths at import time (or read
os.environ directly), pinning every later profile to whichever profile first
imported the module — a cross-profile data leak.
- tools/skills_hub.py: SKILLS_DIR/HUB_DIR/LOCK_FILE/etc. were module constants
frozen at import. Replace with per-call resolver functions; add a PEP 562
module __getattr__ so external 'from tools.skills_hub import SKILLS_DIR'
callers (all function-local) resolve dynamically with no call-site changes.
Convert default-arg bindings (HubLockFile/TapsManager) and the derived
HERMES_INDEX_CACHE_FILE constant too.
- gateway/platforms/base.py: image/audio/video/document cache-dir getters now
re-resolve via get_hermes_dir() per call, falling back to the module
constant when a test has monkeypatched it (preserves the existing test seam).
Media-delivery safe-roots already enumerate all profiles' cache dirs
(#31733), so per-profile resolution does not break delivery.
- gateway/rich_sent_store.py: _store_path() read os.environ['HERMES_HOME']
directly, bypassing the override entirely; route through get_hermes_home().
Teknium