* refactor(skills): shipped-set slim — 15 skills to optional, github six-way merge, pdf absorbs OCR+nano-pdf, channel-gated teams pipeline
Maintainer-directed shipped-skills curation (skills index 1,900 -> ~1,400
tok/call on desktop; every session pays the index, so this is a per-call
diet on all installs):
- optional-skills moves (installable via skills hub, history preserved):
creative comfyui/ascii-art/excalidraw/pretext/sketch/touchdesigner-mcp;
ALL of mlops (huggingface-hub, llama-cpp, serving-llms-vllm,
weights-and-biases, evaluating-llms-harness — subcategory structure
kept); research-paper-writing (55 supporting files, 17.3K-tok load);
openhue; blogwatcher (first taught the cronjob monitor-field watch
pattern + web_extract instead of pre-cron manual workflows)
- DELETED session-librarian (Aug-12 'inspired by Perplexity Computer'
port, never maintainer-intended; session_search covers discovery)
- github: six skills (auth, issues, pr-workflow, issue-to-pr,
code-review, repo-management) merged into ONE software-development/
github skill — routing body + complete per-workflow references;
benbarclay authorship credited; codebase-inspection rides along;
discipline pins from test_github_issue_to_pr_skill.py preserved
against the reference body in the new test_github_skill.py
- pdf absorbs ocr-and-documents + nano-pdf as references/ + scripts
(extract_pymupdf, extract_marker converted to the argparse house
standard its contract test enforces)
- NEW session_platforms frontmatter gate (metadata.hermes): hides a
skill from the index on gateway channels it is not for; fail-open on
unknown platform; teams-meeting-pipeline gated to [teams, cron]
- blocked-page-recovery: research -> new web category; trigger-first
description ('Use when a fetch fails: 403/429, paywall, WAF, bot
wall.') so the model actually reaches for it on blocked fetches
- docs regenerated via generate-skill-docs.py (195 pages); related_skills
swept repo-wide; tests: 1672 passed (2 openclaw failures pre-existing
on clean main, Windows-local)
* chore: ignore .skills_prompt_snapshot.json (local index cache, accidentally committed)
Bundled by #83063 into the always-shipped set, but it serves only
multi-agent kanban campaigns — too niche for every install's skills
index (~20 tok/call for all users). Moved to the optional catalog
(installable via /skills search + hub, official source) and renamed so
the trigger is legible at a glance: 'merge-reconciler' read like a
generic git helper; 'agent-merge-conflict-arbiter' says who it is for.
- skills/autonomous-ai-agents/merge-reconciler -> optional-skills/autonomous-ai-agents/agent-merge-conflict-arbiter
- frontmatter name + description updated (description within the 60-char hardline its own contract test enforces)
- contract test moved/renamed, 9/9 green
- kanban docs (en + zh-Hans) repointed; zero merge-reconciler refs remain
Ports the MIT-licensed 'to-questionnaire' skill from mattpocock/skills as
decision-questionnaire (optional-skills/productivity). Interviews the user
about the send only (recipient + needed outcomes), then drafts a
most-important-first discovery questionnaire with answer stubs to
decision-questionnaire-<slug>.md. Includes staging tests for frontmatter
standards, template sections, and de-upstreaming.
Ports the MIT-licensed 'wizard' skill from mattpocock/skills as
setup-wizard-generator (optional-skills/devops). Generates an interactive
bash wizard that walks a human through manual procedures: opens dashboard
URLs, captures values (hidden entry for secrets), writes .env / GitHub
secrets idempotently, and confirms each stage. Vendors upstream's
template.sh library verbatim (bash -n verified) plus a staging test suite
covering frontmatter, template integrity, and de-upstreaming.
Under #92021 the scanner no longer emits critical agent_config_mod /
hermes_config_mod findings for bare mentions - the migration script's
legitimate references now score as informational _ref findings and the
verdict is "safe" with zero modification-intent findings. Tighten the
assertion accordingly (verdict must be safe, not merely non-dangerous)
and swap the known-false-positive set to the new _ref ids.
Per the 'when in doubt, optional' rule — site publishing is an
on-request capability, not a weekly daily-driver for most users.
Joins cloudflare-temporary-deploy/page-agent under
optional-skills/web-development (existing category, existing
DESCRIPTION.md kept; the new bundled category dir is dropped).
Install via: hermes skills install official/web-development/publish-site
The routing table listed 18 topics and had nothing to say about the rest of
the product, so an agent asked how to get bots to talk to each other answered
that it could not — while user-guide/bot-mode documented four ways to do it.
Point the catch-all at the published index, which is generated from the docs
tree on every build and so cannot fall behind the feature set. website/ is
never packaged, so the URL is the only complete self-knowledge a running
Hermes has; curl covers sessions where the web tools are disabled.
Box cloud content management via the official @box/cli through the
terminal tool: files, folders, sharing, search, metadata, Box AI,
Hubs, bulk operations, webhooks, and a REST fallback via box request.
OAuth-only auth; SKILL.md routes to ten scoped reference files.
Salvaged from PR #52107 by @iskysun96.
Adds scripts/har_capture_cdp.py for browsers reached over CDP -- cloud
backends (Browserbase, Browser-Use, Firecrawl), Camofox-with-CDP, and any
/browser connect endpoint. record_har_path only works on a locally-owned
Playwright context, so the CDP capturer attaches via connect_over_cdp() and
assembles the HAR from page request/response events instead, leaving the
attached browser open (it doesn't own it).
- SKILL.md: pathway->capturer routing table, CDP prerequisites, pitfalls for
wrong-capturer/empty-HAR, headless-UA weakness, and no-close-on-attach
- Validated live: attached to an external CDP Chrome, drove DuckDuckGo
autocomplete, derived the /ac/ endpoint, replayed it browserless
- tests: assert CDP capturer attaches (not launches) and that the skill
documents every browser backend
Record a site's XHR into a HAR with Playwright, derive its private JSON API,
and call it directly over plain HTTP instead of browser-controlling the page
every time. Credit: trick by Jared Longster, popularized by Dax (thdxr).
- scripts/har_capture.py: Playwright HAR recorder with scripted --action steps
and embedded response bodies
- scripts/har_to_client.py: distills the HAR to endpoints (method/path template
/params/body/response) plus User-Agent+cookie+auth replay hints
- Validated live: derived + replayed the Algolia HN-search POST API and the
Wikipedia rest.php search-title GET, both browserless
- tests exercise the real derivation logic on a synthetic HAR fixture
optional-skills placement: heavy Playwright dependency, niche use case.
Teach the kanban reviewer to vary its inspection lens per review round
instead of repeating the same framing: round 1 reads the artifact cold
before the implementer narrative, round 2 checks out and empirically
executes the work, round 3+ audits strictly against the original
acceptance criteria and every prior request_changes item. The round is
derived from the changes_requested entries already visible in the
reviewer's worker context (live-verified against build_worker_context
across two real request_review/request_changes rounds on an isolated
board). Also adds a lens-variation note for parallel delegate_task
review fan-outs. Contract test updated with section order and lens
assertions.
Adds skills/autonomous-ai-agents/merge-reconciler — a bundled skill teaching
a neutral third-party agent to resolve git merge conflicts between two
agents' branches: gather both diffs + intents, classify each hunk
(disjoint-intent / same-question-different-answer / superseded), resolve
under an impartiality contract, verify, and hand back a per-hunk summary.
Procedure was live-tested end-to-end against a real conflict fixture.
Includes contract tests (tests/skills/test_merge_reconciler_skill.py) and a
kanban docs cross-reference (en + zh-Hans): assign a third neutral profile a
reconciliation card with both conflicted cards as parents.
New tests/skills/test_authoring_standards.py parametrizes every bundled +
optional SKILL.md (1148 checks) against the mechanically-verifiable subset
of the hardline standards:
- required frontmatter fields (name/description/version/author/license/
platforms) + tags
- frontmatter name == directory name
- description <= 60 chars, ends with period, no marketing words
- related_skills resolve in-repo
- no machine-local paths
- <= 100k chars
Grandfather dict for legacy debt ships EMPTY — all pre-existing violations
fixed in this PR:
- 13 frontmatter names canonicalized to their directory names (the install
identifier); all related_skills references updated (comfyui -> stable-
diffusion). Fixes the class behind PR #42788's report; also fixes
here.now's invalid dot-name.
- optional-skills/devops/cli -> inference-sh-cli (dir was the generic
'cli'; fm name was right) incl. docs pages (en + zh-Hans), catalog row,
sidebar entry.
- pytorch-fsdp: 157k generated 'Quick Reference' dump moved to
references/common-patterns.md; SKILL.md 159k -> 2.5k with a pointer.
- research-paper-writing: 31.7k Phase 5 drafting section moved to
references/phase5-paper-drafting.md; SKILL.md 103k -> 71k.
Docs regenerated with scope discipline.
- description 210 -> 57 chars; author credits Ben Barclay (benbarclay) first
- optional-skills/creative/ (marketing vertical, narrowest audience of
the batch)
- dropped phantom 'image-generation-workflow' ref; visuals via the
image_generate tool
- honest handoff language: platforms without connectors end at approved
drafts marked handed-off, never claimed as published
- tests (10) incl. phantom-ref and honest-handoff guards
- docs regen scoped: per-skill page + one catalog row + one sidebar line
Skill polish:
- description 208 -> 57 chars; author credits Ben Barclay (benbarclay) first
- connector framing (google-workspace, obsidian, notion, email-inbox-triage)
- modern section order; boilerplate folded into step-local rules
Blueprint wiring (completes the batch's recipe integration):
- weekly-review blueprint loads weekly-review-planning; prompt follows the
skill's seven-section shape, drafts-only
- morning-brief blueprint loads google-workspace; prompt points at
references/daily-brief.md when connected
- important-mail blueprint loads email-inbox-triage
- blueprints index regenerated
Tests: 13 skill tests + two catalog invariants (every blueprint skills=
entry resolves to a real bundled skill; the four task blueprints are wired
to their procedure skills). 32 green across both files.
The salvaged fix covered run_workflow.py and hardware_check.py. The same
locale-default read of user-authored workflow JSON exists in five sibling
scripts (auto_fix_deps, check_deps, extract_schema, health_check,
run_batch) — same bug class, same utf-8-sig fix. Invariant test extended
to pin all nine read sites.
The pdf half of the original PR is superseded: those scripts were
replaced wholesale by the clean-room rewrite (#81890), which ships
UTF-8-explicit I/O enforced by its own invariant test.
The bundled comfyui and pdf skills read and write text files with the
locale-default codec. Both declare platforms: [linux, macos, windows], so
these paths run on hosts where that codec is not UTF-8 (cp1252 on US
Windows, cp936 on Chinese Windows, ASCII under LC_ALL=C).
Readers (the live bugs):
- run_workflow.py load_schema() and the main() workflow read parse
user-authored JSON. A non-ASCII label crashes json.load with
UnicodeDecodeError under a non-UTF-8 locale, and a file saved from a
Windows GUI editor carries a UTF-8 BOM that json.load rejects with
JSONDecodeError. Both are read as utf-8-sig, which is BOM-tolerant and
identical to utf-8 on BOM-less input. This differs from adecb0d1a,
which used plain utf-8 for the pdf form JSON; those payloads are
agent-authored and BOM-free by construction, these are not.
- hardware_check.py reads /proc/version and /proc/meminfo. Both are
Linux-gated so Windows never reaches them, but the C locale defaults to
ASCII, so they pin plain utf-8. No BOM is possible on /proc.
Writers (not currently broken):
- extract_form_structure.py and extract_form_field_info.py write their
JSON with json.dump, whose default ensure_ascii=True keeps the bytes
pure ASCII. Pinned anyway because the codec is the writer's contract,
not a property of what the caller happens to dump.
wf_path.open() is a Path.open() site that check-windows-footguns.py
deliberately does not flag (per the rule comment: "Path.open() is ALSO
affected ... and can be audited separately"). It is fixed here because it
is the same bug 156 lines from a site the checker does flag, and line 623
of the same file already uses read_text(encoding="utf-8").
Adds tests/skills/test_comfyui_skill.py with contract assertions plus two
live regressions that run load_schema in a child interpreter under
LC_ALL=C with PYTHONUTF8=0, and extends the office skill tests with writer
contract assertions. All 8 new tests fail without this change.
Note that pyproject.toml exempts skills/** from ruff PLW1514
(unspecified-encoding) because skill scripts are partly user-authored.
This change does not touch that exemption; the sites are fixed by hand,
the same way adecb0d1a did.
The bundled docx, xlsx, powerpoint, and pdf skills were adapted from
Anthropic's document skills and carried their proprietary LICENSE.txt
(no derivatives, no redistribution). Flagged as critical license
findings by the SkillEvaluator Tier 1 scan of our skill tree.
This replaces all four with clean-room rewrites:
- Authored from scratch against library knowledge only (python-docx,
openpyxl, python-pptx, pypdf/reportlab/pdfplumber — all MIT/BSD) by
isolated subagents given functional specs, with an explicit
prohibition on reading the prior skill content or anthropics/skills;
session transcripts retained as provenance evidence.
- MIT licensed (LICENSE file per skill), author: Nous Research.
- Each skill: SKILL.md to house standards + argparse helper scripts
with UTF-8-explicit I/O + its own e2e pytest suite (fixtures built
on the fly, non-ASCII round-trips run under LC_ALL=C).
- All four pass SkillEvaluator Tier 1 pii+unicode+lint 3/3.
tests/skills/test_office_document_skills.py rewritten against the new
contracts: MIT/no-Anthropic-text invariants, scripts documented in
SKILL.md, argparse CLI shape, and a no-locale-default-open() check
(which caught and fixed a real gap: pdfplumber text reads are fine,
but the invariant scan now guards every future script).
Docs pages regenerated for the four skills (scoped; unrelated
generator drift excluded).
Honest capability deltas vs the old versions are documented per
SKILL.md (e.g. tracked-changes accept/reject and OOXML XSD validation
are not reimplemented; form flattening limits stated).
The brief is single-connector (every command comes from google-workspace),
so it ships as references/daily-brief.md with a pointer + load trigger in
SKILL.md — progressive disclosure instead of a new skill-index entry.
Contributor's procedure preserved (half-open day windows, mail-to-meeting
linking with fuzzy-match discipline, 7-section brief, bounded actions);
credit noted in the reference header. Tests (8) guard the wiring and
disciplines. Version 1.1.0 -> 1.2.0.
Rewrote from a sibling-skill routing table into a skill that carries its
own procedure, and folded in generalized rules from maintainer practice:
- full-thread reads (gh issue view --comments; newest comment = live state)
- duplicate-PR sweep (issue number + keyword variants) before any code
- design-intent check via git log -p -S alongside premise reproduction
- fix the class: sweep sibling call sites into the same PR
- sabotage run: prove the regression test fails without the fix
- open the PR immediately (PR dispatches CI; CI latency is the long pole)
- close the loop: comment the issue with the PR link
Also: description 205 -> 59 chars, author credits Ben Barclay first,
modern section order, boilerplate trimmed, tests (10) incl. a
router-pattern guard, scoped docs regen.
- description 219 -> 58 chars
- author credits Ben Barclay (benbarclay) first
- modern section order; trimmed template safety boilerplate into
step-local rules and a skill-specific verification checklist
- tests at tests/skills/test_email_inbox_triage_skill.py (9 passing)
- docs regen scoped: per-skill page + one catalog row + one sidebar line
Fleet audit showed these task skills are commonly needed across users;
shipping bundled per Teknium's direction. Docs and tests follow the
bundled paths.
- optional-skills/devops/actual-setup: field-tested setup skill contributed
by shl0ms, updated for the first-class 'actual' provider (the original
targeted a custom-provider config that now collides with the built-in name)
- docs: providers.md section + tables, environment-variables.md, quickstart.md
- tests/skills: frontmatter + first-class-provider conformance checks
Live-run findings from a real fact-checking task (ankylosing spondylitis
genetics, 7 authoritative sources) against the new mode:
- Verbatim check rejected a legitimate quote because web_extract returns
markdown: the MedlinePlus sentence is "including _[ERAP1](https://...)_,
_[IL1A](...)_" on the wire but plain prose to a reader. The agent was
forced onto a weaker evidence fragment — the opposite of the point.
Matching now canonicalizes inline links to their label and drops
emphasis/code markers and backslash escapes on both sides, so quoting
the sentence a reader sees works. Paraphrases are still rejected.
- Escaped asterisks (HLA-B\*27) no longer have to be reproduced in the
quote, so extractor artifacts stop leaking into rendered evidence.
- New `render --replace-in <draft>`: rewrites a draft's Sources block in
place, idempotently. Previously the only path was hand-slicing the
file, which also tripped over the emitted heading being `## Sources`
while the prose said "Sources:".
- verify stats: report the provenance total that the percentage is
actually computed from (cited + [unverified], counted once), and print
the line as `info:` instead of `warn:` when nothing is wrong. The old
line printed 17 cited / 2 unverified next to 72%, which does not
reconcile — a sentence can be both.
- SKILL.md documents the emitted heading, --replace-in, and exactly what
counts as a prose sentence for --min-coverage.
7 new tests (47 total) using the real MedlinePlus/Frontiers markup;
6 sabotage runs, all red.
Extends the citation ledger with evidence-backed fact-checking:
- New `quote` subcommand attaches verbatim supporting quotes to a
source; the quote is rejected unless it appears verbatim
(whitespace/case-insensitive) in the fetched page text, so a
paraphrase or misremembered figure cannot masquerade as evidence.
- `verify --evidence` fails a draft whose cited sources carry no
attached quote.
- `render --style evidence` prints each source's quotes beneath its
URL, showing the claim -> source -> exact-text chain.
- `[unverified]` marker declares model-knowledge claims; counts toward
--min-coverage so provenance is declared for every sentence without
forcing fake citations.
- SKILL.md: new Fact-Checking Mode section + pitfalls; version 1.1.0.
- 10 new tests (40 total), all proven live by sabotage runs.
Covers the fact-checking/evidence-transparency half of #28289.
Answers and written deliverables that rest on retrieved information now get
inline numbered citations plus a mechanically-rendered Sources list, with a
persistent ledger that makes a hallucinated citation detectable.
- skills/research/grounded-citations/scripts/sources.py: stdlib citation
ledger (add/ingest/list/render/verify) at
$HERMES_HOME/cache/citations/ledger.json, profile-aware, O_EXCL-locked so
parallel subagents sharing a ledger can't collide on ids
- SKILL.md: cite-while-drafting procedure, register-at-retrieval rule,
pitfalls, verification gate
- references/citation-formats.md: per-target placement (markdown, LaTeX/PDF,
docx, pptx, xlsx, wiki, BibTeX handoff to research-paper-writing)
- references/grounding-rationale.md: why numbered ids (ALCE 2305.14627,
WebGPT 2112.09332, Perplexity marker conventions), and how this relates to
the in-process registry in PR #44833
- tests/skills/test_grounded_citations_skill.py: 30 tests
The inlined temp-file + os.replace in openclaw_to_hermes.dump_yaml_file
replaced a symlinked config.yaml with a regular file, silently detaching
managed deployments that symlink ~/.hermes/config.yaml into a dotfiles repo or
profile package. The bare path.write_text it replaced followed the link, and
utils.atomic_replace -- which the hermes_cli twin reaches through
atomic_write_text -- resolves the link for exactly this reason (#16743).
Mirror that here: resolve the symlink before creating the temp file so the
rename lands on the real file, and fall back to copyfile on EXDEV/EBUSY now
that the target can live on another device. Covered by a regression test that
fails when the resolution is removed.
Also guard the permission-denied test for Windows: os.geteuid does not exist
there and chmod-based denial is unreliable, so skip on non-POSIX.
agent_import.py carries a private load_yaml_file/dump_yaml_file pair that
returned {} for an absent file AND for a present file it could not read or
parse. Three importers -- import_permission_allowlist, import_permission_denylist
and import_mcp_servers -- read config.yaml through it, merge one section into
the result, and write the whole mapping straight back. So a YAML syntax error,
a permission problem or a broken mount meant the importer replaced every
setting the user had with only the one to three keys it merged, and still
reported the item as "imported". The write was a bare path.write_text(), so an
interrupted import truncated the file instead.
Distinguish the two cases at the read. Absent, or present but empty, still
yields {} so first-time creation works. Present but unreadable, unparseable, or
not a mapping raises ConfigReadError; the three sites funnel through a new
load_target_config() that records the refusal as a per-item error and leaves the
file byte-identical. Dry-run refuses too, rather than previewing an "imported"
that would destroy the config. dump_yaml_file now writes through
utils.atomic_write_text, which the module already imports and uses for the
memory store.
This is the invariant hermes_cli/config.py already enforces for its own writers
via require_readable_config_before_write / atomic_config_write, whose docstring
names this exact root cause and calls itself "the single chokepoint every
config-update path should use". agent_import.py has its own helper pair and so
was never covered; it was the last config.yaml writer without the guard.
The identical helper pair lives in openclaw_to_hermes.py, the script this module
was ported from, where twelve config.yaml read-modify-write sites share the same
defect; fixed there too. Its refusal is recorded at the run_if_selected dispatch
point, which flips the existing _config_apply_blocked flag so the remaining
config-mutating options short-circuit instead of each rediscovering the same
unreadable file. The atomic write is inlined with tempfile + os.replace because
that script runs standalone with only the stdlib on its path.
Google API and authentication packages permit vulnerable httplib2 and pyasn1
transitives, while the Workspace and Google Chat runtime installers previously
treated any importable version as sufficient. Existing environments could
therefore remain vulnerable after the project dependency pins were repaired.
Carry the fixed versions through the Google and Vertex extras, lazy feature
requirements, lockfile, and both runtime installers. Route the documented
Google Chat installation path through its maintained secure requirements
instead of an unconstrained direct pip command.
Detect stale distributions, install only unsatisfied requirements, and verify
the result before continuing. Behavioral tests cover those repair invariants
without freezing manifests, lockfiles, or complete package sets.
Related #72108
Extracted from #72840
Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
Replace the AST source-parse of tools/lazy_deps.py with a real import
(the module is importable; only setup.py legitimately needs AST since
it is a side-effectful standalone script), and convert the exact
httplib2==0.32.0 snapshot assertion into a >=0.32.0 floor so routine
future bumps don't break the test.
The previous fix (904ade32b) pinned httplib2==0.32.0 in pyproject.toml's
google extra and tools/lazy_deps.py's skill.google_workspace, but missed
a third install path: skills/productivity/google-workspace/scripts/setup.py
REQUIRED_PACKAGES. A user following the --install-deps path could still
resolve httplib2 via unpinned ranges.
This commit:
1. Exact-pins all four Google packages in REQUIRED_PACKAGES to match
pyproject.toml and lazy_deps.py contracts exactly.
2. Adds a focused regression test that parses setup.py's REQUIRED_PACKAGES
via AST and asserts every pin matches the other two install paths.
Changelog: fix(security), test(security)
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.
Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
(DEFAULT_CONFIG smart-approval leaked in) — pinned approval
mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
silently probed live provider catalogs (~2s/test) — stubbed
cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
MagicMock agents — interrupt.side_effect now clears _running_agents
(22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
(24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
with rationale comment — the watchdog-dump assertion needs the loop
blocked well past deadline+grace under parallel load (flaked once in
the 40-worker verification run at a 0.2s margin).
Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
memories/MEMORY.md is the "§"-delimited store written by MemoryStore, not a
markdown document. parse_existing_memory_entries() fell back to
extract_markdown_entries() -- the *source* parser for CLAUDE.md / AGENTS.md --
whenever the destination held no delimiter, which is exactly the case for a
single-entry store or one that was hand-edited or shell-appended. That
extractor skips fenced code blocks, skips table rows, splits a block into one
entry per bullet and reflows paragraphs. The shredded result was then written
straight back over the user's store and reported as "Imported", with no backup
to recover from.
Parse the destination the way MemoryStore._parse_entries does: split on
ENTRY_DELIMITER only, so a store with no delimiter is one intact entry.
extract_markdown_entries() is unchanged and still used on the sources, where
it is correct.
Also restore the safety net the port dropped. The openclaw migration script
this module was ported from calls maybe_backup(destination) before rewriting a
memory store; the port did not. Snapshot the store to <name>.bak.<unix_ts>
(same naming as MemoryStore._backup_drifted_file), refuse to rewrite when the
snapshot fails, and write via temp file + atomic rename so an interrupted
import cannot leave a truncated store and a symlinked MEMORY.md stays a
symlink.
The identical fallback lives in openclaw_to_hermes.py, where it is reached
from migrate_memory() (memories/MEMORY.md and memories/USER.md) and
migrate_daily_memory(); fixed there too.
The bundled office skills (#68595) read user documents and agent-authored
payloads with the locale-default codec:
- docx/powerpoint validators/base.py opened OOXML part XML in text mode
before handing it to lxml. On Windows (cp1251/GBK) the bytes decode to
mojibake that lxml then parses, so validation runs against silently
corrupted document text; on locales where the UTF-8 bytes don't decode
the validator crashes with UnicodeDecodeError instead of validating.
Opening as bytes lets lxml honor the encoding declared in the XML prolog.
- The pdf form scripts (fill_fillable_fields, fill_pdf_form_with_annotations,
create_validation_image, check_bounding_boxes) read the fields JSON the
agent authors — UTF-8 by construction — with the locale codec, so
non-ASCII form values (any Cyrillic/CJK/accented input) either crash or
get written into the user's PDF as mojibake. The json.dump writers use
ensure_ascii=True and were already safe; only the readers needed pinning.
Adds a contract test asserting every document/payload reader is
locale-independent, plus a live regression test that runs
check_bounding_boxes.py on a non-ASCII fields.json under a forced
non-UTF-8 locale — it fails without the fix on both POSIX (C locale)
and Windows (cp1251 chokes on the 0x98 byte of U+2018).
Follow-up on the salvaged commit: the xurl skill loads even when x_search
isn't registered (check_fn-gated on xAI credentials), so per the
cross-toolset reference rule the skill must not name it. Replaced the
skill's x_search routing block and workflow step with skill-native
wording (raw engageable posts, authenticated context, write-evidence
rule). The cross-surface comparison stays on surfaces where both are
known to exist: x-search feature docs, toolset description, tools-config
setup note, and the x_search tool schema (kept generic, no tool names).
Rewrote the routing tests to pin the placement contract, including that
the skill never names credential-gated surfaces.
Make the x_search / xurl boundary explicit in the skill, feature docs,
toolset metadata, setup note, and reference pages, while keeping the
model-facing x_search schema generic (no static xurl name).
Regression tests assert behavioral routing invariants rather than frozen
prose snapshots. Drop the stale CI-only plugin/hangup hunks already on
main so this rebases cleanly.
Optional skill for driving the tldraw offline desktop app via its local
HTTP control API (the same curl-based path the app's own agent skills use
for Codex/Claude Code/Cursor/Gemini) — read the canvas, make live edits,
and write embedded document scripts.
Grounded in the app's bundled script-context.d.ts and agent playbook, and
in the real tldraw SDK v5 shape schema:
- document-script contract: export default function ({ editor, helpers, signal })
- HTTP API: /api/search, /api/doc/:id/exec, /api/doc/:id/script-workspace,
/api/doc/:id/script-status (bearer token from server.json, re-read per call)
- shape schema table validated against @tldraw/tlschema (scripts/validate_shapes.mjs, 3/3)
- interactive-UI example (scripts/counter.js) + diagram-generation (scripts/main.js)
- honest verification boundary: click->state logic verified via /exec dispatch
(0->1->2->1->0), with documented host caveats (inotify watcher, Electron
background-click rejection)
Tests: tests/skills/test_tldraw_offline_skill.py (15 passing).
from a live blocked-sites pass (no PII):
- posture shift: blind opt-out is the DEFAULT, not a fallback -- submit on every site with an
accessible removal channel even without first confirming a listing (own identifiers to the broker's
own official channel = still least-disclosure). guided flows double as the authoritative search.
- blocked-form rule: when a form is automation-hostile (hard captcha / cloudflare / datadome /
slide-to-verify), default to the broker's CITED rights-email rather than recording blocked.
- captcha policy clarified: never defeat behavioral/token/slider challenges; ok to read a static
distorted-text or plain-arithmetic captcha on the subject's own opt-out; stop if the whole
submission is rejected after a correct answer (fingerprinting the automation, not grading it).
- intelius/peopleconnect: delete-wipes-suppression is field-confirmed -- a deletion-complete email
means the suppression is gone and the subject re-lists cluster-wide; re-run suppression and verify
the Control step reads "suppressed". guided-mode session persists; DOB is an <input type=date>.
- new records: addresses.json (intelius front-end, cluster-covered) and socialcatfish.json
(cited rights-email lane + automation-hostile form).
- new references/site-playbooks.md: per-site game-plan matrix (8 blocked-tail sites), the meta-search
no-op skip-list (idcrawl/lullar/yasni/webmii/namesdir/itools/skipease), and the infopay /
peopleconnect backend clusters. OSINT-list triage taxonomy added to methods.md.
- state-machine.md: fixed doc drift + documented submitted->not_found illegal (resolves as
awaiting_processing), blocked->submitted via action_selected, operator_manual_check, --evidence & pitfall.
tests: standalone 99, PR 97 (+1 cluster-coverage regression); ruff + windows-footguns clean.
Teknium