cf60ebbdfd264c3fc067b9e3230df8752c50d2ec
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 briandevans
|
4541d30181 |
fix(cli): keep newly created SOUL.md and distribution.yaml at 0644
Both files were routed through the shared atomic writers earlier in this branch. tempfile.mkstemp creates the temp file 0600 and the atomic swap carries that mode onto the target, so the *create* paths silently tightened two files that previously landed at the umask default: - web_routers/profiles.py: the dashboard persona editor's first-ever Save has no prior SOUL.md to copy permissions from, so the existing guard skipped the chmod entirely -- contradicting the comment directly below it, which states profile SOUL.md is created 0644 and is not run through _secure_file. - profile_distribution.py: atomic_yaml_write only restores a mode it captured from a file that already existed. _materialize() calls write_manifest() with no manifest on disk whenever a distribution declares an explicit distribution_owned allowlist that omits distribution.yaml, so the staged copy is never placed in the profile. Both are fixed with a local chmod at the two sites this branch regressed; utils.py's public mode semantics are left alone. profiles.py now also distinguishes "no file yet" (FileNotFoundError -> 0644) from "stat failed for some other reason" (-> leave the mode alone rather than guess at it). uninstall.py and xai_retirement.py have no create path and are unchanged: the former captures prior_mode unconditionally after a successful read_text(), and the latter runs require_readable_config_before_write() first. |
||
|
briandevans
|
c005546cbc |
test(cli): skip the permission-preservation cases on Windows
The four new mode-preservation guards assert POSIX permission bits, which Windows does not model (os.chmod only toggles the read-only flag there). Guard them the way the suite already guards POSIX-specific semantics so the tests stay meaningful on Linux/macOS without failing for Windows contributors. The symlink cases stay unguarded, matching the existing symlink tests in tests/hermes_cli/. |
||
|
briandevans
|
67827dd99e |
fix(cli): route the remaining destructive user-file rewrites through atomic writes
`utils.atomic_write_text`'s docstring states the invariant: it exists "so that
every destructive file rewrite in the codebase shares one implementation."
Four full-file rewrites of *existing user-authored files* still bypass it and
use a bare truncating `open(path, "w")` / `Path.write_text()`, which truncates
the target before the new content is produced. A crash, SIGINT, or ENOSPC
mid-write therefore leaves the file empty or half-written.
In all four cases the read half degrades silently to a default rather than
erroring, so the damage is invisible and the next write cements it:
* `xai_retirement.apply_migration()` rewrites the user's config.yaml. Merged
commit
|
||
|
Teknium
|
ee5a66ae3e |
fix(distribution): path-aware allowlist; preserve legacy copy-everything when omitted
Follow-ups to the previous commit (#74414 by @webtecnica, re #74373): - When distribution_owned is OMITTED, restore the legacy contract: every staged entry outside USER_OWNED_EXCLUDE is copied. The cherry-picked filter consulted owned_paths(), which silently narrowed omitted-list distributions to DEFAULT_DIST_OWNED and dropped undeclared payload (extra top-level files/dirs existing distributions legitimately ship). - Make explicit allowlists path-aware so documented nested entries like skills/research/ and cron/digest.json select exactly that subtree/file instead of being dropped by the top-level name comparison. Traversal segments (.., absolute) and USER_OWNED_EXCLUDE roots are still rejected. - Regression tests: omitted-list legacy behavior + nested-path allowlist. |
||
|
webtecnica
|
a42e3e8ba4 |
fix(distribution): respect distribution_owned allowlist in _copy_dist_payload
_copy_dist_payload() in profile_distribution.py iterated all staged entries without consulting the manifest's distribution_owned allowlist, so manifests that restricted distribution_owned only had cosmetic effect. Fix: compute manifest.owned_paths() at the top of _copy_dist_payload() and skip entries not in that set, after the USER_OWNED_EXCLUDE check. The owned_paths() method already existed on DistributionManifest and correctly falls back to DEFAULT_DIST_OWNED when no explicit distribution_owned is set, so the new filter preserves backward compatibility for existing manifests. Closes #74373 |
||
|
Teknium
|
39975613b1 |
test: prune wave 2 + speed fixes — 28,106 → 19,757 test functions, suite wall 315s → 294s
Second, deeper pass over tools/gateway/hermes_cli plus first pass over the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker, dashboard, conformance, monitoring, secret_sources, hermes_state, providers). Same rubric as wave 1 (AGENTS.md test policy); security, alternation/caching invariants, issue-number regressions, and E2E kept. Real test-quality fixes found and rooted out along the way: - tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls (DEFAULT_CONFIG smart-approval leaked in) — pinned approval mode=manual via autouse fixture: 17.4s → 0.4s. - test_model_switch_custom_providers.py / test_user_providers_model_switch.py silently probed live provider catalogs (~2s/test) — stubbed cached_provider_model_ids/provider_model_ids/fetch_api_models. - test_telegram_noise_filter.py: 15-platform copy-paste matrix over shared gateway.run logic → 3 representative platforms (55s → 3.9s). - test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on MagicMock agents — interrupt.side_effect now clears _running_agents (22s → 1.0s). - test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x (24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait 5s → 0.5s. - test_telegram_init_deadline.py: loop-block margin restored to 1.0s with rationale comment — the watchdog-dump assertion needs the loop blocked well past deadline+grace under parallel load (flaked once in the 40-worker verification run at a 0.2s margin). Verification: full hermetic suite via scripts/run_tests.sh — 2,438 files, 21,718 tests passed, 0 failed, 293.9s wall. Suite totals vs original baseline: 46,820 → 19,757 test functions (−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s. |
||
|
Teknium
|
6b81590c55 |
test: prune low-value tests suite-wide (wave 1) — 46,820 → 28,106 test functions
Systematic prune per AGENTS.md test policy, one pass over every major test tree (gateway, hermes_cli, tools, agent, run_agent, plugins, cli, cron, tui_gateway, honcho/openviking, root-level): - DELETE: source-reading tests (read_text/getsource on prod files), change-detector tests (exact catalog counts, model-name snapshots, config version literals), mock-echo tests (assert a mock returns what it was told), assertion-free/trivial tests, near-duplicate parametrizations (boundaries + one representative kept), async/sync twin duplicates, cosmetic within-file variations. - KEEP (mandatory): security/redaction/approval guards, message-role alternation invariants, prompt-caching/deterministic-call-id invariants, issue-number regression tests (deduped), E2E tests. - 6 test files deleted outright (script-style/no-assert or fully redundant); conftest.py, fakes/, fixtures/ untouched. - tests/acp/conftest.py added: autouse fixture stubs the live models.dev/GitHub/Copilot/Anthropic inventory fetches that ACP server tests performed on every session create — test_server.py 147s → 3.4s, and the tests are now genuinely hermetic. - Sleep-based slowness shrunk where safe (codex_ttfb_watchdog, compression_concurrent_fork, etc.); no wall-clock assertion tightened. Verification: full hermetic suite via scripts/run_tests.sh — 2439 files, 31,130 tests passed, 0 failed, 0 flaky retries, 315s wall (baseline: 583s wall, 13,564s subprocess CPU). |
||
|
islam666
|
e53b74c394 |
fix(dist): stop USER_OWNED_EXCLUDE from filtering nested directories
The copytree ignore lambda in _copy_dist_payload applied USER_OWNED_EXCLUDE recursively at every directory depth. This caused nested directories whose names matched exclude entries (bin, logs, cache, etc.) to be silently dropped during distribution install/update. Fix: only apply USER_OWNED_EXCLUDE filtering at the root of the staged tree, matching the two-tier pattern used by _clone_all_copytree_ignore and _default_export_ignore in profiles.py. Add 5 tests covering nested bin/logs/cache preservation and top-level filtering still working. Fixes #37954 |
||
|
kshitijk4poor
|
66827f8947 |
chore: prune unused imports and duplicate import redefinitions
Remove unused imports (F401) and duplicate/shadowed import redefinitions (F811) across the codebase using ruff's safe autofixes. No behavioral changes -- imports only. - ~1400 safe autofixes applied across 644 files (net -1072 lines) - __init__.py re-exports preserved (excluded from F401 removal so public re-export surfaces stay intact) - Re-exports that are imported or monkeypatched by tests but look unused in their defining module are kept with explicit # noqa: F401 (gateway/run.py load_dotenv; run_agent re-exports from agent.message_sanitization, agent.context_compressor, agent.retry_utils, agent.prompt_builder, agent.process_bootstrap, agent.codex_responses_adapter) - Unsafe F841 (unused-variable) fixes deliberately skipped -- those can change behavior when the RHS has side effects - ruff lints remain disabled in pyproject.toml (only PLW1514 is selected); this is a one-time cleanup, not a config change Verification: - python -m compileall: clean - pytest --collect-only: all 27161 tests collect (zero import errors) - core entry points import clean (run_agent, model_tools, cli, toolsets, hermes_state, batch_runner, gateway) - static scan: every name any test imports directly from an edited module still resolves |
||
|
nguyen binh
|
46d8b5dadf | fix(profile): reject symlinks in distributions (#25292) | ||
|
Teknium
|
f209a35859 |
feat(profile): shareable profile distributions via git (#20831)
* feat(profile): shareable profile distributions (pack/install/update/info) Closes #20456. Turns a profile into a portable, versioned artifact. Packs SOUL.md, config, skills, cron, and an env-var manifest into a tar.gz that others can install from a local path, URL, or git repo. Updates re-pull the distribution while preserving user data (memories, sessions, auth.json, .env) and the user's config.yaml overrides. New subcommands (under hermes profile, no parallel tree): hermes profile pack <name> [-o FILE] hermes profile install <source> [--name N] [--alias] [--force] [-y] hermes profile update <name> [--force-config] [-y] hermes profile info <name> Manifest (distribution.yaml at the profile root): name, version, hermes_requires, author, env_requires, distribution_owned. Security: - Installer shows manifest + env-var requirements before mutating disk; confirmation required unless -y. - auth.json and .env are never packed (same exclude set as profile export). - Cron jobs are packed but NOT auto-scheduled — user is pointed at 'hermes -p <name> cron list' to review. - Archive extraction rejects path traversal (../ members). - Alias creation is opt-in via --alias. Update semantics: - Distribution-owned paths (SOUL.md, skills/, cron/, mcp.json, manifest): replaced from the new archive. - config.yaml: preserved by default; --force-config to overwrite. - User-owned paths (memories/, sessions/, auth.json, .env, state.db*, logs/, workspace/, plans/, home/, *_cache/, local/): never touched. Version pin: hermes_requires accepts >=, <=, ==, !=, >, < or a bare version (treated as >=). Install fails with a clear error when the running Hermes version doesn't satisfy the spec. Sources supported by 'install': - Local .tar.gz / .tgz archive - Local directory - HTTP(S) URL pointing to a .tar.gz (uses httpx, already a dep) - Git URL (github.com/user/repo, https://..., git@..., ssh://, git://) Tests: 43 new unit tests (manifest parsing, version checks, env template, pack/install/update round-trip, config-preservation, security). E2E validated via real CLI invocations against an isolated HERMES_HOME covering pack, install with confirmation, update preservation, update --force-config, decline-preview, duplicate-install rejection, and version-requirement rejection. * refactor(profile-dist): git-only — drop tar.gz/HTTP transports and pack Scope-cut on top of the original distribution PR: a profile distribution is now exclusively a git repository (or a local directory during development). The tar.gz / HTTP archive transports and the matching `hermes profile pack` subcommand have been removed. Why: * GitHub tags, branches, and commits are already the right versioning primitive. Tag pushes do for us what 'pack + upload' did. * `hermes profile export` / `import` already cover local backup and restore; they are not a distribution format and stay untouched. * One transport means one install/update code path, one doc page, and one mental model. The extra source types doubled the surface for no real user win — GitHub auto-attaches release tarballs, and `git bundle` / `git clone --mirror` cover the airgap case. Changes: * hermes_cli/profile_distribution.py — removed pack_profile, _fetch_tar_archive (_http_fetch), _safe_extract, _archive_roots, _safe_parts, _find_dist_root, tarfile/io/urlparse imports. The new _stage_source has two arms: git URL → clone, local directory → use in place. * hermes_cli/main.py — removed the 'pack' subparser and action handler. Install help text updated to match the reduced source list. * tests/hermes_cli/test_profile_distribution.py — rewritten around a local-directory staging fixture. The install/update/describe suites now build a distribution tree on disk directly and install from it, which is what a real git clone produces after .git is stripped. Dropped TestPack, TestFindDistRoot, and the tar-specific security test. New tests cover _looks_like_git_url, env_example emission, hermes_requires enforcement, and 'installer does not import credentials if an author mistakenly leaks them in the staging tree'. * website/docs/reference/profile-commands.md — 'Distribution commands' section rewritten around git. Added a 'Publishing a distribution' section. export/import stay documented as local backup/restore. * website/docs/reference/cli-commands.md — dropped 'pack' from the profile subcommand table. * website/package.json — 'lint:diagrams' now passes --exclude-code-blocks to ascii-guard. Without it, markdown tables and box-drawing diagrams inside fenced code blocks were being misidentified as malformed ASCII boxes, blocking the PR's docs-site-checks CI with 8 false-positive errors. Validation: * Targeted suite: tests/hermes_cli/test_profile_distribution.py — 56/56 pass (down from 43 — reorganized to cover the new local-dir paths). * Regression: test_profiles.py + test_profile_export_credentials.py 102/102 still pass. export/import behaviour unchanged. * Docs lint: ascii-guard lint --exclude-code-blocks docs returns 0 errors (was 8 on the PR before the flag bump). * E2E: ran the real `hermes profile install`/`info` against a local staging dir under an isolated HERMES_HOME — install writes SOUL.md + skills to the target profile, info reads the manifest back, a bogus source produces a clear error, and `hermes profile pack` is now rejected by argparse as expected. * feat(profile-dist): distribution-aware list/show/delete + installed_at + env preview Polish pass on top of the git-only scope cut. Five additions, all small, wiring into existing commands rather than adding new surface. 1. `installed_at` timestamp on the manifest * Stamped automatically inside plan_install() on both fresh install and update — ISO-8601 UTC, seconds resolution. * Surfaced in `hermes profile info` as `Installed: <ts>`. * Lets users tell "installed 6 months ago, needs update" from "installed yesterday" without guessing from file mtimes. 2. `hermes profile list` grows a `Distribution` column * Plain profiles: "—" * Distribution profiles: "<name>@<version>" (e.g. `telemetry@1.2.3`) * ProfileInfo gains three optional fields — distribution_name, distribution_version, distribution_source — populated by a new _read_distribution_meta() helper that swallows manifest read errors so a broken distribution.yaml in one profile can't break `list` for the others. 3. `hermes profile show` and `hermes profile delete` surface distribution provenance * show: `Distribution: name@version` + `Installed from: <source>` plus a pointer to `hermes profile info <name>` for the full manifest. * delete: same lines in the pre-confirmation preview, so a user deleting "telemetry" can see it came from `github.com/kyle/telemetry-distribution` before they type `telemetry` to confirm. No change to the confirmation gate itself — deletion semantics are identical to plain profiles. 4. Install preview checks env vars against the current environment * Replaces the "Env vars you'll need to set:" header with a simpler "Env vars:" block. * Each required var is labeled: - `✓ set` — already in `os.environ` OR present as a key in the target profile's existing .env (update case). - `needs setting` — required but not found in either place. - `—` — optional. * Mirrors pip's "Requirement already satisfied" UX: no unnecessary nagging about keys the user already has configured. 5. Docs: private distributions * New "Private distributions" section in website/docs/reference/profile-commands.md explaining that we shell out to the user's `git` binary, so SSH keys / credential helpers / GitHub CLI stored creds all work transparently. One paragraph, two examples. * `hermes profile info` section updated to mention `Installed:`. Module-level hoist: * `from datetime import datetime, timezone` was previously lazy-imported inside plan_install(). Hoisted to module scope so tests can monkeypatch `hermes_cli.profile_distribution.datetime` to freeze time. Tests (+7): * TestInstalledAtStamp.test_install_stamps_installed_at — format check (4-digit year, 'T', +00:00 suffix). * TestInstalledAtStamp.test_update_refreshes_installed_at — freezes datetime.now() to 2099-01-01 and confirms update writes a new stamp. * TestProfileInfoDistribution.test_installed_distribution_shows_in_list — ProfileInfo.distribution_{name,version,source} populated after install. * TestProfileInfoDistribution.test_plain_profile_has_no_distribution_fields — plain profiles have None. * TestProfileInfoDistribution.test_malformed_manifest_does_not_break_list — broken distribution.yaml in one profile doesn't break list_profiles(). Validation: * 163/163 tests pass (56 distribution + 102 profile regression + 5 new from this commit — up from 158). * docs-lint: 0 errors. * E2E verified: install preview shows ✓/needs-setting per env var, `profile list` shows Distribution column, `profile show` + `delete` preview mentions source URL, `info` shows Installed: timestamp. * fix(profile-dist): clean errors + warn when overwriting plain profiles Two small polish fixes found during collision sweeps of the PR: 1. ValueError from validate_profile_name now caught cleanly * A distribution.yaml whose 'name' field can't be used as a profile identifier (spaces, path traversal, etc.) raises ValueError from hermes_cli.profiles.validate_profile_name, which was escaping as a raw Python traceback from 'hermes profile install/update/info'. * Broadened the except clause in all three handlers to catch (DistributionError, ValueError) — users now see: Error: Invalid profile name '../../etc/passwd'. Must match [a-z0-9][a-z0-9_-]{0,63} instead of a stack trace. 2. Install preview distinguishes plain profile overwrite from distribution re-install * When plan.target_dir exists and IS a distribution (has distribution.yaml), preview still shows the mild (profile exists — will overwrite distribution-owned files only) * When plan.target_dir exists but is a HAND-BUILT plain profile (no distribution.yaml), preview now shows a loud warning: ⚠ Profile exists but is NOT a distribution. Installing here will overwrite its SOUL.md, skills/, cron/, and mcp.json. Your memories, sessions, auth.json, and .env will be preserved, but any hand-edits to distribution-owned files will be lost. * Users who type 'hermes profile install foo --force' against a profile they hand-built now see what they're signing up for. User data is still safe (memories, sessions, auth, .env are in USER_OWNED_EXCLUDE), but custom SOUL/skills get stomped. Tests (+2): * TestErrorSurfaces.test_bad_profile_name_raises_valueerror_not_traceback * TestErrorSurfaces.test_path_traversal_name_rejected Validation: * 165/165 tests pass (was 163). * E2E: bad manifest names produce 'Error: Invalid profile name ...' with no traceback; installing over a plain profile shows the warning; re-installing over an existing distribution shows the normal overwrite message. * Bad HTTPS URLs still produce 'Error: git clone failed: ...' — git itself generates a clean enough message that no wrapper is needed. * 'install .' works correctly from any cwd. * fix(profiles): reject reserved names at validate time Before: `hermes profile create hermes` / `profile install` / `profile rename` all silently accepted reserved names like `hermes`, `test`, `tmp`, `root`, `sudo`. The profile directory was created; only alias creation failed (via check_alias_collision), leaving a confusingly-named profile on disk — e.g. `~/.hermes/profiles/hermes/` sitting next to `~/.hermes/` itself. The reserved set already exists (_RESERVED_NAMES, introduced alongside alias collision detection). This commit moves the check up one layer to validate_profile_name so every entry point — create, install, import, rename, dashboard web API — shares the same gate. The error message points the user at the cause without being cryptic: Error: Profile name 'hermes' is reserved — it collides with either the Hermes installation itself or a common system binary. Pick a different name. `default` continues to pass through (it's a special alias for ~/.hermes). _HERMES_SUBCOMMANDS (`chat`, `model`, `gateway`, etc.) stays at alias-collision time only — those are fine as bare profile names with `--no-alias`. Tests (+5): test_reserved_names_rejected parametrized over the full _RESERVED_NAMES set, matching the existing pattern in TestValidateProfileName. No existing test uses a reserved name as a profile identifier (greppped create_profile("hermes|test|tmp|root|sudo") — zero hits). Validation: * 170/170 tests pass in the profile suites. * E2E: `profile create hermes`, `profile install` with manifest name=hermes, and `profile install ... --name hermes` all produce the same clean `Error: Profile name 'hermes' is reserved ...` with rc=1 and no traceback. Normal names (`mybot`) still work. |