Three caches that read config.yaml still keyed change detection on
st_mtime (or st_mtime_ns + st_size), so a same-size replacement that
keeps the old timestamp (cp -p, rsync -t, a timestamp-pinning writer)
was never noticed:
- model_tools._tool_defs_cache_key: get_tool_definitions kept serving
stale dynamic tool schemas / mcp_servers for the process lifetime.
- CLI mcp_servers auto-reload watcher (cli_tui_mixin seed +
cli_info_mixin._check_config_mcp_changes): the replaced mcp_servers
section was never reloaded. The seed is now _config_sig.
- tui_gateway/server._load_cfg_raw / _save_cfg (_cfg_mtime -> _cfg_sig):
the raw-config cache served the stale document and the next _save_cfg
would write it back over the on-disk file.
All three now use utils.file_signature like the rest of the PR. Tests that
reset the renamed module/instance attributes follow the rename; one
pinned-mtime replacement test per cache, red on the previous head.
Also corrects two stale type/comment annotations in hermes_cli/config.py
(_env_cache key shape, _RAW_CONFIG_CACHE record shape).
Review finding: three sibling config.yaml caches (tool-defs memo, CLI mcp watcher, TUI-gateway raw cfg) still compared mtime/size only.
Follow-up to the salvaged commit from #111169 (@KoNit-K):
- Drop HERMES_KANBAN_TASK_TITLE. The worker already has HERMES_KANBAN_TASK
and HERMES_KANBAN_BOARD/HERMES_KANBAN_DB pinned in its env, so
maybe_auto_title reads the card title from the board itself (no new
HERMES_* env var for non-secret config; the dispatcher and the
delegation scrub list stay untouched).
- Unreadable or missing card: the session is named `Kanban task <id>`
with zero auxiliary calls (the fallback the issue asked for; the
#109743 seed left such workers untitled).
- The card title persists at `llm` authority via set_auto_title, so a
manual /title still wins and the upgrade thread never starts.
- Tests trimmed to two invariants against a real board + SessionDB
(card title, unreadable-card fallback), both red on origin/main.
The fallback root when the system temp dir sits inside the operator's
Hermes home was PROJECT_ROOT/.pytest_cache, but the default install checks
the repo out inside that very home (~/.hermes/hermes-agent,
%LOCALAPPDATA%\hermes\hermes-agent), so the relocated basetemp still
resolved under the native home and get_default_hermes_root() pointed the
sandbox back at the live install. mkdtemp under native.parent is outside
the home by construction, and a loud assertion now fails collection if the
chosen basetemp ever resolves inside it.
Review finding: fallback basetemp under PROJECT_ROOT/.pytest_cache stays inside the native home when the repo lives in ~/.hermes.
Every per-test sandbox is <basetemp>/.../hermes_test, and get_default_hermes_root() prefers
the platform-native home whenever HERMES_HOME sits under it. A basetemp inside ~/.hermes
(pytest --basetemp, or TMPDIR/TEMP pointing there — the default on Windows, where the home is
%LOCALAPPDATA%\hermes) therefore turned every sandbox back into the live install, and any
test that resolves the default profile wrote fixtures over the operator's config.yaml, .env
and MEMORY.md.
Hook into pytest_configure after _pytest.tmpdir has built the TempPathFactory and move a
basetemp that resolves under the native home to a fresh tempdir outside it (falling back to
the repo's ignored .pytest_cache when the system temp dir is itself inside the home). The
fix lives at the basetemp seam so it covers every test, not only the two writers in
test_profiles.py, and needs no per-test fixture or opt-out marker.
Fixes#111101
tests/tui_gateway/test_multi_profile_hosting_fail_closed.py — through the
registered RPC handlers: config.get for a secondary resolves only its own
secrets and flips the process fail-closed; llm.oneshot / model.options bodies
see the profile's home + secrets; a launch-profile agent build binds its own
scope once multiplexing is active; CONTROL: a single-profile serve keeps the
os.environ fall-through.
tests/hermes_cli/test_web_multi_profile_scope.py — through the real FastAPI
app: GET /api/config?profile=b expands only B's refs and never mutates
os.environ; console `send` for B lands B's .env in the request scope, not the
process env.
All red on origin/main by source swap (control green), green on head.
test_profile_terminal_scope_entrypoints follows the launch_profile_policy
rename and releases the launch turn's new secret scope.
tests/conftest.py resets the process-global hosting latch
(_MULTIPLEX_ACTIVE, the frozen launch env, _served_profile_homes) per test:
one request routed to a named profile otherwise left every later test in the
file fail-closed.
The per-process address-space cap was defense-in-depth on top of the
SessionDB leak sweep (nobody sets the knob; the sweep removes the leak).
On the 96-worker CI runner it was also the only PR-specific difference
when tests/tools/test_image_source.py hung to the 600s SIGKILL while the
same file passes in ~30s on every sibling branch: RLIMIT_AS counts virtual
reservations, and image/threading libraries reserve far more address
space than they touch. Keep the sweep, remove the cap and its env knob.
Root cause of the 2026-08-16 OOM incidents (three runs of
`python -m pytest -o addopts= -q tests/hermes_cli/` ballooning to
16-25 GB RSS and getting killed): ~40 files under tests/hermes_cli/
construct SessionDB() directly and never close it. Each instance keeps
the writer connection (state.db + -wal fds), up to _READ_POOL_MAX pooled
readers with their SQLite page caches, and — once token accounting has
run — an atexit registration that pins the instance alive until
interpreter exit. In one process over 637 files those accumulate without
bound; the sanctioned per-file runner masks it, so CI never saw it.
Fix the class, not the sites:
* hermes_state: register every successfully constructed SessionDB in a
test-only WeakSet (populated only when HERMES_TEST_ISOLATION is set,
i.e. under this test suite; production never touches it).
* tests/conftest.py: autouse _close_leaked_session_dbs teardown closes
everything left in the registry after each test. close() is idempotent
and unregisters the pinning atexit hook, so instances become
collectable.
* tests/conftest.py: session-scoped _pytest_memory_cap applies a
defensive RLIMIT_AS of 12 GiB (Linux only) so any future in-process
leak fails fast with MemoryError instead of eating the box.
Overridable/disable-able via HERMES_PYTEST_MEM_CAP (documented in
scripts/run_tests_parallel.py).
* tests/hermes_state/test_session_db_leak_sweep.py: behavior contract
for registration, idempotent close, and the cross-test sweep.
Measured (capped single-process `pytest -o addopts= -q tests/hermes_cli/`):
peak RSS 4.16 GiB before -> 1.67 GiB after; per-test open .db fd count
previously climbed monotonically (0 -> 12 -> 17 -> 104 within the
SessionDB-heavy files), now stays bounded (<= 5, transient). Sanctioned
runner over the affected 35 files: 495 passed, 0 failed, no FLAKY.
Incident evidence: ~/.hermes/logs/oom-incidents/20260816-202114
(fd dumps show 100+ open state.db/state.db-wal handles across pytest
tmpdirs; 3rd recurrence that day).
`scripts/run_tests.sh tests/<dir>/` is how a change gets its regression
coverage run, so a test filed under the wrong directory is a test nobody
runs when that code changes. Two kinds of drift had accumulated.
Parallel directories for one source package, folded into the mirror:
tests/acp -> tests/acp_adapter (its __init__/conftest move with it)
tests/cli -> tests/hermes_cli (prompt_toolkit fixture merged into
hermes_cli/conftest.py)
tests/run_agent -> tests/agent (backoff fixture becomes
agent/conftest.py)
tests/relay -> tests/gateway/relay
tests/state -> tests/hermes_state
246 loose files at tests/ root, routed by the package they import/patch:
hermes_cli, hermes_state, agent, gateway, tools, plugins, tui_gateway, cron.
Installer and desktop-update script tests go to tests/scripts/{install,
desktop_update}/. 43 tests of root-level modules (batch_runner, utils,
hermes_constants, packaging) stay at the root.
Filenames drop their issue numbers (95 files: test_89315_x.py -> test_x.py);
the number stays in the module docstring where it has context.
Collisions: test_cli_skin_integration.py existed in both tests/ and tests/cli
with different subsets — merged into one (10 tests, all kept);
run_agent/test_pre_compress_memory_context.py -> agent/..._handoff.py;
tests/test_account_usage.py -> agent/test_account_usage_fetch.py;
tests/test_web_server.py -> hermes_cli/test_web_server_ws_ping.py.
Deleted: test_minisweagent_path.py (empty since PR #2804),
test_model_picker_scroll.py (tested a private copy of the logic, imported
nothing), test_process_loop_event_loop_warning.py (asserted asyncio behaviour,
imported nothing from Hermes).
Repo-root path arithmetic (Path(__file__).parents[N], dirname chains) is
bumped for the 202 files that changed depth and verified by evaluating every
such expression against the new location. classify_changes' desktop-updater
lane prefix, tests-os.yml's ignore glob and every in-tree path comment follow
the moves. tests/test_tests_tree_layout.py keeps the tree from drifting back.
computer_use kept its own approval decision: two module dicts
(_session_auto_approve / _always_allow) mirroring tools.approval's
session store and _persist_choice, a private verdict vocabulary
(approve_once/approve_session/always_approve) that hermes_cli mapped
back to once/session/always, and — the real problem — `if
_approval_callback is None: return None`. Only the interactive CLI ever
installed that callback, so every other host (gateway turns, cron,
api_server, tui_gateway, ACP) ran destructive desktop input with no
approval at all, ignoring cron_mode / unattended_mode / the permanent
allowlist, and "always" grants were invisible to `is_approved`,
`clear_session` and the messaging-platform approval buttons.
_request_approval now calls tools.approval._run_approval_gate with
pattern_key `cua:<action>:<background|foreground>` (the old scope shape,
so a background grant still never covers the visible foreground variant)
and fail_closed_when_no_human=True, the same posture as
request_tool_approval / the SSH-config write gate. The private dicts,
their release/atexit clearing, the verdict mapping in
hermes_cli/cli_modal_mixin.py and the extra callback install in cli.py
are deleted: the CLI's terminal_tool callback answers computer_use
prompts like any other tool. set_approval_callback stays as an optional
explicit-callback hook with the shared callback contract
(cb(command, description, **kw) -> once|session|always|deny|timeout);
no in-tree host uses it.
Behavior change:
- No approval callback and no gateway (cron, api_server/webhook,
headless -q, plain library use): destructive actions are now REFUSED
with a BLOCKED error and never reach the backend. Previously they
silently ran. cron honors approvals.cron_mode, unattended platforms
approvals.unattended_mode, -q approvals.single_query_mode.
- --yolo / gateway /yolo / approvals.mode: off still allow (unchanged).
- Gateway sessions (Telegram/Discord/Slack/...) now get a real pending
approval with once/session/always buttons instead of default-allow.
- session/always grants live in tools.approval's store; "always" is one
command_allowlist entry (`cua:click:background`) and is scoped to that
action+mode — the old blanket "always_approve unlocks everything for
the session" no longer exists.
- Denial wording is the shared gate's ("BLOCKED: User denied ...",
"BLOCKED: Action timed out ..."); the error JSON keeps `action`.
Tests: tests/tools/test_computer_use_approval_isolation.py
::test_no_callback_refuses_unless_yolo (blocked + no backend call, then
yolo executes) and ::test_always_grant_lands_in_the_shared_store
(is_approved sees the cua:<action>:<mode> key; second call served from
the store). Sabotage: restoring the `callback is None -> allow`
short-circuit fails the first; swapping the shared gate for a private
grant set fails the second plus the three delivery-ladder scope tests.
tests/tools/conftest.py gains `grant_computer_use_approvals` for
dispatch tests that only care about routing.
noninteractive_git_env() now spawns `git config --get-all safe.directory` before
building the env. Eight tests fake subprocess.run/Popen with a fixed sequence of
expected git calls (update check, plugin pull, MCP install, bounded probe) and the
extra spawn tripped them in CI. An autouse fixture stubs the read to "no entries";
the two carve-out invariant tests opt back in with @pytest.mark.real_safe_directory
(and were confirmed to still exercise the real read: the ordering test would fail
against the stub).
contributors/emails: pry@privacydied.net -> privacydied (check-attribution).
#106623 (ca16cafee4) blocks any test subprocess whose argv resolves to
`hermes gateway run|start|restart`, so the harness cannot spawn a runtime that
outlives the worker and restarts the developer's gateway. The matcher strips the
`-p <profile>` selector and reads through `sh -c`, so it also fired on
`docker exec -u hermes <ctr> sh -c 'hermes -p x gateway start'` and broke
tests/docker/test_profile_gateway.py on every Docker build since. A gateway
launched inside a container cannot reach the host unit or webhook port; the
guard now skips commands whose argv[0] is a container runtime (docker, podman,
nerdctl). Host-side spawns stay blocked.
Tests that exercise the dashboard's gateway-restart path can end up
spawning a REAL `python -m hermes_cli.main gateway restart` child when
the spawn seam is not intercepted. `_spawn_hermes_action` launches it
with start_new_session=True, so it outlives the pytest worker; the
child inherits the pytest-tmp HERMES_HOME, which is not a profile and
hashes to no service suffix, so `get_service_name()` resolves the
DEVELOPER's `hermes-gateway` unit, `systemd_restart` restarts the live
gateway, and without systemd the fallback runs `run_gateway()`
in-process forever and squats the webhook port.
Live repro on this machine (origin/main): an unintercepted spawn of
["gateway", "restart"] from a test restarted the production gateway
(MainPID 136820 -> 1689090, NRestarts=1). On 2026-09-03 a sibling
refactor moved `_spawn_hermes_action` from the `hermes_cli.web_server`
facade to `hermes_cli.web_server_gateway` ~10 minutes before the tests
were repointed; runs in that window patched a name production never
read and left 39 orphans alive for six days.
The live-system guard now rejects any subprocess primitive whose
command line the canonical matcher (`gateway.status.
_gateway_command_subcommand`) classifies as `gateway run|start|restart`.
Argv substrings are never consulted, so `gateway status`, `gateway
--help`, `hermes_cli.main serve`, etc. pass through. Three files that
deliberately spawn and reap a stub child with a gateway-shaped argv
(flock holders, sleep sleepers with an argv tail) opt out with the new
`spawns_gateway_lookalike` marker, which lifts only this check and keeps
os.kill guarded. Two canary tests pin the block and the pass-through.
Adds plugins/web/perplexity — a keyed-only WebSearchProvider over httpx:
- search: POST https://api.perplexity.ai/search (documented Search API),
search_context_size=low so `snippet` stays description-sized;
results[].snippet -> description, max_results capped at the API's 20.
- extract: POST /sdk/content/snippets — the query-relevant page-excerpt
route behind `pplx content snippets` (the CLI's `content fetch` is
deprecated upstream). web_extract has no query, so the URLs' path words
serve as the relevance query; per-URL `error` entries survive a 200.
- Wired into the same touchpoints as the other keyed vendors: legacy
backend set + credential ladder + availability probe (web_tools),
registry preference walk, OPTIONAL_ENV_VARS, `hermes config`/status/
dump key lists, nous_subscription direct-credential detection, setup
summary, test conftests, docs.
Not a keyless-ring member (Perplexity has no anonymous tier). Related
closed PRs #9192 / #23981 / #45225 predate the plugin ABC.
hermes_state.py: delete every '# noqa: F401 (re-exported...)' import block (hermes_state_common/errors/guard/
readpool/sessions/fts/dbfile/wal/repair/registry + agent.context_compressor _DB_PERSISTED_MARKER_KEY); keep
only the names hermes_state.py itself uses, without noqa.
hermes_state_registry.py: drop get_shared_session_db/release_shared_session_db/close_shared_session_dbs
aliases; every caller (gateway/, tools/, tui_gateway/, cron/, mcp_serve, run_agent, tests) now imports
acquire/release/close_all/release_or_close from hermes_state_registry.
hermes_state_titles.py: drop set_auto_title_if_empty shim (title_generator keeps its getattr fallback).
Re-remove shim-only names restored by 34abf954bd: latest_user_message_row_id (tests call
latest_message_row_id(key, role='user'); role-targeting assertions kept) and get_session_activity (tests
build the snapshot via agent.session_activity.build_activity_snapshot over db.get_session(sid)).
hermes_state_wal._log_once resolves its dedupe sets as module globals instead of via hermes_state;
hermes_state_repair helpers call module globals directly (tests patch hermes_state_repair.<name>).
Frozen updater surface untouched (update_cmd_maint imports only SessionDB from hermes_state).
- Tavily plugin deleted (plugins/web/tavily), keyless endpoints and
ring entry removed from keyless_mcp, legacy backend set / credential
ladder / preference walks / rescue key map scrubbed.
- TAVILY_API_KEY deregistered across config, setup, status, dump, and
nous_subscription surfaces. The tvly- redaction pattern stays --
legacy keys in user envs still deserve masking.
- Sibling test pins migrated (keenable/exa stand in where tavily was
the fixture vendor); tavily test suite deleted.
- Docs updated: web-search, configuration, integrations,
environment-variables, tools-reference, web-dashboard, provider
plugin dev guide.
Live-verified from an isolated HERMES_HOME with all web creds blanked:
zero-config resolution lands in the 4-vendor ring, live keyless ring
search succeeds, no tavily anywhere in resolution order.
The adapter godfile split moved credential resolution into
agent/anthropic_credentials.py, which silently disarmed two repository
guards still pointed at the old seam:
- tests/conftest.py::_neutralize_macos_keychain_creds patched only the
adapter re-export, so the default suite lost its protection against
reading the operator's real macOS Keychain. Patch the implementation
owner AND the adapter alias.
- test_oauth_setup_token_keeps_inherited_stdin read only the old source
file; it now scans both seams and fails loudly if the call moves again.
- test_hermetic_side_effect_guards isolates the owner module directly.
The /goal post-turn hook constructs a real SessionDB on an executor
thread at the turn boundary. On a cold or loaded CI runner that
state.db init can exceed send_and_capture's 2s poll window, so the
send lands after the assertion and the test reports the bare
'Expected mock to have been called once. Called 0 times.' (#92130).
Mock _run_post_turn_hooks in the e2e runner — these tests exercise
gateway command dispatch, not goal hooks.
Also scrub TELEGRAM_GROUP_ALLOWED_CHATS / *_GROUP_ALLOWED_USERS / QQ
allowlist env vars in the hermetic conftest: a developer shell with
those set flips _get_unauthorized_dm_behavior to 'ignore' and fails
the pairing e2e test locally.
Three local-environment leaks made tests red locally while green on CI:
- tests/conftest.py: blank HERMES_REAL_HOME and TERMINAL_HOME_MODE per
test. The terminal tool injects both into subprocess envs, so any
pytest run launched from a Hermes session inherits them and the
hermes_constants home-resolution helpers prefer HERMES_REAL_HOME over
the monkeypatched HOME (4 failures in
test_subprocess_home_isolation.py).
- test_modal_sandbox_fixes.py: reset the import-time _YOLO_MODE_FROZEN
flag and pin approval mode to manual in _isolate_approval_state().
HERMES_YOLO_MODE=1 in the launching shell froze True at collection
time and every guard auto-approved (2 failures).
- test_noninteractive_git.py: strip GIT_ASKPASS/VS Code askpass vars in
the fail-fast clone E2E. noninteractive_git_env() intentionally keeps
a working askpass helper, but this test asserts the no-helper path;
under VS Code the helper blocks on the editor until the 30s timeout
(1 failure).
Verified: all 49 tests in the three files pass both in a plain dev
shell (with HERMES_YOLO_MODE=1, HERMES_REAL_HOME, and VS Code askpass
set) and inside an unshare -rn network namespace.
Two production incidents (OOF-77 "larrikin-lollies", OOF-30
"synclare-task-manager") followed the same shape: no
kanban.max_in_progress configured, a busy board, and a 1 GiB hosted VM.
The dispatcher fanned out 26-31 concurrent workers, the host went into
swap-thrash/OOM, and the whole machine — dashboard included — became
unreachable. NAS restart loops then masked the problem: each restart
"recovered" briefly before the kanban dispatcher immediately respawned
unbounded workers.
Building on the cherry-picked max_in_progress-across-both-lanes fix
(PR #28695, credit @Dusk1e), this adds two complementary safeguards to
hermes_cli/kanban_db.py:
1. Memory-DERIVED default concurrency cap. When kanban.max_in_progress
is unset, resolve_max_in_progress() derives a default of
clamp(MemTotal / 512 MiB, 2, 8) — e.g. 2 workers on a 1 GiB VM,
8 on 4 GiB+. Explicit config always wins in either direction. On
hosts where total memory can't be read (macOS/Windows dev machines),
the default stays None (no cap — unchanged behaviour). Wired into
both dispatch entry points (gateway/kanban_watchers.py and
hermes kanban dispatch) so behaviour matches regardless of path.
2. Live memory-PRESSURE guard inside dispatch_once. A static cap can't
see the host's actual memory state (other tenants, bloated
long-lived workers). The dispatcher now samples system memory each
tick via gateway.lifecycle_ledger.sample_memory() and classifies it
with gateway.memory_status.classify_pressure() (same thresholds as
the dashboard memory banner and OOM-suspicion heuristics from
NS-608/NS-656): critical -> spawn nothing this tick; elevated ->
at most one new worker; unknown -> no restriction (fail-open).
Reclaim/promotion bookkeeping still runs under pressure, and
deferred tasks stay queued — nothing is dropped. Restriction is
surfaced on DispatchResult.memory_pressure and logged.
Tests: tests/hermes_cli/test_kanban_memory_guard.py (14 tests) covers
the derived cap (floor/ceiling/fail-open/explicit-config-wins), the
pressure classifier, and dispatch behaviour under critical/elevated/
unknown pressure including defer-not-drop and bookkeeping-still-runs.
An autouse fixture in tests/conftest.py pins the memory sample to
"no data" suite-wide so existing dispatch tests don't depend on the
CI runner's live memory state (opt-out marker: real_memory_guard).
The hermetic conftest now exports HERMES_TEST_ISOLATION (value = the tmp
isolation root) before any test module imports, and re-pins it per test in
_hermetic_environment. hermes_state._running_under_pytest() honors the
marker as a test-context signal alongside PYTEST_CURRENT_TEST /
PYTEST_VERSION.
Why a third layer: PYTEST_* belongs to pytest, and tests that spawn
children routinely rebuild the child env and strip it ("the subprocess
must look like a real CLI" — tests/cli/test_exit_watchdog_signal_arm.py,
tests/hermes_cli/test_config_loader_e2e.py do exactly this on purpose).
Such a child loses the HERMES_HOME redirect and the guard's arming signal
in one step, which is how 700+ zero-message fixture rows (dm:123, chat-1,
wx-chat, ...) landed in a developer's production state.db. The marker is
OURS: stripping it is never required to make a child "look real" (no
production code branches on it except the guard), it inherits by default,
and children that genuinely need a real DB use the sanctioned
HERMES_STATE_DB_GUARD_BYPASS=1 hatch instead.
The ancestry-walk layer (previous commits) stays: it covers children whose
env was rebuilt from a completely empty dict. The marker layer covers the
common **os.environ-derived rebuilds cheaply (one dict lookup, no psutil),
and — unlike ancestry — also covers detached/daemonized children that
escape the process tree.
tests/hermes_state/test_isolation_marker_env.py pins: the conftest export,
the marker-alone arming, the rebuilt-env child refusing the production
path, and the bypass hatch. Sabotage-verified: 3/6 fail without the fix.
test_live_db_guard_ancestry._scrubbed_env now strips the marker too, so
the ancestry tests keep proving ancestry rather than riding the marker.
Six test files still selected an OS branch with a faked host. Each one now
carries the marker for the host that owns the branch, or derives the
expectation from the real host:
- test_clipboard: macos_only on the has_clipboard_image dispatch. The fake
picked the branch, but _macos_has_image needs osascript.
- test_claw: windows_only on the tasklist/powershell scan, with return_value
in place of a side_effect list that pinned the call count.
- test_linux_desktop_entry: the parametrize over "darwin"/"win32" becomes one
marked test per host. A fake left POSIX paths and a POSIX XDG layout.
- test_graphical_browser_detection: linux_only on the display-server arm. The
$BROWSER check runs before the platform branch, so its test stays unmarked.
- test_auth_nous_provider: the fixture pinned linux so the macOS certifi
fallback could not change the result. The assertion now reads the host, so
the macOS lane covers the fallback too.
- test_tts_macos_output and test_voice_mode: the afplay policy exists because
CoreAudio init raises a TCC prompt, which no Linux runner reproduces.
tests/conftest.py refuses collection when one test carries two OS markers.
Each marker skips on all but one host, so two of them make a test that runs
nowhere while every lane reports green. tests/test_os_marker_gating.py pins
that behavior.
The docstring on TestConfirmDestructiveSlash said the Windows job runs it.
The class has no marker, so -m windows_only deselects it.
many tests patched sys.platform or a module's _IS_WINDOWS flag, then
ran on linux ci. the patch selects the branch under test, but the host
does not have the behavior the branch exists for. the test proves the
patch, not the platform. some gated assertions never ran on any host.
this commit adds three markers: linux_only, macos_only, windows_only.
a conftest hook skips a marked test on the other hosts, with a clear
reason. no test fakes a host now. two documented fakes remain
(android/termux, freebsd) because no ci runner exists for them.
each fake site got one of four treatments:
- gate it: the real host supplies the platform; mocks cover real
dependencies only, never host identity
- patch the module's own probe when the subject is the probe's consumer
- assert against the real host when the fake stood in for any non-x host
- delete the patch when it set the value the host already has
bare skipif(sys.platform != ...) guards became markers too. the lane
model skips these on linux and never imports them on windows, so they
ran on no host. platform parametrize tables are now one marked test
per os.
running on real hosts found real errors: a chrome-sandbox failure in
test_gui_command that main hides, and two windows failures fixed here.
the agents.md testing section now documents the policy.
Forensics on a live developer machine found pytest fixture rows inside the
REAL ~/.hermes/state.db — sessions with chat_id 'chat-1', '123', 'wx-chat',
and gateway_routing rows whose scope was literally under /tmp/pytest-of-*/.
A pytest-spawned process also opened the live DB and flipped its journal
mode (journal_mode=DELETE fallback on SQLite 3.50.4) under the WAL-mode
gateway writer, destroying committed transcripts ("Persisted transcript
lagged live cached history ... possible FTS write corruption", 15+
occurrences). The existing live-system guard covers kill primitives but not
the SessionDB/SessionStore write paths.
Root cause (leak vector): the session-level HERMES_HOME sandbox in
tests/conftest.py only created a tempdir when HERMES_HOME was UNSET. On a
machine where the shell (e.g. gateway-launched, or an exported
HERMES_HOME=~/.hermes) hands pytest the production home, the sandbox was
skipped entirely — every argless SessionDB()/SessionStore() and every
collection-time DEFAULT_DB_PATH froze onto the real state.db.
Fixes (fail the class, one owner):
* hermes_state._ensure_test_isolation(): single choke point wired into
SessionDB.__init__ (every construction, incl. read_only). Under pytest
(PYTEST_CURRENT_TEST / PYTEST_VERSION — inherited by subprocess
children), a db path resolving to <real-root>/state.db or
<real-root>/profiles/<name>/state.db raises RuntimeError('live-system
guard: ...') before any connection, mkdir, or journal-mode pragma.
* tests/conftest.py: session sandbox now also tempdir-redirects a pre-set
HERMES_HOME that points at the production root (the actual escape
vector); kanban deny-list capture updated to match. New autouse
_state_db_write_guard fixture honors the existing
@pytest.mark.live_system_guard_bypass marker as the escape hatch and
feeds custom (non-~/.hermes) production roots into the guard deny-list.
* gateway/session.py: SessionStore.__init__ no longer swallows the guard's
RuntimeError into the JSONL fallback — guard trips are loud.
* tests/hermes_state/test_live_db_isolation_guard.py: behavioral
regression tests — production paths (direct, profile, read-only,
unnormalized, default-resolution) raise; tmp HERMES_HOME works; bypass
marker works; SessionStore re-raises guard errors but still degrades on
ordinary failures; subprocess child without HERMES_HOME is refused while
a hermetic child succeeds.
No new HERMES_* env vars; no hardcoded ~/.hermes (platform root comes from
hermes_constants._get_platform_default_hermes_home()).
Cross-file leaks made tests/tui_gateway + tests/test_tui_gateway_server.py
fail when run in one process (issue #57068):
- conftest: _hermetic_environment now re-pins hermes_state.DEFAULT_DB_PATH
to the fake home so no test ever touches the developer's real state.db
- conftest: new autouse _reset_tui_gateway_server_state snapshots/restores
_methods, cfg cache, db handle and _real_stdout, and tears down leftover
sessions via the production _close_session_by_id(..., end_reason=
"test_cleanup") boundary
- per-file fixtures scope patch.dict(sys.modules) to the import only
- drop reload()-based teardowns that duplicated atexit hooks
Conflict resolution vs main: kept main's mod._live_transports.clear() in
the test_protocol.py server fixture alongside the snapshot/restore logic.
Combined run now 792 passed / 0 failed.
Salvaged from PR #57066 by @lEWFkRAD. Fixes#57068.
Guard the .lazy-refresh-incomplete marker writer (update_cmd), launch-time
recovery (main.py), and _early_recovery repair paths behind a two-condition
check: running under pytest AND the target is this live checkout. Sandboxed
tmp_path tests still exercise the real code paths.
Salvaged from PR #72002 by @fcavalcantirj. Fixes#72000.
Co-authored-by: fcavalcantirj <felipe.cavalcanti.rj@gmail.com>
The guard's sys.modules.get() can observe hermes_cli.kanban_db while a
lazy import is still executing on a fixture boundary — the partially
initialized module has no .connect yet (AttributeError flake in the
full-suite verification run, 1/2460 files). A half-imported module has
no callers to guard; skip this round and let the next fixture patch the
completed module.
hermes_cli/main.py calls setup_logging() at module scope. That resolves
get_hermes_home() and attaches rotating file handlers to the ROOT logger via
a QueueHandler. So merely importing it - which many test modules do, directly
or transitively - points the whole pytest session's logging at
<HERMES_HOME>/logs/agent.log and errors.log.
The _isolate_env fixture already sandboxes HERMES_HOME, but fixtures run
after collection has imported the test modules, and by then the handler holds
an absolute path to the real file. Verified by importing hermes_cli.main in a
clean interpreter and walking the queue listener: both handlers pointed at the
developer's own ~/.hermes/logs/.
Measured on a live install: 126 warnings in a personal agent.log came from
test runs rather than the running gateway - phantom 'FakeTree' Discord
registration failures and 'rejected invalid API key' entries whose paths only
exist in tests/gateway/test_api_server_runs.py. That noise makes genuine
warnings hard to find exactly when someone is debugging.
conftest is imported before any test module, so sandboxing HERMES_HOME there
closes the window. The per-test fixture still applies afterwards.
Also fixes 4 pre-existing failures: tests/gateway/test_channel_directory.py
TestBuildFromSessions was reading the operator's real sessions data for the
same reason.
Full gateway+tools suites: 66 failures on clean origin/main, 62 with this
change, 0 new. The regression guard asserts the value captured AT conftest
import - reading os.environ inside a test passes even with the fix removed,
because the per-test fixture has sandboxed it by then.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Re-port of PR #35464 onto the rewritten hermetic conftest:
- autouse _neutralize_webbrowser fixture records open/open_new/open_new_tab
and webbrowser.get() instead of launching a real browser
- autouse _neutralize_macos_keychain_creds defaults the Anthropic Keychain
reader to None, with an opt-in allow_macos_keychain marker
- regression tests in tests/test_hermetic_side_effect_guards.py
- tests/agent/test_anthropic_keychain.py opts in via pytestmark
Salvaged-from: #35464
Co-authored-by: y0shualee <yuxiangl490@gmail.com>
Autouse conftest fixture patches kanban_db.connect to refuse writes whose
resolved DB path lands under the REAL kanban root (captured at conftest
import time, before fixtures rewire the environment). Deny-list, not
allow-list, so hermetic tests moving HERMES_HOME to sibling tempdirs are
unaffected. Lazily attaches only when hermes_cli.kanban_db is already in
sys.modules.
Salvaged from PR #69385 by @smfworks; rebased by hand onto the pruned
conftest and adapted to guard on the resolved DB path (explicit db_path
or kanban_db_path()) rather than kanban_home() alone.
Co-authored-by: Jasmine Naderi <jasmine@smfworks.com>
tools/computer_use/tool.py keeps the CLI approval flow in module-globals:
_approval_callback plus the per-session unlock stores _always_allow /
_session_auto_approve. Any test that installs a callback (or drives CLI
init far enough that the real one is registered) and does not reset it
poisons every later computer-use test in the same process:
* a leaked callback that raises — dead UI infra or a stale two-argument
signature (the contract is (action, args, summary)) — becomes
verdict='deny' in _request_approval, so dispatch tests fail with an
empty backend call list;
* a leaked callback that blocks (the real CLI one waits on an answer
queue) hangs a single-process run forever.
Both are order-dependent: tests/tools/test_computer_use.py passes 220/220
in isolation but shows dispatch failures in single-process full-suite runs
(and, with a blocking leak, a permanent hang observed via py-spy inside
_request_approval -> callback -> queue.get with no timeout).
Fix: an autouse teardown-only fixture resets callback + unlock stores
after every test; tests that install their own callback keep it for their
own duration. Regression pair included: a 'forgetful' test leaves a stale
two-arg callback behind, the next test asserts dispatch still routes to
the backend — red without the fixture (1 failed), green with it (225
passed together with the whole computer-use file).
Running the suite could take over the machine it ran on. Two real
effects, both now closed:
- **The suite spoke through the speakers.** Once any test drove the
`voice.toggle` RPC with `action="tts"`, the handler set
`HERMES_VOICE_TTS=1` in the *live process environment*, and the flag
outlived that test. Every later test that drove a turn to completion
then fed its final response text to `hermes_cli.voice.speak_text` on a
background thread - real synthesis, real playback, no API key needed
(the default `edge` provider is keyless). A developer heard the fixture
string "partial answer complete" out loud. Because the flag is set from
inside the process, `scripts/run_tests.sh`'s `env -i` never protected
against this.
`tests/conftest.py` now blanks `HERMES_VOICE`/`HERMES_VOICE_TTS` per
test, and a new autouse `_audio_playback_guard` stubs `speak_text` and
its playback binding outright, so the speakers stay shut even inside the
test that sets the flag itself. `@pytest.mark.real_audio_playback` opts
out.
- **The suite launched Chrome.** `tests/tools/test_browser_supervisor.py`
spawned a real browser on any machine with Chrome on `PATH`. Its
docstring promised a `HERMES_E2E_BROWSER=1` gate that existed nowhere in
the code. That gate is now real, and the file is marked `integration`
so the default marker filter excludes it. `scripts/run_tests.sh`
forwards `HERMES_E2E_BROWSER` so the documented manual run still works.
Miscellanea
- `tests/test_audio_playback_guard.py`: regression cover for both defences,
driving the real `voice.toggle` handler rather than a stand-in.
Two tests fail deterministically on main depending only on which SQLite the
test interpreter links — nothing about the code under test. Both are green in
isolation and red in the suite / on an older library, the worst diagnostic
shape.
Root cause A — WAL is not always WAL. Hermes refuses journal_mode=WAL on
SQLite builds carrying the upstream WAL-reset corruption bug (3.7.0–3.51.2,
excluding backports 3.50.7 / 3.44.6) and falls back to DELETE. On such a
build NO -wal sidecar is ever created, so
test_wal_checkpoint_truncates_wal_file asserts on a file that cannot exist.
Invisible locally when the repo .venv and the Hermes managed runtime link
different versions (observed: .venv 3.50.4 → DELETE, runtime 3.53.1 → WAL),
so the same test passes for one interpreter and fails for the other.
- tests/conftest.py: add a `requires_wal` marker plus a
pytest_collection_modifyitems hook that skips such tests when the linked
library will fall back to DELETE. The skip reason names the actual
version so it is diagnosable rather than mysterious.
- pyproject.toml: register the marker.
- test_kanban_db_repair.py: mark the -wal-sidecar test.
Root cause B — process-global warn-once dedup. The WAL-fallback warning is
emitted at most once per (process, db_label). Any earlier test in
test_kanban_db.py that opens a kanban.db consumes that one-shot, so
test_connect_falls_back_to_delete_on_locking_protocol sees zero warnings and
fails — but only as part of the file, never alone.
- test_kanban_db.py: clear both dedup sets in the test that asserts on the
warning, with a comment explaining the isolation trap.
The gate deliberately does NOT import hermes_state. That module computes
DEFAULT_DB_PATH from get_hermes_home() at import time, so importing it during
collection — before the per-test _isolate_hermes_home fixture redirects
HERMES_HOME — permanently caches the developer's REAL ~/.hermes/state.db for
the whole session. The first version of this change did exactly that and made
tests read a live 31,881-session production database (test_console_engine
asserted "Total sessions: 2" and got 31881). The version predicate is
duplicated instead, and tests/test_conftest_wal_gate.py pins the two
implementations in agreement across every documented upstream boundary plus
guards against the import coming back.
Verified: on SQLite 3.50.4 the sidecar test SKIPS naming the version; on
3.53.1 it RUNS and passes, so coverage is not lost where WAL works. Clean
main fails exactly these 2 tests under
`scripts/run_tests.sh tests/hermes_cli/ tests/test_hermes_state.py`
(9926 passed, 2 failed); with this change the same scope is green.
Tests: 726 passed across the two kanban files, test_hermes_state.py, and the
new gate tests.
Follow-up to the #51899 pick, folding in the config-bridge half of the
competing PR #46925 (@bhanusharma, earliest submitter for the
ignored-channel gate):
- _apply_yaml_config: translate config.yaml slack.ignored_channels into
SLACK_IGNORED_CHANNELS (list or CSV), env-var-wins like every other
bridged Slack key.
- SlackAdapter._slack_ignored_channels / gateway.run's
_slack_ignored_channels_from_gateway_config: fall back to the
SLACK_IGNORED_CHANNELS env var when PlatformConfig.extra carries no
value, so top-level slack: blocks (which flow through the env bridge,
not extra) are honored at both the adapter and runner gates.
- conftest: force-clear SLACK_ALLOWED_CHANNELS / SLACK_IGNORED_CHANNELS /
SLACK_DISABLE_DMS between tests (config-loader side-effect leak class).
- Tests: env-bridge translation + precedence in test_config.py, env
fallback + extra-wins in test_slack_runner_ignored_channels.py.
Credit: #46925 by @bhanusharma proposed the same gate with the YAML→env
bridge; #51899 (picked as the base) carries the wider outbound/runner
coverage. Closes#46925 as consolidated here with first-submitter credit.
- Add SLACK_THREAD_REQUIRE_MENTION, SLACK_IGNORE_OTHER_USER_MENTIONS and
SLACK_REQUIRE_MENTION_CHANNELS to the conftest behavioral-env force-clear
list so config-loader side effects can't leak between tests (same class
of leak the existing SLACK_* entries guard against).
- Document thread_require_mention and require_mention_channels in the
Slack messaging guide next to the other mention-gating options.
* feat(attribution): conflict-free contributor mappings via contributors/emails/ directory
The AUTHOR_MAP dict in scripts/release.py was a merge-conflict magnet:
every concurrent salvage PR appended entries to the same lines of the
same file, so parallel PRs re-conflicted on every merge to main.
New system: one file per email under contributors/emails/ — filename is
the commit-author email, first non-comment line is the GitHub login.
File additions never conflict, so any number of PRs can add mappings
concurrently.
- scripts/release.py: AUTHOR_MAP is now LEGACY_AUTHOR_MAP (frozen)
merged with the directory at import time (directory wins). All
existing consumers (resolve_author, contributor_audit.py) unchanged.
- scripts/add_contributor.py: idempotent CLI to add a mapping; refuses
conflicting reassignments (incl. against the legacy map), validates
email/login shapes.
- contributor-check.yml: attribution gate now accepts a mapping file OR
a legacy entry; failure message prints the exact add_contributor
command. Also auto-resolves bare <login>@users.noreply.github.com
emails is intentionally NOT added (kept id+login form only, matching
previous behavior).
- contributor_audit.py: guidance now points at add_contributor.py.
- tests/scripts/test_contributor_map.py: 12 tests covering loader,
merge precedence, CLI idempotency/conflict/validation, subprocess E2E.
* feat(ci): one-shot per-file flake retry in the parallel test runner
A failing test FILE is re-run once in a fresh subprocess. Pass-on-retry
counts as green but is loudly reported in a '⚠ FLAKY' summary section
(with both attempts' output preserved) so the flake gets fixed instead
of eating a full-run rerun. Deterministic failures fail both attempts —
regressions cannot be laundered green.
- --file-retries N / HERMES_TEST_FILE_RETRIES (default 1, 0 disables)
- E2E verified: simulated first-run-fail flake goes green with banner;
deterministic failure still exits 1; retries=0 restores old behavior.
This converts the dominant CI failure mode (one timing-sensitive test
flaking a 4600-test shard, requiring a manual 10-minute rerun and an
agent triage loop) into a self-healing retry that costs one file's
runtime.
* test(approval): loosen wall-clock perf bounds 0.15s -> 2.0s
These guard against catastrophic regex backtracking (seconds-to-minutes
class), but 0.15s is within scheduler-stall noise on loaded shared CI
runners — test_max_accepted_separator_free_input_is_fast failed a CI
shard this week on runner load alone. 2.0s still catches the regression
class with zero flake surface.
* fix(ci): job timeouts everywhere + retries on all network installs
Reliability pass over every workflow:
- timeout-minutes on all 21 jobs that lacked one (a hung job previously
burned the 6-hour default runner budget)
- ./.github/actions/retry wrapped around every network-fetching install
that lacked it: pip installs (deploy-site, skills-index), npm ci
(deploy-site website, upload_to_pypi web + ui-tui), uv sync (docker
test deps). Deterministic build steps (npm run build) deliberately
NOT retried — split into separate steps so a real build failure fails
fast instead of retrying 3x.
* docs(agents): document the file-retry flake policy
* fix(ci): curl retries on deploy hook + skills-index probe
* fix(ci): kill the remaining transient-failure classes in workflows + Dockerfile
From the workflow reliability audit:
- tests.yml: duration-cache restore had NO restore-keys while saves use
run_id-suffixed keys — the cache never matched once, so LPT slicing
always ran blind and unbalanced slices pushed heavy files toward the
per-file timeout. One-line restore-keys fixes slice balancing.
- Label gates (lint ci-reviewed, supply-chain mcp-catalog-reviewed):
'gh pr view || true' turned an API blip into 'label absent' → false
BLOCKING failure. Now 3x retry, and API failure is reported as an API
failure instead of a missing label.
- detect-changes action: compare API retried before failing open (was
silently running all lanes on any blip).
- uv-lockfile-check: 'uv lock --check' resolves against PyPI — retried
so registry blips don't read as 'lockfile stale'.
- docker.yml merge job: imagetools create retried (Docker Hub eventual
consistency on just-pushed digests).
- Dockerfile: apt-get Acquire::Retries=3; s6-overlay ADDs converted to
curl --retry 3 (ADD cannot retry; checksums still enforced); npm
--fetch-retries=5; playwright chromium fetch retried 3x.
- Advisory artifact uploads (per-slice durations, ci-timings report)
get continue-on-error so an artifact-service blip can't fail a green
test slice.
* fix(tests): kill the two root-cause flakes — leaking pre-warm timer + env-dependent provider list
- test_tui_gateway_server.py: session.create / non-eager session.resume
arm a 50ms threading.Timer (_schedule_agent_build) that outlives its
test and fires into the NEXT test's _make_agent mock, racily
corrupting captured state (the recurring session_resume shard
failures). Replaced the per-test whack-a-mole stub with a module-wide
autouse fixture; the 3 worker-lifecycle tests that genuinely need the
deferred build opt back in via @pytest.mark.real_agent_prewarm (new
marker in pyproject).
- test_api_key_providers.py: PROVIDER_ENV_VARS is now derived from the
live PROVIDER_REGISTRY instead of a hand-list that had drifted
(missing HF_TOKEN / DEEPINFRA_API_KEY) — resolve_provider('auto')
tests failed on any machine with HF_TOKEN exported. E2E-verified with
HF_TOKEN/DEEPINFRA_API_KEY set: 42/42 pass.
* test: de-flake 30 timing-sensitive test files for loaded CI runners
Root-cause fixes from the flake audit (session-DB mining + repo sweep):
Event-based sync instead of sleep-sync:
- title_generator: mock sets threading.Event, wait(10) replaces
sleep(0.3) hoping the daemon thread got scheduled
- docker zombie_reaping / profile_gateway: poll-for-state helpers
replace fixed 1-3s sleeps (s6 transitions + SIGCHLD reaping are async)
- process_registry tree test: select()-bounded readline replaces an
unbounded blocking read (parent wedge now fails THIS test with a clear
message instead of an opaque rc=124 file kill); SIGTERM grace 1s->2s
(the 1s partition window mid-interpreter-startup is how a child PID
escaped the live-system guard in CI)
Timeout raises (loaded 8-way-sliced runners see ~5s scheduling floors;
all of these complete in ms-to-1s when healthy so the raises cost
nothing on green runs):
- subprocess/thread waits <= 2s raised to 10-15s across mcp_tool,
mcp_circuit_breaker, mcp_reconnect_retry_reset, mcp_parked_self_probe,
mcp_cancelled_error_propagation, registry, clarify_gateway, interrupt,
voice_cli_integration, docker_environment, session_store_lock_io,
planned_stop_watcher, cli_interrupt_subagent, thread_scoped_output
(joins now also assert not is_alive() so stragglers fail loudly)
- wall-clock discrimination ceilings loosened where the guarded hang is
10x larger: local_background_child_hang 4s->10s, interrupt_cleanup
setup 5s->20s + pgid-exit 30s->60s, mcp_stability grandchild spinup
5s->15s, protocol/gil-starvation fast-handler 0.5s->2s,
iso_certify_seam 1.5s->5s, wait_for_mcp_discovery 0.1s->1s
- narrow assertion windows widened: honcho first-turn wait 0.4..0.65 ->
0.25..2.0 (property is bounded-not-hung, not an exact wall-clock);
compression fork-lock TTL 1s->3s (12 refresh chances per lease);
compression-lock expiry margins symmetric (ttl 0.05->0.5, sleep 1.0)
- telegram hung-DNS bound 1.0->1.4 (fake hang is 1.5s — must stay under)
* fix(tests): repair indentation from de-flake batch edit
* fix(tests): harden env isolation and replace remaining sleep-sync races
The full 42k-test run and complete npm check surfaced three more classes:
- Environment isolation: local ~/.honcho defaultHost and SSH_* variables
leaked into Python/TUI tests. Pin the default Honcho host in the
hermetic fixture, isolate the one fallback test from ~/.honcho, and
blank SSH_* around terminalSetup tests. This flipped 20 false failures
back to deterministic behavior on developer machines.
- Background-thread sleep-sync: Honcho async writer tests patched
time.sleep globally, then busy-polled with that same mocked sleep. Under
full-suite load the poller could starve the writer. Each test now waits
on an Event emitted by the exact flush/retry transition; 30/30 passed
under 15-way contention.
- Desktop streaming: the test slept 80ms and assumed a 500ms timer could
not fire before its assertion. A loaded runner descheduled the test for
>500ms and both chunks arrived. Producer controls now gate second-chunk
and completion transitions explicitly.
Also make file-retry observability complete: a self-healed flaky file now
prints BOTH attempts' full output in the FLAKY summary. Two behavioral
runner tests prove pass-on-retry is green+loud+traceback-preserving, while
a deterministic failure remains red.
* refactor(ci): use gh bot pat, better retries
refactor(ci): use retry action for PR label fetch
the retry action now captures stdout as a step output, so it can serve
double duty: retry + output capture for commands like 'gh pr view' whose
result must be consumed by later steps.
Retry action gains:
- 'stdout' output (heredoc-delimited to preserve newlines)
- tee to temp file so stdout still streams to the job log
- step id 'retry' for output reference
Both lint.yml and supply-chain-audit.yml now use the retry action
directly with 'command: gh pr view ...' and read
steps.<id>.outputs.stdout.
ci: use AUTOFIX_BOT_PAT for all gh CLI / GitHub API auth
Replace secrets.GITHUB_TOKEN and github.token with
secrets.AUTOFIX_BOT_PAT across all workflows and composite actions
that use the gh CLI or GitHub API. The PAT has consistent permissions
across fork PRs (where GITHUB_TOKEN is read-only), avoids API rate
limit sharing with the default token, and is already used by
js-autofix.yml for the same reasons.
19 sites swapped across 9 files:
- lint.yml (3): label fetch, comment post/edit, comment update
- supply-chain-audit.yml (5): scan, critical comment, unbounded dep
comment, label fetch, mcp-catalog comment
- lockfile-diff.yml (1): PR comment post/update
- skills-index-freshness.yml (1): issue creation on degraded probe
- skills-index.yml (2): index build, trigger deploy workflow
- upload_to_pypi.yml (2): release view poll, release upload
- ci.yml (1): timings report
- deploy-site.yml (2): skills index crawl
- detect-changes/action.yml (1): compare API call
---------
Co-authored-by: ethernet <arilotter@gmail.com>
* test: deflake CI and dev-machine flaky tests in bulk
Fixes ten distinct flake sources found by mining recent CI failures and
running the full suite on a dev machine with real user state:
CI-observed races:
- tests/conftest.py live-system guard: allow signal 0 (pure liveness
probe) through _guarded_kill/_guarded_killpg. psutil.pid_exists()
probes a just-killed grandchild reparented to init; the subtree check
fails for it and the guard RuntimeError'd
test_entire_tree_is_sigkilled_not_just_parent intermittently on
unrelated PRs.
Hermeticity flakes (fail on dev machines with real state, pass on CI):
- agent/coding_context.py: _marker_root() now skips the shared temp
root (tempfile.gettempdir()) like it skips $HOME — a stray
/tmp/package.json flipped every tmp_path test into the coding
posture (9 failures in test_coding_context.py).
- test_agent_guardrails.py: pin MAX_CONCURRENT_CHILDREN=3 via autouse
monkeypatch instead of freezing the user's real config value at
import time (import-time vs call-time config mismatch).
- test_web_tools_config.py: TestCheckWebApiKey now neutralizes the
ddgs package probe and registry providers — the optional ddgs
package in a dev venv lit up the fallback backend.
- test_credential_pool.py: block claude_code/hermes-oauth credential
autodiscovery in the two pool-merge tests that assert exact id
lists (a real ~/.claude/.credentials.json seeded an extra entry).
- test_modal_sandbox_fixes.py: clear _permanent_approved /
_session_approved — the user's real command_allowlist silently
approved the guard-escalation commands under test.
- test_setup_irc.py: stub prompt_checklist to select only the IRC row;
the non-TTY cancel fallback re-ran the real configured platforms'
interactive setup_fn, which hit input() under captured stdin.
- test_doctor.py: TestGitHubTokenCheck now patches the module-level
HERMES_HOME constant (the file's established pattern) instead of
only setenv — doctor was running PRAGMA integrity_check against the
real multi-GB state.db and blowing the 300s per-file budget.
Latent atexit-duplication (same _enter_buffered_busy class as #34217):
- test_undo_command.py: drop importlib.reload(tui_gateway.server) in
fixture teardown; reload re-registers the module's atexit hooks.
- test_session_platform_resolution.py: drop per-test reload of
tui_gateway.server; every resolver reads env at call time.
* test: sentinel model value in ignore-user-config fallback assertion
With HERMES_IGNORE_USER_CONFIG=1, load_cli_config() falls back to the
repo-root cli-config.yaml (untracked, gitignored). On a dev machine that
file can legitimately set the same popular model the test hardcoded
(anthropic/claude-sonnet-4.6), flipping the != assertion locally while
CI (no cli-config.yaml) stayed green. Use an impossible sentinel model
name instead.
New hermes_cli/managed_scope.py resolves a system-level managed directory
(HERMES_MANAGED_DIR override > /etc/hermes), parses managed config.yaml/.env
with fail-open semantics, and exposes is_key_managed/is_env_managed helpers.
The system default is ignored under pytest and HERMES_MANAGED_DIR is added to
the conftest env scrub so a real managed scope can't leak into the suite.
Not wired into the load paths yet (Phases 2-3).
The pyproject addopts pin `--timeout-method=signal` relies on signal.SIGALRM,
which doesn't exist on Windows. pytest-timeout raised AttributeError at timer
setup and aborted the entire run before any test executed, so the suite was
unrunnable on Windows by default. Override timeout_method to "thread" on
Windows in pytest_configure; POSIX keeps the more reliable signal method.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends _live_system_guard in tests/conftest.py to block any subprocess
call that would run 'hermes update' (or 'python -m hermes_cli.main update')
against the real checkout.
These commands run git fetch origin + git pull, overwriting repo files
like pyproject.toml mid-test-run and corrupting every subsequent
subprocess that reads them. The spawned process uses setsid /
start_new_session=True so it's invisible to pytest's process tree
(PPid=1) — the corruption was essentially undetectable without
explicit inotify/SHA watchdogs.
Root cause of #43703 CI failures: tests in TestUpdateCommandPlatformGate
called _handle_update_command() with HERMES_MANAGED='' and no Popen mock,
causing the code to fall through and spawn a real 'hermes update --gateway'
that overwrote pyproject.toml with origin/main's content (which still
had '--timeout=30 --timeout-method=thread' in addopts while the PR had
already removed pytest-timeout).
The guard covers all three invocation patterns:
- 'hermes update' / 'hermes update --gateway' (direct or via setsid bash -c)
- 'python -m hermes_cli.main update --gateway'
- '.venv/bin/hermes update' (absolute path variant)
Does not false-positive on: git update-index, apt-get update,
pip install --upgrade, or any command lacking 'hermes'/'hermes_cli'.
teknium1