Commit Graph

16 Commits

Author SHA1 Message Date
kshitijk4poor f003e449be refactor(cron): trim the scope-degrade dispatch to its invariants
Follow-up to the cherry-picked #102431 fix, addressing the review findings:

- The two real-helper scheduler tests ran the Linux-only helper unmarked
  and failed on macOS/Windows; the surviving one is now `linux_only`.
- `_warn_scope_degraded_once` used a bare `os.getuid()` that tripped
  scripts/check-windows-footguns.py --all (lint lane red). The remedy text
  is now built once in the helper and passed into the warning, so the
  "scope binary vanished" case no longer warns about a missing D-Bus.
- Tests trimmed to the invariant bar: degraded != in_process and warns
  once (helper level); default config still Popens externally and
  `require_restart_safe_scope: true` raises (scheduler level, real helper).
  Dropped the stubbed duplicate, the standalone config-raise test and the
  in_process half already covered by the existing passthrough test.
- `GatewayChildDispatch.reason` had no reader outside a test; removed.
- Both degrade branches share one local `_degrade(detail)`.
- The per-fire config read uses `load_config_readonly()` (no deepcopy)
  with the same `except Exception` guard as the sibling
  `failure_nudge_threshold` read, so a config error no longer escapes
  the launcher.
- Kanban's no-run-id guard fails closed for any non-`in_process` mode
  instead of matching one enum value.
- Rationale restated in six places collapsed to the helper docstring;
  `cron.require_restart_safe_scope` documented in the cron user guide.
2026-09-12 23:17:12 +05:30
Paul Robertson 560b6d2e81 fix(cron): degrade gracefully when systemd user scopes are unavailable
A systemd-supervised gateway (INVOCATION_ID set) with no user D-Bus
session (containers, minimal LXCs, supervisors without linger) fails
EVERY scheduled job at dispatch: restart_safe_gateway_child_argv()
raises, run_one_job() records a failure, and the only symptom is
silently skipped executions (a missed nightly backup, dead watchdogs,
no alert).

Cron now degrades to a direct external subprocess with a
once-per-process warning instead of raising, unless
cron.require_restart_safe_scope=true (config.yaml, default false)
restores fail-closed. Degraded jobs keep process separation and the
full #101940 ownership handoff - only cgroup isolation is lost, so a
mid-job gateway restart kills the worker and the execution ledger
records exactly that.

The dispatch is a GatewayChildDispatch NamedTuple (in_process /
scoped / degraded) so the degraded case can never collapse into the
"not managed, stay in-process" sentinel - the failure mode that would
recreate the restart-interruption edge #101940 closed.

Kanban stays fail-closed (require_restart_safe_scope=True at its call
sites): its workers are long-lived agentic runs, so the degrade policy
is limited to bounded cron jobs in this PR.

Addresses the #102431 review: the env-var flag became a config key per
AGENTS.md (no new HERMES_* non-secret vars), Kanban keeps fail-closed
instead of updating its tests to a degraded contract, main's
enable-linger remedy message is preserved, and the degrade warning
fires once per process.
2026-09-12 23:17:12 +05:30
Teknium 284d220ba4 fix(multiplex): cron, kanban, /loop and completion paths for a served profile match its standalone gateway
Under gateway.multiplex_profiles a secondary profile X is ticked, dispatched
and notified from the default profile's process, where os.environ holds the
DEFAULT profile's .env and X's values live only in the per-turn secret scope /
HERMES_HOME override. Every remaining read that skipped that scope made X
behave differently from `hermes -p X gateway run`:

- cron: HERMES_CRON_TIMEOUT, HERMES_MODEL (job/preflight fallback),
  HERMES_CRON_MAX_PARALLEL, inflight allowance, prefill file and the script
  timeout were bare os.getenv → the default profile's values; a job without a
  model silently ran on the default's HERMES_MODEL instead of refusing.
  cron/env_settings.py::cron_env_setting reads the scope (fire) or the ticked
  home's .env (tick thread), plain environ when multiplexing is off.
- child env: the restart-safe cron worker, the Bot Chat delivery child and the
  kanban worker inherited the launch profile's non-credential .env settings
  and bridged TERMINAL_* policy (TERMINAL_ENV=docker, default's image,
  HERMES_MODEL) — X's worker ran in the default's docker image on the
  default's model. tools/environments/local.py::strip_launch_profile_env drops
  them when the child targets another served profile.
- kanban: the worker --toolsets pin was silently dropped for every served
  assignee (toolset probes call get_secret without a scope → swallowed
  UnscopedSecretError); notifier pings, artifact uploads and the wake text ran
  under the default's media policy / display language (only wake() was scoped).
- /loop: _post_turn_loop_completion hopped to the executor without contextvars,
  writing the completed tick into the DEFAULT profile's state.db and leaving
  X's row awaiting_response forever; the --until judge ran with the default's
  aux credentials.
- background processes: a secondary's processes.json (scope-relative since
  adf23550f5) was never read at startup; its processes were not re-adopted
  and notify_on_complete notices were lost. Startup recovers every served
  home under its scope; recovery adopts each session once.
- completion delivery: background_process_notifications was evaluated once per
  drain for the ambient profile (default's mode for everyone; X's `off`
  dropped a sibling's `all` event), recovered watchers used the default's
  mode, HERMES_BACKGROUND_NOTIFICATIONS was read raw from environ;
  _deliver_platform_notice used the default's GatewayConfig so a secondary's
  notice_delivery: private went public.

Not changed: gateway/run.py and tools/async_delegation.py (PR #106742
rewrites both). Known residue left for the env-bridge lane:
HERMES_SESSION_STALL_TIMEOUT is bridged once from the launch config.
scheduler bug, not a parity gap; unchanged here.
2026-09-11 19:58:07 -07:00
kshitijk4poor 6f11a3296e refactor(gateway): wait on the target user's bus socket, not the generic control-socket predicate
_user_systemd_socket_ready() accepts systemd/private alone, which is enough for
systemctl --user but not for the systemd-run --user that restart-safe workers
need; systemd_user_bus_env() requires the bus socket. Replace the uid threading
through five helpers with one _wait_for_target_user_bus(uid) that polls
/run/user/<uid>/bus, and move the post-enable wait + restart hint out of
_ensure_linger_enabled into _ensure_system_service_linger so the activity probe
runs only when linger was actually just enabled. Kanban applies the bus env
unconditionally like the cron sibling. Refs #104893.
2026-09-08 22:35:39 +05:30
ckomma be9c2bd19c fix(cron): derive the user bus env per probe and scoped spawn
A system-level gateway unit has no ordering against user@<uid>.service and
linger may be enabled after boot, so the bus can appear after the one-shot
adoption in run_gateway() ran. Derive XDG_RUNTIME_DIR/DBUS_SESSION_BUS_ADDRESS
fresh for the availability probe and every scoped spawn (cron worker, Kanban
worker, PTY/pipe terminal spawns, scope cleanup) so the 60s failure TTL can
actually recover. Refs #104893.
2026-09-08 22:35:39 +05:30
Teknium b578261584 fix: keep Kanban worker scope out of descendant processes
Carry the existing write fence across Hermes-owned spawn boundaries without
dropping board routing or changing credential policy. Grant dispatcher and
managed tool runtimes explicit task scope; align CLI task mutations with tools.

Verify real shell/CLI descendants, dispatcher startup, and supervised stdio
transport against isolated SQLite boards. This is cooperative runtime scoping,
not OS confinement.

Refs #103974, #104058, #104904
2026-09-07 07:10:28 -07:00
Teknium e3ab65fe80 simplify(compat): kanban_db — drop 73 re-exports/aliases, repoint 794 callers 2026-09-03 13:48:14 -07:00
Teknium e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium 0071ba9965 Merge origin/main (561b053f79) into simp/forwardport: forward-port 220 main commits into the simplified tree 2026-09-03 03:31:03 -07:00
Teknium b6838e8135 refactor(kanban_db_*): hand-compact docstrings/comments keeping every invariant; heartbeat SQL built once (4476->4390 LOC) 2026-09-02 21:59:24 -07:00
Teknium e66429eb0a refactor(kanban_db_dispatch,connect): split god functions into phase helpers, unify probe/kill/profile helpers, AST-neutral line joins
- detect_crashed_workers (239) -> _reclaim_dead_workers/_classify_dead_worker(_DeadWorker)/_account_crashes(_CrashSweep)
- _dispatch_once_locked (193) -> _run_reclaim_phase/_tick_spawn_budget/_lane_rows/_any_spawnable_review/_resolve_default_assignee
- _default_spawn (164) -> _worker_argv/_open_worker_log; TERMINAL_* overrides looped
- connect: _probe_for_corruption/_missing_or_empty shared by guard+repair_db, _backfill_legacy_inflight_runs, _column_names/_table_exists, unified rebuild INSERT
- dispatch: _kill_fn/_poll_worker_exit/_sigkill shared by enforce_max_runtime and _terminate_reclaimed_worker; dead _log loggers dropped
- SQL trace parity identical; 934/934 kanban tests green
2026-09-02 21:33:44 -07:00
Teknium 464ee35248 refactor(kanban-db): compact docstrings/comments in the four split modules (AST-identical, every WHY kept) 2026-09-02 17:27:14 -07:00
Teknium 41a26c8ba9 refactor(kanban-db): compact long docstrings (rules kept) across origin/dispatch/connect 2026-09-02 17:04:54 -07:00
Teknium 693893df5c refactor(kanban-db): compact Task field/constant comments that duplicated SCHEMA_SQL; contextlib.suppress for try/except/pass 2026-09-02 17:02:26 -07:00
Teknium a11464423a refactor(kanban-db): drop dead _clear_spawn_failures alias, unify per-task row/link accessors, compact three docstrings 2026-09-02 16:55:53 -07:00
Teknium e03a680592 refactor(kanban-db): split into connect/workspace/dispatch/notify modules
Byte-identical bodies moved out of hermes_cli/kanban_db.py into four sibling
modules, re-exported from the origin so kanban_db.<name> keeps resolving and
stays the single monkeypatch target; origin-resident helpers are reached via a
late-bound _kb namespace. AST-identity verified for all 329 moved symbols; SQL
statement multiset parity vs base. Three source-inspection tests repointed at
kanban_db_dispatch; the _add_column_if_missing alias test now imports the real
owner (hermes_cli.sqlite_util).
2026-09-02 16:46:52 -07:00