From 3272fb35aa20041af55ce7ef68432d1a7527755a Mon Sep 17 00:00:00 2001 From: teknium1 <127238744+teknium1@users.noreply.github.com> Date: Tue, 15 Sep 2026 04:00:53 -0700 Subject: [PATCH] =?UTF-8?q?docs:=20profile-scope=20invariant=20in=20AGENTS?= =?UTF-8?q?.md=20=E2=80=94=20one=20process=20serves=20many=20profiles;=20o?= =?UTF-8?q?ut-of-turn=20code=20binds=20its=20scope?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Root AGENTS.md § Code Shape Rules replaces "module-level constants are fine — they cache after _apply_profile_override() sets HERMES_HOME" (true for `hermes -p x `, inverted under the multiplex gateway and the Desktop/dashboard `serve` backend, where os.environ holds the LAUNCH profile) with the invariant: a profile = home + secret scope + terminal scope, bound per profile ACTIVITY, and every execution point with no turn on the stack binds it explicitly. Names the real seams: gateway/run.py::_profile_runtime_scope, tui_gateway @_profile_scoped + _session_profile_runtime_scope (+ _profile_runtime_scope_tokens, launch_profile_policy -> set_multiplex_active), cron/scheduler_provider.py::_profile_cron_scope, gateway/run_agent_cache.py::_run_release_in_profile_scope, tools/environments/local.py:: served_profile_child_env, agent/memory_provider.py::spawn_context_thread. Adds a routing-table row for profiles / multiplex / secret scope. Area AGENTS.md paragraphs, one per seam, for gateway/ (activity-not-turn binding, hooks per profile, adapter YAML never reaches os.environ, unserved shared-ingress reported via _note_unserved_secondary_platform + needs_attention at the single writer), tui_gateway/ (RPC binding is home AND secret AND terminal; HOME-only is half-bound; teardown chokepoint), cron/ (per-home tick lock, ticker scope incl. pre-loop code, kanban notifier routing, worker liveness by (pid, worker_started_at) fingerprint, descendant fence as a path), hermes_cli/ (DEFAULT_CONFIG key <-> reader parity, service-install matrix, -p vs multiplex home binding), tools/ (check_fn reads through get_secret and is cached per hermes_home_key, one env builder per spawn, MCP trust per profile), plugins/ (lifecycle hooks are bound by the caller; never cache the home from initialize()), apps/desktop/src/ (pooled serve per (connection, profile); remote topologies), agent/ (end-of-session flush is caller-bound; set_multiplex_active gates fail-closed). Corrects the statements the multiplex model made wrong, in the same PR: root module-constant sentence; hermes_cli "sets HERMES_HOME before any import" (+ cli-internals.md); ADDING_A_PLATFORM.md §2 raw os.getenv loader (now an _ENV_STEPS row through config.py::_getenv) and §4 platform_env_map in gateway/run.py (now _PLATFORM_ALLOWLIST_ENV in pairing.py + registry allowed_users_env); platform_registry.py "may set os.environ (guard with not os.getenv)"; cron/AGENTS.md hardcoded ~/.hermes/cron/.tick.lock; gateway-internals.md agent:main as THE key format, ~/.hermes/hooks/, single-profile `gateway stop`, plus a new "Multiplexed profiles" section; tools/AGENTS.md os.getenv check_fn sample; "installed per turn" wording; "one temp HERMES_HOME" E2E wording; multi-profile-gateways.md intro lists system units, Windows tasks, s6 and the Desktop backend. --- AGENTS.md | 20 ++++++++-- agent/AGENTS.md | 13 +++++++ apps/desktop/src/AGENTS.md | 10 +++++ cron/AGENTS.md | 29 ++++++++++++++- gateway/AGENTS.md | 29 ++++++++++++++- gateway/platform_registry.py | 6 ++- gateway/platforms/ADDING_A_PLATFORM.md | 37 +++++++++++-------- hermes_cli/AGENTS.md | 24 +++++++++++- plugins/AGENTS.md | 12 ++++++ tools/AGENTS.md | 37 +++++++++++++++---- tui_gateway/AGENTS.md | 24 ++++++++++++ website/docs/developer-guide/cli-internals.md | 11 ++++-- .../docs/developer-guide/gateway-internals.md | 28 +++++++++++--- .../docs/user-guide/multi-profile-gateways.md | 11 ++++-- 14 files changed, 245 insertions(+), 46 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b8db958cd9..28bd535616 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -64,7 +64,8 @@ grow: expansive at the edges, conservative at the waist. freeze a current value (see Testing). - **E2E validation, not just green unit mocks.** Anything touching resolution chains, config propagation, security boundaries, remote backends, or file/network I/O must exercise the - real path with real imports against a temp `HERMES_HOME`. Mocks hide integration bugs. + real path with real imports against a temp `HERMES_HOME` — two of them (A→B→A) when the + change touches profile scope. Mocks hide integration bugs. - **Cache-, alternation-, and invariant-safe.** Preserve prompt caching, strict role alternation (never two same-role messages in a row; never a synthetic user message injected mid-loop), and a system prompt byte-stable for the life of a conversation. @@ -269,10 +270,22 @@ families: `hermes_state.py` (21), `gateway/run.py` (15), `tools/mcp_tool.py` (15 display. Details: `hermes_cli/AGENTS.md`. - **Never hardcode `~/.hermes`.** `get_hermes_home()` for code paths, `display_hermes_home()` for user-facing text (both from `hermes_constants`). Hardcoding breaks profiles (5 bugs in - PR #3575). Module-level constants are fine — they cache after `_apply_profile_override()` - sets `HERMES_HOME`. Profile operations themselves are HOME-anchored + PR #3575). Profile operations themselves are HOME-anchored (`_get_profiles_root()` = `Path.home()/.hermes/profiles`) so `hermes -p x profile list` sees all profiles — intentional, not a bug. +- **One process may serve many profiles; code that runs outside a turn binds the owning + profile scope explicitly.** A profile = home + secret scope + terminal scope, bound by + `gateway/run.py::_profile_runtime_scope` (turn), `tui_gateway/server.py::@_profile_scoped` + + `model_switch.py::_session_profile_runtime_scope` (RPC, teardown), `cron/scheduler_provider.py:: + _profile_cron_scope` (ticker), `gateway/run_agent_cache.py::_run_release_in_profile_scope` + (eviction). `os.environ`, module globals and import-time values hold the *launch* profile's, so + an unbound read is a silent default-profile leak, never an error: home/config/`.env`-derived + module constants are a bug class — key slots by `hermes_home_key()` or resolve at call time. + Needs a binding: boot probes (`check_fn`, MCP discovery, hooks), session end/eviction, tickers, + deferred callbacks, RPC methods, config readers, thread hops (`spawn_context_thread`), child + spawns (`served_profile_child_env`, never `os.environ.copy()`). Fail-closed reads exist only after + `set_multiplex_active(True)`. Prove live with two homes (A→B→A) under multiplex, not one temp + `HERMES_HOME`. Advisory lint: `scripts/check_profile_scope_patterns.py`. - **Argparse alias dispatch:** `add_parser("list", aliases=["ls"])` sets `dest` to the literal the user typed (`"ls"`). Dispatch must accept both (caught PTY-testing `hermes webhook ls`). - **Don't wire in dead code without E2E validation.** Unshipped code was dead for a reason; @@ -425,6 +438,7 @@ extract, not to regex around it. | `skills/`, `optional-skills/`, `agent/curator*.py` | `skills/AGENTS.md` | Frontmatter, HARDLINE authoring standards, curator | | `cron/`, kanban (`hermes_cli/kanban*.py`, `tools/kanban_tools.py`, `plugins/kanban/`) | `cron/AGENTS.md` | Scheduler invariants, job fields, kanban board/dispatcher | | `gateway/platforms/` new adapter | `gateway/platforms/ADDING_A_PLATFORM.md` | Step-by-step adapter guide | +| profiles / multiplex / secret scope (any area) | `gateway/AGENTS.md` § Profile scope, `website/docs/user-guide/multi-profile-gateways.md` § What is isolated per profile | which execution points bind scope, what is isolated per profile | Long-form background lives in `website/docs/developer-guide/` (agent-loop, prompt-assembly, context-compression-and-caching, gateway-internals, tools-runtime, plugins/, cron-internals, diff --git a/agent/AGENTS.md b/agent/AGENTS.md index 0aad5c8772..77cd5f0bf2 100644 --- a/agent/AGENTS.md +++ b/agent/AGENTS.md @@ -104,6 +104,19 @@ image-gen plugins (all in `plugins/AGENTS.md`). `agent/curator.py` + `curator_ba the skill curator (`skills/AGENTS.md`). Cron sessions pass `skip_memory=True` by default — memory providers intentionally do not run during cron. +- End-of-session memory extraction and provider `on_session_end` run wherever the session ends — + turn, eviction, shutdown, `tui_gateway` teardown — and the CALLER binds the owning profile's scope + first (`_run_release_in_profile_scope`, `_session_profile_runtime_scope`); the agent never derives + its home from `os.environ` at flush time (`Path(_session_db.db_path).parent` is the ground truth). + Provider background work starts through `memory_provider.py::spawn_context_thread` (copies the + contextvars), never a bare `threading.Thread`; `title_generator.py` is the shape. +- `agent/secret_scope.py::get_secret` fails closed (`UnscopedSecretError`) only after + `set_multiplex_active(True)`; the gateway, cron, migrate and `serve` set it. A new multi-home host + must too, or every guard is silently off. Isolation is BETWEEN profiles; children inherit via + `copy_context`; a child's `UnscopedSecretError` is a spawn-site bug, never grounds for an + `os.getenv` fallthrough. Delegated children carry `delegation_context.py:: + DELEGATED_CHILD_ENV_MARKER` valued as the fenced Kanban board root, not a bare flag. + ## Tests Loop/phase tests go in `tests/agent/`; patch the binding the phase actually reads (siblings often diff --git a/apps/desktop/src/AGENTS.md b/apps/desktop/src/AGENTS.md index 7c642586a8..536e3b24f4 100644 --- a/apps/desktop/src/AGENTS.md +++ b/apps/desktop/src/AGENTS.md @@ -23,6 +23,16 @@ unknown subcommand and bricks every mid-upgrade user. Keep it narrow and tested. Lifecycle: `serve` dies with the app by design; the messaging gateway survives it (spawned detached via `/api/gateway/*`). Never re-parent the gateway under the backend — `gateway/AGENTS.md`. +The backend the app spawns is a **pooled `hermes serve --port 0` per (connection, profile)**: its +launch home is that profile, `HERMES_DESKTOP=1` is set, and its in-process cron ticker stands down +for homes a running gateway already serves. One process may still host sessions from several homes +(`tui_gateway/AGENTS.md` § Profile scope); the first non-launch home flips `set_multiplex_active`. +Remote connections (SSH, URL+token, Cloud) reach a backend with no desktop env var that may serve +several profiles from one process. Every lifecycle/status/settings REST call against a pooled +backend carries `?profile=` (or the `profile` param) and every new-session tile records an owner +route; a backend-side scope fix is probed twice — with the profile as the launch home of a pooled +backend (env-bound) and as a secondary served by one process (override-bound). + ## Slash commands: curated client-side, dispatched to the backend - The backend already provides everything: `commands.catalog` and `complete.slash` include built-ins, diff --git a/cron/AGENTS.md b/cron/AGENTS.md index c1b0a8e660..2032bc57df 100644 --- a/cron/AGENTS.md +++ b/cron/AGENTS.md @@ -22,7 +22,18 @@ Hardening invariants — each guards a real failure; don't weaken without answer finds the stamp with a dead owner restores the instant ONCE (`cron/occurrences.py`), the executions ledger's `scheduled_instant` blocks a second fire, `cron.catch_up_missed: false` skips past-grace misses with a logged reason. Never drop a slot silently (#107485). -- File lock `~/.hermes/cron/.tick.lock` prevents duplicate ticks across processes. +- Per-home tick lock `/cron/.tick.lock` prevents duplicate ticks across processes for + that profile's store; never a `~/.hermes/...` literal. +- **The ticker binds each served profile's scope for the whole tick, including pre-loop code.** + `scheduler_provider.py::_start_multiplex` is ONE ticker iterating `profiles_to_serve()` + sequentially under `_profile_cron_scope(home)` (home + secret scope + terminal scope) — never N + threads (module globals race). Everything a tick touches lives inside that guard: store open, + lock path, backoff/failure counters (`_note_tick_failure`), job env construction, and the + `on_session_end` flush of a finished job. Supervision (`scheduler_thread.py:: + SupervisedTickerThread`; start on gateway boot, stand down for homes another gateway already + serves, re-enumerate when a profile dir appears or is tombstoned) is per served home, not per + process. Why: a store opened before the scope was entered wrote a secondary profile's run + records into the launch profile's `jobs.json`. - Cron sessions pass `skip_memory=True`; memory providers intentionally do not run during cron. - Cron execution has its own session. Eligible continuable deliveries may mirror or seed the reply-facing conversation: origin, origin-less home fallback, user-written bare-platform home, @@ -62,7 +73,21 @@ cannot see other boards; **tenant** is a soft namespace within a board (workspac isolation, one fleet serving several businesses). After `kanban.failure_limit` consecutive non-success attempts on a task (default 2) the dispatcher auto-blocks it to stop spin loops. Process-identity note: `kanban --preserve-cache` contains "serve" — never classify processes by argv -substring (root). +substring (root). Worker liveness is `(worker_pid, worker_started_at)` — the start-time fingerprint +(`gateway.status.get_process_start_time`) recorded at claim time — never bare PID existence, or a +recycled PID gets killed on reclaim. + +- **Notifications leave through the task's owning profile.** `hermes_cli/kanban_db_notify.py` + subscriptions carry the profile; `gateway/kanban_watchers_notifier.py` delivers via THAT + profile's adapter under its scope (`_notify_profile_filter`), never the multiplexer's launch + adapter; a fail-closed skip logs once at WARNING with the remedy, never a bare `continue`. + Dispatched workers get `HERMES_KANBAN_BOARD` and the assignee's `HERMES_HOME` pinned in a + scrubbed child env (`build_subprocess_env` + `strip_launch_profile_env`); they never inherit the + default profile's `.env`. +- **Descendant fence is a path, not a flag.** A delegated child's Kanban marker + (`agent/delegation_context.py::DELEGATED_CHILD_ENV_MARKER`) carries the fenced board ROOT; + `kanban_path_is_fenced(path)` denies mutations only on the dispatcher-pinned `HERMES_KANBAN_DB` + or under that root, so a child working against a scratch `HERMES_HOME` keeps a writable board. ## Tests diff --git a/gateway/AGENTS.md b/gateway/AGENTS.md index 14a4a0154e..db0534fae6 100644 --- a/gateway/AGENTS.md +++ b/gateway/AGENTS.md @@ -124,7 +124,7 @@ replaced by #92091's `pause-for-update`. Do NOT "fix" gateway-dies-with-app by r gateway under the backend, and do NOT "fix" update locks by widening the tree-kill. Gateways stamp `code_sha`/`code_version` into `gateway_state.json` (`status.py`) so the updater can verify a fleet. -## Profiles and secrets in adapters +## Profile scope (adapters, turns, and everything between turns) - **Token locks.** An adapter that connects with a unique credential (bot token, API key) calls `acquire_scoped_lock()` from `gateway.status` in `connect()`/`start()` and `release_scoped_lock()` @@ -133,7 +133,8 @@ gateway under the backend, and do NOT "fix" update locks by widening the tree-ki - **Multiplex profile-scoped env reads MUST fail closed — never borrow from `os.environ`** (`agent/secret_scope.py`; #72348, #86905). Under `gateway.multiplex_profiles`, `os.environ` holds the DEFAULT profile's values; a secondary profile's `.env` exists only in its secret scope, - installed per turn by `_profile_runtime_scope`. All profile-level env config — credentials + bound per profile *activity* by `_profile_runtime_scope` (turn, callback, eviction, tick — never + only "per turn"). All profile-level env config — credentials (`app_secret`, tokens) AND authorization (`FEISHU_ALLOWED_USERS`, `{PLATFORM}_ALLOW_ALL_USERS`, `GATEWAY_ALLOW_ALL_USERS`, `group_policy`, `allow_bots`) — is read scope-aware: adapters via `gateway.platforms._shared.get_scoped_secret` (the ONE implementation; adapters import it as @@ -146,6 +147,30 @@ gateway under the backend, and do NOT "fix" update locks by widening the tree-ki adapter (the `try get_secret / except UnscopedSecretError: os.getenv` shape drifts into a fallback-after-miss leak); import the shared one. `tests/gateway/test_shared_platform_boilerplate.py` asserts every plugin's `_env_enablement` reads only through it. +- **Scope is bound per profile activity, not per turn.** `run.py::_profile_runtime_scope(home)` + wraps a routed turn (`run_turn.py::_profile_scope_for_source`); the same binding wraps every path + that touches a session's home, secrets or terminal scope with no turn on the stack: release and + eviction (`run_agent_cache.py::_run_release_in_profile_scope` — TTL, LRU and memory-pressure + eviction all route through it so `on_session_end`/memory flush hit the OWNING profile's provider), + shutdown (`run_shutdown.py::_finalize_session`), post-turn media delivery + (`platforms/base.py::_media_delivery_scope`), deferred callbacks (pickers, reactions — capture the + routed home at command time, re-enter the scope in the callback), notifiers and outbound webhooks. + Resolve the owning home from the session record (`profile_home`, `agent::` key), never + from `os.environ`, which holds the launch profile. Why: eviction that flushed under the launch scope + wrote a secondary profile's memories into the default profile's store, silently. +- **Hooks and observers register per served profile.** `builtin_hooks/`, `agent/shell_hooks.py:: + register_from_config` and lifecycle observers are prepared under each profile's scope at startup + and on profile add/remove; idempotence keys include the profile, and `hooks/` paths resolve at call + time (a module constant freezes to the first importer). +- **Adapter YAML never reaches `os.environ` under multiplex.** `platforms/_shared.py:: + apply_yaml_bridge` seeds `PlatformConfig.extra` and skips the environ write under a secondary + profile's scope; gate/allowlist reads go through `platform_gate_env`. A `if not os.getenv(X): + os.environ[X] = …` bridge is first-profile-wins across the process — test two profiles with + conflicting flags before touching precedence. +- **Unserved is reported, never silent.** Shared-ingress platforms (WhatsApp bridge, Relay) run on + the default profile only; a secondary enabling one is logged once with the remedy and stamped + into runtime status (`run_adapters.py::_note_unserved_secondary_platform`). `needs_attention` is + set and cleared at the single writer (`_update_platform_runtime_status`) on the connect path. ## Tests diff --git a/gateway/platform_registry.py b/gateway/platform_registry.py index 4e6fc5eb15..0e7b2ffec8 100644 --- a/gateway/platform_registry.py +++ b/gateway/platform_registry.py @@ -81,8 +81,10 @@ class PlatformEntry: # ``_apply_env_overrides`` BEFORE adapter construction so ``gateway status`` sees it. env_enablement_fn: Optional[Callable[[], Optional[dict]]] = None # YAML->env bridge ``(yaml_cfg, platform_cfg) -> Optional[dict]`` merged into ``extra``; runs - # after the shared-key loop, before ``_apply_env_overrides``. May set ``os.environ`` (guard - # with ``not os.getenv(...)`` to keep env > YAML). Contract: docs/developer-guide/adding-platform-adapters.md. + # after the shared-key loop, before ``_apply_env_overrides``. Build it with + # ``gateway.platforms._shared.apply_yaml_bridge`` — it writes env only when unset (env > YAML) + # and never under a multiplexed secondary's scope; a hand-rolled ``os.environ[...] =`` is + # first-profile-wins. Contract: docs/developer-guide/adding-platform-adapters.md. apply_yaml_config_fn: Optional[Callable[[dict, dict], Optional[dict]]] = None cron_deliver_env_var: str = "" # home-channel env var read for cron ``deliver=`` # ``(target_ref) -> Optional[(chat_id, thread_id)]`` run before channel-directory diff --git a/gateway/platforms/ADDING_A_PLATFORM.md b/gateway/platforms/ADDING_A_PLATFORM.md index 257ed06e52..9887ad5dd9 100644 --- a/gateway/platforms/ADDING_A_PLATFORM.md +++ b/gateway/platforms/ADDING_A_PLATFORM.md @@ -163,18 +163,21 @@ class Platform(Enum): YOUR_PLATFORM = "your_platform" ``` -Add env var loading in `_apply_env_overrides()`: +Add a row to `_ENV_STEPS` in `gateway/config_env.py` (source order = application order); +`_Cred` enables the platform when the named env vars resolve and copies them into `extra`: ```python -# Your Platform -your_token = os.getenv("YOUR_PLATFORM_TOKEN") -if your_token: - if Platform.YOUR_PLATFORM not in config.platforms: - config.platforms[Platform.YOUR_PLATFORM] = PlatformConfig() - config.platforms[Platform.YOUR_PLATFORM].enabled = True - config.platforms[Platform.YOUR_PLATFORM].token = your_token +_Cred(Platform.YOUR_PLATFORM, ("YOUR_PLATFORM_TOKEN",), token="YOUR_PLATFORM_TOKEN"), +_Home(Platform.YOUR_PLATFORM, "YOUR_PLATFORM_HOME_CHANNEL"), ``` +Every read goes through `gateway/config.py::_getenv` (the active profile's secret scope when one +is bound, `os.environ` otherwise). **Never `os.getenv` here and never write `os.environ`**: under +`gateway.multiplex_profiles` the process env is the DEFAULT profile's, so a raw read enables your +platform for the wrong profile with the wrong credentials, and a write pins one profile's policy +process-wide (first profile wins). Adapter-side reads use `gateway.platforms._shared.get_scoped_secret` +/ `extra_or_secret`. + Update `get_connected_platforms()` if your platform doesn't use token/api_key (e.g., WhatsApp uses `enabled` flag, Signal uses `extra` dict). @@ -200,21 +203,23 @@ before `connect()`. --- -## 4. Authorization Maps (`gateway/run.py`) +## 4. Authorization Maps (`gateway/pairing.py`, `gateway/authz_mixin.py`) -Add to BOTH dicts in `_is_user_authorized()`: +Add the allowlist var to `_PLATFORM_ALLOWLIST_ENV` in `gateway/pairing.py`; +`authz_mixin.py` derives `_ALLOWED_USERS_ENV` / `_ALLOW_ALL_ENV` from it (the `*_ALLOW_ALL_USERS` +name is computed, not hand-listed): ```python -platform_env_map = { +_PLATFORM_ALLOWLIST_ENV = { ... - Platform.YOUR_PLATFORM: "YOUR_PLATFORM_ALLOWED_USERS", -} -platform_allow_all_map = { - ... - Platform.YOUR_PLATFORM: "YOUR_PLATFORM_ALLOW_ALL_USERS", + "your_platform": "YOUR_PLATFORM_ALLOWED_USERS", } ``` +Plugin adapters declare `allowed_users_env` / `allow_all_env` on `ctx.register_platform` instead. +`_is_user_authorized()` reads every gate through `_shared.platform_gate_env` (`_auth_env`), which +answers from the routed profile's secret scope under multiplex — never add an `os.getenv` here. + --- ## 5. Session Source (`gateway/session.py`) diff --git a/hermes_cli/AGENTS.md b/hermes_cli/AGENTS.md index 625e288235..72aa68ac91 100644 --- a/hermes_cli/AGENTS.md +++ b/hermes_cli/AGENTS.md @@ -74,6 +74,12 @@ Do not add a surface-specific goal parser. ACP has no goal command or goal loop canon, NO defaults — for presence-sensitive readers). If the CLI sees a key and the gateway doesn't (or vice versa), you're on the wrong loader — check `DEFAULT_CONFIG` coverage. Never hand-roll raw-read → overlay → expand; `read_user_config_raw` is for write-back round-trips only. +- **Every `DEFAULT_CONFIG` key has a runtime reader, and every reader a registry entry.** Both drift + modes are silent: a registered key nothing reads (a knob that does nothing) and a reader of a key + never registered (never shown, never migrated; new roots also go in `_KNOWN_ROOT_KEYS`). For a new + key: `rg -n '""' hermes_cli/config_defaults.py` AND `rg -n '' --glob '!tests' .` both hit, + and one invariant test sets it in a temp `config.yaml` and asserts the behaviour through the loader + the consuming surface uses. Under multiplex the process env never overrides a profile's YAML. - **Working directory:** CLI uses `os.getcwd()`; messaging uses `terminal.cwd`, bridged to `TERMINAL_CWD` for child tools. @@ -130,8 +136,12 @@ matchers; parser-derived flag sets; never blanket-exclude gateway ancestors, #87 ## Profiles (multi-instance) -`_apply_profile_override()` in `hermes_cli/main.py` sets `HERMES_HOME` before any module import, so -every `get_hermes_home()` scopes to the active profile (rules in root). Profiles are independent +`_apply_profile_override()` in `hermes_cli/main.py` sets `HERMES_HOME` before any module import for +single-profile commands (`hermes -p x `), so there `get_hermes_home()` scopes to the active +profile. The multiplex gateway and the Desktop/dashboard `serve` backend instead bind the active +profile per activity via a contextvar override while `os.environ["HERMES_HOME"]` keeps the launch +profile — a module constant or import-time read there freezes to the launch profile (rules in +root). Profiles are independent islands by design — no live config inheritance; `--clone` copies at creation, minus messaging channels (`profile_channels.py`: ownership-based inventory evaluated in the SOURCE's plugin scope — adapter-declared keys + canonical/alias prefixes + `GATEWAY_ALLOW*`/`GATEWAY_RELAY_*`; prefixes shared @@ -161,6 +171,16 @@ supervisor (control-socket `identify` answering anything but `manual`, OR the ar for a fresh supervised PID, never stop + foreground `run_gateway` (that stamps the CLI's PID and wedges every KeepAlive respawn, #110637). +Service installs are a matrix, not a unit file: `gateway.py::generate_systemd_unit(system=, +run_as_user=)` (user unit AND `--system` unit with `User=`; an unresolvable `User=` is a blocker, +never a dir-owner fallback), `generate_launchd_plist` (`gui/` then `user/` domains, never a +`~/Library/LaunchAgents` glob), Windows Scheduled Task and the Desktop-spawned backend all carry the +profile's `HERMES_HOME` (and `HOME` for the service user) explicitly — a supervisor starts with an +empty environment, so the env override that makes `-p` work interactively does not exist there. A +change to install/restart/status regenerates and diffs every kind; both user and system units are +recorded when both exist. Process liveness is `(pid, start_time)` or the canonical matchers +(`gateway.status.live_gateway_pid_for_home`), never bare PID existence. + ## Nous free tier (`hermes_cli/anon_auth.py`) Sign-in completion is one function, `settle_after_upgrade`, called by every caller that persists an diff --git a/plugins/AGENTS.md b/plugins/AGENTS.md index a78538fb5a..c446de6e2d 100644 --- a/plugins/AGENTS.md +++ b/plugins/AGENTS.md @@ -58,6 +58,18 @@ bare names resolve through the catalog or error. tool) and `run_agent.py` (lifecycle). When a plugin changes a default, add a migration guard keyed on an "existing config" signal (`_explicitly_configured`) so existing users keep the old default. +**Lifecycle hooks fire under the owning profile's scope, and the caller binds it.** +`on_session_start`/`on_session_end`/`sync_turn`/`shutdown` are invoked from the turn (bound) AND +from eviction, shutdown, `tui_gateway` teardown and cron completion (bound by the caller via +`_run_release_in_profile_scope`, `_session_profile_runtime_scope`, `_profile_cron_scope`). One +process serves several profiles, so a provider never caches `hermes_home` from `initialize()` as +"the" home — key state by the home it is handed per call (`hermes_home_key()`) — and never reads +`os.environ` for credentials (`agent.secret_scope.get_secret`; a `check_fn` too). Background +work starts via `agent.memory_provider.spawn_context_thread`, never a bare `threading.Thread`, +or the worker runs with no scope and fails closed (or writes into the launch profile's tenant). +Platform plugins never mutate `os.environ`: YAML goes to `PlatformConfig.extra` through +`_shared.apply_yaml_bridge`, gates through `platform_gate_env` (`gateway/AGENTS.md`). + ## Native plugin compatibility contract (summary — canonical text in the docs page) Compatibility is a **behavior contract**, not a monolithic `PLUGIN_API_VERSION`, a manifest-wide diff --git a/tools/AGENTS.md b/tools/AGENTS.md index 0f34ecf07b..02ba803521 100644 --- a/tools/AGENTS.md +++ b/tools/AGENTS.md @@ -25,8 +25,9 @@ toggled without touching `tools/` or `toolsets.py` (`plugins/AGENTS.md`). 1. `tools/your_tool.py`: ```python + from agent.secret_scope import get_secret from tools.registry import registry - def check_requirements() -> bool: return bool(os.getenv("EXAMPLE_API_KEY")) + def check_requirements() -> bool: return bool(get_secret("EXAMPLE_API_KEY")) def example_tool(param: str, task_id: str = None) -> str: return json.dumps({"success": True, ...}) registry.register(name="example_tool", toolset="example", schema={"name": "example_tool", "description": "...", "parameters": {...}}, @@ -42,14 +43,20 @@ Rules for tool code: web_search"). Those tools may be unavailable (missing key, disabled toolset) and the model hallucinates calls to them. Cross-references are added dynamically in `get_tool_definitions()` in `model_tools.py` — see the `browser_navigate` / `execute_code` post-processing blocks. -- **Paths in schema descriptions use `display_hermes_home()`** (schema is built at import, after - `_apply_profile_override()` set `HERMES_HOME`). **State files use `get_hermes_home()`**, never - `Path.home()/.hermes`, so each profile gets its own state. +- **Paths in schema descriptions use `display_hermes_home()`** (schema is built at import; under + multiplex it shows the launch home, which is display-only). **State files use `get_hermes_home()`** + at call time, never `Path.home()/.hermes` and never a module constant, so each served profile gets + its own state. - **No `offset`/`limit` on instructional tools** (skills, prompts, playbooks) — models read page 1 and skip the rest (root rubric). -- **`check_fn` answers reachability/opt-in, never surface.** It is TTL-cached process-wide, and one - process serves many sessions; GUI-only tools go in a named toolset (`desktop_ui`, `project`) - folded in by `_load_enabled_toolsets(platform)` (root: capability is a property of the SESSION). +- **`check_fn` answers reachability/opt-in for the profile it runs under, never surface.** Results + are TTL-cached in `registry.py::_check_fn_cache` keyed by `hermes_home_key()`, and one process + serves many sessions AND many profiles: a probe reads credentials through + `agent.secret_scope.get_secret`, never bare `os.getenv` (that answers with the launch profile's + `.env` for everyone). The registry classifies an `UnscopedSecretError` from + `current_secret_scope()` at the catch site — a boot-time probe with no scope is DEBUG, not a + traceback. GUI-only tools go in a named toolset (`desktop_ui`, `project`) folded in by + `_load_enabled_toolsets(platform)` (root: capability is a property of the SESSION). - **Agent-level tools** (`todo`, `memory`) are intercepted before `handle_function_call()` via the `INLINE_TOOL_EXECUTORS` table (`agent/inline_tool_executors.py`; `agent/AGENTS.md`). - **`_last_resolved_tool_names`** is a process-global in `model_tools.py`; `_run_single_child()` in @@ -77,6 +84,22 @@ client (`mcp_tool_*.py`: config, discovery, transport, registration, content, er never an `elif` on a backend name (root shape rules). Remote-backend file visibility problems are fixed at the mount, not by adding a tool. +**Every spawn goes through one env builder.** `environments/local.py::build_subprocess_env` (+ +`hermes_constants.apply_subprocess_home_env`, `env_passthrough.py::resolve_passthrough_value`) is +how a terminal, `execute_code`, background process, delegation child, ACP or MCP stdio child gets +its environment; a child that acts FOR the served profile (`hermes -p X` workers, `key_cmd` +helpers, browser drivers, Bot Chat relay turns) uses `environments/local.py:: +served_profile_child_env(target_home=, inherit_credentials=)`: launch-profile `.env` / +`TERMINAL_*` residue dropped (`strip_launch_profile_env`), the target home pinned, only the +target's own secrets overlaid. `os.environ.copy()` / `dict(os.environ)` pins the launch profile; +contextvars do not cross process boundaries, so resolve before `Popen`. A child's +`UnscopedSecretError` is a spawn-site bug, never a reason to add environ fallthrough. New threads +from scoped code use `agent.memory_provider.spawn_context_thread` (a bare `threading.Thread` +drops the scope). **MCP trust is a per-profile record:** `mcp_tool_registration.py:: +_record_scope_trust` keys trust on the home; a secondary never adopts the launch profile's trust +for a same-named server, and `mcp_tool_handlers.py::_trust_gate_check` consults the calling +session's profile. + ## Delegation (`tools/delegate_tool.py`) Spawns a subagent with isolated context + terminal session; the parent waits for the summary unless diff --git a/tui_gateway/AGENTS.md b/tui_gateway/AGENTS.md index 6d3b9ae33f..4cdd0948bb 100644 --- a/tui_gateway/AGENTS.md +++ b/tui_gateway/AGENTS.md @@ -46,6 +46,30 @@ New question for the user = `_ask("", sid, params, timeout)` in the emit and a `server_request(...)` in `contracts/server_requests.py`. New event = `event("", Payload)` in `contracts/events.py`; the emitter is checked against it. +## Profile scope in RPC methods + +One `serve` process may host sessions from several profile homes (Desktop pooled backends launch +under a profile; the dashboard serves several). The launch profile is a profile: "default" means +the launch home, never `~/.hermes`. The first non-launch home hosted flips +`launch_profile_policy.py` → `set_multiplex_active(True)`; without it every fail-closed guard is +silently off. Every method that reads or writes home-, config- or `.env`-derived state runs under +`server.py::@_profile_scoped` (resolved from the live session's `profile_home`, or the explicit +`profile` argument for sessionless calls) and, for tool/agent construction, +`methods_tools.py::_profile_scoped_rpc`; the tokens come from `model_switch.py:: +_profile_runtime_scope_tokens(profile_home)` — home + secret scope + terminal scope together. +**A method that sets only `get_hermes_home_override()` is half-bound**: config paths resolve to the +right profile while credentials and `TERMINAL_*` policy still come from the launch profile. +Off-turn paths bind the same way: `session_lifecycle.py::_finalize_session` / `_teardown_session` +enter `_session_profile_runtime_scope(session)` around `on_session_end`, the memory commit and +`agent.close()` (their callers are unscoped reapers, Timers, atexit and pool threads); background +threads start via `agent.memory_provider.spawn_context_thread`, never bare `threading.Thread`; +children act for the served profile through `tools/environments/local.py::served_profile_child_env` +(`hermes -p X` workers, `key_cmd` helpers, browser drivers), never `dict(os.environ)`. Grep for +unscoped handlers before adding one: `rg -n "^(async )?def " tui_gateway/methods_*.py | rg -v +_profile_scoped`. Probe with two on-disk homes and a `.env` name present only in the secondary: +call the method for that session and assert the secondary's value resolves and the launch +profile's does not, and that `os.environ` is unchanged afterwards. + ## Key surfaces | Surface | Ink component | Gateway method / event | diff --git a/website/docs/developer-guide/cli-internals.md b/website/docs/developer-guide/cli-internals.md index 58792e4e51..f5145a3767 100644 --- a/website/docs/developer-guide/cli-internals.md +++ b/website/docs/developer-guide/cli-internals.md @@ -71,9 +71,14 @@ User skins are `~/.hermes/skins/.yaml` with the same keys, activated with ## Profiles: multi-instance support Hermes supports profiles — fully isolated instances, each with its own `HERMES_HOME` (config, API -keys, memory, sessions, skills, gateway). `_apply_profile_override()` in `hermes_cli/main.py` sets -`HERMES_HOME` before any module imports, so every `get_hermes_home()` reference scopes to the active -profile. Profile operations are HOME-anchored (`_get_profiles_root()` returns +keys, memory, sessions, skills, gateway). For single-profile commands (`hermes -p x `), +`_apply_profile_override()` in `hermes_cli/main.py` sets `HERMES_HOME` before any module imports, so +every `get_hermes_home()` reference scopes to the active profile. The multiplex gateway and the +Desktop/dashboard `serve` backend serve several profiles from one process instead: the active +profile is a contextvar override bound per activity, `os.environ["HERMES_HOME"]` stays the launch +profile's, and a module-level constant derived from the home freezes to that launch profile (see +[Gateway Internals § Multiplexed profiles](./gateway-internals.md#multiplexed-profiles)). Profile +operations are HOME-anchored (`_get_profiles_root()` returns `Path.home() / ".hermes" / "profiles"`, not `get_hermes_home() / "profiles"`) so `hermes -p coder profile list` sees all profiles regardless of which one is active — intentional. Profile-safe coding rules are in the root `AGENTS.md`; multiplex secret-scope rules in diff --git a/website/docs/developer-guide/gateway-internals.md b/website/docs/developer-guide/gateway-internals.md index d397d16402..066d86a4dc 100644 --- a/website/docs/developer-guide/gateway-internals.md +++ b/website/docs/developer-guide/gateway-internals.md @@ -60,7 +60,7 @@ When a message arrives from any platform: - If agent is running for this session → queue message, set interrupt event - If `/approve`, `/deny`, `/stop` → bypass guard (dispatched inline) 3. **GatewayRunner._handle_message()** receives the event: - - Resolve session key via `_session_key_for_source()` (format: `agent:main:{platform}:{chat_type}:{chat_id}`) + - Resolve session key via `_session_key_for_source()` (format: `agent:{namespace}:{platform}:{chat_type}:{chat_id}`; the namespace is `main` for the default profile, `` under multiplexing — see [Multiplexed profiles](#multiplexed-profiles)) - Check authorization (see Authorization below) - Check if it's a slash command → dispatch to command handler - Check if agent is already running → intercept commands like `/stop`, `/status` @@ -72,10 +72,12 @@ When a message arrives from any platform: Session keys encode the full routing context: ``` -agent:main:{platform}:{chat_type}:{chat_id} +agent:{namespace}:{platform}:{chat_type}:{chat_id} ``` -For example: `agent:main:telegram:private:123456789` +For example: `agent:main:telegram:private:123456789` for the default profile, or +`agent:work:telegram:private:123456789` when the multiplexer routes that chat to profile `work` +(`gateway/session.py::_session_key_namespace`; a profile literally named `main` is marked `main~`). Thread-aware platforms (Telegram forum topics, Discord threads, Slack threads) may include thread IDs in the chat_id portion. **Never construct session keys manually** — always use `build_session_key()` from `gateway/session.py`. @@ -227,7 +229,7 @@ Gateway hooks are Python modules that respond to lifecycle events: | `agent:end` | Agent finishes and returns response | | `command:*` | Any slash command is executed | -Hooks are discovered from `gateway/builtin_hooks/` (an extension point — currently empty in the shipped distribution; `_register_builtin_hooks()` is a no-op stub) and `~/.hermes/hooks/` (user-installed). Each hook is a directory with a `HOOK.yaml` manifest and `handler.py`. +Hooks are discovered from `gateway/builtin_hooks/` (an extension point — currently empty in the shipped distribution; `_register_builtin_hooks()` is a no-op stub) and `/hooks/` (user-installed; `~/.hermes/hooks/` for the default profile, one directory per served profile under multiplexing — paths resolve at call time, never at import). Each hook is a directory with a `HOOK.yaml` manifest and `handler.py`. ## Memory Provider Integration @@ -268,7 +270,23 @@ The gateway runs as a long-lived process, managed via: - `systemctl` (Linux) or `launchctl` (macOS) — service management - PID file at `~/.hermes/gateway.pid` — profile-scoped process tracking -**Profile-scoped vs global**: `start_gateway()` uses profile-scoped PID files. `hermes gateway stop` stops only the current profile's gateway. `hermes gateway stop --all` uses global `ps aux` scanning to kill all gateway processes (used during updates). +**Profile-scoped vs global**: `start_gateway()` uses profile-scoped PID files. Standalone (one gateway per profile), `hermes -p x gateway stop` stops only that profile's gateway. Under multiplexing there is ONE gateway process, owned by the default profile: `hermes gateway stop` on the default takes every served profile down, and `hermes -p x gateway stop` for a served secondary refuses with exit 78 (it has no gateway of its own). `hermes gateway stop --all` uses global `ps aux` scanning to kill all gateway processes (used during updates). Liveness is decided by `gateway.status.live_gateway_pid_for_home` (PID + start-time fingerprint), never bare PID existence. + +## Multiplexed profiles + +With `gateway.multiplex_profiles: true` one process serves the default profile plus every live directory under `profiles/` (`hermes_cli/profiles.py::profiles_to_serve(multiplex=True)`). `os.environ` and module globals hold the **launch** profile's values, so every activity for a secondary binds its scope explicitly — a profile is home + secret scope + terminal scope together: + +| Activity | Binding | +|---|---| +| Routed turn | `gateway/run.py::_profile_runtime_scope(home)` via `run_turn.py::_profile_scope_for_source` | +| Agent release / eviction (TTL, LRU, memory pressure) | `gateway/run_agent_cache.py::_run_release_in_profile_scope` | +| Shutdown | `gateway/run_shutdown.py::_finalize_session` | +| Post-turn media delivery | `gateway/platforms/base.py::_media_delivery_scope` | +| Cron tick | `cron/scheduler_provider.py::_profile_cron_scope(home)` (one ticker, profiles in sequence) | +| Child processes (`hermes -p X` workers, relay turns, browser drivers) | `tools/environments/local.py::served_profile_child_env` | +| Background threads | `agent/memory_provider.py::spawn_context_thread` | + +Secret reads fail closed (`agent.secret_scope.get_secret` raises `UnscopedSecretError`) only after `set_multiplex_active(True)`, which the gateway, cron, `gateway migrate` and the Desktop/dashboard `serve` backend set. Adapter YAML never reaches `os.environ` under multiplex: `gateway/platforms/_shared.py::apply_yaml_bridge` seeds `PlatformConfig.extra` and skips the environ write under a secondary's scope; gates read through `platform_gate_env`. Shared-ingress platforms (WhatsApp bridge, Relay) run on the default profile only; a secondary that enables one is logged once and stamped into runtime status (`run_adapters.py::_note_unserved_secondary_platform`). Per-profile isolation as the user sees it: [Multi-profile gateways § What is isolated per profile](/user-guide/multi-profile-gateways#what-is-isolated-per-profile). ## Related Docs diff --git a/website/docs/user-guide/multi-profile-gateways.md b/website/docs/user-guide/multi-profile-gateways.md index 24d592b6bf..f21142980a 100644 --- a/website/docs/user-guide/multi-profile-gateways.md +++ b/website/docs/user-guide/multi-profile-gateways.md @@ -26,10 +26,13 @@ be online at the same time. Common reasons: - A research agent + a writing agent + a cron-driven bot — each with isolated memory and skills -Every profile already gets its own per-platform LaunchAgent -(`ai.hermes.gateway-.plist`) or systemd user service -(`hermes-gateway-.service`). This guide adds the patterns for managing -them collectively. +Every profile already gets its own per-platform supervisor entry: a LaunchAgent +(`ai.hermes.gateway-.plist`), a systemd user service +(`hermes-gateway-.service`), a systemd **system** service when installed with +`sudo hermes gateway install --system` (runs as the invoking user via `User=`), a +Windows Scheduled Task, or an s6/Docker service — and the Desktop app spawns its own +per-profile `hermes serve` backend. This guide adds the patterns for managing them +collectively. ## Quick start