af7dceaf77bbcd5dcafe4f65982c7cd7df5f4c4a
31 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 teknium1
|
a0972b9748 |
fix: widen None-deref guards to config-derived sibling sites + tests
Sibling sites of the salvaged #55997 fix, all reading user-editable config values through .get(key, '').method(): MoA slot provider/model labels, gateway quick-command alias targets (2 sites), gateway.proxy_url, and gateway.relay_url. Regression tests for the contributor's two sites plus the MoA labels. |
||
|
Ben Barclay
|
f64e4f4f57 |
feat(gateway): generic OIDC client-credentials relay provisioning (NAS-free) (#60730)
For air-gapped / self-hosted-IdP deploys with NO Nous Portal, let the gateway
obtain its caller-identity bearer from a generic OAuth2 client_credentials grant
against the operator's own IdP (e.g. Microsoft Entra ID) instead of only
resolve_nous_access_token(). The connector's OIDC tenant resolver reads a claim
(default tid) off that token as the tenant.
- gateway/relay: new canonical _resolve_relay_identity_token() — client_credentials
when gateway.idp.token_url (or GATEWAY_RELAY_IDP_* env) is set, else Nous Portal
(unchanged default). Wired into self_provision_relay().
- hermes_cli/gateway_enroll: _resolve_identity_token() delegates to the canonical
resolver so the enroll CLI and the runtime self-provision path share ONE impl.
Config via gateway.idp.{token_url,client_id,client_secret,scope} in config.yaml
(env override GATEWAY_RELAY_IDP_*). No behaviour change when unset.
Tests: tests/gateway/relay/test_identity_token_resolver.py (6 — mode selection,
request shape, config/env precedence, fail-closed). Relay suite 162 pass.
Validated via the cross-repo gateway<->connector live E2E (provision, managed
self-provision, inbound round-trip, /link) against a connector running the OIDC
tenant resolver with zero NAS config.
|
||
|
Ben Barclay
|
4e4a69cbf7 |
feat(relay): carry routed profile from the connector wire source (#60586)
The multiplex machinery already routes an inbound message to a profile via SessionSource.profile (build_session_key namespacing + the per-turn config/credential scope in SessionStore._resolve_profile_for_key). But the relay path never populated it: _event_from_wire rebuilt the SessionSource field-by-field and dropped any 'profile' the connector sent, so a Team-Gateway (connector + relay) message could not be routed to a specific profile the way the /p/<profile>/ HTTP prefix and per-credential polling adapters already can. Stamp source.profile from the wire payload in _event_from_wire. This is the last missing link for NAS-driven per-profile routing over the relay in multiplex mode; the connector populating the field ships separately (gateway-gateway contract adds the optional wire field). Back-compat: absent 'profile' → None → legacy agent:main namespace, byte-identical to today for every single-profile gateway. |
||
|
Ben Barclay
|
729bbb7a30 |
refactor(relay): purge platform-specific scope terminology from the relay adapter (D-Q2.5c) (#56016)
The gateway HALF of the D-Q2.5c cleanup (connector half: gateway-gateway #92). Scope is STRICTLY the relay adapter (gateway/relay/) — session.py and every native platform adapter are untouched (SessionSource.guild_id remains for their use; it is NOT relay-only). Within gateway/relay/, drop the D-Q2.5 wire dual-write/dual-read alias AND genericize all platform-specific (Discord "guild") scope terminology: - ws_transport._event_from_wire: read scope_id only (drop the ?? guild_id fallback). - adapter._with_scope: emit scope_id only on outbound metadata (drop the guild_id dual-write); genericize the "GUILD reply" docstring to "SCOPED reply". - adapter._capture_scope: read source.scope_id only; rename the local `guild` var to `scope`; genericize the docstring + the _scope_by_chat/_dm_user_by_chat field comments ("guild_id (Discord)" -> "scope_id (server/workspace scope)"). - __init__.relay_route_keys docstring: "guild_ids" -> "scope_ids". - The ONE real Discord `guild_id` kept: the raw inbound interaction payload field (payload.get("guild_id")), which is Discord's own wire field, mapped straight into the generic scope_id slot — unchanged. Contract doc (docs/relay-connector-contract.md): reframe the `guild_id` row as a legacy alias the connector no longer reads (session.py's agent-wide to_dict() still emits it for non-relay persistence, so it stays documented + wire-present but ignored) — accurate, and keeps the to_dict()-vs-doc conformance test green. Tests (relay only): migrate the wire-key writes + assertions guild_id -> scope_id across test_relay_adapter / _ws_transport / _passthrough / _roundtrip / _roundtrip_telegram / _multiplatform; keep raw Discord `type:2` interaction payloads' guild_id (real Discord field) and the conformance test's guild_id parametrize (validates the kept legacy field stays wire-reachable). Gate: 156 relay tests pass, ruff clean. Cross-repo E2E — all 14 drivers pass BOTH ways: connector#92 (scope_id-only) x agent-main (still dual-reads) AND connector#92 x this worktree (scope_id-only). Deploy-order-safe either way. |
||
|
Ben Barclay
|
3a55f66602 |
refactor(relay): adopt scope_id wire key (guild_id → scope_id dual-read/write) (#55289)
Gateway half of relay-platform-parity Phase 2.5 (D-Q2.5). The relay wire's platform-neutral scope discriminator is renamed guild_id → scope_id; this is the hermes-agent side of the cross-repo wire-compatible migration. - SessionSource: scope_id is canonical; guild_id kept as @deprecated alias. __post_init__ mirrors the two so all existing SessionSource(guild_id=...) constructors across native adapters keep working unchanged. to_dict dual-WRITES scope_id+guild_id; from_dict dual-READS scope_id ?? guild_id. - relay/adapter.py: capture + outbound metadata dual-read/write scope_id. - relay/ws_transport.py: _frame_to_event dual-reads scope_id ?? guild_id. - docs/relay-connector-contract.md: document scope_id (canonical) + guild_id (deprecated alias) in the §3 SessionSource field table (conformance test). 250 relay+session+contract tests green. Solo lane (relay). |
||
|
Ben Barclay
|
6e4e5967f7 |
feat(relay): multi-platform-per-agent — list identity, provision-loop, N-hello, per-frame egress (Phase 1.5) (#52830)
Cut over the agent half of Shape A (D-Q1.5a/b.1/c) to front a SET of platforms on
one relay WS:
- relay_platform_identities() parses GATEWAY_RELAY_PLATFORMS (list) +
GATEWAY_RELAY_BOT_IDS (JSON keyed map {platform:{botId,username?}}). Cut over
from the scalar GATEWAY_RELAY_PLATFORM/_BOT_ID (no fallback, D-Q1.5c).
- self_provision_relay() loops one /relay/provision per platform under one
gatewayId+secret, partial-failure-tolerant.
- WebSocketRelayTransport takes the identity SET, sends one hello per identity
(connector accumulates the advertised set), and stamps the per-frame
OutboundFrame.platform + its matching advertised botId on outbound.
- RelayAdapter remembers each chat's underlying source.platform (mirroring the
existing guild/dm scope capture) and tags the reply's egress platform.
- send_relay_policy() declares one relevance policy per fronted platform (the
connector keys policy by (tenant,platform,instanceId)).
Single-platform deploys are byte-identical on the wire (1-element list, no per-frame
tag -> connector session-default fallback). typecheck/ruff clean; relay unit 221 pass
(+10 new); all 15 cross-repo E2E drivers green vs connector origin/main.
|
||
|
Ben Barclay
|
cb7d1f68f8 |
fix(relay): accept is_reconnect kwarg in RelayAdapter.connect (#52911)
The gateway reconnect watcher (gateway/run.py) recovers a platform after a
fatal adapter error by building a fresh adapter and calling
connect(is_reconnect=True). Every BasePlatformAdapter implements
connect(*, is_reconnect: bool = False) for this — except RelayAdapter, whose
connect() was bare. So the watcher's recovery path raised:
TypeError: connect() got an unexpected keyword argument 'is_reconnect'
Observed live on a hosted staging agent: after a fatal relay adapter error the
watcher could never re-establish relay, so the shared-bot inbound never reached
the gateway and Discord DMs stopped (dashboard surfaced the TypeError).
Relay deliberately ignores the flag: the #46621 server-side-queue-preservation
concern doesn't apply, because relay's outage buffer is the connector's durable
buffer (replayed on the transport's re-handshake), not a gateway-side queue the
adapter owns. Routine WS drops are already handled by the transport's own
reconnect supervisor (WebSocketRelayTransport, reconnect=True); the watcher path
is fatal-error recovery, and the fatal handler disconnect()s the old adapter
(cancelling its supervisor) before a fresh adapter+transport is built, so there
is no double-dial.
Adds two regression tests (both proven red without the fix): connect(is_reconnect=True)
reaches the same transport-less RuntimeError instead of TypeError, and the
signature matches BasePlatformAdapter.connect.
|
||
|
Ben Barclay
|
72ae163250 |
fix(relay): authorize relay-delivered events by delivery, not source.platform (#52306)
* fix(relay): authorize relay-delivered events by delivery, not source.platform The #52190 upstream-authz fix keyed _is_user_authorized off source.platform via _adapter_authorization_is_upstream(source.platform). But a relay *message* inbound carries the UNDERLYING platform (source.platform == discord/telegram/...), NOT Platform.RELAY, because ws_transport._event_from_wire maps the connector's wire payload (platform="discord") straight onto SessionSource for session-keying and egress. The relay adapter is registered only under Platform.RELAY, so adapters.get(Platform.DISCORD) misses, the trusted-upstream branch is skipped, and the user hits the env-allowlist default-deny: WARNING gateway.run: Unauthorized user: <id> (<name>) on discord (Live staging bug: alpha tester linked successfully, then every follow-up DM was silently dropped.) Fix: the authentic trust signal is that the event was delivered over the per-instance-authenticated relay WS, not which platform it underlies. Add a wire-INVISIBLE SessionSource.delivered_via_upstream_relay flag, stamped by the relay transport in _event_from_wire, and authorize on it. The flag is excluded from to_dict/from_dict so a peer can neither forge it across the wire nor have it restored from persistence. The existing adapter-flag check is retained for events whose source.platform IS Platform.RELAY (interaction-passthrough). A direct Discord event on a multiplexing gateway (direct + relay adapters) is unmarked and still default-denies. * fix(relay): use identity check on delivery marker to avoid MagicMock fail-open A MagicMock() source (used by test_signal.py and other gateway tests) auto- vivifies source.delivered_via_upstream_relay as a truthy Mock, which a bare truthiness check would treat as authorized — flipping test_signal_in_allowlist_maps from False to True. The marker is a real bool on SessionSource, so check 'is True' explicitly: refuses to authorize any non-bool stand-in, defensive against accidental fail-open. |
||
|
Ben
|
0c3f197cff |
fix(relay): re-attach DM author user_id on outbound for connector egress
A DM reply carries no guild_id, so the connector's egress guard cannot resolve the owning tenant from metadata.guild_id and declines the send with "discord egress declined: target not routed to an onboarded tenant" — the bug behind "the bot never replies in DMs". Guild replies are unaffected (they carry guild_id), which is why the guild path worked end-to-end while DMs looked broken. The connector now resolves a DM reply's tenant from the recipient's author binding (gateway-gateway #67, resolveByUser keyed on metadata.user_id) — the outbound counterpart to inbound Phase 7a author-first resolution. But it needs the recipient user_id ON the outbound action, and the adapter only re-attached guild_id (_capture_scope/_with_scope), no-op for DMs (the docstring even said so). This extends the adapter's inbound-scope capture: for a DM (no guild_id) remember chat_id -> the authentic author user_id we observed, and re-attach it as metadata.user_id on outbound. Guild capture is unchanged and wins when present; user_id is the DM-only fallback. The id is the one the connector observed inbound (never gateway-asserted), so the trust invariant holds. +4 unit tests (DM reply re-attaches user_id + no guild_id; unknown chat invents nothing; explicit user_id preserved; guild reply never carries user_id). Proved load-bearing (reverting the re-attach fails the DM test). 144 relay tests pass, ruff clean. Pairs with gateway-gateway #67 (the connector-side resolver). Together they close the DM-reply egress gap end-to-end. |
||
|
Ben
|
96af4bec30 |
feat(relay): add go_dormant() transport mode for scale-to-zero (0.E0)
Net-new WebSocketRelayTransport.go_dormant() + RelayAdapter.go_dormant() — the third transport mode the scale-to-zero behaviour layer needs, distinct from both disconnect() and an unexpected close (decisions.md D12/F14): - disconnect() sets _closing=True and CANCELS the reconnect supervisor (terminal "shutting down for good") -> a suspended machine never re-dials on wake, stranding its buffered backlog. - an unexpected close re-dials IMMEDIATELY -> the socket never stays down, so the platform proxy never suspends the machine. go_dormant(): going_idle->ack (reuse go_idle), then close the socket WITHOUT setting _closing, so the reader's fall-through still arms the reconnect supervisor (wake path stays live) but on the longer _dormant_redial_s cadence so it doesn't fight the platform suspend window. A successful re-dial clears _dormant. Honors the §3.4 wake->reconnect->drain contract. Tests: 6 new in test_relay_going_idle.py incl. the F14 regression guard (routing dormancy through disconnect() fails exactly the 4 wake-path tests). Full relay suite 140 passed. |
||
|
Ben
|
d335164833 |
fix(relay): authorize relay inbound via connector-enforced upstream authz
A hosted instance fronted by the Team Gateway connector dropped EVERY relay
message as "Unauthorized user" and the agent never replied — despite the
message routing correctly through the connector to the instance.
Root cause: gateway authorization (_is_user_authorized) had no notion of
upstream-enforced authz. Platform.RELAY matches no {PLATFORM}_ALLOWED_USERS
allowlist and isn't in the HA/WEBHOOK always-authorized set, so a relay user
with no env allowlist configured hit the default-deny ("No user allowlists
configured. All unauthorized users will be denied."). The message was received,
then silently denied before reaching the agent.
This is incorrect for relay: the connector authenticates the gateway's WS with
a per-instance secret and performs owner-only author-binding resolution BEFORE
delivering. A message only reaches this gateway because the connector resolved
it to THIS instance's bound user (user_instance_binding), keyed on the author id
the connector OBSERVED off the event — never a gateway claim. The authorization
decision is already made by a trusted, authenticated upstream; there is no local
RELAY_ALLOWED_USERS allowlist to consult, and default-denying for its absence is
the bug.
Fix: add a generic BasePlatformAdapter.authorization_is_upstream capability
(default False) that the relay adapter overrides to True, plus a dedicated
trusted branch in _is_user_authorized that honors it. This is delegation to a
trusted upstream, NOT a fail-open: it fires only for an adapter that explicitly
declares the flag; every direct network-exposed adapter leaves it False and the
env-allowlist default-deny (SECURITY.md §2.6) is unchanged. Distinct from
enforces_own_access_policy, which mirrors a LOCAL config-driven allowlist —
this delegates to an authenticated upstream's decision.
Tests: behavior contract that the base defaults False, the relay adapter
declares True, a relay user (group + DM) is authorized with no env allowlist,
and crucially a non-upstream adapter with no allowlist still default-denies
(guards against the fix becoming a blanket fail-open). 6 new tests; relay +
authz + config-policy suites green (134 + 90).
Found via live staging debug of the Discord self-serve onboarding flow.
|
||
|
Ben
|
c93b9f9057 |
feat(relay): terminal 4401 (opt-out) → clean "Relay disabled" state
Phase 7 Unit 7d-B. When an operator opts an instance OUT of the Team Gateway
relay (Unit 7b deprovision), the connector revokes the per-gateway secret and
closes the gateway's WS with 4401. The reconnect supervisor previously treated
EVERY close as retryable, so the live process spun "retrying 4401" forever and
the dashboard showed a red error — opt-out looked like a failure.
Now a 4401 close that arrives AFTER a successful handshake is recognized as a
terminal credential revocation:
- ws_transport.py: track `_handshake_succeeded` (set when a descriptor is
received); on a 4401 close after a prior success, latch `auth_revoked` and do
NOT spawn the reconnect supervisor. A 4401 BEFORE any successful handshake
stays retryable (cold-start / not-yet-provisioned race, not a revocation).
New `auth_revoked` property + a websockets-version-safe close-code reader
(prefers `.rcvd`/`.sent` Close frames; `.code` is deprecated in websockets 13+).
- adapter.py: a revocation monitor turns `transport.auth_revoked` into a clean,
NON-retryable `relay_disabled` fatal and notifies the gateway's fatal-error
handler (so the adapter is removed and NOT queued for reconnection — the
credential is dead until the instance is recreated). Monitor is cancelled on
disconnect; only started when the transport exposes `auth_revoked` (prod WS).
- run.py: `_handle_adapter_fatal_error` maps the `relay_disabled` code to a
`disabled` platform_state (not `fatal`/`retrying`).
- web: PlatformsCard renders the `disabled` state with a neutral outline badge,
a PowerOff icon, and muted (not destructive-red) text + message. New optional
`status.disabled` i18n string ("Disabled").
Also bundles the Phase 7 contract-doc update (this doc is authoritative in
hermes-agent): docs/relay-connector-contract.md gains an "Author-first
resolution + the account-link (DM) path" section documenting the
multi-tenant-guild rule (D-7.2 — route by authenticated author binding, never by
guild; unlinked → fail-closed), the `/link <code>` DM flow, and the
connector-authoritative opt-out + terminal-4401 behavior this PR implements.
Tests: +2 ws_transport (4401-after-handshake terminal / no-reconnect;
4401-before-handshake stays retryable) and +2 adapter (revocation → non-retryable
relay_disabled fatal + handler fired; no-revocation → no fatal). 138 relay tests
pass (incl. the contract-doc conformance test); ruff clean; web tsc clean.
Phase 7 Unit 7d-B (relay-adapter solo lane). Q17 → Option 2; Option 3 (live
de-register, no recreate) + the restart-re-provision hole deferred post-alpha.
|
||
|
Ben Barclay
|
6e88f7b6f7 |
feat(relay): Phase 5 Unit C — wake primitive (gateway side) (#51595)
Register a per-instance wakeUrl and forward it to the connector at self-provision so a suspended gateway can be poked awake when buffered work arrives (pairs with the connector-side WakePoker). - relay_wake_url() resolver (env GATEWAY_RELAY_WAKE_URL, then gateway.relay_wake_url in config.yaml), mirroring relay_instance_id() - thread wake_url through _post_provision (adds wakeUrl to the body only when set) + self_provision_relay (resolve, forward, log) - hermes gateway enroll --wake-url <url> persists GATEWAY_RELAY_WAKE_URL - document the §5.2 wake poke in relay-connector-contract.md §3.3 - tests: relay_wake_url resolution (env/config/absent), provision forwarding, body-only-when-set (6 new; 130 relay tests pass) The actual reconnect+drain on wake is Unit B's loop; this unit only wires the wake SIGNAL. Opt-in: absent wakeUrl => connector never pokes. |
||
|
Ben Barclay
|
40fddc9e4c |
feat(relay): Phase 5 §5.3 going-idle / buffered-flip primitive (gateway side) (#51572)
The gateway half of the going-idle/buffered-flip primitive (scale-to-zero PRIMITIVE, not the behaviour). Integrates with the EXISTING drain transition: - ws_transport: `go_idle()` sends `going_idle` + awaits the connector's `going_idle_ack` (connector-authoritative flip-then-ack, Q-5.3c — stays serving until the ack so nothing is lost in the flip window); acks a buffered inbound (bufferId present) via `inbound_ack` after the handler runs (drain-without-dup on the delivery leg); NET-NEW reconnect loop re-dials + re-handshakes after an unexpected close (off by default, on in production). - adapter: emits `going_idle` from its existing `disconnect()` drain seam before tearing down the socket; best-effort + guarded (never blocks shutdown). - transport Protocol + contract doc §3.2 document the 3 new frames. +6 relay tests (124 pass). NOT in scope: the autonomous idle timer / machine suspend / NAS health model (deferred behaviour). Ben's relay-adapter solo lane. |
||
|
Ben Barclay
|
45bc4fb37f |
feat(relay): declare relevance policy to the connector + document the management plane (#51248)
The gateway half of Phase 6 Unit ζ: project the agent's existing relevance
knobs into the connector's platform-agnostic vocabulary and declare them at boot
over the /relay/policy route, so the SAME mention-gating / free-response /
allow-bots behavior the agent applies directly also governs relay delivery (and
excluded chatter never wakes a scaled-to-zero agent).
- gateway/relay/__init__.py:
- relay_relevance_policy(): project require_mention -> requireAddress,
free_response_channels -> freeResponseScopes, {PLATFORM}_ALLOW_BOTS in
{mentions,all} -> allowOtherBots. Reads the fronted platform's config block
+ bridged top-level keys. Returns None when all-default (the connector's
quiet default already matches) or no concrete platform is fronted.
- send_relay_policy(): POST /relay/policy authenticated with the gateway's own
per-gateway upgrade token (make_upgrade_token — same bearer as the WS
upgrade), so the connector attaches it to the authenticated instance, never
a body-asserted id. Re-declares every boot (self-healing, full replace).
NEVER raises, NEVER blocks boot — relevance is an optimization layered on
the δ/ε authorization gate. Reuses the per-gateway secret + the
/relay/provision host; no new inbound surface, no new credential.
- _policy_url(): ws(s)://…/relay -> http(s)://…/relay/policy.
- gateway/run.py: call send_relay_policy() after register_relay_adapter()
succeeds (the secret is resolved by then).
- docs/relay-connector-contract.md: new §7 documenting per-instance delivery +
the management plane (/manage/* + /relay/policy) + the relevance-declaration
contract; versioning renumbered to §8. Contract conformance test stays green
(§2/§3 tables untouched).
Tests: +12 (projection mapping incl. comma-string + top-level fallback; send
auth/skip/fail-soft/non-200). Full relay suite 118 pass. The connector route is
already E2E-proven (connector repo gateway_policy_driver.py); this adds the real
gateway send-path it pairs with.
This completes Phase 6 (Team Gateway per-user isolation) end to end.
|
||
|
Ben Barclay
|
75a70d98f3 |
feat(relay): forward a stable instance id at self-provision (Phase 6 Unit α) (#50772)
Add relay_instance_id() (env GATEWAY_RELAY_INSTANCE_ID first, then gateway.relay_instance_id in config.yaml, mirroring the other relay readers) and forward it in the /relay/provision body so the connector can bind gatewayId -> instanceId and route inbound per-instance once Phase 6 delivery lands. The value is gateway-asserted but safely scoped: the org/tenant stays NAS-token-verified at the connector, so a dishonest gateway can only bind its OWN tenant's instance — same posture as relay_endpoint(). instanceId is only added to the body when present, so omitting it lets the connector store null (back-compat: self-hosted / pre-Phase-6 gateways simply have no binding yet). For a managed (NAS-hosted) agent the id is NAS's AgentInstance.id, stamped into the container env beside GATEWAY_RELAY_URL. Tests: reader (env/config/absent), self_provision_relay forwards the id (set + absent), and the real _post_provision body includes instanceId ONLY when set. Refs: ~/nous/specs/gateway-gateway plan.md Phase 6 Unit α; decisions.md Q11. |
||
|
Ben Barclay
|
64a507da44 |
feat(relay): handle passthrough_forward over the WS (Phase 5 §5.1, gateway half) (#50702)
The connector half (gateway-gateway) moves the passthrough plane's post-ACK forward off the HTTP gatewayEndpoint onto the gateway's outbound /relay WS via a new passthrough_forward frame. This is the gateway side: the relay adapter now RECEIVES and handles that frame, so a hosted gateway (no public IP) can process forwarded Class-2/3 traffic (Discord interactions, Twilio) over the socket it already holds — closing the "passthrough inbound doesn't work for hosted gateways" gap. - ws_transport.py: decode the passthrough_forward frame; PassthroughForward dataclass + _passthrough_from_wire (base64 body -> exact bytes, byte parity with the connector's toPassthroughForward); set_passthrough_handler mirrors set_interrupt_inbound_handler. - transport.py: PassthroughHandler type + set_passthrough_handler on the RelayTransport protocol. - adapter.py: connect() wires the passthrough handler; _on_passthrough decodes the (already-sanitized, token-free) forward and, for a Discord interaction, converts it to a MessageEvent routed through the normal agent path (handle_message) — the reply egresses over the outbound / token-less follow_up path, so the gateway never holds the interaction credential. Never raises (a bad forward can't kill the read loop). Non-discord forwards (Twilio) are logged + dropped for now. - docs/relay-connector-contract.md: document the passthrough_forward frame + PassthroughForward shape + §3.1. The interaction -> MessageEvent CONVERSION semantics (slash-command vs button UX, option rendering) are the open sub-design flagged in the spec; the TRANSPORT + receive mechanism (this) is settled per Ben's Gate-2 decision: "the relay adapter handles receiving these events over the WS." Tests (tests/gateway/relay/test_relay_passthrough.py): byte-preservation round-trip (+ malformed-body tolerance), connect() wiring, application-command and message-component interactions route through handle_message with correct session source + scope capture, malformed/non-discord forwards dropped cleanly. 100 relay tests green. Pairs with the connector PR (gateway-gateway). |
||
|
Ben Barclay
|
a64fc490fe |
fix(relay): make hosted gateways actually connect AND complete the inbound/outbound round-trip (#48828)
* fix(relay): enable RELAY platform + normalize dial URL so hosted gateways actually connect Three bugs blocked a self-provisioned hosted gateway from ever establishing its inbound relay WS (found while standing up the live staging end-to-end). Each masked the next; all three are needed for inbound to work. 1. RELAY platform never enabled in config.platforms (gateway/config.py). register_relay_adapter() puts the adapter in the platform_registry, but start_gateway()'s connect loop iterates self.config.platforms — which never contained Platform.RELAY. So the adapter was "registered" but never connected (logs showed "relay adapter registered" then "No messaging platforms enabled"). Fix: _apply_env_overrides now enables Platform.RELAY (mirroring relay_url into extra for the connected-checker) when GATEWAY_RELAY_URL (env) or gateway.relay_url (yaml) is set. Absent -> no RELAY entry (direct/ single-tenant gateways unaffected). 2. URL scheme not converted for the WS dial (gateway/relay/ws_transport.py). The relay URL is configured once as the http(s):// base (used as-is for the provision POST), but websockets.connect rejects http(s):// with "scheme isn't ws or wss". Fix: _ws_dial_url converts https->wss / http->ws. 3. /relay path not appended (same helper). The connector mounts its WebSocketServer at path "/relay" and returns HTTP 400 on an upgrade to any other path. GATEWAY_RELAY_URL is the base (no /relay), so the dial hit "/" -> 400. Fix: _ws_dial_url ensures the path ends in /relay. Idempotent — a URL already carrying ws(s):// and/or /relay is unchanged, so provision's _provision_url (which derives /relay/provision from either form) still works. Why the cross-repo E2E missed #2/#3: the stub connector binds ws://host:port and its websockets.serve accepts ANY path, so neither the scheme nor the /relay path was exercised. Real connector needs both. Verified live on staging hermes-agent-stg-automated-perception-5054: after the fixes the gateway logs "Connecting to relay..." -> "✓ relay connected" -> "Gateway running with 1 platform(s)" against wss://gateway-gateway.staging-nousresearch.com/relay, stable. Tests: added _ws_dial_url scheme+path+idempotency cases (test_ws_transport.py) and RELAY-platform-enablement cases for env + yaml + absent (test_config.py). Full gateway/relay + config suites green (191 passed). Relay-adapter lane. EXPERIMENTAL. * fix(relay): re-attach guild_id to outbound so connector egress resolves the tenant The final bug in the hosted-relay round-trip. Inbound worked end to end (Discord -> connector -> bus -> agent WS -> agent runs -> reply), but the reply's egress was declined by the connector: "discord egress declined: target not routed to an onboarded tenant". Cause: the connector's routedEgressGuard resolves the owning tenant from the OUTBOUND action's metadata.guild_id (Discord's routing discriminator). The gateway's generic delivery path builds outbound metadata via run.py _thread_metadata_for_source, which only carries thread_id (and returns None entirely for a non-threaded message) — so guild_id never reached the connector, tenant resolution failed, and the shared bot refused to post. Fix (relay-adapter-local, no perturbation of the generic delivery path or other platforms): RelayAdapter learns chat_id -> guild_id from each inbound event (_capture_scope) and re-attaches it to the outbound action's metadata in send() (_with_scope) when not already present. No-op for chats we never saw inbound (e.g. DMs) and never overwrites an explicit guild_id. Verified live on staging hermes-agent-stg-automated-perception-5054: an @mention in #general now produces a visible bot reply — full multi-tenant relay round-trip (real Discord -> shared connector bot -> tenant routing -> agent WS -> reply egress -> Discord). Tests: _capture_scope/_with_scope reattach, no-scope no-op, explicit-guild_id preserved (test_relay_adapter.py). Full relay + config suites green (160 passed). Relay-adapter lane. EXPERIMENTAL. |
||
|
Ben Barclay
|
2c6e266e88 |
fix(relay): trigger self-provision on relay-config + NAS token, not is_managed() (#48724)
self_provision_if_managed() gated on is_managed(), but is_managed() means
"NixOS/package-manager-managed" (it keys on HERMES_MANAGED or a ~/.hermes/.managed
marker) — NOT "NAS-hosted". A NAS-provisioned Fly agent sets NEITHER, so the gate
was always False and relay self-provision SILENTLY no-oped on exactly the hosted
agents it was built for. Caught live: a staging agent with GATEWAY_RELAY_URL
correctly stamped logged "No messaging platforms enabled" and never dialed the
connector; HERMES_MANAGED was unset on the machine. The unit tests had mocked
is_managed()->True, so they passed while the real trigger never fired (mocked-
trigger blind spot).
Fix: drop the is_managed() gate and rename self_provision_if_managed ->
self_provision_relay. The real trigger is now "relay_url() set + no pinned secret
+ a resolvable NAS token", which is both NAS-independent and self-guarding:
- NAS-hosted agent: GATEWAY_RELAY_URL + no pinned secret + bootstrapped NAS
token -> self-provisions.
- Self-hosted + `hermes gateway enroll`: pinned GATEWAY_RELAY_SECRET -> skipped
(existing secret-present guard).
- Self-hosted, unenrolled, no NAS identity: resolve_nous_access_token() fails
-> graceful no-op (existing fail-soft path).
Security: unchanged trust model. The connector still derives tenant from the
validated NAS token; this only broadens WHEN the provision attempt fires, and
every broadened case is still guarded by token-resolution + pinned-secret-skip.
Tests: replaced the (wrong) "skips when not managed" test with a regression test
proving a NAS host where is_managed()==False STILL provisions; renamed all call
sites; added a "no NAS token -> non-fatal skip" test for the self-hosted branch.
88 relay tests pass.
Relay-adapter lane. EXPERIMENTAL.
|
||
|
Ben Barclay
|
d2c53ff558 |
feat(relay): WS-only inbound on the gateway adapter (Phase 3) (#48294)
The connector now delivers inbound (messages + interrupts) over the gateway's OUTBOUND /relay WebSocket, not a signed HTTP POST to an inbound endpoint. The gateway needs no inbound HTTP port — which is what makes hosted gateways (no public IP) able to receive inbound at all. - gateway/relay/adapter.py: connect() wires set_interrupt_inbound_handler( self.on_interrupt) so connector->gateway interrupt_inbound frames bridge into the existing per-session interrupt path (the inbound message handler was already wired). Removed _maybe_start_inbound_receiver() + the _inbound_runner lifecycle — there is no HTTP receiver anymore. - gateway/relay/inbound_receiver.py: deleted (the signed-HTTP InboundDelivery receiver). - gateway/relay/__init__.py: removed relay_inbound_config() (dead with the receiver gone). The delivery key is still set in-process by self-provision for forward-compat but is no longer consumed for inbound. - docs/relay-connector-contract.md: §3 rewritten — inbound is the WS back-channel routed cross-instance via the connector's relay bus; §5 interrupt + §6 auth table updated; the old signed-HTTP-POST + per-tenant-delivery-key-signing path is documented as superseded. gatewayEndpoint noted as passthrough-plane only. Tests: stub_connector grows set_interrupt_inbound_handler + push_interrupt; new test_relay_interrupt case proves connect() wires BOTH inbound handlers and an interrupt_inbound frame over the WS cancels the right session. Removed the HTTP-receiver test; updated the crypto-shedding scan + self-provision delivery-key assertion. 88 relay tests pass. EXPERIMENTAL. Pairs with gateway-gateway (relay bus + WsGatewayDelivery) and the NAS GATEWAY_RELAY_URL stamp. The cross-repo E2E (connector repo) proves the full multi-instance path against this production adapter code. |
||
|
Ben Barclay
|
0ddd21c74e |
feat(relay): managed-boot self-provision client (Phase 3, gateway side) (#48242)
The gateway half of relay Phase 3. On a MANAGED boot with relay configured and no secret pinned, the runtime self-provisions its relay credentials IN-PROCESS: resolve the agent's own Nous access token (resolve_nous_access_token) -> POST the connector's /relay/provision asserting its own endpoint + route keys -> set GATEWAY_RELAY_ID/SECRET/DELIVERY_KEY into os.environ so the immediately- following register_relay_adapter() reads them and dials out authenticated. No human, no enrollment token, no disk write — the creds live only in process memory (save_env_value refuses under managed anyway, and keeping the secret off any volume is the stronger posture). Stateless: process-env creds don't survive a restart, so a managed container re-provisions every boot; the connector's rotation window covers a still-connected prior instance. An explicitly-pinned GATEWAY_RELAY_SECRET is respected (skip). Self-hosted is unchanged: humans keep using `hermes gateway enroll`. Endpoint provenance is gateway-asserted (GATEWAY_RELAY_ENDPOINT + GATEWAY_RELAY_ROUTE_KEYS, env or gateway.relay_* config) — uniform code path whether the operator sets it (self-hosted) or NAS stamps it (hosted, the only case NAS knows the public URL). Both absent -> outbound-only provisioning (credentials, no inbound routes). The connector scopes the asserted endpoint to the verified tenant, so it stays within the security model. - gateway/relay/__init__.py: relay_endpoint(), relay_route_keys(), _provision_url(), _post_provision(), self_provision_if_managed() (never raises — a provision failure logs and boots without relay auth). - gateway/run.py: call self_provision_if_managed() immediately before register_relay_adapter() in the startup path. Tests: 12 unit (trigger logic, respect-pinned-secret, in-process env wiring, endpoint+routes vs outbound-only, fail-soft on token/connector failure); mutation-checked (drop is_managed guard / pinned-secret guard -> tests fail). Cross-repo live E2E driver lands on the connector side (depends on this). EXPERIMENTAL: relay auth scheme may change until >=2 Class-1 platforms validate. |
||
|
Ben Barclay
|
c276b017ad |
feat(relay): connector⇄gateway channel auth + signed-HTTP inbound receiver + enroll CLI (#48147)
* feat(relay): authenticate the connector⇄gateway WS channel
The relay gateway may be customer-managed and internet-exposed, so the
connector⇄gateway channel is itself authenticated (distinct from the
platform crypto the relay path sheds). Add gateway/relay/auth.py — a
Python port of the connector's HMAC token + delivery-signature schemes
(relayAuthToken.ts / deliverySigning.ts), verified byte-for-byte against
the connector's compiled TypeScript via cross-language test vectors.
Present an Authorization bearer on the /relay WS upgrade keyed by the
per-gateway secret (resolved from GATEWAY_RELAY_ID / GATEWAY_RELAY_SECRET
in env or config). The connector rejects an unauthenticated/invalid/
revoked upgrade with close 4401.
* feat(relay): signed-HTTP inbound delivery receiver
The connector delivers normalized inbound events to a tenant's gateway
over a signed HTTP POST, not the outbound /relay WS: the connector
instance owning a platform socket is generally not the instance a given
gateway dialed out to, so inbound targets a tenant endpoint that may
load-balance across gateway instances.
Add gateway/relay/inbound_receiver.py — verifies x-relay-signature /
x-relay-timestamp over the EXACT raw request bytes (re-serializing would
break the HMAC: JS JSON.stringify is compact, Python json.dumps spaces)
against the per-tenant delivery key verify list within a 300s replay
window, then dispatches messages to handle_message and interrupts to the
interrupt handler. Wire it into the adapter lifecycle (start in connect()
when a delivery key + bind port are configured, tear down in disconnect();
a purely-outbound dev gateway runs without it).
Refine test_relay_sheds_crypto to distinguish PLATFORM crypto (Discord
ed25519, Twilio/WeCom HMAC — still shed) from the connector⇄gateway
CHANNEL auth (intended): auth.py / inbound_receiver.py are exempt from
the platform-symbol scan but still banned from importing platform-crypto
modules, plus a positive guard that auth.py uses only stdlib hmac/hashlib.
* feat(relay): hermes gateway enroll CLI
Add the gateway half of zero-touch enrollment. `hermes gateway enroll`
resolves a fresh Nous Portal access token (the tenant-proving identity),
POSTs {enrollmentToken, gatewayId} to the connector's /relay/enroll, and
persists GATEWAY_RELAY_ID / GATEWAY_RELAY_SECRET / GATEWAY_RELAY_DELIVERY_KEY
to ~/.hermes/.env. The per-gateway secret authenticates the WS upgrade;
the per-tenant delivery key verifies signed inbound deliveries.
Refuses under is_managed() (hosted installs get the secret stamped in by
the orchestrator). Added as an 'enroll' subcommand on the existing
gateway subparser — not a new top-level command.
* docs(relay): inbound is signed HTTP, not WS; document channel auth
Fix the stale contract: §3/§5 said inbound rode the WS socket (single-
instance only, predates the multi-instance socket-ownership + channel-auth
model). Inbound + connector→gateway interrupt are signed HTTP POSTs to the
tenant endpoint. Add §6.1 documenting the two channel-auth schemes (per-
gateway WS-upgrade secret, per-tenant inbound delivery key) and how they
differ from the platform crypto the relay path sheds.
* test(relay): update build_gateway_parser callers for cmd_gateway_enroll
The enroll subcommand added cmd_gateway_enroll as a required keyword-only
arg to build_gateway_parser, but two existing parser-extraction tests still
called it with only cmd_gateway/cmd_proxy — failing CI with TypeError.
Thread the new handler through both call sites and add a test asserting
`gateway enroll` dispatches to cmd_gateway_enroll with its flags parsed.
|
||
|
Ben
|
237fa7d29c |
feat(gateway): register relay adapter from config; drop HERMES_GATEWAY_RELAY gate
Wire the relay adapter into gateway startup and make activation config-driven instead of a dark-launch flag. - gateway/relay/__init__.py: replace relay_enabled()/HERMES_GATEWAY_RELAY with relay_url() (GATEWAY_RELAY_URL env or gateway.relay_url in config.yaml) — the same shape as gateway.proxy_url. register_relay_adapter() registers when a URL is configured and builds a live WebSocketRelayTransport; with no URL it's a no-op (direct/single-tenant deployments unaffected). force=True keeps the transport-less adapter for unit tests. relay_platform_identity() reads the hello platform/botId from GATEWAY_RELAY_PLATFORM/GATEWAY_RELAY_BOT_ID. - gateway/run.py: call register_relay_adapter() during GatewayRunner.start(), right after plugin discovery, so a configured connector relay is registered on every boot. Failures are logged, never block startup. This removes the dark-launch posture: the relay is on whenever it's configured, shipping the production end state rather than hiding it behind a flag. |
||
|
Ben
|
6b03874d07 |
feat(gateway): production WebSocketRelayTransport + descriptor negotiation
Adds the concrete transport behind the RelayTransport Protocol — the missing
'later-phase work' the relay scaffold deferred. The gateway dials OUT to the
connector over a WebSocket and speaks the newline-delimited JSON frame protocol
(docs/relay-connector-contract.md; connector src/relay/protocol.ts):
- connect(): opens the ws, sends hello{platform,botId}, starts a background
read loop, and resolves handshake() when the connector's descriptor frame
arrives.
- inbound frames -> the registered InboundHandler (rebuilt into a MessageEvent
via _event_from_wire, mapping the snake_case SessionSource wire form back
onto the gateway dataclasses).
- send_outbound / send_follow_up / get_chat_info: request/response correlated
by a uuid requestId against a per-request future, with a timeout so a caller
never hangs; send_interrupt is fire-and-forget.
- disconnect(): cancels the reader, closes the ws, and fails any in-flight
outbound waiters with a structured error.
RelayAdapter.connect() now negotiates the real CapabilityDescriptor from the
transport and adopts it (_apply_descriptor updates MAX_MESSAGE_LENGTH +
markdown surface), replacing the construction-time placeholder. Lazy
'import websockets' mirrors gateway/platforms/feishu.py; WEBSOCKETS_AVAILABLE
gates construction.
|
||
|
Ben
|
3db9b3e616 |
feat(gateway): token-less follow_up outbound op (A2 capability action)
The relay outbound surface had send/edit/typing but no way to act on a SHARED-identity capability (e.g. a Discord interaction follow-up token, ~15min) that the connector captured + stripped at the edge. Under A2 that credential never reaches the gateway, so the gateway can't just 'send with the token' — it needs a semantic op naming the session it's already in. Adds the follow_up op end to end on the gateway side: - RelayTransport.send_follow_up(action): protocol method. Action carries op='follow_up' + session_key + kind + content (+ metadata) and NO token. - RelayAdapter.send_follow_up(session_key, kind, content, metadata): builds that action and returns a SendResult. The connector resolves the real capability (its resolveOutboundCapability), enforces the tenant match so tenant B can't wield tenant A's capability, and egresses; success=False when the capability is absent/expired/mismatched (nothing to retry — a leaked gateway holds zero capability material). - StubConnector records follow_ups + a canned next_follow_up_result. Tests: round-trips without a token; the wire action carries only session refs (no credential value field — the 'kind' string is a type ref, not the secret); failure surfaces when the connector can't resolve; no-transport fails cleanly. 55 passed. §4 doc entry follows in the contract-rewrite commit. |
||
|
Ben
|
a3cdd8c39d |
feat(relay): route mid-turn /stop over relay interrupt channel
RelayAdapter.on_interrupt(session_key, chat_id) bridges a connector-delivered mid-turn /stop into the existing interrupt_session_activity path, setting the per-session _active_sessions Event and clearing typing — cancelling exactly the targeted session's turn without touching siblings (mirrors test_stop_thread_ sibling isolation). Transport.send_interrupt carries the gateway-side egress to the connector for socket-owner routing. Phase 1, Task 1.4 of the gateway-relay plan. |
||
|
Ben
|
d0133fd8e4 |
feat(relay): register RelayAdapter through platform registry (flagged off by default)
register_relay_adapter() registers the generic 'relay' platform via the same PlatformRegistry path as plugin adapters — no core dispatch changes. OFF by default (dark-launch): only registers when HERMES_GATEWAY_RELAY is truthy (or force=True for tests), so existing single-tenant/direct deployments are unaffected. Factory builds a transport-less RelayAdapter with a placeholder descriptor; the real descriptor is negotiated at handshake. Phase 1, Task 1.3 of the gateway-relay plan. |
||
|
Ben
|
259e78e175 |
feat(relay): transport protocol + test-only stub connector
Defines RelayTransport (lifecycle/handshake/inbound/outbound/interrupt) as the gateway<->connector wire contract; RelayAdapter.connect now registers an inbound handler that bridges connector-delivered MessageEvents into handle_message. Adds an in-memory StubConnector under tests/ and an E2E round-trip proving: connect registers the handler, inbound events reach the adapter, guild_id drives build_session_key isolation (two guilds -> two keys; same guild/channel/user -> one), outbound send round-trips, get_chat_info is proxied. Phase 1, Task 1.2 of the gateway-relay plan. |
||
|
Ben
|
b0999c82f3 |
feat(relay): generic RelayAdapter advertising negotiated capabilities
One BasePlatformAdapter subclass that reads its capability profile from a CapabilityDescriptor: MAX_MESSAGE_LENGTH attribute, message_len_fn (table-driven by len_unit: chars=len, utf16=Telegram-style code units), supports_draft_streaming. Implements the four abstract methods (connect/disconnect/send/get_chat_info) by delegating to an injected RelayTransport (full protocol lands in Task 1.2). Adds Platform.RELAY enum member. No per-platform gateway code. Phase 1, Task 1.1 of the gateway-relay plan. |
||
|
Ben
|
3db49381d6 |
feat(relay): derive descriptor from PlatformEntry
CapabilityDescriptor.from_platform_entry() projects an existing PlatformEntry
(label, max_message_length, emoji, platform_hint, pii_safe, name) into a
descriptor, proving the descriptor is a projection of existing config rather
than a parallel concept. Runtime-only capabilities (len_unit, draft/edit/
thread/markdown) are caller-supplied. max_message_length==0 ('no limit') maps
to the stream_consumer 4096 default.
Phase 0 complete. Task 0.3 of the gateway-relay plan.
|
||
|
Ben
|
53d9b98305 |
feat(relay): experimental CapabilityDescriptor schema
Frozen, JSON-serializable handshake payload the connector hands the future RelayAdapter: char limit, draft-streaming/edit/threading flags, markdown dialect, len_unit. Mostly a wire projection of PlatformEntry + the adapter capability methods. contract_version gates additive-only evolution; declared EXPERIMENTAL until >=2 Class-1 platforms validate it. from_json ignores unknown keys (forward-compat) and fills optional defaults. Phase 0, Task 0.2 of the gateway-relay plan. |