Commit Graph

61 Commits

Author SHA1 Message Date
Teknium c4f96565d0 refactor(adapters/whatsapp_webhook): 6449->4465; whatsapp_cloud/common/plugin send + media unification, webhook route/filter dispatch tables 2026-09-02 14:06:38 -07:00
Teknium dc7e1b7ab9 fix(webhook): load URL-resolved profile's skills under multiplex
A `/p/<profile>/webhooks/<route>` request resolved the profile from the URL
but ran the route script, prompt render and `skills:` lookup with no
profile scope — the runner only enters `_profile_runtime_scope` later,
around `handle_message` — so routed webhooks loaded the launch (default)
profile's skills and logged "Skill not found" for the routed profile's own.

- gateway/platforms/webhook.py: add `_profile_scope(profile)` (nullcontext
  when no prefix was resolved; `_profile_runtime_scope(get_profile_dir(p))`
  otherwise, same helper the runner uses) and wrap the script / render /
  skill-injection block in it. Bare routes are unchanged.
- agent/skill_commands.py: `scan_skill_commands` scanned the import-time
  `SKILLS_DIR` (frozen to the launch home), so even a correctly scoped call
  listed default's skills; the #88023 home-keyed cache alone could not fix
  that. Use the call-time `_skills_dir()` there and at the two other
  SKILLS_DIR-relative sites in the module.
- agent/skill_utils.py: `normalize_skill_lookup_name` used the same frozen
  root, so a routed profile's absolute skill_dir was rejected by
  `skill_view` ("must be a relative path within the skills directory").
  Resolve against `_skills_dir()` — the root `skill_view` itself enforces.

Fixes #67277

Co-authored-by: Juani Lezcano <tky.juani@gmail.com>
Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
2026-09-02 07:00:13 -07:00
kshitijk4poor c0ff25a1f8 fix(gateway): stop blocking the event loop — off-loop hot sites + ASYNC lint ratchet
Pattern-A architectural fix: blocking calls inside async functions freeze
the gateway/uvicorn event loop for every adapter, timer, and health check.
Known incidents: 17-minute getaddrinfo freeze (#91912 class), 10s restart
freeze in start_gateway (#36163).

Fixes at the four unguarded core sites:
- gateway/platforms/webhook.py: `gh pr comment` subprocess (30s timeout)
  now runs via asyncio.to_thread — a webhook delivery no longer freezes
  every other platform for the duration of a network call.
- gateway/run.py start_gateway --replace: two time.sleep() waits (10s +
  5s worst case) become await asyncio.sleep() (re-lands #36163 at current
  line numbers, credit AhmetArif0).
- gateway/slash_commands.py /save: session render + file write move off
  the loop (scales with transcript size).
- hermes_cli/web_server.py voice TTS: multi-MB audio file read + unlink
  move off the loop.

Prevention gate so the bug class cannot re-enter:
- pyproject.toml [tool.ruff.lint] select gains ASYNC210/220/221/251
  (blocking HTTP / Popen / subprocess.run / time.sleep in async def).
  These run in the existing blocking `ruff check .` CI job.
- Frozen ratchet baseline in per-file-ignores for the remaining legacy
  sites (detached restart watchers; router sweep in flight via #84376;
  two platform adapters), each documented for burn-down. New files or
  new violations fail CI immediately.
- tests/** keeps the relaxation (deliberate sleeps in fixtures).

Verification:
- ruff check . green on this branch; sabotage file with time.sleep +
  subprocess.run in async def fails the gate with 2 errors.
- New behavioral test test_webhook_offloop_delivery.py asserts loop
  liveness DURING delivery (ticker coroutine): 1 tick on the old
  blocking code (fails), 21 ticks off-loop (passes).
- 50 webhook/replace gateway tests + 26 save/export tests pass.

Co-authored-by: AhmetArif0 <147827411+AhmetArif0@users.noreply.github.com>
2026-08-28 07:50:51 -07:00
teknium1 272f4e4abe feat(plugins): generalize native platform handler registration to every gateway platform
ctx.register_platform_handler(platform, factory) — the generic surface for
plugins to wire native handlers into any platform adapter at connect()
time. Factories receive (native, adapter): the platform's client/app
object (PTB Application, discord.py Bot, slack_bolt AsyncApp, Teams App,
DingTalkStreamClient, aiohttp web.Application) or None for adapters with
no separate native object.

- BasePlatformAdapter._wire_plugin_handlers(native): shared, isolated
  invocation helper — a raising plugin cannot block a platform connect.
- All 27 connectable adapters call it: telegram/slack/teams/line/
  api_server/msgraph_webhook wire before their dispatch tables freeze;
  the rest hook at connect success.
- register_telegram_handler and get_telegram_handler_factories retained
  as thin back-compat aliases over the telegram bucket.
- Source-invariant test guarantees every adapter with connect() keeps
  calling the hook.
2026-08-27 07:51:37 -07:00
Teknium 6cb1085d3d fix(gateway): /p/<profile>/ on a non-multiplex gateway fails closed instead of serving the owner profile
A /p/<profile>/ URL prefix on a gateway with multiplex_profiles off was
silently ignored: the request was handled as the gateway-owning profile,
so /p/lokaj/v1/toolsets reported the OWNER's platform_toolsets (and every
other profile-owned config read — skills, capabilities, model options,
agent-run toolset resolution — resolved from the owner too). That is the
exact repro in #91583 defect 2: enabling computer_use with
'hermes -p lokaj tools enable computer_use --platform api_server' showed
enabled in lokaj's config while /p/lokaj/v1/toolsets stayed false, and
enabling it on the owner profile flipped it true.

Per-profile capability isolation is the intended design (ruling on
a different profile's config. Multiplexed gateways were already correct —
the profile-prefix middleware enters _profile_runtime_scope and every
canonical config loader honors the HERMES_HOME override contextvar
(verified empirically for load_config, get_config_path and
_load_gateway_config) — the leak was only the non-multiplex fallthrough.

Fix at the one seam both adapters share: _resolve_request_profile now
rejects (404) a prefix naming any profile other than the one the gateway
actually serves. A self-referential prefix (/p/default/ on the default
gateway, /p/lokaj/ on a gateway launched for lokaj) still falls through
so existing well-formed clients keep working. Same change in the webhook
adapter, which had the identical fallthrough. New shared helper
hermes_cli.profiles.profile_matches_home does the home comparison,
fail-closed.

Tests: tests/gateway/test_multiplex_toolsets_profile_isolation.py —
E2E-style with two real profile homes + config.yamls under a temp
HERMES_HOME, real aiohttp routing through the profile-prefix middleware:
per-profile /p/<x>/v1/toolsets isolation for both owner and secondary
(the #91583 repro asserts computer_use true under /p/lokaj only),
cross-profile key rejection, and the fail-closed non-multiplex prefix
for both adapters. Sabotage-verified: reverting the adapter change fails
the 3 fail-closed tests.

Fixes #91583 (defect 2). Repro and live validation by @kubaboski.
2026-08-23 03:57:47 -07:00
liuhao1024 f13f3401a1 fix(webhook): authenticate Linear deliveries via linear-signature HMAC (#87348) 2026-08-16 01:45:37 -07:00
Teknium e4aeb65599 feat(webhook): per-route toolset overrides for webhook agent runs
Webhook agent runs default to the constrained hermes-webhook toolset
(web/vision/clarify) because payloads can carry untrusted third-party
content. That default is right for public webhooks but wrong for trusted
local pushes (e.g. an OOM monitor daemon that needs the agent to run
ps/free/py-spy): the only workaround was widening platform_toolsets.webhook,
which elevates EVERY webhook route at once.

This adds a 'toolsets' key on individual webhook route configs (static
routes in config.yaml and dynamic subscriptions in
webhook_subscriptions.json) that replaces the platform-level resolution
for that route only:

- BasePlatformAdapter.toolsets_for_source(): per-source override hook,
  default None (no behavior change for any other platform).
- WebhookAdapter.toolsets_for_source(): maps the session chat_id
  (webhook:{route}:{delivery_id}) back to its route config and returns
  the route's toolsets list.
- GatewayRunner._resolve_enabled_toolsets_for_source(): shared resolver
  used by both agent-run call sites; validates the override through the
  SAME _get_platform_tools path as platform config, so unknown names and
  platform-restricted toolsets (e.g. discord_admin) are dropped rather
  than trusted.

Deliberately NOT exposed via 'hermes webhook subscribe': granting elevated
tools is a manual config edit only, so an agent-created subscription
cannot self-grant terminal at runtime.

Cache-safe: the toolset list is resolved before agent construction and is
constant for a route, so the per-session agent signature and frozen system
prompt are unaffected mid-conversation.
2026-08-13 01:51:19 -07:00
Trevin Chow c8f235a106 feat(gateway): allow selective multiplex profile serving 2026-08-10 22:48:24 -07:00
teknium1 5b751dc0ad chore: remove unused imports and dead locals (ruff F401/F841 sweep)
Cleans F401 unused imports and F841 dead local assignments across
root *.py, agent/, hermes_cli/, tools/, gateway/, cron/, tui_gateway/
(tests/, plugins/, skills/ excluded).

Intentionally KEPT (false positives / test-patch surfaces):
- agent/transports/__init__.py package re-exports
- cli.py browser_connect re-exports (DEFAULT_BROWSER_CDP_URL area,
  used by tests/cli/test_cli_browser_connect.py)
- hermes_cli/main.py _prompt_auth_credentials_choice /
  _model_flow_bedrock_api_key (accessed via main_mod attr in tests)
- gateway/run.py aliased replay_cleanup + whatsapp_identity re-exports
  and _PORT_BINDING_PLATFORM_VALUES (test-referenced)
- hermes_cli/web_server.py get_running_pid (tests monkeypatch it) and
  _OAUTH_TOKEN_URL availability probe
- hermes_cli/config.py get_process_hermes_home re-export (noqa'd F811
  chain) and yaml availability-probe import
- hermes_cli/nous_subscription.py managed_nous_tools_enabled
  (tests patch hermes_cli.nous_subscription.managed_nous_tools_enabled)
- try/except ImportError availability probes (env_loader, tts_tool,
  mcp_tool, web_server anthropic OAuth block)
- tools/web_tools.py noqa F401 re-exports
- hermes_cli/setup_whatsapp_cloud.py:263 'proceed' skipped: possible
  missing-guard bug, flagged for separate review
- unused function parameters (signature changes out of scope)

Side-effect RHS calls preserved where only the binding was dead
(e.g. web_server proc = _spawn_hermes_action -> bare call).
2026-07-29 11:53:39 -07:00
Atakan 8c5e846536 fix(gateway): bind HTTP auth to routed profiles 2026-07-28 14:22:18 -07:00
teknium1 136f8dab67 refactor(gateway): promote autonomous silence matcher to shared response_filters helper
Follow-up to the salvaged #71756: instead of webhook importing cron's
private _is_cron_silence_response, the loose autonomous-lane matcher now
lives in gateway/response_filters.py as is_autonomous_silence_response,
sharing LIVE_GATEWAY_SILENT_MARKERS with the interactive exact-marker
rule so the marker sets can never drift. Cron and webhook both delegate
to it. Interactive gateway behavior unchanged.
2026-07-26 16:55:42 -07:00
Gumclaw 55d3272286 fix(webhook): honor [SILENT] when the agent explains its own silence
A webhook route that answered `[SILENT]` still delivered, whenever the model
added a sentence saying why it was staying quiet:

    [SILENT]

    The new inbound was the same email quoted back a second time, on a ticket
    we already answered. Nothing new to reply to, so I closed it.

Webhook subscription prompts tell the agent to answer `[SILENT]` on a tick that
produced no story — a duplicate inbound, a stand-down because a sibling lane
already replied, a routine close. Nobody is waiting on the other end of a
webhook, so a "nothing happened" message has no reader.

Delivery went through the live gateway's `is_intentional_silence_response`,
which requires the response to be EXACTLY a marker. That rule is right for an
interactive chat: swallowing a real answer because it opens with a marker is
much worse than showing a stray marker. It is the wrong trade for an autonomous
lane, where a leaked non-story is a pointless notification on every tick and
models reliably append the explanation that flips the check back to "deliver".
Cron already resolved this the other way — `cron/scheduler.py` treats a marker
on its own first or last line as silence — so the two autonomous lanes
disagreed while the interactive path was fine.

Suppress in `WebhookAdapter.send`, before the deliver-type switch, so every
route (log, github_comment, cross-platform) behaves the same. Reuses cron's
`_is_cron_silence_response` rather than restating the rule, so the two lanes
cannot drift; prose that merely mentions a marker mid-sentence still delivers.
The interactive gateway path is untouched.

Tests: six cases in tests/gateway/test_webhook_adapter.py — bare marker,
marker + trailing prose (the reported shape), marker on the last line, a real
report, a report quoting a marker mid-sentence, and a `log` route. Verified
red-first: with the suppression removed the three silence cases fail
("Expected send to not have been awaited") while the three delivery cases still
pass, so the tests assert the fix rather than the framework.
2026-07-26 16:55:42 -07:00
Stoltemberg c89481db5e fix: add explicit UTF-8 encoding to all subprocess text=True calls (#53428)
On Windows with Chinese locale (GBK), subprocess.run(text=True) without
explicit encoding causes UnicodeDecodeError crashes. This fix adds
encoding='utf-8', errors='replace' to all subprocess.run() and
subprocess.Popen() calls that use text=True across 76 non-test Python files.

Fixes #53428 (master tracker for Windows GBK locale crash).

Note: credential_pool.py and electron changes excluded per reviewer request —
those will be submitted as separate focused PRs.
2026-07-24 11:45:57 -07:00
Teknium 9fc0074bac fix(gateway): unify reset boundaries vs recovery — promote accidental ends, honor mode=none, adapter-aware resume guidance
Unifies the two gateway subsystems that were fighting each other: the
'never lose a session' recovery machinery (#54878 stale-route self-heal,
find_latest_gateway_session_for_peer reopening agent_close/ws_orphan_reap
rows) and the session reset/expiry machinery (expiry watcher, /new,
/resume, resume_pending freshness gate).

The unified contract:
- INTENTIONAL boundaries (expiry finalization, auto-reset, /new,
  /resume switch) are recorded durably via promote_to_session_reset(),
  which upgrades accidental recoverable end_reasons (agent_close,
  ws_orphan_reap) to the explicit boundary while preserving other
  explicit reasons (compression, etc.). Recovery then correctly refuses
  to resurrect them.
- ACCIDENTAL ends (crash, cleanup bug, mistaken reaper) stay
  recoverable — genuine crash recovery is untouched.

On top of the cherry-picked contributor commits:
- promote_to_session_reset widened to ws_orphan_reap + parameterized
  reason so auto-reset paths stay auditable (idle/daily/suspended/
  resume_pending_expired) (#61220, #61993, #63539)
- get_or_create_session auto-reset, reset_session (/new), and
  switch_session (/resume) all write through the promote path — the
  first-reason-wins end_session no-op could previously leave a reset
  session resurrectable behind a stale agent_close row (#61993)
- resume_pending freshness gate now honors session_reset.mode=none:
  explicit opt-out of automatic resets also opts out of the zombie
  gate (#61052)
- resume recovery note extracted to build_resume_recovery_note() and
  made adapter-aware via a new interactive_resume capability flag:
  webhook/api_server auto-resume turns now CONTINUE the interrupted
  task instead of emitting an unanswerable 'session restored'
  acknowledgement that abandoned the work (#57056)
- tests updated to call the real note builder instead of mirroring it

E2E-validated against a real SessionDB + SessionStore in a temp
HERMES_HOME: expiry->agent_close->no-resurrection, /new promote,
crash recovery preserved, mode=none opt-out, routing-table flag sync.
2026-07-17 04:51:39 -07:00
Drexuxux 1b69c47e97 fix(webhook): reject a non-ASCII signature header instead of crashing the endpoint
_validate_signature backs the public webhook receiver. It compared each
attacker-supplied signature/token header (GitHub X-Hub-Signature-256,
GitLab X-Gitlab-Token, generic X-Webhook-Signature / -V2, and the Svix v1
header) against a computed hex/base64 digest with hmac.compare_digest on
two str values. compare_digest raises TypeError on a str containing
non-ASCII characters, and the header is raw client input on an
unauthenticated endpoint — so any internet client could POST a single
non-ASCII byte in the signature header and raise out of the handler,
returning a 500 instead of a clean 401. Fail-closed, but an on-demand
crash of the request path.

Route all five comparisons through a small _hmac_str_equal() helper that
encodes both sides to UTF-8 bytes before the constant-time compare
(compare_digest has no ASCII restriction on bytes). Semantics are
unchanged for valid signatures; a hostile non-ASCII header now fails
closed with a rejection instead of raising.

Adds regression tests: non-ASCII GitHub/GitLab/generic/V2 signature
headers return False (no raise), and a non-ASCII configured secret still
matches its exact token value.

Also maps drexux0@gmail.com in scripts/release.py AUTHOR_MAP.
2026-07-16 07:22:24 -07:00
Agung Subastian 8fc989b416 fix(gateway): multiplex secret_scope for authz, Slack, webhooks
Secondary profiles under gateway multiplex keep tokens/allowlists in
profile secret_scope, not process os.environ. Auth and Slack were still
reading os.getenv, so Slack on a secondary profile failed allowlist and
socket mode. Webhook deliver also only looked at default adapters.

- Prefer get_secret for allowlists / allow-all flags (authz_mixin)
- Slack app token + allowlist via secret_scope with getenv fallback
- Wrap secondary profile message handlers in _profile_runtime_scope
  before auth runs
- Resolve home-channel env from secret_scope / PlatformConfig
- Webhook deliver falls back to _profile_adapters for target platform
- Template key event_type for webhook prompts
2026-07-16 05:39:58 -07:00
Teknium 9420ad946a fix(webhook): scope reuse_address=False to macOS only (#65482)
On Linux, SO_REUSEADDR only allows rebinding past TIME_WAIT (a second
live listener would need SO_REUSEPORT, which we never set), so
disabling it bought no protection there while making a quick gateway
restart fail to rebind for up to ~60s. Keep the BSD silent-split guard
on darwin, default semantics elsewhere.

E2E verified: dual-stack v4+v6 bind on one port, immediate rebind
after disconnect, and live-listener conflict still rejected.
2026-07-16 01:28:18 -07:00
kshitijk4poor 92876effe2 fix(webhook): make dual-stack bind exclusive
Disable address reuse so an existing family-specific listener cannot silently split traffic with the webhook server. Normalize wildcard bind hosts for local CLI URLs and align setup documentation with the dual-stack default.
2026-07-16 12:36:51 +05:30
Ben d542894adf fix(webhook): default to dual-stack bind so 6PN (IPv6) can reach the adapter
The webhook adapter defaulted to host='0.0.0.0' — IPv4 only. On Fly.io
hosted agents the edge router (hermes-agent-router) reverse-proxies public
webhook traffic to <app>.internal:8644 over 6PN, Fly's private network,
which is IPv6-only (.internal resolves to an fdaa:… address). An IPv4-only
listener is unreachable there, so public webhook POSTs to
https://<agent>.agents.nousresearch.com/webhooks/<route> never landed on
the adapter — the router's dial was refused.

Fix: DEFAULT_HOST = None, which makes aiohttp/asyncio create_server bind
BOTH address families. '::' is NOT a valid substitute: on hosts where the
kernel sets bindv6only=1 (verified on Fly machines) it yields an IPv6-only
socket, breaking the IPv4 loopback /health check and the AF_INET
port-conflict probe in connect(). None binds per-family regardless of the
sysctl. An explicit empty-string/null host in config now also normalises to
None (dual-stack) rather than an invalid host=''. Users can still pin a
specific host via platforms.webhook.extra.host.

Validated live on a Fly staging agent: with this default and no config
override, the adapter binds both v4 and v6 (127.0.0.1:8644 and [::1]:8644
both answer), and a public signed webhook POST through the router returns
202 (valid sig) / 401 (bad sig) instead of the router's 502.

Tests: new TestDualStackBind asserts the None default, config resolution
(missing/empty→None, pinned preserved), and a real dual-stack bind opens
both AF_INET and AF_INET6 listeners. Red-proof: these fail on the old
'0.0.0.0' default.
2026-07-16 12:36:51 +05:30
teknium1 ae5e39005b fix(gateway): run webhook route scripts off the event loop + AUTHOR_MAP entry
- run_route_script shells out with subprocess.run (up to 30s timeout); wrap
  the call in asyncio.to_thread so a slow script can't stall every other
  webhook and gateway task on the loop.
- scripts/release.py: map grace@weeb.onl -> evelynburger for the salvaged
  contributor commit.
2026-07-08 08:10:55 -07:00
Grace 0cf2e39c41 feat(gateway): add webhook payload filters 2026-07-08 08:10:55 -07:00
MorAlekss d577408f3f fix(webhook): reject generic V2 signature missing timestamp instead of falling back to V1 2026-07-05 02:25:38 -07:00
teknium1 708b57e009 fix(webhook): rate-limit V1 deprecation warning + document V2 signature
- warn once per route instead of on every request (busy senders would
  spam the log)
- document X-Webhook-Signature-V2 / X-Webhook-Timestamp in the webhooks
  user guide

Follow-ups for salvaged #58461.
2026-07-05 01:36:53 -07:00
MorAlekss 70449a4939 fix(security): add timestamp-bound V2 signature for generic webhook replay protection 2026-07-05 01:36:53 -07:00
Gutslabs ec29590a0f fix(webhook): enforce body-size limit on chunked requests
The webhook adapter enforced max_body_bytes only via the Content-Length
header; a Transfer-Encoding: chunked request (content_length=None) or a
spoofed small Content-Length bypassed the cap entirely and read the full
body (bounded only by aiohttp's implicit 1 MiB default, above any
operator-configured smaller limit).

- web.Application(client_max_size=max_body_bytes): aiohttp enforces the
  cap on every read path, chunked included
- catch HTTPRequestEntityTooLarge -> 413 (was swallowed into generic 400)
- post-read length re-check as defense in depth
- chunked-upload regression test

Manual port of PR #3955 by @Gutslabs onto current main (handler had
been restructured since); authorship preserved.
2026-07-04 15:33:31 -07:00
Teknium 64ed99a6e6 fix(webhook): close per-delivery session at the true end of the run (#57423)
The merged webhook session-close fix (#57370, salvaging #57322) wrapped
handle_message in a try/finally — but BasePlatformAdapter.handle_message
is fire-and-forget: it spawns _process_message_background and returns
before the agent run starts. The finally-close therefore ran BEFORE
get_or_create_session created the session row, found no session_id, and
silently no-op'd — the ghost-session leak persisted on the real path.
(The shipped test masked this by stubbing handle_message with a fake
that created the row synchronously.)

Move the close to an on_processing_complete override — the lifecycle
hook the base class fires at the TRUE end of the run, on the success,
failure, and cancellation paths alike. Empirically verified through the
real fire-and-forget pipeline: before, ended_at stayed NULL; after,
ended_at is set with end_reason=webhook_complete and the row is
prunable.

Tests now stub only the runner-side _message_handler (the seam the live
gateway injects) so handle_message / _process_message_background /
on_processing_complete all run for real; adds an AsyncSessionDB-facade
coverage test for the coroutine-await branch.
2026-07-02 17:39:09 -07:00
kshitijk4poor 65cb70b8d0 refactor(gateway): add SessionStore.peek_session_id public accessor for webhook close
Replace the webhook delivery-close path's direct reach into private
SessionStore._entries (which also bypassed the store lock) with a public,
lock-held peek_session_id(session_key) accessor. Mirrors the existing
lookup_by_session_id inverse helper. Keeps a getattr fallback for older
stores / test doubles. Adds a unit test for the accessor.
2026-07-03 03:26:53 +05:30
Gumclaw 14882bab7e fix(gateway): close webhook sessions on delivery completion so prune can reap them
Webhook deliveries created a unique one-shot session (delivery_id baked into
the session key at gateway/platforms/webhook.py:668) but the adapter fired
handle_message via asyncio.create_task WITHOUT ever ending the session
(webhook.py:713, pre-fix). Nothing else closes it: the gateway caches/expires
the agent per session_key but never calls end_session for the webhook path,
and _end_session_on_close teardown doesn't run for these fire-and-forget tasks.

SessionDB.prune_sessions (hermes_state.py:4965) only deletes rows WHERE
ended_at IS NOT NULL. So every webhook session stayed with ended_at NULL ->
unprunable -> unbounded state.db growth. This was the primary driver of the
SQLite lock-contention gateway outage.

Fix: wrap the delivery in _run_delivery_and_close, which awaits
handle_message and then (in finally, so failures still reap) calls
_end_webhook_session -> SessionDB.end_session(session_id, 'webhook_complete').
This mirrors how cron closes its session with 'cron_complete'
(cron/scheduler.py:3065). end_session is first-reason-wins and no-ops on an
already-ended row, so it never clobbers a compression/agent_close reason.

Adds tests/gateway/test_webhook_session_close.py asserting the invariant
(a completed webhook session has ended_at set + is prunable), including the
error-path case, against a real SessionStore + SessionDB.
2026-07-03 03:26:53 +05:30
VolodymyrBg bd4007396d fix(webhook): remove unused payload from delivery state 2026-07-02 03:00:17 -07:00
aaronagent 27ddd8fd80 fix(gateway): sanitize agent error messages, validate webhook gh args
Two of the three fixes from PR #6660 (the cli.py reopen_session change is
moot — that raw _conn.execute reopen block no longer exists on main).

- gateway/run.py: stop sending raw type(e).__name__ and str(e)[:300] to
  end users on chat platforms. Exception text from LLM providers can leak
  API URLs, file paths, and partial credentials. Return a generic message;
  keep curated status hints for known HTTP codes; full detail stays in logs.
- gateway/platforms/webhook.py: validate pr_number (positive int) and repo
  (owner/name regex) before passing to the 'gh pr comment' subprocess.
  Payload-controlled values could otherwise inject gh flags (--help, a
  different --repo). List-form subprocess means this is arg injection, not
  shell injection, but validation is still correct.

Co-authored-by: aaronagent <1115117931@qq.com>
2026-06-28 18:53:26 -07:00
Teknium d3d621f7c3 revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853)
* Revert "fix(windows): capture is not a no-window boundary; route flashing spawns through chokepoint (#53829)"

This reverts commit 2ecca1e7d3.

* Revert "fix(windows): stop terminal-window popups from background spawns (#53810)"

This reverts commit 5db1430af9.

* Revert "fix(windows): stop subprocess console-window popups + add CI guard (#53791)"

This reverts commit ef17cd204d.
2026-06-27 15:59:00 -07:00
Teknium 2ecca1e7d3 fix(windows): capture is not a no-window boundary; route flashing spawns through chokepoint (#53829)
Follow-up to #53791 addressing review feedback: the footgun checker treated
capture_output=/stdout=/stderr=/check_output as proof a subprocess can't pop a
Windows console. That invariant is false — stream redirection controls where a
child's output goes, not whether a console is allocated. From a console-less
parent (Desktop/Electron, pythonw.exe, detached gateway/cron) a console-subsystem
child still flashes a window even when fully captured.

- check-windows-footguns.py: capture/redirect/check_output is no longer a blanket
  safe-pass. Added _WINDOWS_FLASHING_PROGRAMS (git/gh/npm/node/python/uv/ffmpeg/
  docker/powershell/…); calls to those are flagged even when captured. Non-flashing
  programs keep the capture exemption (no 271-site noise). _subprocess_compat.run/
  popen calls are inherently safe (wrapper injects CREATE_NO_WINDOW).
- Routed the 35 genuine flashing git/gh/npm/uv/ffmpeg/docker spawns through the
  _subprocess_compat.run/popen chokepoint (Brooklyn's wrapper from #53810) — the
  durable fix, not per-site annotations. cmd.exe /c start stays # ok (intentional).
- Updated tests + CONTRIBUTING.md rule #17 to the corrected invariant.
2026-06-27 14:49:41 -07:00
teknium1 43b8ba4181 fix(telegram): preserve Bot API update queue on watcher reconnect
After a prolonged outage the in-process network-error ladder escalates to
fatal and GatewayRunner._platform_reconnect_watcher rebuilds a fresh adapter
that reconnects through the bootstrap path. That path called
start_polling(drop_pending_updates=True), discarding every update Telegram
queued during the outage — all messages sent while the bot was down were
silently lost. The in-process ladder and 409-conflict handler already passed
drop_pending_updates=False; only bootstrap did not distinguish a cold first
boot from a reconnect.

Thread an is_reconnect signal from the watcher through
_connect_adapter_with_timeout into adapter.connect(). The base
BasePlatformAdapter.connect() gains a keyword-only is_reconnect=False so every
adapter inherits a tolerant signature (no per-platform breakage when the
runner forwards the kwarg). Telegram translates is_reconnect into
drop_pending_updates=not is_reconnect on both the polling and webhook bootstrap
calls. Cold boot still drops the stale queue; a watcher reconnect preserves it.

Fixes #46621.

Co-authored-by: annguyenNous <annguyen@nousresearch.com>
Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
Co-authored-by: Kewe63 <Kewe63@users.noreply.github.com>
2026-06-25 21:29:57 -07:00
Ben Barclay f35abb122a feat(gateway): multiplex phase 1 — HTTP-inbound /p/<profile>/ routing (webhook)
Serve webhook inbound for multiple profiles off the one shared listener via a
URL prefix, with no second port bound.

- SessionSource gains a 'profile' field (round-trips through to_dict/from_dict;
  omitted when unset so existing serialization is unchanged). It carries which
  profile an inbound message was routed to.
- WebhookAdapter registers /p/{profile}/webhooks/{route_name} alongside the
  existing /webhooks/{route_name}. _resolve_request_profile validates the
  prefix against profiles_to_serve(): None when absent or multiplexing is off
  (ignored, handled as default — no spurious 404), the profile name when valid,
  _PROFILE_REJECTED (→ 404) when the profile isn't served. The resolved profile
  is stamped onto the SessionSource.
- session-key namespacing and the per-turn home/credential scope now prefer
  source.profile: SessionStore._resolve_profile_for_key(source),
  _session_key_for_source fallback, and _resolve_profile_home_for_source all
  honor it (→ the agent turn resolves that profile's config/skills/credentials
  via the Phase 2 _profile_runtime_scope).

Constraint: routing inbound needs no per-profile platform credential, but the
agent still needs the routed profile's provider key — delivered by Phase 2's
secret scope. api_server (OpenAI-compatible surface) profile routing is a
focused follow-on; its source-construction path differs from webhook's.

Tests: SessionSource.profile round-trip + namespace drive; _resolve_request_
profile accept/reject/ignore matrix.
2026-06-19 07:34:15 -07:00
Teknium afc8615509 perf(webhook): prune request caches incrementally (#46065) 2026-06-14 02:40:54 -07:00
Teknium bd8e2ec1a6 feat(dashboard): complete admin panel — MCP catalog, enable/disable toggles, hook creation, system stats (#36736)
* feat(dashboard): MCP catalog + enable/disable, webhook toggle, hook create/delete, system stats

Backend for the comprehensive admin pass:
- MCP: GET /api/mcp/catalog (browse Nous-approved optional-mcps), POST
  /api/mcp/catalog/install, PUT /api/mcp/servers/{name}/enabled
- Webhooks: PUT /api/webhooks/{name}/enabled; gateway rejects disabled routes
  with 403 (hot-reloaded, no restart)
- Hooks: POST/DELETE /api/ops/hooks — create (with consent approval) + remove;
  list now reports accurate allowlist status + valid events
- System: GET /api/system/stats — OS/arch/python/cpu + psutil memory/disk/
  uptime/process, stdlib fallback

All gated by dashboard auth; secrets never returned.

* feat(dashboard): MCP catalog UI, enable/disable toggles, hook create, system stats

- McpPage: catalog section (browse Nous-approved MCPs, one-click install with
  env prompts) + per-server enable/disable toggle with gateway-restart note
- WebhooksPage: per-subscription enable/disable toggle (muted + badge when off)
- SystemPage: new Host stats section (OS/arch/python/cpu/mem/disk/uptime/load),
  shell-hook create modal + delete, 'Create backup' label
- api.ts: client methods + types for catalog, toggles, hook CRUD, system stats

* test(dashboard): cover catalog, toggles, hook CRUD, system stats, webhook toggle

Adds tests for the comprehensive pass: MCP enable/disable + catalog list +
catalog-install-unknown, hook create/delete with consent, system stats shape,
and webhook enable/disable. 26 tests total, all green.

* docs(dashboard): document the comprehensive admin pass + fresh screenshots

Updates the MCP/Webhooks/Pairing/System sections for catalog browse+install,
enable/disable toggles, hook creation, and host system stats; adds the new
endpoints to the API table; replaces the screenshots with live captures of
the rebuilt pages (real data, no dummies) including the hook-create modal.

* feat(dashboard): curator, portal status, and prompt-size/dump/migrate ops

Closes the last in-scope CLI gaps from the coverage audit:
- Curator: GET /api/curator (status), PUT /api/curator/paused, POST
  /api/curator/run (background)
- Portal: GET /api/portal (Nous auth + Tool Gateway routing, read-only)
- Diagnostics: POST /api/ops/prompt-size, /api/ops/dump, /api/ops/config-migrate
  (backgrounded, tailed via action status)

Host-bound commands (secrets/proxy/lsp/acp/computer-use/desktop/completion/
postinstall/uninstall/claw) remain CLI-only by design.

* feat(dashboard): curator + portal + diagnostics UI, tests

- SystemPage: Nous Portal status section (auth + Tool Gateway routing),
  Skill curator card (status + pause/resume + run now), and three new
  Operations buttons (prompt size, support dump, migrate config)
- api.ts: client methods + CuratorStatus/PortalStatus types
- tests: curator pause/resume, portal shape, system-stats shape, + auth-gate
  coverage for the new GET endpoints (31 tests total)

* docs(dashboard): document curator, portal, and diagnostics + refresh System screenshots

Updates the System section for the Nous Portal status, Skill curator
controls, and the new prompt-size/dump/migrate operations; adds them to the
API table; refreshes the System screenshots (now showing Portal + Curator)
and adds a dedicated curator/gateway/memory capture.

* feat(dashboard): session stats/export/prune + skills hub search endpoints

Completes the existing tabs' backend depth (audit vs CLI):
- Sessions: GET /api/sessions/stats (store stats), GET /api/sessions/{id}/export,
  POST /api/sessions/prune. /stats is registered before /{session_id} so the
  literal path isn't captured by the parameterized route.
- Skills: GET /api/skills/hub/search — parallel multi-source hub search (threaded),
  returns installable identifiers
- (rename via PATCH and cron-edit via PUT already existed; now surfaced in UI)

* feat(dashboard): complete existing tabs — sessions mgmt, skills hub browse, cron edit

Audited every existing tab against its CLI command and filled the gaps:
- Sessions: store stats bar, per-row rename + export (JSON download), and a
  prune-old-sessions control (mirrors hermes sessions rename/export/prune/stats)
- Skills: new 'Browse hub' view — search the skill hub across all sources,
  install by identifier with a live install log, and 'Update all' (mirrors
  hermes skills search/install/update)
- Cron: per-job Edit modal (pre-filled) calling updateCronJob (hermes cron edit)
- api.ts: renameSession/getSessionStats/exportSessionUrl/pruneSessions,
  updateCronJob, searchSkillsHub + types

Models tab was already comprehensive (provider+model picker, dynamic per-provider
lists, main + all 11 aux-task assignments, reset) — verified, no change needed.

* test(dashboard): cover session stats/rename/export/prune + skills hub search

Adds the route-shadowing guard for /api/sessions/stats (must not be captured
by /api/sessions/{session_id}), rename/export/prune, and the empty-query
short-circuit for hub search. 36 tests total, all green.

* docs(dashboard): document enhanced Sessions, Skills hub, and Cron edit

Sessions: stats bar, rename, export, prune (+ screenshot). Skills: new Browse
hub view for search/install/update (+ screenshot). Cron: edit action. API
table updated with the new endpoints.
2026-06-02 00:16:11 -04:00
teknium1 15aa6884a2 fix(webhook): use 403 not 500 for missing-secret rejection
Operator misconfiguration is a client/setup error, not an internal server
exception. 403 "forbidden" more accurately reflects "this route refuses
to authenticate" than 500 "internal server error" — the latter triggers
incident alerting on operator monitoring and conflates real bugs with
config drift.

Follow-up tweak to PR #29629 by @m0n3r0.
2026-05-24 04:47:45 -07:00
m0n3r0 dbf73e90fa fix: fail closed for webhook routes without secrets
Reject unsigned webhook requests when a route has no effective HMAC secret, even if the request handler is reached without the normal connect-time validation. Add regression coverage for the direct-handler path.
2026-05-24 04:47:45 -07:00
BaxBit bbf02c3224 fix(gateway): validate Svix webhook signatures (#30200) 2026-05-24 04:45:13 -07:00
Zyrixtrex 61ac118724 fix(webhook): enforce INSECURE_NO_AUTH safety rail on dynamic route reloads 2026-05-23 02:39:12 -07:00
memosr 9c90b3a597 fix(security): validate secret in _reload_dynamic_routes to prevent HMAC bypass 2026-05-22 03:45:21 -07:00
0xchainer 60531889d5 fix: remove unused import and hoist module-level constant
- Remove unused  from tools/tts_tool.py (dead code)
- Move _BUILTIN_DELIVER_PLATFORMS set from send() method to module
  scope in gateway/platforms/webhook.py to avoid reallocation on
  every call
2026-05-16 22:49:54 -07:00
teknium1 898b6d7d55 fix(webhook): widen INSECURE_NO_AUTH loopback check + tests + docs
Follow-up to the previous commit:
- Add _is_loopback_host() helper covering 127.0.0.1, localhost, ::1,
  ip6-localhost, ip6-loopback (case-insensitive). Empty/None host is
  treated as non-loopback since unset usually means public default bind.
- Fix mixed-indent comment in the safety rail (comment now aligned with
  the if-block) and collapse the nested-if into one condition.
- Add TestInsecureNoAuthSafetyRail covering rejection on 0.0.0.0, a LAN
  IP, and empty host; allowance on 127.0.0.1/localhost; plus unit-level
  parametrized coverage of _is_loopback_host for spellings we can't bind
  in the hermetic test env (::1, ip6-localhost, ip6-loopback).
- Pin test_connect_starts_server + test_webhook_deliver_only defaults
  to 127.0.0.1 so they keep passing under the new rail.
- Document the behavior in website/docs/user-guide/messaging/webhooks.md.
2026-05-07 07:38:43 -07:00
0z! fb4f953569 fix: block INSECURE_NO_AUTH on non-localhost webhook bindings 2026-05-07 07:38:43 -07:00
Teknium e464cde58f feat: final platform plugin parity — webhook delivery, platform hints, docs
Closes remaining functional gaps and adds documentation.

webhook.py: Cross-platform delivery now checks the plugin registry
  for unknown platform names instead of hardcoding 15 names in a tuple.
  Plugin platforms can receive webhook-routed deliveries.

prompt_builder: Platform hints (system prompt LLM guidance) now fall
  back to the plugin registry's platform_hint field. Plugin platforms
  can tell the LLM 'you're on IRC, no markdown.'

PlatformEntry: Added platform_hint field for LLM guidance injection.

IRC adapter: Added acquire_scoped_lock/release_scoped_lock in
  connect/disconnect to prevent two profiles from using the same IRC
  identity. Added platform_hint for IRC-specific LLM guidance.

Removed dead token-empty-warning extension for plugin platforms
  (plugin adapters handle their own env vars via check_fn).

website/docs/developer-guide/adding-platform-adapters.md:
  - Added 'Plugin Path (Recommended)' section with full code examples,
    PLUGIN.yaml template, config.yaml examples, and a table showing all
    18 integration points the plugin system handles automatically
  - Renamed built-in checklist to clarify it's for core contributors

gateway/platforms/ADDING_A_PLATFORM.md:
  - Added Plugin Path section pointing to the reference implementation
    and full docs guide
  - Clarified built-in path is for core contributors only
2026-04-29 21:56:51 -07:00
Tranquil-Flow 6a228d52f7 fix(webhook): validate HMAC signature before rate limiting (#12544) 2026-04-19 22:45:08 -07:00
Teknium 206a449b29 feat(webhook): direct delivery mode for zero-LLM push notifications (#12473)
External services can now push plain-text notifications to a user's chat
via the webhook adapter without invoking the agent. Set deliver_only=true
on a route and the rendered prompt template becomes the literal message
body — dispatched directly to the configured target (Telegram, Discord,
Slack, GitHub PR comment, etc.).

Reuses all existing webhook infrastructure: HMAC-SHA256 signature
validation, per-route rate limiting, idempotency cache, body-size limits,
template rendering with dot-notation, home-channel fallback. No new HTTP
server, no new auth scheme, no new port.

Use cases: Supabase/Firebase webhooks → user notifications, monitoring
alert forwarding, inter-agent pings, background job completion alerts.

Changes:
- gateway/platforms/webhook.py: new _direct_deliver() helper + early
  dispatch branch in _handle_webhook when deliver_only=true. Startup
  validation rejects deliver_only with deliver=log.
- hermes_cli/main.py + hermes_cli/webhook.go: --deliver-only flag on
  subscribe; list/show output marks direct-delivery routes.
- website/docs/user-guide/messaging/webhooks.md: new Direct Delivery
  Mode section with config example, CLI example, response codes.
- skills/devops/webhook-subscriptions/SKILL.md: document --deliver-only
  with use cases (bumped to v1.1.0).
- tests/gateway/test_webhook_deliver_only.py: 14 new tests covering
  agent bypass, template rendering, status codes, HMAC still enforced,
  idempotency still applies, rate limit still applies, startup
  validation, and direct-deliver dispatch.

Validation: 78 webhook tests pass (64 existing + 14 new). E2E verified
with real aiohttp server + real urllib POST — agent not invoked, target
adapter.send() called with rendered template, duplicate delivery_id
suppressed.

Closes the gap identified in PR #12117 (thanks to @H1an1 / Antenna team)
without adding a second HTTP ingress server.
2026-04-19 05:18:19 -07:00
Teknium 4654f75627 fix: QQBot missing integration points, timestamp parsing, test fix
- Add Platform.QQBOT to _UPDATE_ALLOWED_PLATFORMS (enables /update command)
- Add 'qqbot' to webhook cross-platform delivery routing
- Add 'qqbot' to hermes dump platform detection
- Fix test_name_property casing: 'QQBot' not 'QQBOT'
- Add _parse_qq_timestamp() for ISO 8601 + integer ms compatibility
  (QQ API changed timestamp format — from PR #2411 finding)
- Wire timestamp parsing into all 4 message handlers
2026-04-14 00:11:49 -07:00
Teknium 8d023e43ed refactor: remove dead code — 1,784 lines across 77 files (#9180)
Deep scan with vulture, pyflakes, and manual cross-referencing identified:
- 41 dead functions/methods (zero callers in production)
- 7 production-dead functions (only test callers, tests deleted)
- 5 dead constants/variables
- ~35 unused imports across agent/, hermes_cli/, tools/, gateway/

Categories of dead code removed:
- Refactoring leftovers: _set_default_model, _setup_copilot_reasoning_selection,
  rebuild_lookups, clear_session_context, get_logs_dir, clear_session
- Unused API surface: search_models_dev, get_pricing, skills_categories,
  get_read_files_summary, clear_read_tracker, menu_labels, get_spinner_list
- Dead compatibility wrappers: schedule_cronjob, list_cronjobs, remove_cronjob
- Stale debug helpers: get_debug_session_info copies in 4 tool files
  (centralized version in debug_helpers.py already exists)
- Dead gateway methods: send_emote, send_notice (matrix), send_reaction
  (bluebubbles), _normalize_inbound_text (feishu), fetch_room_history
  (matrix), _start_typing_indicator (signal), parse_feishu_post_content
- Dead constants: NOUS_API_BASE_URL, SKILLS_TOOL_DESCRIPTION,
  FILE_TOOLS, VALID_ASPECT_RATIOS, MEMORY_DIR
- Unused UI code: _interactive_provider_selection,
  _interactive_model_selection (superseded by prompt_toolkit picker)

Test suite verified: 609 tests covering affected files all pass.
Tests for removed functions deleted. Tests using removed utilities
(clear_read_tracker, MEMORY_DIR) updated to use internal APIs directly.
2026-04-13 16:32:04 -07:00
chqchshj 5f0caf54d6 feat(gateway): add WeCom callback-mode adapter for self-built apps
Add a second WeCom integration mode for regular enterprise self-built
applications.  Unlike the existing bot/websocket adapter (wecom.py),
this handles WeCom's standard callback flow: WeCom POSTs encrypted XML
to an HTTP endpoint, the adapter decrypts, queues for the agent, and
immediately acknowledges.  The agent's reply is delivered proactively
via the message/send API.

Key design choice: always acknowledge immediately and use proactive
send — agent sessions take 3-30 minutes, so the 5-second inline reply
window is never useful.  The original PR's Future/pending-reply
machinery was removed in favour of this simpler architecture.

Features:
- AES-CBC encrypt/decrypt (BizMsgCrypt-compatible)
- Multi-app routing scoped by corp_id:user_id
- Legacy bare user_id fallback for backward compat
- Access-token management with auto-refresh
- WECOM_CALLBACK_* env var overrides
- Port-in-use pre-check before binding
- Health endpoint at /health

Salvaged from PR #7774 by @chqchshj.  Simplified by removing the
inline reply Future system and fixing: secrets.choice for nonce
generation, immediate plain-text acknowledgment (not encrypted XML
containing 'success'), and initial token refresh error handling.
2026-04-11 15:22:49 -07:00