feat(webhook): per-route toolset overrides for webhook agent runs

Webhook agent runs default to the constrained hermes-webhook toolset
(web/vision/clarify) because payloads can carry untrusted third-party
content. That default is right for public webhooks but wrong for trusted
local pushes (e.g. an OOM monitor daemon that needs the agent to run
ps/free/py-spy): the only workaround was widening platform_toolsets.webhook,
which elevates EVERY webhook route at once.

This adds a 'toolsets' key on individual webhook route configs (static
routes in config.yaml and dynamic subscriptions in
webhook_subscriptions.json) that replaces the platform-level resolution
for that route only:

- BasePlatformAdapter.toolsets_for_source(): per-source override hook,
  default None (no behavior change for any other platform).
- WebhookAdapter.toolsets_for_source(): maps the session chat_id
  (webhook:{route}:{delivery_id}) back to its route config and returns
  the route's toolsets list.
- GatewayRunner._resolve_enabled_toolsets_for_source(): shared resolver
  used by both agent-run call sites; validates the override through the
  SAME _get_platform_tools path as platform config, so unknown names and
  platform-restricted toolsets (e.g. discord_admin) are dropped rather
  than trusted.

Deliberately NOT exposed via 'hermes webhook subscribe': granting elevated
tools is a manual config edit only, so an agent-created subscription
cannot self-grant terminal at runtime.

Cache-safe: the toolset list is resolved before agent construction and is
constant for a route, so the per-session agent signature and frozen system
prompt are unaffected mid-conversation.
This commit is contained in:
Teknium
2026-08-13 01:37:40 -07:00
parent f4749a77a5
commit e4aeb65599
6 changed files with 272 additions and 6 deletions
+20
View File
@@ -7118,6 +7118,26 @@ class BasePlatformAdapter(ABC):
- type: "dm", "group", "channel"
"""
pass
def toolsets_for_source(self, source: "SessionSource") -> Optional[List[str]]:
"""Per-source toolset override for agent runs triggered by this adapter.
Return a list of configurable toolset keys (e.g. ``["terminal",
"file", "web"]``) to REPLACE the platform-level toolset resolution
for this specific source, or ``None`` to use the normal
``platform_toolsets.<platform>`` resolution (the default).
The gateway validates the returned list through the same
``_get_platform_tools`` path as platform-level config, so unknown or
platform-restricted toolset names are dropped rather than trusted.
Currently used by the webhook adapter so individual routes can pin
their own toolsets (a trusted local monitoring route can get
``terminal`` without widening every webhook route's default-safe
toolset). See ``platforms.webhook.extra.routes.<name>.toolsets`` and
the ``toolsets`` key in ``webhook_subscriptions.json``.
"""
return None
def format_message(self, content: str) -> str:
"""
+31 -1
View File
@@ -42,7 +42,7 @@ import subprocess
import sys
import time
from collections import deque
from typing import Any, Deque, Dict, Optional
from typing import Any, Deque, Dict, List, Optional
try:
from aiohttp import web
@@ -463,6 +463,36 @@ class WebhookAdapter(BasePlatformAdapter):
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
return {"name": chat_id, "type": "webhook"}
def toolsets_for_source(self, source) -> Optional[List[str]]:
"""Per-route toolset override.
Webhook session chat_ids are ``webhook:{route}:{delivery_id}``.
When the matching route config carries a ``toolsets`` list, that list
replaces the platform-level ``platform_toolsets.webhook`` resolution
for this run only. Routes without the key keep the platform default
(the intentionally constrained webhook-safe toolset), so a single
trusted route (e.g. a localhost monitoring push) can be granted
``terminal`` without widening every other webhook route.
Set via ``platforms.webhook.extra.routes.<name>.toolsets`` in
config.yaml or a ``toolsets`` key on a subscription in
``webhook_subscriptions.json`` (manual edit — deliberately NOT
exposed through `hermes webhook subscribe`, so an agent-created
subscription cannot self-grant elevated tools).
"""
chat_id = str(getattr(source, "chat_id", "") or "")
parts = chat_id.split(":", 2)
if len(parts) < 2 or parts[0] != "webhook":
return None
route_config = self._routes.get(parts[1])
if not isinstance(route_config, dict):
return None
toolsets = route_config.get("toolsets")
if not isinstance(toolsets, list) or not toolsets:
return None
cleaned = [str(t).strip() for t in toolsets if str(t).strip()]
return cleaned or None
# ------------------------------------------------------------------
# HTTP handlers
# ------------------------------------------------------------------
+41 -4
View File
@@ -20699,6 +20699,41 @@ class GatewayRunner(GatewayAuthorizationMixin, GatewayKanbanWatchersMixin, Gatew
prompt, source, task_id, event_message_id, media_urls, media_types,
)
def _resolve_enabled_toolsets_for_source(
self,
user_config: dict,
source: "SessionSource",
platform_key: str,
) -> list:
"""Resolve enabled toolsets for an agent run, honoring per-source overrides.
Asks the receiving adapter for a ``toolsets_for_source()`` override
(e.g. per-route webhook toolsets). When present, the override list is
validated through the SAME ``_get_platform_tools`` path as normal
platform config — by substituting it as the platform's toolset list —
so unknown names and platform-restricted toolsets are dropped rather
than trusted. When absent, falls back to standard
``platform_toolsets.<platform>`` resolution.
"""
from hermes_cli.tools_config import _get_platform_tools
override = None
try:
adapter = self._adapter_for_source(source)
if adapter is not None:
override = adapter.toolsets_for_source(source)
except Exception:
override = None
if override and isinstance(override, list):
cfg = dict(user_config)
pts = dict(cfg.get("platform_toolsets") or {})
pts[platform_key] = [str(t) for t in override]
cfg["platform_toolsets"] = pts
return sorted(_get_platform_tools(cfg, platform_key))
return sorted(_get_platform_tools(user_config, platform_key))
async def _run_background_task_inner(
self,
prompt: str,
@@ -20737,8 +20772,9 @@ class GatewayRunner(GatewayAuthorizationMixin, GatewayKanbanWatchersMixin, Gatew
platform_key = _platform_config_key(source.platform)
from hermes_cli.tools_config import _get_platform_tools
enabled_toolsets = sorted(_get_platform_tools(user_config, platform_key))
enabled_toolsets = self._resolve_enabled_toolsets_for_source(
user_config, source, platform_key
)
agent_cfg = user_config.get("agent") or {}
disabled_toolsets = agent_cfg.get("disabled_toolsets") or None
@@ -25837,8 +25873,9 @@ class GatewayRunner(GatewayAuthorizationMixin, GatewayKanbanWatchersMixin, Gatew
user_config = _load_gateway_config()
platform_key = _platform_config_key(source.platform)
from hermes_cli.tools_config import _get_platform_tools
enabled_toolsets = sorted(_get_platform_tools(user_config, platform_key))
enabled_toolsets = self._resolve_enabled_toolsets_for_source(
user_config, source, platform_key
)
agent_cfg_local = user_config.get("agent") or {}
disabled_toolsets = agent_cfg_local.get("disabled_toolsets") or None
@@ -0,0 +1,137 @@
"""Per-route webhook toolset overrides (adapter.toolsets_for_source).
A webhook route config may carry a ``toolsets`` list that replaces the
platform-level ``platform_toolsets.webhook`` resolution for runs triggered by
that route only. The gateway validates the override through the same
``_get_platform_tools`` path as platform config, so restricted/unknown names
behave identically to a manually configured platform toolset list.
"""
from gateway.platforms.base import BasePlatformAdapter
from gateway.platforms.webhook import WebhookAdapter
from gateway.run import GatewayRunner
from hermes_cli.tools_config import _get_platform_tools
class _Src:
def __init__(self, chat_id):
self.chat_id = chat_id
def _make_adapter(routes):
wa = object.__new__(WebhookAdapter)
wa._routes = routes
return wa
def _make_runner(adapter):
gr = object.__new__(GatewayRunner)
gr._adapter_for_source = lambda source: adapter
return gr
BASE_CONFIG = {"platform_toolsets": {"webhook": ["web", "vision", "clarify"]}}
class TestWebhookAdapterToolsetsForSource:
def test_route_with_toolsets_returns_list(self):
wa = _make_adapter({"mon": {"secret": "x", "toolsets": ["terminal", "file"]}})
assert wa.toolsets_for_source(_Src("webhook:mon:d1")) == ["terminal", "file"]
def test_route_without_toolsets_returns_none(self):
wa = _make_adapter({"plain": {"secret": "x"}})
assert wa.toolsets_for_source(_Src("webhook:plain:d1")) is None
def test_unknown_route_returns_none(self):
wa = _make_adapter({})
assert wa.toolsets_for_source(_Src("webhook:ghost:d1")) is None
def test_non_webhook_chat_id_returns_none(self):
wa = _make_adapter({"mon": {"secret": "x", "toolsets": ["terminal"]}})
assert wa.toolsets_for_source(_Src("telegram:123")) is None
def test_empty_or_non_list_toolsets_returns_none(self):
wa = _make_adapter(
{
"empty": {"secret": "x", "toolsets": []},
"str": {"secret": "x", "toolsets": "terminal"},
"blank": {"secret": "x", "toolsets": [" ", ""]},
}
)
assert wa.toolsets_for_source(_Src("webhook:empty:d")) is None
assert wa.toolsets_for_source(_Src("webhook:str:d")) is None
assert wa.toolsets_for_source(_Src("webhook:blank:d")) is None
def test_base_adapter_default_is_none(self):
# Non-webhook adapters inherit a None default: no override anywhere.
wa = _make_adapter({})
assert (
BasePlatformAdapter.toolsets_for_source(wa, _Src("webhook:mon:d")) is None
)
class TestGatewayResolveEnabledToolsetsForSource:
def test_override_replaces_platform_resolution(self):
wa = _make_adapter(
{"mon": {"secret": "x", "toolsets": ["terminal", "file", "web"]}}
)
gr = _make_runner(wa)
res = GatewayRunner._resolve_enabled_toolsets_for_source(
gr, BASE_CONFIG, _Src("webhook:mon:d"), "webhook"
)
assert "terminal" in res and "file" in res and "web" in res
assert "vision" not in res # platform list fully replaced, not merged
def test_override_validated_like_platform_config(self):
# Contract: resolving with an override is byte-identical to resolving
# the same list configured as platform_toolsets.webhook.
override = ["terminal", "file", "web", "discord_admin"]
wa = _make_adapter({"mon": {"secret": "x", "toolsets": override}})
gr = _make_runner(wa)
res = GatewayRunner._resolve_enabled_toolsets_for_source(
gr, BASE_CONFIG, _Src("webhook:mon:d"), "webhook"
)
expected = sorted(
_get_platform_tools(
{"platform_toolsets": {"webhook": list(override)}}, "webhook"
)
)
assert res == expected
# discord_admin is platform-restricted to discord — must be dropped.
assert "discord_admin" not in res
def test_no_override_uses_platform_resolution(self):
wa = _make_adapter({"plain": {"secret": "x"}})
gr = _make_runner(wa)
res = GatewayRunner._resolve_enabled_toolsets_for_source(
gr, BASE_CONFIG, _Src("webhook:plain:d"), "webhook"
)
assert res == sorted(_get_platform_tools(BASE_CONFIG, "webhook"))
assert "terminal" not in res
def test_adapter_exception_falls_back_to_platform_resolution(self):
wa = _make_adapter({})
wa.toolsets_for_source = lambda source: (_ for _ in ()).throw(
RuntimeError("boom")
)
gr = _make_runner(wa)
res = GatewayRunner._resolve_enabled_toolsets_for_source(
gr, BASE_CONFIG, _Src("webhook:mon:d"), "webhook"
)
assert res == sorted(_get_platform_tools(BASE_CONFIG, "webhook"))
def test_missing_adapter_falls_back_to_platform_resolution(self):
gr = _make_runner(None)
res = GatewayRunner._resolve_enabled_toolsets_for_source(
gr, BASE_CONFIG, _Src("webhook:mon:d"), "webhook"
)
assert res == sorted(_get_platform_tools(BASE_CONFIG, "webhook"))
def test_original_config_not_mutated(self):
cfg = {"platform_toolsets": {"webhook": ["web"]}}
wa = _make_adapter({"mon": {"secret": "x", "toolsets": ["terminal"]}})
gr = _make_runner(wa)
GatewayRunner._resolve_enabled_toolsets_for_source(
gr, cfg, _Src("webhook:mon:d"), "webhook"
)
assert cfg["platform_toolsets"]["webhook"] == ["web"]
@@ -88,7 +88,7 @@ platforms:
| `deliver_extra.pr_number` | Resolves to the PR number from the payload. |
:::note The payload does not contain code
The GitHub webhook payload includes PR metadata (title, description, branch names, URLs) but **not the diff**. The prompt above instructs the agent to run `gh pr diff` to fetch the actual changes. The `terminal` tool is included in the default `hermes-webhook` toolset, so no extra configuration is needed.
The GitHub webhook payload includes PR metadata (title, description, branch names, URLs) but **not the diff**. The prompt above instructs the agent to run `gh pr diff` to fetch the actual changes. The default `hermes-webhook` toolset is deliberately constrained (web search/extract, vision, clarify — **no terminal**) because webhook payloads can carry untrusted content. To let this route run `gh`, add a per-route toolset grant: `toolsets: ["terminal", "web"]` on the route config — see [Per-route toolsets](/docs/user-guide/messaging/webhooks#per-route-toolsets).
:::
---
@@ -85,6 +85,7 @@ Routes define how different webhook sources are handled. Each route is a named e
| `filters` | No | Declarative payload filters evaluated after auth/body/event filtering and before agent or direct delivery work. Non-matches return `{"status":"ignored","reason":"filter"}` with HTTP 200. |
| `script` | No | Filter/transform script under `~/.hermes/scripts/`. The webhook payload is passed as JSON on stdin. JSON object stdout replaces the payload before templating; text stdout is exposed as `script_output`; empty stdout, `[SILENT]`, or a nonzero exit code ignores the webhook. |
| `skills` | No | List of skill names to load for the agent run. |
| `toolsets` | No | List of toolset keys (e.g. `["terminal", "file", "web"]`) that **replaces** the platform-level webhook toolset for runs triggered by this route only. Manual config edit only — not settable via `hermes webhook subscribe`, so agent-created subscriptions cannot self-grant elevated tools. Names are validated the same way as `platform_toolsets` entries (unknown or platform-restricted names are dropped). See [Per-route toolsets](#per-route-toolsets). |
| `deliver` | No | Where to send the response: `github_comment`, `telegram`, `discord`, `slack`, `signal`, `sms`, `whatsapp`, `matrix`, `mattermost`, `homeassistant`, `email`, `dingtalk`, `feishu`, `wecom`, `weixin`, `bluebubbles`, `qqbot`, or `log` (default). |
| `deliver_extra` | No | Additional delivery config — keys depend on `deliver` type (e.g. `repo`, `pr_number`, `chat_id`). Values support the same `{dot.notation}` templates as `prompt`. |
| `deliver_only` | No | If `true`, skip the agent entirely — the rendered `prompt` template becomes the literal message that gets delivered. Zero LLM cost, sub-second delivery. See [Direct Delivery Mode](#direct-delivery-mode) for use cases. Requires `deliver` to be a real target (not `log`). |
@@ -446,6 +447,47 @@ The agent can create subscriptions via the terminal tool when guided by the `web
---
## Per-route toolsets {#per-route-toolsets}
Webhook agent runs default to a deliberately constrained toolset (`web_search`, `web_extract`, `vision_analyze`, `clarify`) because webhook payloads can carry untrusted third-party content — a public PR title or issue comment should never be able to prompt-inject its way into your terminal.
For **trusted** routes — a localhost monitoring daemon pushing system alerts, an internal CI system — you can grant a wider toolset to that route only, without widening every other webhook route:
```yaml
platforms:
webhook:
enabled: true
extra:
routes:
oom-emergency:
secret: "monitor-secret"
prompt: "Memory emergency: {detail}. Diagnose with ps/free/py-spy and report."
toolsets: ["terminal", "file", "code_execution", "web"]
deliver: "telegram"
```
For dynamic subscriptions, add the `toolsets` key by editing `~/.hermes/webhook_subscriptions.json` directly:
```json
{
"oom-emergency": {
"secret": "...",
"prompt": "...",
"toolsets": ["terminal", "file", "web"],
"deliver": "telegram"
}
}
```
Behavior and safety properties:
- The route list **replaces** the platform-level webhook toolset resolution for that route's runs (it is not merged).
- Names are validated through the same path as `platform_toolsets` config — unknown names and platform-restricted toolsets are dropped.
- `hermes webhook subscribe` deliberately does **not** accept a toolsets flag. Granting elevated tools is a manual config-file edit, so an agent creating its own subscription at runtime cannot self-grant `terminal`.
- Only grant elevated toolsets to routes whose senders you fully control, with a real HMAC secret. Anyone who can POST a validly-signed payload to that route is effectively running an agent with those tools.
---
## Security {#security}
The webhook adapter includes multiple layers of security: