Commit Graph

13 Commits

Author SHA1 Message Date
teknium1 9b1990583d refactor(gateway): adapters share helpers.cancel_task / MessageDeduplicator / bounded_put
Six adapters defined their own `_cancel_task` and nine more inlined the same
cancel + suppress(CancelledError) + await block; five kept a hand-rolled TTL-dict
`_is_duplicate` next to the existing `helpers.MessageDeduplicator`; three carried a
`_bounded_put`. Each copy fixed the same bugs on its own schedule (self-cancel deadlock,
done-task re-await, eviction under load).

- `helpers.cancel_task`: None/done no-op, never awaits the current task, swallows the
  task's own exception at teardown. Replaces qqbot/signal/yuanbao/buzz/photon/simplex
  definitions and the inline copies in weixin, discord, email, irc, line, mattermost,
  whatsapp and telegram.
- `helpers.MessageDeduplicator` replaces `_is_duplicate` in qqbot, ntfy, photon,
  wecom_callback and LINE's `_MessageDeduplicator`; every site keeps its own
  max_size/TTL (qqbot and ntfy 1000/300s, photon 4000/48h, wecom_callback 2000/300s,
  LINE 1000/no TTL).
- `helpers.bounded_put` replaces photon/wecom/whatsapp_cloud copies; a re-put now
  refreshes the key to the newest slot at every site.
- telegram gmail-triage scripts resolve under `get_hermes_home()` instead of a hard
  `~/.hermes`, so profiles with HERMES_HOME set find them.

Not changed: `get_chat_info` stays `@abstractmethod` because
tests/gateway/test_relay_capability_surface.py locks the abstract set to exactly
{connect, disconnect, send, get_chat_info} as a cross-repo contract, so the ~17 no-op
overrides remain.

Behavior change: whatsapp_cloud `_bounded_put` was a pure FIFO (no refresh on re-put);
it now refreshes like the other two sites. Task cancellation at the migrated sites
swallows a task's terminal exception where a few copies previously only suppressed
CancelledError (all are shutdown/disconnect paths).
2026-09-13 05:32:38 -07:00
Teknium d76856cc69 fix(migrate): shared-ingress adapters declare serves_profile_prefix so migration reports them as notices
#108952 taught sms/line/teams/bluebubbles/whatsapp_cloud/msgraph_webhook/feishu/wecom-callback to
serve a secondary at /p/<profile>/ on the default listener; #108928's preflight derives its
port-binder blocker from the adapter class's serves_profile_prefix flag, which those adapters never
set. Merged together, migrate would have blocked every profile the ingress work just unblocked.
Declare the flag on each shared-ingress adapter and run plugin discovery before consulting the
registry (plugin adapters are absent from a bare CLI process otherwise).
2026-09-12 02:02:20 -07:00
Teknium 6fc76ce751 feat(platforms): bind through shared_ingress.bind_listener in every inbound-port adapter
sms, line, teams, bluebubbles, msgraph_webhook, whatsapp_cloud, wecom_callback and
feishu (webhook mode) build their aiohttp app exactly as before and hand it to
bind_listener(): standalone and default-profile behaviour is unchanged (same host,
port, reuse_address, access_log), while a multiplex secondary publishes the app for
/p/<profile>/ forwarding instead of binding. BlueBubbles registers the /p/<profile>/
URL with its server; LINE builds media URLs off the shared listener's prefix when no
LINE_PUBLIC_URL is set; WeCom skips its own port-in-use probe in shared mode.
2026-09-12 01:53:15 -07:00
kshitijk4poor ab2f4602de refactor: MessageEvent to gateway/platforms/event.py; ElicitationHandler takes a call_context thunk
Breaks the two import cycles that forced Protocol stand-ins in the F821 sweep, so the two
sites now name the real types.

gateway/platforms/event.py (new leaf): MessageType, ProcessingOutcome, MessageEvent moved
out of base.py verbatim. Their only dependency is gateway.session.SessionSource; base.py
imported helpers.py at module level, so helpers could not name MessageEvent. Now
TextBatchAggregator is typed by the real MessageEvent. 249 importers repointed
(`from gateway.platforms.base import` -> `.event`, preserving each import's layout);
gateway.platforms.__init__ re-exports from .event. The three revert-scheduled PLUGIN-COMPAT
pointers that named these symbols (gateway.slash_commands → MessageType, dingtalk → MessageType,
photon → ProcessingOutcome) and their COMPAT_MANIFEST rows now target gateway.platforms.event.
Docs updated: ADDING_A_PLATFORM.md, adding-platform-adapters.md (en + zh-Hans).

tools/mcp_tool_sampling.py: ElicitationHandler no longer holds a back-reference to its
MCPServerTask (mcp_tool imports sampling, so the task type cannot be named there). It only
ever read owner._pending_call_context, so it takes `call_context: Callable[[], Context | None]`
and MCPServerTask passes `lambda: self._pending_call_context`. The consent call is one
`functools.partial`, run directly or inside the captured Context.

ty on the 11 touched production files vs origin/main: 0 new diagnostics, 14 resolved.
(The one `source: SessionSource = None` diagnostic moves with the class; typing it Optional
exposes ~60 unguarded call sites — separate follow-up.)

Tests: tests/gateway + tests/plugins + tests/tools + touched files, 18,235 passed; the 31
failures reproduce identically on origin/main (macOS /private/tmp, systemd socket,
long-path fixtures, live-service tests).
2026-09-07 22:47:33 +05:30
Teknium e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium ce08af6139 refactor(plugins/whatsapp,wecom): dedupe send/media/stream paths, drop dead helpers, compact docs (4357->3048 LOC, bridge/WeCom payload + crypto parity verified) 2026-09-03 00:40:56 -07:00
Teknium b8bea7199c refactor(adapters/cn_group): 12368->8549; wecom split into streaming/media/send_queue mixins, dingtalk inbound parsers -> inbound.py, google_chat cards.py + setup_files.py, teams summary_writer.py; dedupe token/URL/credential helpers 2026-09-02 14:06:34 -07:00
kshitij f5784617e8 refactor: fold simplify-code review findings
- matrix/dingtalk: extract deps-only installers (ensure_matrix_deps,
  ensure_dingtalk_deps) and register THOSE as ensure_deps_fn — the prior
  check_*_requirements combined credential env checks with the install,
  so a platform configured via PlatformConfig.extra (which is_connected
  accepts) would pass enablement, reach create_adapter(), and have the
  'installer' veto on env-var grounds before installing anything —
  re-creating the #79812 deadlock for extra-configured setups.  The
  combined deps+credentials functions remain for setup/status callers.
- matrix/feishu passive probes: use the existing lazy_deps.is_available()
  instead of hand-rolling 'not feature_missing(...)' (reuse finding).
- teams: module docstring no longer recommends bare system pip (the
  PEP 668 trap purged everywhere else); docs troubleshooting row updated
  to match the new hint text.
- wecom_callback: drop dead 'global ET, DEFUSEDXML_AVAILABLE'
  (ensure_and_bind mutates the module dict directly; nothing assigns).
- tests: parametrized wiring contract for all 8 lazy-installable
  platforms — ensure_deps_fn present and distinct from check_fn
  (behavior contract, not identity snapshot, so renames don't churn it).
2026-08-07 13:28:43 +05:30
kshitij 0d32607c62 fix(gateway): split check_fn (passive probe) from ensure_deps_fn (active installer)
PlatformEntry.check_fn served three contradictory roles: adapter-creation
gate, config auto-enablement gate, and status display. Plugins had to pick
one function for all three:

- Active installer as check_fn (discord/slack/telegram/matrix/dingtalk/
  feishu): every status display could pip-install SDKs as a side effect
  (the desktop 94% boot-loop class).
- Passive probe as check_fn (teams, wecom_callback): create_adapter()
  returned None before connect() could lazy-install, so the SDK never
  installed (#79812 deadlock; wecom_callback's platform.wecom_callback
  LAZY_DEPS entry was dead code).

The split makes both call sites correct by construction:

- check_fn is now contractually PASSIVE (probe only, never installs).
- New optional PlatformEntry.ensure_deps_fn is the ACTIVE installer;
  create_adapter() runs it exactly when check_fn is False — the platform
  is enabled+configured and the gateway is about to connect it.
- Config enablement keeps a configured platform whose deps are missing
  but installable; the install itself is deferred to create_adapter().
- Status surfaces (_platform_status, hermes status) read only the
  passive probe and can never trigger pip.

Migrated all lazy-installable platform plugins to the split; platforms
with no optional deps (irc/ntfy/buzz/simplex/line/a2a/...) are unchanged
— no ensure_deps_fn means a False check_fn stays a hard block.
wecom_callback gains a working installer for the first time.

Builds on @xxxigm's #79812 (both commits cherry-picked with authorship
preserved), reworking the check_fn swap into the two-field split so the
Teams fix doesn't reintroduce install-on-status.
2026-08-07 13:28:43 +05:30
Teknium 2c771be406 fix(gateway): dual-stack webhook bind for wecom/msgraph/whatsapp_cloud/teams/telegram siblings
Same class of bug as the LINE adapter (NS-603): defaulting the webhook
bind to "0.0.0.0" (or hardcoding it) binds IPv4 ONLY, so the listener
is unreachable over IPv6-only private networks such as Fly.io 6PN.

- wecom callback_adapter: DEFAULT_HOST None; config.py env seed no
  longer forces 0.0.0.0 when WECOM_CALLBACK_HOST is unset.
- msgraph_webhook: DEFAULT_HOST None; the allowed_source_cidrs
  requirement still fires for the all-interfaces default (host=None is
  treated as network-accessible).
- whatsapp_cloud: DEFAULT_WEBHOOK_HOST None.
- teams: hardcoded 0.0.0.0 TCPSite bind → _DEFAULT_HOST=None with new
  TEAMS_HOST / extra.host override (mirrors LINE_HOST pattern).
- telegram: hardcoded listen="0.0.0.0" → default "" (tornado
  bind_sockets opens one socket per address family; verified against
  PTB 22.6/tornado) with new TELEGRAM_WEBHOOK_HOST / extra.webhook_host
  override.

Explicit host overrides everywhere are preserved; empty/unset collapses
to the dual-stack default. "::" remains a bad substitute on
bindv6only=1 hosts (see LINE adapter comment).
2026-07-28 22:42:41 -07:00
lemonwan 0f8603c571 test(gateway): regression: every adapter.connect() must accept is_reconnect
The gateway reconnect watcher forwards is_reconnect=True to every
adapter.connect() call on every retry. Adapters whose signature omits
the kwarg raise TypeError at every reconnect attempt and stay silently
disconnected — the exact bug that shipped for QQAdapter and only
surfaced after messages stopped flowing on the QQ channel for hours.

This test statically parses every adapter.py under gateway/platforms/
and plugins/platforms/ (via AST, so third-party SDKs like slack_sdk,
matrix-nio, aiohttp, telegram, etc. are NOT required in the test env)
and asserts every *Adapter class with an async connect() accepts
is_reconnect — either as a keyword-only argument or absorbed by
**kwargs.

Also fixes plugins/platforms/wecom/callback_adapter.py:WecomCallbackAdapter,
which the new test caught as a second offender. Same class of bug: bare
'async def connect(self)' signature would die on the first reconnect.

Companion to #59429 (which fixed the original QQAdapter offender).
2026-07-09 19:09:38 -07:00
Teknium 74541beb9c fix(security): cap WeCom callback body size before pre-auth XML parse (#54615)
The WeCom callback endpoint (internet-facing, 0.0.0.0) parsed untrusted
request bodies before signature verification. defusedxml already guards
the entity-expansion class on main, but there was no cap on raw body
size, so an unauthenticated POST could still force unbounded read work
pre-auth.

Set client_max_size=64KB on the aiohttp app (413 at the framework layer)
plus an explicit length guard in _handle_callback as defense in depth.
WeCom callbacks are small encrypted XML envelopes — media is delivered
out-of-band via MediaId, never inline — so 64KB is ample for legitimate
traffic. Adds tests for oversized (413) and normal-sized (not 413) bodies.

Salvaged from #10192 by @memosr (body-size limit half; defusedxml half
already superseded on main).
2026-06-28 22:35:43 -07:00
Teknium 5600105478 refactor(gateway): migrate slack/dingtalk/whatsapp/matrix/feishu/telegram/wecom/email/sms adapters to bundled plugins
Salvage of PR #41284 onto current main. Relocates the last 9 inline messaging
adapters (+ satellites: telegram_network, feishu_comment/_rules/meeting_invite,
wecom_crypto, wecom_callback) from gateway/platforms/ into self-contained
bundled plugins under plugins/platforms/<x>/, discovered via the platform
registry. Strips the per-platform core touchpoints from gateway/run.py,
gateway/config.py, hermes_cli/gateway.py, hermes_cli/setup.py, and
tools/send_message_tool.py.

Carries forward the migration fixes (explicit enabled:false honored,
get_connected_platforms forces discovery, plugin is_connected via
gateway.get_env_value, logs --component gateway matches plugins.platforms.*,
matrix hidden on Windows).

Additionally ports config keys main added since the PR base: the matrix
plugin's _apply_yaml_config now also covers allowed_users,
ignore_user_patterns, process_notices, and session_scope (the inline
gateway/config.py matrix block gained these in the 1340 commits the PR sat
open; they would otherwise have been silently dropped on deletion).
2026-06-20 10:26:45 -07:00