ab2f4602de93ca1c864275e53ca107b158da740a
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 kshitijk4poor
|
ab2f4602de |
refactor: MessageEvent to gateway/platforms/event.py; ElicitationHandler takes a call_context thunk
Breaks the two import cycles that forced Protocol stand-ins in the F821 sweep, so the two sites now name the real types. gateway/platforms/event.py (new leaf): MessageType, ProcessingOutcome, MessageEvent moved out of base.py verbatim. Their only dependency is gateway.session.SessionSource; base.py imported helpers.py at module level, so helpers could not name MessageEvent. Now TextBatchAggregator is typed by the real MessageEvent. 249 importers repointed (`from gateway.platforms.base import` -> `.event`, preserving each import's layout); gateway.platforms.__init__ re-exports from .event. The three revert-scheduled PLUGIN-COMPAT pointers that named these symbols (gateway.slash_commands → MessageType, dingtalk → MessageType, photon → ProcessingOutcome) and their COMPAT_MANIFEST rows now target gateway.platforms.event. Docs updated: ADDING_A_PLATFORM.md, adding-platform-adapters.md (en + zh-Hans). tools/mcp_tool_sampling.py: ElicitationHandler no longer holds a back-reference to its MCPServerTask (mcp_tool imports sampling, so the task type cannot be named there). It only ever read owner._pending_call_context, so it takes `call_context: Callable[[], Context | None]` and MCPServerTask passes `lambda: self._pending_call_context`. The consent call is one `functools.partial`, run directly or inside the captured Context. ty on the 11 touched production files vs origin/main: 0 new diagnostics, 14 resolved. (The one `source: SessionSource = None` diagnostic moves with the class; typing it Optional exposes ~60 unguarded call sites — separate follow-up.) Tests: tests/gateway + tests/plugins + tests/tools + touched files, 18,235 passed; the 31 failures reproduce identically on origin/main (macOS /private/tmp, systemd socket, long-path fixtures, live-service tests). |
||
|
Ben Barclay
|
29033a3fd5 |
fix(relay): restore voice-note STT — wire media[] MIMEs, message_type "voice", and a User-Agent for CDN downloads (#95274)
* fix(relay): map wire media[] → event.media_types; accept message_type voice A relayed voice note arrived as MessageType.AUDIO with media_types=[] — the STT gate (_event_media_is_stt_input) excludes AUDIO unconditionally and its per-attachment MIME rescue was unreachable, so STT never fired and the agent fell back to the "user sent an audio file attachment" context note (live-verified on staging 2026-08-26, Discord + Telegram). Two wire-boundary fixes, both additive within contract_version 1: - "voice" parses to MessageType.VOICE: the enum already had it — pinned by test so a future refactor can't collapse the two. - media[] is now mapped into event.media_types (positional alignment with media_urls; mime-less entries keep their slot as ""). This is what run.py's per-attachment classifiers key off, so EVERY relayed attachment — image vs document, audio vs voice — now routes like its native-adapter equivalent, not just voice notes. Behaviour pinned: new-connector voice → STT-eligible; legacy audio-typed events unchanged (no STT); music uploads never STT-eligible (direct _event_media_is_stt_input assertions on real wire-parsed events, not mocks). Pairs with the gateway-gateway PR that puts "voice" on the wire. * review: pin the STT gate by test; fail safe on media/media_urls mismatch Addresses independent review of #95274. 1. The PR's acceptance criterion is STT ROUTING, but no committed test called _event_media_is_stt_input — it was only asserted ad-hoc. Adds TestSttGate: voice→eligible, voice-without-media_types→eligible (the new-connector/old-gateway shape), legacy audio-typed voice note→not eligible, music→not eligible. Mutation-verified: removing the VOICE branch from the gate turns these RED. 2. media_urls and media[] are INDEPENDENT wire fields that consumers index by the same i. Mapping MIMEs positionally without checking agreement means a disagreeing producer misassociates a MIME with the wrong URL and mis-routes that attachment — strictly worse than no MIME, which degrades safely to message-level classification. _media_types_from_wire() now maps only when the lengths agree, warns and returns [] otherwise. Note for the record: MessageType.VOICE predates this PR and the gate's VOICE branch ignores media_types, so a NEW connector against an OLD gateway ALREADY fires STT. That is desirable, but it is not "unchanged" — the PR body's rollout matrix said otherwise and is corrected. * fix(relay): send a User-Agent on relay media requests (Discord CDN 403) Discord's CDN rejects urllib's default "Python-urllib/x.y" User-Agent with HTTP 403, and RelayMediaClient never set one. Every Discord CDN pass-through download therefore failed; _localize_inbound_media then kept the raw URL (its "a public URL still has value" branch), and the consumer tried to open a URL as a FILE PATH: WARNING gateway.relay.media: relay media download failed for https://cdn.discordapp.com/...voice-message.ogg: HTTP Error 403 INFO gateway.run: Voice transcription failed for https://cdn.discord... : Audio file not found: https://cdn.discordapp.com/... This killed ALL Discord relay media inbound — voice notes, images and documents alike — not just the voice lane. Telegram/WhatsApp were unaffected because their media is connector-re-hosted (/relay/media/{id}, fetched from our own host) and localizes to real /tmp paths. Reproduced from a clean shell against a live CDN URL: curl (own UA) -> 200 urllib, no UA -> 403 Forbidden urllib + descriptive UA -> 200, 14583 bytes, OggS magic Fix: a module-level _MEDIA_USER_AGENT sent on both download() and upload(). upload() only ever targets our own connector so it was not broken, but a single client should identify itself consistently. Validated on staging: hot-patched hermes-agent-stg-test-6698, restarted the gateway service, and Ben's Discord voice note transcribed successfully — zero new 403s and zero new transcription failures after the patch (last 403 predates it). Test is mutation-verified: removing the UA from download() turns it RED while the other five media tests stay green. * fix(relay): keep url↔mime pairing through media localization Addresses a blocking review finding on my own change: mapping media[] into media_types created a POSITIONAL contract that the rest of the inbound path then broke. 1. _localize_inbound_media (adapter.py) filtered media_urls without filtering media_types. Dropping a dead connector re-host is a NORMAL best-effort path, so every surviving attachment inherited its neighbour's mime. Reproduced through the real functions: before urls [.../relay/media/dead, .../kept.png] types [application/pdf, image/png] after urls [.../kept.png] types [application/pdf, image/png] <-- PNG reads as PDF _event_media_is_image(ev, 0) -> False The loop now carries (url, mime) as PAIRS, so a dropped URL drops its mime with it. 2. _media_types_from_wire compared LENGTHS only, which is not alignment: equal-length-but-reordered wire fields were accepted and paired wrongly, and an absent media_urls skipped the check entirely while still emitting types. Resolution is now BY URL (url -> mime lookup over media_urls); an unmatched URL degrades to "" and falls back to message-level classification. Tests: 4 new cases driving the real chain (wire parse -> localization -> run.py classifier), incl. the dropped-first-attachment case the existing localization test could not catch (it builds events without media_types). The obsolete length-mismatch test now asserts the stronger by-url guarantee. Both fixes mutation-verified: reinstating the URL-only filter fails 1 test, reverting to positional resolution fails 3. Relay suite 258 passed; media/voice/stt selection 685 passed; ruff clean; cross-repo integration payload re-verified. * fix(relay): media_types is always one slot per media_url Self-review after two review rounds flagged this bug class in adjacent seams: I checked the function I edited, not every consumer of the parallel arrays I created. Grepping ALL writers found a third instance. merge_pending_message_event (gateway/platforms/base.py:2725-2735) EXTENDS media_urls and media_types together when a second media message merges into a pending one. My mapping could emit a POPULATED media_urls with an EMPTY media_types (an older connector sends media_urls but no media[]), so extend() concatenated lists of different lengths: A urls [old1.png, old2.png] types [] B urls [new.pdf] types [application/pdf] merged urls [old1.png, old2.png, new.pdf] types [application/pdf] -> old1.png reads as application/pdf; the real PDF gets '' Fix: media_types is now ALWAYS len(media_urls), padded with '' — the url-keyed lookup runs even when media[] is absent, and the localizer rewrites the list unconditionally (no short-circuit that could leave a stale/short list behind). Tests: 4 new cases — padding with no media[], the merge shift above driven through the real merge_pending_message_event, localization preserving the invariant while dropping an entry, and normalization of a short/empty media_types arriving from a non-wire source. All mutation-verified: removing the padding fails 4; restoring the guard fails 1. Relay 262 passed; media/voice/stt selection 689 passed; ruff clean; cross-repo integration payload re-verified. |