29033a3fd5
* fix(relay): map wire media[] → event.media_types; accept message_type voice A relayed voice note arrived as MessageType.AUDIO with media_types=[] — the STT gate (_event_media_is_stt_input) excludes AUDIO unconditionally and its per-attachment MIME rescue was unreachable, so STT never fired and the agent fell back to the "user sent an audio file attachment" context note (live-verified on staging 2026-08-26, Discord + Telegram). Two wire-boundary fixes, both additive within contract_version 1: - "voice" parses to MessageType.VOICE: the enum already had it — pinned by test so a future refactor can't collapse the two. - media[] is now mapped into event.media_types (positional alignment with media_urls; mime-less entries keep their slot as ""). This is what run.py's per-attachment classifiers key off, so EVERY relayed attachment — image vs document, audio vs voice — now routes like its native-adapter equivalent, not just voice notes. Behaviour pinned: new-connector voice → STT-eligible; legacy audio-typed events unchanged (no STT); music uploads never STT-eligible (direct _event_media_is_stt_input assertions on real wire-parsed events, not mocks). Pairs with the gateway-gateway PR that puts "voice" on the wire. * review: pin the STT gate by test; fail safe on media/media_urls mismatch Addresses independent review of #95274. 1. The PR's acceptance criterion is STT ROUTING, but no committed test called _event_media_is_stt_input — it was only asserted ad-hoc. Adds TestSttGate: voice→eligible, voice-without-media_types→eligible (the new-connector/old-gateway shape), legacy audio-typed voice note→not eligible, music→not eligible. Mutation-verified: removing the VOICE branch from the gate turns these RED. 2. media_urls and media[] are INDEPENDENT wire fields that consumers index by the same i. Mapping MIMEs positionally without checking agreement means a disagreeing producer misassociates a MIME with the wrong URL and mis-routes that attachment — strictly worse than no MIME, which degrades safely to message-level classification. _media_types_from_wire() now maps only when the lengths agree, warns and returns [] otherwise. Note for the record: MessageType.VOICE predates this PR and the gate's VOICE branch ignores media_types, so a NEW connector against an OLD gateway ALREADY fires STT. That is desirable, but it is not "unchanged" — the PR body's rollout matrix said otherwise and is corrected. * fix(relay): send a User-Agent on relay media requests (Discord CDN 403) Discord's CDN rejects urllib's default "Python-urllib/x.y" User-Agent with HTTP 403, and RelayMediaClient never set one. Every Discord CDN pass-through download therefore failed; _localize_inbound_media then kept the raw URL (its "a public URL still has value" branch), and the consumer tried to open a URL as a FILE PATH: WARNING gateway.relay.media: relay media download failed for https://cdn.discordapp.com/...voice-message.ogg: HTTP Error 403 INFO gateway.run: Voice transcription failed for https://cdn.discord... : Audio file not found: https://cdn.discordapp.com/... This killed ALL Discord relay media inbound — voice notes, images and documents alike — not just the voice lane. Telegram/WhatsApp were unaffected because their media is connector-re-hosted (/relay/media/{id}, fetched from our own host) and localizes to real /tmp paths. Reproduced from a clean shell against a live CDN URL: curl (own UA) -> 200 urllib, no UA -> 403 Forbidden urllib + descriptive UA -> 200, 14583 bytes, OggS magic Fix: a module-level _MEDIA_USER_AGENT sent on both download() and upload(). upload() only ever targets our own connector so it was not broken, but a single client should identify itself consistently. Validated on staging: hot-patched hermes-agent-stg-test-6698, restarted the gateway service, and Ben's Discord voice note transcribed successfully — zero new 403s and zero new transcription failures after the patch (last 403 predates it). Test is mutation-verified: removing the UA from download() turns it RED while the other five media tests stay green. * fix(relay): keep url↔mime pairing through media localization Addresses a blocking review finding on my own change: mapping media[] into media_types created a POSITIONAL contract that the rest of the inbound path then broke. 1. _localize_inbound_media (adapter.py) filtered media_urls without filtering media_types. Dropping a dead connector re-host is a NORMAL best-effort path, so every surviving attachment inherited its neighbour's mime. Reproduced through the real functions: before urls [.../relay/media/dead, .../kept.png] types [application/pdf, image/png] after urls [.../kept.png] types [application/pdf, image/png] <-- PNG reads as PDF _event_media_is_image(ev, 0) -> False The loop now carries (url, mime) as PAIRS, so a dropped URL drops its mime with it. 2. _media_types_from_wire compared LENGTHS only, which is not alignment: equal-length-but-reordered wire fields were accepted and paired wrongly, and an absent media_urls skipped the check entirely while still emitting types. Resolution is now BY URL (url -> mime lookup over media_urls); an unmatched URL degrades to "" and falls back to message-level classification. Tests: 4 new cases driving the real chain (wire parse -> localization -> run.py classifier), incl. the dropped-first-attachment case the existing localization test could not catch (it builds events without media_types). The obsolete length-mismatch test now asserts the stronger by-url guarantee. Both fixes mutation-verified: reinstating the URL-only filter fails 1 test, reverting to positional resolution fails 3. Relay suite 258 passed; media/voice/stt selection 685 passed; ruff clean; cross-repo integration payload re-verified. * fix(relay): media_types is always one slot per media_url Self-review after two review rounds flagged this bug class in adjacent seams: I checked the function I edited, not every consumer of the parallel arrays I created. Grepping ALL writers found a third instance. merge_pending_message_event (gateway/platforms/base.py:2725-2735) EXTENDS media_urls and media_types together when a second media message merges into a pending one. My mapping could emit a POPULATED media_urls with an EMPTY media_types (an older connector sends media_urls but no media[]), so extend() concatenated lists of different lengths: A urls [old1.png, old2.png] types [] B urls [new.pdf] types [application/pdf] merged urls [old1.png, old2.png, new.pdf] types [application/pdf] -> old1.png reads as application/pdf; the real PDF gets '' Fix: media_types is now ALWAYS len(media_urls), padded with '' — the url-keyed lookup runs even when media[] is absent, and the localizer rewrites the list unconditionally (no short-circuit that could leave a stale/short list behind). Tests: 4 new cases — padding with no media[], the merge shift above driven through the real merge_pending_message_event, localization preserving the invariant while dropping an entry, and normalization of a short/empty media_types arriving from a non-wire source. All mutation-verified: removing the padding fails 4; restoring the guard fails 1. Relay 262 passed; media/voice/stt selection 689 passed; ruff clean; cross-repo integration payload re-verified.
232 lines
8.5 KiB
Python
232 lines
8.5 KiB
Python
"""Relay Phase 2 media tests — send_media egress lanes + inbound media localization.
|
|
|
|
Covers:
|
|
- the five ``send_*`` overrides route through ONE ``send_media`` op with the
|
|
right ``media_kind`` and honor op-level capability gating (a connector not
|
|
advertising ``send_media`` falls back to the base-class behaviour);
|
|
- local-path sources upload through the RelayMediaClient first (the
|
|
connector cannot reach our filesystem) and public URLs pass through;
|
|
- a connector decline / failed upload degrades to the pre-media fallback;
|
|
- inbound ``media_urls`` are localized to temp paths (re-hosts downloaded
|
|
with the per-gateway bearer; dead re-host refs dropped; public URLs kept
|
|
when no client is available);
|
|
- the RelayMediaClient URL derivation + auth header shape.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
from typing import Optional
|
|
|
|
import pytest
|
|
|
|
from gateway.config import PlatformConfig
|
|
from gateway.relay.adapter import RelayAdapter
|
|
from gateway.relay.descriptor import CONTRACT_VERSION, CapabilityDescriptor
|
|
from gateway.relay.media import RelayMediaClient, media_base_url
|
|
|
|
from tests.gateway.relay.stub_connector import StubConnector
|
|
|
|
|
|
def make_desc(**kw) -> CapabilityDescriptor:
|
|
base = dict(
|
|
contract_version=CONTRACT_VERSION,
|
|
platform="telegram",
|
|
label="Telegram",
|
|
max_message_length=4096,
|
|
supports_draft_streaming=False,
|
|
supports_edit=True,
|
|
supports_threads=True,
|
|
markdown_dialect="markdown_v2",
|
|
len_unit="utf16",
|
|
supported_ops=(
|
|
"send",
|
|
"edit",
|
|
"typing",
|
|
"get_chat_info",
|
|
"send_media",
|
|
),
|
|
)
|
|
base.update(kw)
|
|
return CapabilityDescriptor(**base)
|
|
|
|
|
|
class FakeMediaClient:
|
|
"""In-memory stand-in for RelayMediaClient (no HTTP)."""
|
|
|
|
def __init__(self) -> None:
|
|
self.enabled = True
|
|
self.uploads: list[tuple[str, Optional[str]]] = []
|
|
self.downloads: list[str] = []
|
|
self.upload_result: Optional[str] = "https://conn.example/relay/media/aa11"
|
|
self.download_result: Optional[str] = "/tmp/relay_media_fake.png"
|
|
|
|
async def upload(self, file_path, *, mime=None, filename=None):
|
|
self.uploads.append((str(file_path), filename))
|
|
return self.upload_result
|
|
|
|
async def download(self, url, *, suggested_name=None):
|
|
self.downloads.append(url)
|
|
return self.download_result
|
|
|
|
def is_relay_media_url(self, url: str) -> bool:
|
|
return "/relay/media/" in (url or "")
|
|
|
|
|
|
def _adapter(**desc_kw) -> tuple[RelayAdapter, StubConnector, FakeMediaClient]:
|
|
stub = StubConnector(make_desc(**desc_kw))
|
|
adapter = RelayAdapter(PlatformConfig(), make_desc(**desc_kw), transport=stub)
|
|
fake = FakeMediaClient()
|
|
adapter._media_client = fake # bypass env-derived construction
|
|
return adapter, stub, fake
|
|
|
|
|
|
# ── egress: the five overrides ───────────────────────────────────────────
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_send_image_url_passes_through_without_upload():
|
|
adapter, stub, fake = _adapter()
|
|
result = await adapter.send_image(
|
|
"chat1", "https://fal.media/x.png", caption="a pic", reply_to="m9"
|
|
)
|
|
assert result.success is True
|
|
assert result.message_id == "md1"
|
|
assert fake.uploads == [] # public URL → no upload leg
|
|
action = stub.sent[-1]
|
|
assert action["op"] == "send_media"
|
|
assert action["media_kind"] == "image"
|
|
assert action["source_url"] == "https://fal.media/x.png"
|
|
assert action["content"] == "a pic"
|
|
assert action["reply_to"] == "m9"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_local_path_lanes_upload_first(tmp_path: Path):
|
|
adapter, stub, fake = _adapter()
|
|
f = tmp_path / "clip.ogg"
|
|
f.write_bytes(b"oggbytes")
|
|
result = await adapter.send_voice("chat1", str(f), caption="listen")
|
|
assert result.success is True
|
|
assert fake.uploads == [(str(f), None)]
|
|
action = stub.sent[-1]
|
|
assert action["op"] == "send_media"
|
|
assert action["media_kind"] == "voice"
|
|
# The wire carries the RE-HOST reference, never the local path.
|
|
assert action["source_url"] == fake.upload_result
|
|
assert str(f) not in str(action)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_op_gating_falls_back_when_not_advertised(tmp_path: Path):
|
|
# Connector advertises only the legacy ops — send_media must never hit the wire.
|
|
adapter, stub, fake = _adapter(
|
|
supported_ops=("send", "edit", "typing", "get_chat_info")
|
|
)
|
|
result = await adapter.send_image("chat1", "https://x.io/a.png", caption="hi")
|
|
# Base-class fallback: caption + URL as a text send.
|
|
assert result.success is True
|
|
ops = [a["op"] for a in stub.sent]
|
|
assert "send_media" not in ops
|
|
assert ops[-1] == "send"
|
|
assert "https://x.io/a.png" in stub.sent[-1]["content"]
|
|
|
|
|
|
# ── inbound localization ─────────────────────────────────────────────────
|
|
|
|
|
|
def _make_event(media_urls):
|
|
from gateway.platforms.base import MessageEvent, MessageType
|
|
from gateway.session import SessionSource
|
|
|
|
return MessageEvent(
|
|
text="look",
|
|
message_type=MessageType.TEXT,
|
|
source=SessionSource(
|
|
platform="telegram", chat_id="c1", chat_type="dm", user_id="u1"
|
|
),
|
|
media_urls=list(media_urls),
|
|
)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_inbound_without_client_keeps_public_drops_rehost():
|
|
adapter, _stub, _fake = _adapter()
|
|
adapter._media_client = None
|
|
adapter._get_media_client = lambda: None # type: ignore[method-assign]
|
|
event = _make_event(
|
|
[
|
|
"https://conn.example/relay/media/deadbeef",
|
|
"https://cdn.discordapp.com/attachments/a/b.png",
|
|
]
|
|
)
|
|
await adapter._localize_inbound_media(event)
|
|
assert event.media_urls == ["https://cdn.discordapp.com/attachments/a/b.png"]
|
|
|
|
|
|
# ── RelayMediaClient unit surface ────────────────────────────────────────
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_client_upload_rejects_oversize_and_missing(tmp_path: Path):
|
|
c = RelayMediaClient("https://c.example", "gw1", "sec")
|
|
# Missing file → None (no network attempted).
|
|
assert await c.upload(str(tmp_path / "nope.bin")) is None
|
|
# Empty file → None.
|
|
empty = tmp_path / "empty.bin"
|
|
empty.write_bytes(b"")
|
|
assert await c.upload(str(empty)) is None
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_download_sends_a_user_agent_on_every_request():
|
|
"""Discord's CDN 403s the default ``Python-urllib/x.y`` User-Agent.
|
|
|
|
Live-verified on staging 2026-08-26: every Discord CDN pass-through
|
|
download failed with ``HTTP Error 403: Forbidden``, the localizer then kept
|
|
the raw URL, and the transcriber tried to open a URL as a file path
|
|
("Audio file not found") — so voice notes, images and documents were ALL
|
|
silently dead on the Discord relay lane. Reproduced from a clean shell:
|
|
urllib with no UA → 403; the same URL with any descriptive UA → 200.
|
|
|
|
Assert the header on BOTH url classes (public pass-through and
|
|
bearer-authenticated re-host), because they take different header paths.
|
|
"""
|
|
seen: list[dict] = []
|
|
|
|
class _Resp:
|
|
headers = {"Content-Type": "audio/ogg", "Content-Length": "4"}
|
|
|
|
def read(self, *_a):
|
|
return b"OggS"
|
|
|
|
def __enter__(self):
|
|
return self
|
|
|
|
def __exit__(self, *_a):
|
|
return False
|
|
|
|
def _fake_urlopen(req, timeout=None): # noqa: ARG001
|
|
seen.append(dict(req.headers))
|
|
return _Resp()
|
|
|
|
import urllib.request as _ur
|
|
|
|
orig = _ur.urlopen
|
|
_ur.urlopen = _fake_urlopen # type: ignore[assignment]
|
|
try:
|
|
c = RelayMediaClient("https://conn.example", "gw1", "sec")
|
|
assert await c.download("https://cdn.discordapp.com/attachments/1/2/v.ogg")
|
|
assert await c.download("https://conn.example/relay/media/deadbeef")
|
|
finally:
|
|
_ur.urlopen = orig # type: ignore[assignment]
|
|
|
|
assert len(seen) == 2
|
|
for headers in seen:
|
|
# urllib title-cases header keys on Request.
|
|
ua = headers.get("User-agent") or headers.get("User-Agent")
|
|
assert ua, f"no User-Agent sent; urllib would default to Python-urllib (403s on Discord CDN): {headers}"
|
|
assert "python-urllib" not in ua.lower()
|
|
# The re-host request must still carry its bearer (no regression).
|
|
rehost_headers = seen[1]
|
|
assert (rehost_headers.get("Authorization") or "").startswith("Bearer ")
|