Files
EvoScientist-Multi/EvoScientist/channels/telegram/channel.py
T
Faych 9bd7a37a77 fix: _extract_retry_after returns None for non-retryable errors (#394)
* fix: _extract_retry_after returns None for non-retryable errors

* fix: update _extract_retry_after to handle generic transient errors with default retry delay

* fix: extend base _non_retryable_patterns in channel subclasses

* fix(channels): merge structured SDK error check into non-retryable step

* fix(channels): decouple status code and SDK error code extraction in retry logic

- Independently evaluate HTTP status codes and structured SDK error codes
- Fix misleading doc comments for Feishu and DingTalk patterns
- Remove redundant try-except AttributeError on getattr with default
- Expand test coverage for dual-signal matrix and header parsing

* refactor(channels): simplify status code and SDK error extraction via channel overrides

- Handle httpx and aiohttp exceptions in base Channel class
- Override _extract_status_code and _extract_sdk_error_code in SlackChannel and DiscordChannel
- Replace mock exception types in comprehensive test suite with real httpx and aiohttp errors
- Add dedicated Slack and Discord retry error extraction test suites

* fix(channels): clean up Slack and Discord error code extraction

- Remove defensive string checks and attribute guards in SlackChannel
- Directly access exc.response.status_code and exc.response.get('error') in SlackChannel
- Remove unnecessary _extract_sdk_error_code override in DiscordChannel
- Use real SlackApiError, SlackResponse, and discord.HTTPException in unit tests

* refactor: reorder retry logic to prioritize non-retryable checks, remove aiohttp dependency, and clean up exception handling in base and channel modules.

* test(channels): skip Slack/Discord retry tests when the SDK extra is absent

The retry-extraction tests build real SlackApiError / discord.HTTPException
objects, but slack-sdk and discord.py are optional extras that the dev
dependency group does not install. Under CI's `uv sync --dev` all nine
tests failed with ModuleNotFoundError raised from the channel override.

Gate both test classes with skipif(find_spec(...) is None) so the suite is
green without the extras and the tests still run wherever they are installed.

* refactor(channels): replace retry-delay lookup with _extract_retry_delay

_extract_retry_after still read the server-supplied delay by probing
exc.retry_after and exc.response.headers via getattr/hasattr, the last
remnant of the pattern the extractors moved away from. Replace both steps
with one overridable hook, _extract_retry_delay, implemented against the
real exception types:

- base: httpx.HTTPStatusError -> Retry-After header (httpx.Headers is
  case-insensitive; HTTP-date form remains unsupported)
- SlackChannel: SlackApiError -> Retry-After, matched case-insensitively
  because SlackResponse.headers is a plain dict whose casing depends on the
  HTTP client (same approach as slack_sdk's RateLimitErrorRetryHandler)
- TelegramChannel: telegram.error.RetryAfter.retry_after (int, or timedelta
  under PTB_TIMEDELTA)
- DiscordChannel: discord.RateLimited.retry_after, which the old duck-typed
  getattr matched and would otherwise have been lost

Drop the isinstance(retry, bool) and val >= 0 guards; no SDK produces those.
Delete the test that asserted the duck-typed attribute; add real-object tests
for each override, guarded like the existing SDK-dependent classes.

* ci: install the all-channels extra so SDK-dependent channel tests run

The Slack, Discord, and Telegram retry tests build real SDK exception
objects and are skipped when the SDK is absent. CI only ran `uv sync --dev`,
so those tests never executed there. Install the existing all-channels
extra alongside the dev group; the skipif guards remain for lean local runs.

* fix(channels): honor HTTP-date Retry-After and tolerate malformed values

RFC 9110 allows Retry-After as either delay-seconds or an HTTP-date. The
httpx path treated a date as unparseable and fell back to the 1.0 s default,
so a 503 asking for a specific wait was retried too early. Add
Channel._parse_retry_after, which returns delay-seconds as-is and converts
an HTTP-date to the non-negative seconds until it (tz-less dates read as
UTC).

SlackChannel used a bare float() on the header. A non-numeric value raised
inside the retry predicate, which escapes retry_async and drops the chunk
instead of retrying. Route Slack through the same helper so a bad header
falls back to _rate_limit_delay.

Addresses CodeRabbit review comments on base.py:866 and slack/channel.py:229.

* fix(channels): treat HTTP 400 and 404 as non-retryable

Both are permanent for a given request, so retrying burns the attempt
budget for nothing. Add them to _non_retryable_status_codes alongside
401/403.

Deliberately not a 4xx range check: 408 and 425 are retryable by
definition and 429 is handled by the rate-limit path. A test pins 408 as
still retryable so the range shortcut is not reintroduced later.

Partially addresses CodeRabbit's outside-diff comment on base.py:749-750.

* fix(channels): guard Slack retry extractors against raw aiohttp responses

slack_sdk attaches the bare aiohttp.ClientResponse to SlackApiError when a
JSON-declared body fails to parse. That object has neither status_code nor
get(), so _extract_status_code raised AttributeError inside should_retry,
replacing the original error and skipping the remaining attempts. Narrow
both extractors to SlackResponse/AsyncSlackResponse so such errors fall
through to the message patterns and retry as before. Add a wire-level
regression test against a local aiohttp server.

---------

Co-authored-by: Dinos Papakostas <dinospk1999@gmail.com>
Co-authored-by: X-iZhang <zacharyzhang2022@gmail.com>
2026-09-10 22:25:12 +00:00

362 lines
12 KiB
Python

"""Telegram channel implementation using python-telegram-bot."""
import logging
from dataclasses import dataclass
from datetime import datetime, timedelta
from pathlib import Path
from typing import ClassVar
from ..base import (
AUDIO_EXTS,
IMAGE_EXTS,
VIDEO_EXTS,
Channel,
ChannelError,
RawIncoming,
)
from ..capabilities import TELEGRAM as TELEGRAM_CAPS
from ..config import BaseChannelConfig
logger = logging.getLogger(__name__)
@dataclass
class TelegramConfig(BaseChannelConfig):
bot_token: str = ""
text_chunk_limit: int = 4096
class TelegramChannel(Channel):
"""Telegram channel using python-telegram-bot with long polling."""
name = "telegram"
capabilities = TELEGRAM_CAPS
_typing_interval: float = 4.0
_ready_attrs = ("_app",)
_non_retryable_patterns = (
*Channel._non_retryable_patterns,
"parse",
"can't parse",
)
_mention_pattern = r"(?i)@{bot_id}\s*"
def __init__(self, config: TelegramConfig):
super().__init__(config)
self._app = None
self._bot_username: str = ""
async def start(self) -> None:
if not self.config.bot_token:
raise ChannelError("Telegram bot token is required")
try:
from telegram.ext import (
ApplicationBuilder,
MessageHandler,
filters,
)
except ImportError:
raise ChannelError(
"python-telegram-bot not installed. "
"Install with: pip install evoscientist[telegram]"
) from None
builder = ApplicationBuilder().token(self.config.bot_token)
if self.config.proxy:
builder = builder.proxy(self.config.proxy).get_updates_proxy(
self.config.proxy
)
self._app = builder.build()
# Accept text and media message types
media_filter = filters.TEXT
if self.config.include_attachments:
media_filter = (
filters.TEXT
| filters.PHOTO
| filters.VOICE
| filters.AUDIO
| filters.Document.ALL
| filters.VIDEO
| filters.Sticker.ALL
| filters.LOCATION
)
self._app.add_handler(MessageHandler(media_filter, self._on_message))
await self._app.initialize()
# Cache bot username for @mention detection in groups
bot_info = await self._app.bot.get_me()
self._bot_username = (bot_info.username or "").lower()
await self._app.start()
await self._app.updater.start_polling(drop_pending_updates=True)
self._running = True
logger.info("Telegram channel started (polling)")
async def _cleanup(self) -> None:
app = self._app
self._app = None
if app is None:
return
if app.updater and app.updater.running:
await app.updater.stop()
if app.running:
await app.stop()
await app.shutdown()
logger.info("Telegram channel stopped")
# ── Typing indicator (override base) ────────────────────────────
async def _send_typing_action(self, chat_id: str) -> None:
"""Send typing action via Telegram Bot API."""
if self._app:
await self._app.bot.send_chat_action(
chat_id=int(chat_id),
action="typing",
)
# ── Retry delay extraction (override base) ─────────────────────
def _extract_retry_delay(self, exc: Exception) -> float | None:
"""Honor Telegram flood control (``telegram.error.RetryAfter``).
``retry_after`` is an ``int`` by default and a ``timedelta`` when the
``PTB_TIMEDELTA`` opt-in is enabled.
"""
from telegram.error import RetryAfter
if isinstance(exc, RetryAfter):
ra = exc.retry_after
return ra.total_seconds() if isinstance(ra, timedelta) else float(ra)
return super()._extract_retry_delay(exc)
# ── Send (template method overrides) ──────────────────────────
async def _send_chunk(self, chat_id, formatted_text, raw_text, reply_to, metadata):
reply_id = int(reply_to) if reply_to else None
async def _send(text):
await self._app.bot.send_message(
chat_id=int(chat_id),
text=text,
parse_mode="HTML" if text == formatted_text else None,
reply_to_message_id=reply_id,
)
await self._send_with_format_fallback(_send, formatted_text, raw_text)
_MEDIA_SENDERS: ClassVar[dict] = {
IMAGE_EXTS: ("send_photo", "photo"),
VIDEO_EXTS: ("send_video", "video"),
AUDIO_EXTS: ("send_audio", "audio"),
}
async def _send_media_impl(
self,
recipient: str,
file_path: str,
caption: str = "",
metadata: dict | None = None,
) -> bool:
"""Send a media file through Telegram."""
chat_id = int(self._resolve_media_chat_id(recipient, metadata))
cap = caption or None
ext = Path(file_path).suffix.lower()
for exts, (method, param) in self._MEDIA_SENDERS.items():
if ext in exts:
await getattr(self._app.bot, method)(
chat_id=chat_id,
caption=cap,
**{param: file_path},
)
return True
await self._app.bot.send_document(
chat_id=chat_id,
document=file_path,
caption=cap,
)
return True
def _get_bot_identifier(self) -> str | None:
return self._bot_username or None
@staticmethod
def _command_target(text: str) -> str | None:
"""Return a Telegram command's target username.
An empty string represents a bare command; ``None`` means the message
is not command-shaped.
"""
parts = text.lstrip().split(None, 1)
if not parts or not parts[0].startswith("/"):
return None
command_token = parts[0][1:]
if "@" not in command_token:
return ""
return command_token.rsplit("@", 1)[1].lower()
async def _send_ack_reaction(
self, chat_id: str, message_id: str, emoji: str = "👀"
) -> None:
"""Send an acknowledgment reaction via Telegram."""
if self._app:
try:
from telegram import ReactionTypeEmoji
await self._app.bot.set_message_reaction(
chat_id=int(chat_id),
message_id=int(message_id),
reaction=[ReactionTypeEmoji(emoji)],
)
except Exception as e:
logger.debug(f"Telegram ACK reaction failed: {e}")
async def _remove_ack_reaction(
self, chat_id: str, message_id: str, emoji: str = "👀"
) -> None:
"""Remove the ack reaction by setting empty reaction list."""
if self._app:
try:
await self._app.bot.set_message_reaction(
chat_id=int(chat_id),
message_id=int(message_id),
reaction=[],
)
except Exception as e:
logger.debug(f"Telegram remove ACK reaction failed: {e}")
async def _on_message(self, update, context) -> None:
"""Handler callback for text, photos, voice, audio, documents, video."""
if not update.message:
return
message = update.message
user_id = str(message.from_user.id)
chat_id = str(message.chat_id)
# Detect group and mention status for centralized gating
is_group = message.chat.type in ("group", "supergroup")
was_mentioned = not is_group
if is_group:
text_check = (message.text or message.caption or "").lower()
command_target = self._command_target(text_check)
if command_target is not None:
# A bare command that Telegram delivered to this bot is
# actionable. Commands explicitly addressed to another bot
# must remain ignored.
was_mentioned = not command_target or (
bool(self._bot_username) and command_target == self._bot_username
)
elif self._bot_username:
was_mentioned = f"@{self._bot_username}" in text_check
content_parts: list[str] = []
media_paths: list[str] = []
# Text content
if message.text:
content_parts.append(message.text)
if message.caption:
content_parts.append(message.caption)
# Handle media files
annotations: list[str] = []
if self.config.include_attachments:
media_file = None
media_type = None
if message.photo:
media_file = message.photo[-1] # Largest size
media_type = "image"
elif message.voice:
media_file = message.voice
media_type = "voice"
elif message.audio:
media_file = message.audio
media_type = "audio"
elif message.video:
media_file = message.video
media_type = "video"
elif message.document:
media_file = message.document
media_type = "file"
elif message.sticker:
media_file = message.sticker
media_type = "sticker"
# Location is not a downloadable file — handle separately
if message.location and not media_file:
loc = message.location
annotations.append(f"[位置] ({loc.latitude}, {loc.longitude})")
if media_file and self._app:
file_size = getattr(media_file, "file_size", 0) or 0
too_large = self._check_attachment_size(file_size, media_type)
if too_large:
annotations.append(too_large)
else:
try:
file = await self._app.bot.get_file(
media_file.file_id,
)
ext = self._get_extension(
media_type,
getattr(media_file, "mime_type", None),
)
file_path = self._media_path(f"{media_file.file_id[:16]}{ext}")
await file.download_to_drive(str(file_path))
media_paths.append(str(file_path))
annotations.append(f"[{media_type}: {file_path}]")
logger.debug(f"Downloaded {media_type} to {file_path}")
except Exception as e:
logger.error(f"Failed to download media: {e}")
annotations.append(f"[{media_type}: download failed]")
text_content = "\n".join(content_parts) if content_parts else ""
await self._enqueue_raw(
RawIncoming(
sender_id=user_id,
chat_id=chat_id,
text=text_content,
media_files=media_paths,
content_annotations=annotations,
timestamp=message.date or datetime.now(),
message_id=str(message.message_id),
metadata={"chat_id": chat_id},
is_group=is_group,
was_mentioned=was_mentioned,
)
)
_MIME_TO_EXT: ClassVar[dict[str, str]] = {
"image/jpeg": ".jpg",
"image/png": ".png",
"image/gif": ".gif",
"image/webp": ".webp",
"audio/ogg": ".ogg",
"audio/mpeg": ".mp3",
"audio/mp4": ".m4a",
"video/mp4": ".mp4",
"video/quicktime": ".mov",
}
_TYPE_TO_EXT: ClassVar[dict[str, str]] = {
"image": ".jpg",
"voice": ".ogg",
"audio": ".mp3",
"video": ".mp4",
"file": "",
"sticker": ".webp",
}
@staticmethod
def _get_extension(media_type: str, mime_type: str | None) -> str:
"""Get file extension based on media type and MIME type."""
if mime_type and mime_type in TelegramChannel._MIME_TO_EXT:
return TelegramChannel._MIME_TO_EXT[mime_type]
return TelegramChannel._TYPE_TO_EXT.get(media_type, "")