9bd7a37a77
* fix: _extract_retry_after returns None for non-retryable errors
* fix: update _extract_retry_after to handle generic transient errors with default retry delay
* fix: extend base _non_retryable_patterns in channel subclasses
* fix(channels): merge structured SDK error check into non-retryable step
* fix(channels): decouple status code and SDK error code extraction in retry logic
- Independently evaluate HTTP status codes and structured SDK error codes
- Fix misleading doc comments for Feishu and DingTalk patterns
- Remove redundant try-except AttributeError on getattr with default
- Expand test coverage for dual-signal matrix and header parsing
* refactor(channels): simplify status code and SDK error extraction via channel overrides
- Handle httpx and aiohttp exceptions in base Channel class
- Override _extract_status_code and _extract_sdk_error_code in SlackChannel and DiscordChannel
- Replace mock exception types in comprehensive test suite with real httpx and aiohttp errors
- Add dedicated Slack and Discord retry error extraction test suites
* fix(channels): clean up Slack and Discord error code extraction
- Remove defensive string checks and attribute guards in SlackChannel
- Directly access exc.response.status_code and exc.response.get('error') in SlackChannel
- Remove unnecessary _extract_sdk_error_code override in DiscordChannel
- Use real SlackApiError, SlackResponse, and discord.HTTPException in unit tests
* refactor: reorder retry logic to prioritize non-retryable checks, remove aiohttp dependency, and clean up exception handling in base and channel modules.
* test(channels): skip Slack/Discord retry tests when the SDK extra is absent
The retry-extraction tests build real SlackApiError / discord.HTTPException
objects, but slack-sdk and discord.py are optional extras that the dev
dependency group does not install. Under CI's `uv sync --dev` all nine
tests failed with ModuleNotFoundError raised from the channel override.
Gate both test classes with skipif(find_spec(...) is None) so the suite is
green without the extras and the tests still run wherever they are installed.
* refactor(channels): replace retry-delay lookup with _extract_retry_delay
_extract_retry_after still read the server-supplied delay by probing
exc.retry_after and exc.response.headers via getattr/hasattr, the last
remnant of the pattern the extractors moved away from. Replace both steps
with one overridable hook, _extract_retry_delay, implemented against the
real exception types:
- base: httpx.HTTPStatusError -> Retry-After header (httpx.Headers is
case-insensitive; HTTP-date form remains unsupported)
- SlackChannel: SlackApiError -> Retry-After, matched case-insensitively
because SlackResponse.headers is a plain dict whose casing depends on the
HTTP client (same approach as slack_sdk's RateLimitErrorRetryHandler)
- TelegramChannel: telegram.error.RetryAfter.retry_after (int, or timedelta
under PTB_TIMEDELTA)
- DiscordChannel: discord.RateLimited.retry_after, which the old duck-typed
getattr matched and would otherwise have been lost
Drop the isinstance(retry, bool) and val >= 0 guards; no SDK produces those.
Delete the test that asserted the duck-typed attribute; add real-object tests
for each override, guarded like the existing SDK-dependent classes.
* ci: install the all-channels extra so SDK-dependent channel tests run
The Slack, Discord, and Telegram retry tests build real SDK exception
objects and are skipped when the SDK is absent. CI only ran `uv sync --dev`,
so those tests never executed there. Install the existing all-channels
extra alongside the dev group; the skipif guards remain for lean local runs.
* fix(channels): honor HTTP-date Retry-After and tolerate malformed values
RFC 9110 allows Retry-After as either delay-seconds or an HTTP-date. The
httpx path treated a date as unparseable and fell back to the 1.0 s default,
so a 503 asking for a specific wait was retried too early. Add
Channel._parse_retry_after, which returns delay-seconds as-is and converts
an HTTP-date to the non-negative seconds until it (tz-less dates read as
UTC).
SlackChannel used a bare float() on the header. A non-numeric value raised
inside the retry predicate, which escapes retry_async and drops the chunk
instead of retrying. Route Slack through the same helper so a bad header
falls back to _rate_limit_delay.
Addresses CodeRabbit review comments on base.py:866 and slack/channel.py:229.
* fix(channels): treat HTTP 400 and 404 as non-retryable
Both are permanent for a given request, so retrying burns the attempt
budget for nothing. Add them to _non_retryable_status_codes alongside
401/403.
Deliberately not a 4xx range check: 408 and 425 are retryable by
definition and 429 is handled by the rate-limit path. A test pins 408 as
still retryable so the range shortcut is not reintroduced later.
Partially addresses CodeRabbit's outside-diff comment on base.py:749-750.
* fix(channels): guard Slack retry extractors against raw aiohttp responses
slack_sdk attaches the bare aiohttp.ClientResponse to SlackApiError when a
JSON-declared body fails to parse. That object has neither status_code nor
get(), so _extract_status_code raised AttributeError inside should_retry,
replacing the original error and skipping the remaining attempts. Narrow
both extractors to SlackResponse/AsyncSlackResponse so such errors fall
through to the message patterns and retry as before. Add a wire-level
regression test against a local aiohttp server.
---------
Co-authored-by: Dinos Papakostas <dinospk1999@gmail.com>
Co-authored-by: X-iZhang <zacharyzhang2022@gmail.com>
362 lines
12 KiB
Python
362 lines
12 KiB
Python
"""Telegram channel implementation using python-telegram-bot."""
|
|
|
|
import logging
|
|
from dataclasses import dataclass
|
|
from datetime import datetime, timedelta
|
|
from pathlib import Path
|
|
from typing import ClassVar
|
|
|
|
from ..base import (
|
|
AUDIO_EXTS,
|
|
IMAGE_EXTS,
|
|
VIDEO_EXTS,
|
|
Channel,
|
|
ChannelError,
|
|
RawIncoming,
|
|
)
|
|
from ..capabilities import TELEGRAM as TELEGRAM_CAPS
|
|
from ..config import BaseChannelConfig
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
@dataclass
|
|
class TelegramConfig(BaseChannelConfig):
|
|
bot_token: str = ""
|
|
text_chunk_limit: int = 4096
|
|
|
|
|
|
class TelegramChannel(Channel):
|
|
"""Telegram channel using python-telegram-bot with long polling."""
|
|
|
|
name = "telegram"
|
|
|
|
capabilities = TELEGRAM_CAPS
|
|
_typing_interval: float = 4.0
|
|
_ready_attrs = ("_app",)
|
|
_non_retryable_patterns = (
|
|
*Channel._non_retryable_patterns,
|
|
"parse",
|
|
"can't parse",
|
|
)
|
|
_mention_pattern = r"(?i)@{bot_id}\s*"
|
|
|
|
def __init__(self, config: TelegramConfig):
|
|
super().__init__(config)
|
|
self._app = None
|
|
self._bot_username: str = ""
|
|
|
|
async def start(self) -> None:
|
|
if not self.config.bot_token:
|
|
raise ChannelError("Telegram bot token is required")
|
|
|
|
try:
|
|
from telegram.ext import (
|
|
ApplicationBuilder,
|
|
MessageHandler,
|
|
filters,
|
|
)
|
|
except ImportError:
|
|
raise ChannelError(
|
|
"python-telegram-bot not installed. "
|
|
"Install with: pip install evoscientist[telegram]"
|
|
) from None
|
|
|
|
builder = ApplicationBuilder().token(self.config.bot_token)
|
|
|
|
if self.config.proxy:
|
|
builder = builder.proxy(self.config.proxy).get_updates_proxy(
|
|
self.config.proxy
|
|
)
|
|
self._app = builder.build()
|
|
|
|
# Accept text and media message types
|
|
media_filter = filters.TEXT
|
|
if self.config.include_attachments:
|
|
media_filter = (
|
|
filters.TEXT
|
|
| filters.PHOTO
|
|
| filters.VOICE
|
|
| filters.AUDIO
|
|
| filters.Document.ALL
|
|
| filters.VIDEO
|
|
| filters.Sticker.ALL
|
|
| filters.LOCATION
|
|
)
|
|
|
|
self._app.add_handler(MessageHandler(media_filter, self._on_message))
|
|
|
|
await self._app.initialize()
|
|
# Cache bot username for @mention detection in groups
|
|
bot_info = await self._app.bot.get_me()
|
|
self._bot_username = (bot_info.username or "").lower()
|
|
await self._app.start()
|
|
await self._app.updater.start_polling(drop_pending_updates=True)
|
|
self._running = True
|
|
logger.info("Telegram channel started (polling)")
|
|
|
|
async def _cleanup(self) -> None:
|
|
app = self._app
|
|
self._app = None
|
|
if app is None:
|
|
return
|
|
|
|
if app.updater and app.updater.running:
|
|
await app.updater.stop()
|
|
if app.running:
|
|
await app.stop()
|
|
await app.shutdown()
|
|
logger.info("Telegram channel stopped")
|
|
|
|
# ── Typing indicator (override base) ────────────────────────────
|
|
|
|
async def _send_typing_action(self, chat_id: str) -> None:
|
|
"""Send typing action via Telegram Bot API."""
|
|
if self._app:
|
|
await self._app.bot.send_chat_action(
|
|
chat_id=int(chat_id),
|
|
action="typing",
|
|
)
|
|
|
|
# ── Retry delay extraction (override base) ─────────────────────
|
|
|
|
def _extract_retry_delay(self, exc: Exception) -> float | None:
|
|
"""Honor Telegram flood control (``telegram.error.RetryAfter``).
|
|
|
|
``retry_after`` is an ``int`` by default and a ``timedelta`` when the
|
|
``PTB_TIMEDELTA`` opt-in is enabled.
|
|
"""
|
|
from telegram.error import RetryAfter
|
|
|
|
if isinstance(exc, RetryAfter):
|
|
ra = exc.retry_after
|
|
return ra.total_seconds() if isinstance(ra, timedelta) else float(ra)
|
|
return super()._extract_retry_delay(exc)
|
|
|
|
# ── Send (template method overrides) ──────────────────────────
|
|
|
|
async def _send_chunk(self, chat_id, formatted_text, raw_text, reply_to, metadata):
|
|
reply_id = int(reply_to) if reply_to else None
|
|
|
|
async def _send(text):
|
|
await self._app.bot.send_message(
|
|
chat_id=int(chat_id),
|
|
text=text,
|
|
parse_mode="HTML" if text == formatted_text else None,
|
|
reply_to_message_id=reply_id,
|
|
)
|
|
|
|
await self._send_with_format_fallback(_send, formatted_text, raw_text)
|
|
|
|
_MEDIA_SENDERS: ClassVar[dict] = {
|
|
IMAGE_EXTS: ("send_photo", "photo"),
|
|
VIDEO_EXTS: ("send_video", "video"),
|
|
AUDIO_EXTS: ("send_audio", "audio"),
|
|
}
|
|
|
|
async def _send_media_impl(
|
|
self,
|
|
recipient: str,
|
|
file_path: str,
|
|
caption: str = "",
|
|
metadata: dict | None = None,
|
|
) -> bool:
|
|
"""Send a media file through Telegram."""
|
|
chat_id = int(self._resolve_media_chat_id(recipient, metadata))
|
|
cap = caption or None
|
|
ext = Path(file_path).suffix.lower()
|
|
for exts, (method, param) in self._MEDIA_SENDERS.items():
|
|
if ext in exts:
|
|
await getattr(self._app.bot, method)(
|
|
chat_id=chat_id,
|
|
caption=cap,
|
|
**{param: file_path},
|
|
)
|
|
return True
|
|
await self._app.bot.send_document(
|
|
chat_id=chat_id,
|
|
document=file_path,
|
|
caption=cap,
|
|
)
|
|
return True
|
|
|
|
def _get_bot_identifier(self) -> str | None:
|
|
return self._bot_username or None
|
|
|
|
@staticmethod
|
|
def _command_target(text: str) -> str | None:
|
|
"""Return a Telegram command's target username.
|
|
|
|
An empty string represents a bare command; ``None`` means the message
|
|
is not command-shaped.
|
|
"""
|
|
parts = text.lstrip().split(None, 1)
|
|
if not parts or not parts[0].startswith("/"):
|
|
return None
|
|
command_token = parts[0][1:]
|
|
if "@" not in command_token:
|
|
return ""
|
|
return command_token.rsplit("@", 1)[1].lower()
|
|
|
|
async def _send_ack_reaction(
|
|
self, chat_id: str, message_id: str, emoji: str = "👀"
|
|
) -> None:
|
|
"""Send an acknowledgment reaction via Telegram."""
|
|
if self._app:
|
|
try:
|
|
from telegram import ReactionTypeEmoji
|
|
|
|
await self._app.bot.set_message_reaction(
|
|
chat_id=int(chat_id),
|
|
message_id=int(message_id),
|
|
reaction=[ReactionTypeEmoji(emoji)],
|
|
)
|
|
except Exception as e:
|
|
logger.debug(f"Telegram ACK reaction failed: {e}")
|
|
|
|
async def _remove_ack_reaction(
|
|
self, chat_id: str, message_id: str, emoji: str = "👀"
|
|
) -> None:
|
|
"""Remove the ack reaction by setting empty reaction list."""
|
|
if self._app:
|
|
try:
|
|
await self._app.bot.set_message_reaction(
|
|
chat_id=int(chat_id),
|
|
message_id=int(message_id),
|
|
reaction=[],
|
|
)
|
|
except Exception as e:
|
|
logger.debug(f"Telegram remove ACK reaction failed: {e}")
|
|
|
|
async def _on_message(self, update, context) -> None:
|
|
"""Handler callback for text, photos, voice, audio, documents, video."""
|
|
if not update.message:
|
|
return
|
|
|
|
message = update.message
|
|
user_id = str(message.from_user.id)
|
|
chat_id = str(message.chat_id)
|
|
|
|
# Detect group and mention status for centralized gating
|
|
is_group = message.chat.type in ("group", "supergroup")
|
|
was_mentioned = not is_group
|
|
if is_group:
|
|
text_check = (message.text or message.caption or "").lower()
|
|
command_target = self._command_target(text_check)
|
|
if command_target is not None:
|
|
# A bare command that Telegram delivered to this bot is
|
|
# actionable. Commands explicitly addressed to another bot
|
|
# must remain ignored.
|
|
was_mentioned = not command_target or (
|
|
bool(self._bot_username) and command_target == self._bot_username
|
|
)
|
|
elif self._bot_username:
|
|
was_mentioned = f"@{self._bot_username}" in text_check
|
|
|
|
content_parts: list[str] = []
|
|
media_paths: list[str] = []
|
|
|
|
# Text content
|
|
if message.text:
|
|
content_parts.append(message.text)
|
|
if message.caption:
|
|
content_parts.append(message.caption)
|
|
|
|
# Handle media files
|
|
annotations: list[str] = []
|
|
if self.config.include_attachments:
|
|
media_file = None
|
|
media_type = None
|
|
|
|
if message.photo:
|
|
media_file = message.photo[-1] # Largest size
|
|
media_type = "image"
|
|
elif message.voice:
|
|
media_file = message.voice
|
|
media_type = "voice"
|
|
elif message.audio:
|
|
media_file = message.audio
|
|
media_type = "audio"
|
|
elif message.video:
|
|
media_file = message.video
|
|
media_type = "video"
|
|
elif message.document:
|
|
media_file = message.document
|
|
media_type = "file"
|
|
elif message.sticker:
|
|
media_file = message.sticker
|
|
media_type = "sticker"
|
|
|
|
# Location is not a downloadable file — handle separately
|
|
if message.location and not media_file:
|
|
loc = message.location
|
|
annotations.append(f"[位置] ({loc.latitude}, {loc.longitude})")
|
|
|
|
if media_file and self._app:
|
|
file_size = getattr(media_file, "file_size", 0) or 0
|
|
too_large = self._check_attachment_size(file_size, media_type)
|
|
if too_large:
|
|
annotations.append(too_large)
|
|
else:
|
|
try:
|
|
file = await self._app.bot.get_file(
|
|
media_file.file_id,
|
|
)
|
|
ext = self._get_extension(
|
|
media_type,
|
|
getattr(media_file, "mime_type", None),
|
|
)
|
|
file_path = self._media_path(f"{media_file.file_id[:16]}{ext}")
|
|
await file.download_to_drive(str(file_path))
|
|
|
|
media_paths.append(str(file_path))
|
|
annotations.append(f"[{media_type}: {file_path}]")
|
|
logger.debug(f"Downloaded {media_type} to {file_path}")
|
|
except Exception as e:
|
|
logger.error(f"Failed to download media: {e}")
|
|
annotations.append(f"[{media_type}: download failed]")
|
|
|
|
text_content = "\n".join(content_parts) if content_parts else ""
|
|
|
|
await self._enqueue_raw(
|
|
RawIncoming(
|
|
sender_id=user_id,
|
|
chat_id=chat_id,
|
|
text=text_content,
|
|
media_files=media_paths,
|
|
content_annotations=annotations,
|
|
timestamp=message.date or datetime.now(),
|
|
message_id=str(message.message_id),
|
|
metadata={"chat_id": chat_id},
|
|
is_group=is_group,
|
|
was_mentioned=was_mentioned,
|
|
)
|
|
)
|
|
|
|
_MIME_TO_EXT: ClassVar[dict[str, str]] = {
|
|
"image/jpeg": ".jpg",
|
|
"image/png": ".png",
|
|
"image/gif": ".gif",
|
|
"image/webp": ".webp",
|
|
"audio/ogg": ".ogg",
|
|
"audio/mpeg": ".mp3",
|
|
"audio/mp4": ".m4a",
|
|
"video/mp4": ".mp4",
|
|
"video/quicktime": ".mov",
|
|
}
|
|
_TYPE_TO_EXT: ClassVar[dict[str, str]] = {
|
|
"image": ".jpg",
|
|
"voice": ".ogg",
|
|
"audio": ".mp3",
|
|
"video": ".mp4",
|
|
"file": "",
|
|
"sticker": ".webp",
|
|
}
|
|
|
|
@staticmethod
|
|
def _get_extension(media_type: str, mime_type: str | None) -> str:
|
|
"""Get file extension based on media type and MIME type."""
|
|
if mime_type and mime_type in TelegramChannel._MIME_TO_EXT:
|
|
return TelegramChannel._MIME_TO_EXT[mime_type]
|
|
return TelegramChannel._TYPE_TO_EXT.get(media_type, "")
|