Breaks the two import cycles that forced Protocol stand-ins in the F821 sweep, so the two
sites now name the real types.
gateway/platforms/event.py (new leaf): MessageType, ProcessingOutcome, MessageEvent moved
out of base.py verbatim. Their only dependency is gateway.session.SessionSource; base.py
imported helpers.py at module level, so helpers could not name MessageEvent. Now
TextBatchAggregator is typed by the real MessageEvent. 249 importers repointed
(`from gateway.platforms.base import` -> `.event`, preserving each import's layout);
gateway.platforms.__init__ re-exports from .event. The three revert-scheduled PLUGIN-COMPAT
pointers that named these symbols (gateway.slash_commands → MessageType, dingtalk → MessageType,
photon → ProcessingOutcome) and their COMPAT_MANIFEST rows now target gateway.platforms.event.
Docs updated: ADDING_A_PLATFORM.md, adding-platform-adapters.md (en + zh-Hans).
tools/mcp_tool_sampling.py: ElicitationHandler no longer holds a back-reference to its
MCPServerTask (mcp_tool imports sampling, so the task type cannot be named there). It only
ever read owner._pending_call_context, so it takes `call_context: Callable[[], Context | None]`
and MCPServerTask passes `lambda: self._pending_call_context`. The consent call is one
`functools.partial`, run directly or inside the captured Context.
ty on the 11 touched production files vs origin/main: 0 new diagnostics, 14 resolved.
(The one `source: SessionSource = None` diagnostic moves with the class; typing it Optional
exposes ~60 unguarded call sites — separate follow-up.)
Tests: tests/gateway + tests/plugins + tests/tools + touched files, 18,235 passed; the 31
failures reproduce identically on origin/main (macOS /private/tmp, systemd socket,
long-path fixtures, live-service tests).
Second review pass on the flood-retry change.
- sweep_recoverable returns a dead owner's not-yet-due flood row flagged
`adopted` (with its `not_before`) instead of dropping it from the result,
so _claim_pending_obligations clears its session's resume_pending flag like
every other claimed row; the answer is in the ledger and the turn must not
be re-run at boot. _redeliver_claimed_obligations skips adopted rows and
still arms the timer for them.
- A legacy row without adapter_profile is normalised to 'default' when the
boot sweep claims or adopts it: the runtime sweep matches profiles exactly
and the timer asks for 'default', so an adopted NULL row could only wake the
timer without ever being sent.
- A sleeping timer is replaced by a shorter refusal's timer; the shorter one
re-arms for the longer row after its sweep. A timer already sweeping is
never cancelled from outside.
- pending_flood_retries is read in a worker thread like the other ledger calls.
- Adoption tests seed a distinct dead-owner stamp and assert ownership moves.
- Scratch tags and long comment lines removed.
The Telegram adapter fails a flood-controlled final send closed as
flood_control:<seconds> so the send coroutine never sleeps a long penalty
(#91969), on the understanding that the delivery ledger owns the wait. The
ledger did not: sweep_failed_for_runtime only replayed send_path_degraded
rows, so a flood-refused row sat in 'failed' until the next restart, whose
sweep_recoverable then redelivered it hours late prefixed with "Recovered
reply, the gateway restarted during delivery, so this may be a duplicate".
Observed: a reply refused at 08:58 UTC (both the MarkdownV2 send and the
plain fallback got flood_control:185) arrived at 12:15 UTC after a restart,
labelled as a possible duplicate although the platform had never accepted it.
Ledger (gateway/delivery_ledger.py):
- flood_control rows are runtime-retryable, but only once their own deadline
has passed: the refusal's updated_at plus the platform's wait
(flood_not_before). Neither an early timer nor a reconnect sweep spends a
redelivery attempt inside the penalty window.
- A flood refusal of a reply that fits in one Telegram message (4096 UTF-16
units) proves non-delivery, so that redelivery carries no duplicate marker.
A chunked reply may have had its first chunk accepted before the refusal
and keeps the marker.
- Claiming a flood row clears the stale refusal (last_error NULL, state
'attempting'), so a resend interrupted before mark_delivered is seen as
uncertain by the next boot and gets the marker.
- At boot, a dead owner's flood row that is not yet due is adopted (owner
re-stamped, no attempt spent) instead of being resent early.
- pending_flood_retries() lists this process's waiting flood rows per adapter
identity with the earliest deadline.
Runner (gateway/run_startup.py):
- _schedule_flood_redelivery arms one timer per adapter identity that runs
the existing runtime sweep after the wait (capped at 15 minutes per sleep;
the row's deadline, not the timer, decides eligibility, so a capped timer
wakes early, sends nothing, and re-arms for the remainder). The slot stays
occupied until the timer ends and only the running timer may arm its
successor into it, so a refusal during the sweep can never strand the row.
- _arm_flood_timers_for_waiting_rows runs after every redelivery pass (boot
and runtime), covering adopted rows, rows skipped as not yet due, and rows
refused again.
Adapter (gateway/platforms/base.py): _finalize_delivery_obligation arms the
timer on a flood_control failure, best-effort inside the existing try.
Tests: 35 in tests/gateway/test_delivery_ledger_flood_retry.py, with a
controllable clock shared by ledger and runner. Each guard was mutation
tested. Existing ledger, reconnect and redelivery suites (131 tests) pass.
A rate-limited error classifies as network above and the retry loop only breaks
on a non-transient, non-rate-limited error, so the fallback-path guard could
never fire. Tests: the classifier delegator's vocabulary tests were a
change-detector; two rate-limit loop tests were subsumed by the exhaustion
contract (3 sends, no notice, no fallback).
Review follow-up on the salvaged #100072 flood handling. Honouring retry_after
is right, but sleeping it verbatim in _send_with_retry would pin the send
coroutine for a long penalty (a 97-minute FloodWait once froze inbound on
every platform, #91969 -- the Telegram adapter already fails closed at 5s for
the same reason and hands the wait to this loop). Past 60s the typed failure
goes back to the delivery ledger, which owns redelivery after the cooldown.
The six classifier tests collapse into one parametrized contract.
Flood-capped / rate-limited sends (Telegram FloodWait, Weixin bare RuntimeError)
were not honored as retryable when retry_after was absent, so _send_with_retry
fell through to the truncating plain-text fallback, which re-entered the server
ban and dropped the message tail — silently burning the turn with no delivery.
Three changes to _send_with_retry:
1. Route rate-limit checks through _is_rate_limited_error (single wrapper over
classify_send_error) so rate limits are treated as transient even when the
platform omits retry_after.
2. Reclassify rate-limit per retry attempt, not just the initial send, so the
in-loop break/continue reflects the CURRENT attempt (covers transient->flood
and rate-limited->formatting transitions).
3. Never take the truncating plain-text fallback for a rate-limited send — return
the typed failure so the delivery ledger owns redelivery after the cooldown.
And when retries exhaust on a rate-limited / retry_after-carrying failure,
return BEFORE sending the delivery-failure notice: the notice send would land
inside the same flood penalty and re-enter the ban ([0, 189, 378, 378] -> 3
sends, no 4th). Ordinary exhausted network errors keep the existing notice.
Honors server retry_after (when present) as the backoff delay instead of the
default exponential schedule.
Rebased onto current main (was 5163 commits behind, CONFLICTING) and folded in
reviewer feedback. Regression coverage in tests/gateway/test_send_retry.py:
rate-limit classifier, rate-limited-without-retry_after retries & succeeds,
rate-limited exhaustion returns a typed failure with no fallback and no notice
inside the active flood penalty, and failure-kind transitions between attempts.
'Skipping unsafe MEDIA directive path' fired for every non-existent file too, sending
operators down a security-policy rabbit hole for what was a hallucinated or untranslated
path. The warning now names the reason: 'not found on this host' vs 'denied by the
delivery policy'.
`MEDIA:/path` only delivered when the file existed on the gateway host.
With the ssh / modal / daytona / singularity / vercel backends the agent's
artifact sits on another filesystem, so `validate_media_delivery_path`
rejected it and the attachment vanished with a "Skipping unsafe" log line —
the #1 gap for sandboxed deployments.
`BaseEnvironment.fetch_file` pulls a regular file out of any backend over
the exec channel (base64, marker-fenced, size-bounded INSIDE the sandbox so
/dev/zero cannot flood host memory — the same shape image_source already
uses). `gateway/media_fetch.py` runs only when a remote backend is active
and the host lookup failed: the sandbox path is screened against the SAME
denylist as host deliveries, again after `readlink -f`, then copied into
the document cache (an allowlisted root) and validated like any host file.
No new tool; the existing `MEDIA:` tag is the interface.
Salvaged from #68506 by @tokou (design and denylist mirroring); redone on
current main without the send_file tool, the per-backend transports and
the undeliverable-notice plumbing (the #66797 failure notice already covers
that surface).
_kanban_root/_kanban_board_dirs replace the two inline HERMES_KANBAN_HOME + iterdir copies;
the allow side keeps its strict slug/symlink/db filter on top. _sqlite_files also covers
the rollback journal, matching hermes_state_repair's sidecar set.
`MEDIA:~/.hermes/state.db` was deliverable in default mode: the denylist
named individual credential files but not the SQLite stores that hold the
whole cross-session conversation history (and every secret ever pasted
into a chat), nor their WAL/SHM sidecars — whose mtime is always fresh, so
strict mode's recency trust would have handed them over too. Same gap for
legacy `sessions/` transcripts, the copied `browser-profile/` cookie store,
and named-board `kanban/boards/<slug>/kanban.db`, which sits right beside
the board attachments the gateway already allowlists.
Add them to the per-file denylist; the rest of HERMES_HOME (skills/, logs/,
memories/, ad-hoc artifacts) stays deliverable — the whole-tree deny from
#32090/#35939 is still deliberately not reintroduced.
Salvaged from #41071 by @pprism13 (root-level state.db/kanban.db entries);
named-board and sessions/browser-profile coverage added on top.
Re-applies the gateway compat removal byte-for-byte; see 92d0bd0d73 for the
full inventory (30 re-exports/aliases + 2 shim modules dropped, 3 shim-only
names re-removed, 24 callers + 34 test files repointed). No new changes.
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
Partial salvage of #100887 by @jwilson411. The restore-path half (raise
TranscriptReadError from load_transcript, fail the turn closed) already landed
via #100910; this keeps the complementary half: every slash-command handler and
platform helper that reads the transcript now catches TranscriptReadError and
tells the user the history exists but is unreadable, instead of letting the
exception reach the dispatch wrapper, which logs it and sends no reply.
(cherry picked from commit 2a132c903f51c8d21f6eea64ddeef688c9619a18, run.py/session.py
hunks dropped as already on main; notice-path tests replaced accordingly)
Follow-ups on the #101406 salvage (#101391):
- The issue's measured incident was a healthy connect followed by silent
polling loss; nothing republished for 11 h. `_schedule_polling_recovery`
(the single entry to the recovery ladder) now publishes "retrying" while
the adapter is running; `_record_polling_progress` already flips it back.
- `BasePlatformAdapter.send_path_degraded` property (default False) replaces
the runner's `getattr(adapter, "_send_path_degraded")` reach into a
plugin-private attribute; Telegram overrides it. `_mark_connected()` reads
the property instead of taking a `degraded=` kwarg, and `_mark_degraded()`
+ `DEGRADED_STATUS_MESSAGE` replace three copies of the same write/string.
- The startup connect stamp (sibling of the reconnect stamp) honours the
same flag.
- Tests: mid-session death publishes retrying; pre-connect death does not;
progress never flips while not running / fatal; runner reconnect stamp
honours the adapter flag.
connect() intentionally returns True when Telegram polling starts in
degraded mode (or a reconnect's require_progress=False skips the strict
readiness gate), so background recovery can retry without failing
gateway startup. But _mark_connected() published platform_state
"connected" unconditionally, and the reconnect watcher in gateway/run.py
stamped "connected" again right after -- so gateway_state.json was
indistinguishable from a healthy adapter for as long as recovery took
(observed ~11h on one seat).
_send_path_degraded already tracks exactly this (set at polling
generation start, cleared on the first confirmed getUpdates round-trip),
so:
- BasePlatformAdapter._mark_connected() takes a `degraded` flag and
publishes "retrying" instead of "connected" when set.
- TelegramAdapter passes its current _send_path_degraded into
_mark_connected().
- The gateway/run.py reconnect watcher checks the same flag before
overwriting the adapter's own status write.
- _record_polling_progress() republishes "connected" the moment
polling actually proves progress, so the state does not stay wedged
at "retrying" until the next disconnect.
Fixes#101391
Under `multiplex_profiles` the primary adapter's message handler is a
profile closure, so the Telegram inline-button gate (and the early
message prefilter) cannot recover the runner via `_message_handler.__self__`
and fell to env-only auth. #65589 made the gate prefer the injected
`_authorization_check`, but `_make_adapter_auth_check` built a bare
`(user_id, chat_type, chat_id)` source: never route-stamped, never
`is_bot`.
- `_make_adapter_auth_check`: for the shared primary adapter under
multiplex, mirror the inbound message path exactly — stamp the
`profile_routes` match so the routed profile's pairing store is
consulted, and authorize under the TRANSPORT home via
`_is_user_authorized_for_source` (same split as
`_make_default_profile_message_handler`, 2afed50863). A rejected route
fails closed like the ingress gate. Retain the receiving adapter as
`_transport_adapter_ref` so config.yaml policy reads stay on it.
Accept `is_bot` / `thread_id` keywords. (#86296)
- `BasePlatformAdapter._is_sender_authorized`: forward `is_bot` /
`thread_id` as keywords only when set, so legacy 3-positional callbacks
keep working.
- Telegram `_source_from_message_for_auth` carries `from_user.is_bot`;
the prefilter forwards it so `TELEGRAM_ALLOW_BOTS=mentions|all` is
honored at the early gate under multiplex. (#92840)
- Telegram `_should_pass_unauthorized_dm_for_pairing`: same `__self__`
introspection class — fall back to the injected `gateway_runner` and
the adapter's owner profile.
Fixes#86296Fixes#92840
Co-authored-by: PRATHAMESH75 <118293218+PRATHAMESH75@users.noreply.github.com>
Co-authored-by: Ahmett101 <297889955+Ahmett101@users.noreply.github.com>
Address hermes-sweeper review on #76487:
- Prefer hermes_profile from send metadata when pruning stale topic
bindings so profile_routes cannot delete the transport adapter's
namespace instead of the routed runtime's
- Namespace lobby/capability cooldowns and /topic off cleanup by
(profile, chat_id)
- Document profile_name PKs and scoped cleanup SQL in telegram.md
- Regression: primary-adapter stamp + routed metadata prune isolation
A multiplexed Hermes process (gateway.multiplex_profiles, unified
dashboard/TUI, or cron) serves several profiles at once, but terminal.*
resolved through process-global TERMINAL_* env vars bridged ONCE at
startup from the launch profile (gateway/run.py ~2700-2760) plus the
one-shot _ensure_terminal_env_bridged() guard. Every routed profile
therefore inherited the launch profile's backend, cwd, docker volumes,
SSH target and shared-container key: a local profile ran inside another
profile's docker sandbox (or a docker profile escaped to the host), and a
container labeled profile A carried profile B's RW bind mounts.
Fix: an authoritative per-profile terminal policy seam, mirroring
agent/secret_scope.py:
- tools/terminal_scope.py: ContextVar holding the routed profile's
COMPLETE effective TERMINAL_* policy (defined defaults <- profile .env
TERMINAL_* <- config.yaml terminal:). While bound, terminal_env()
resolves ONLY from it - an omitted key yields the defined default,
never os.environ. Unreadable/malformed policy installs a refusal
scope; terminal_tool / execute_code refuse instead of running under
ambient launch-process policy (fail closed).
- Installed at every in-process profile boundary: gateway
_profile_runtime_scope, tui_gateway session/build/turn scopes, cron
per-job fire. The unscoped single-process path is byte-identical.
- Every terminal.* consumer reads through the scope: terminal_tool
(_get_env_config, _resolve_container_task_id shared key, orphan
reaper lifetime, degraded mode), gateway/platforms/base.py docker
media translation (volumes, shared key, persistence), runtime_cwd /
agent_init / skill_utils / code_execution_tool / file_tools cwd
anchors, prompt_builder / browser_tool / env_probe backend checks,
gateway footer, @-refs and slash-command cwd. env_probe resolves the
backend in the caller's context, since the probe worker thread does
not inherit the ContextVar.
Salvage of #99225 onto current main: adds the three ambient reads the PR
missed (tools/file_tools.py TERMINAL_CWD, tools/browser_tool.py and
tools/env_probe.py TERMINAL_ENV; shape from #79117) and trims the test
module to the leak matrix driven through the real gateway boundary,
omitted-key defaults, refusal, and boundary reset.
Fixes#68559Fixes#94200Fixes#101132Fixes#95470
Co-authored-by: x7peeps <9640837+x7peeps@users.noreply.github.com>
Co-authored-by: Eva <239388517+100yenadmin@users.noreply.github.com>
Co-authored-by: ExitMaster <292490062+ExitMaster@users.noreply.github.com>
Every gateway/plugin platform adapter hard-coded aiohttp.ClientSession(trust_env=True)
(~20 sites), so a gateway launched by a Windows Scheduled Task that inherits a stale
HTTP_PROXY (Clash/V2Ray on 127.0.0.1:7890) looped on 'Cannot connect to host' with no
way to opt out short of NO_PROXY hacks per vendor host.
- gateway/platforms/base.py: gateway_trust_env() reads gateway.trust_env (default true);
resolve_proxy_url() skips generic HTTP(S)_PROXY/ALL_PROXY + macOS system-proxy
auto-detect when false (explicit per-platform vars still win).
- All aiohttp ClientSession sites in weixin, qqbot, matrix, line, wecom, slack, sms,
teams, google_chat now pass trust_env=gateway_trust_env(); mattermost + homeassistant
bare sessions gain the same kwarg (intent of #70119 / #56229).
- DEFAULT_CONFIG + cli-config.yaml.example + messaging docs.
- tests/gateway/test_gateway_trust_env.py: config flip + no-bare-literal sweep.
Reported-by: @ranlingfeng (#48820), @frontnopipe-cloud (#76309)
Co-authored-by: rcarrata <rcarratalasanchez@gmail.com>
Co-authored-by: Backroads4Me <TEDLANHAM@GMAIL.COM>
Localizing inbound relay media spends the agent's own Buzz credentials on
a URL chosen by the sender, so it must not run on the strength of the
adapter's local allow-list alone. Require the gateway's authorization
callback to return an explicit True before any `buzz media get` runs; a
denial, a missing callback, or a raising callback fails closed and leaves
the message text exactly as it arrived.
`_is_sender_authorized` previously wrapped the callback result in
`bool()`, so a truthy non-boolean (a status string, a sentinel) would
satisfy an `is True` gate's intent while bypassing its guarantee. Only
the literal booleans now propagate; anything else is "unknown", which the
existing Slack and Discord callers already treat as trust-unknown.
Reviewers asked for this boundary on the sibling inbound-media PRs
(#77734, #78051); it applies equally to the retrieval path in #75614,
which this change builds on.
/bg (formerly /background, which is retired) keeps the existing semantics:
spawn a fresh, independent agent session in the background.
/btw is now its own command matching the convention other harnesses use:
ask a quick side question ABOUT the current conversation without
interrupting it. A one-shot auxiliary LLM call (main model by default,
overridable via auxiliary.side_question.* in config.yaml) answers from a
read-only transcript snapshot — the live session's history, role
alternation, and prompt cache are untouched, and the current turn keeps
running.
Surfaces wired: CLI (inline mid-run dispatch), gateway (all messengers,
busy-dispatch table + idle dispatch, i18n across all 17 locales), TUI
(prompt.btw RPC + btw.complete event), Discord native slash, relay
command manifest, desktop exec routing, docs (EN + zh-Hans).
ctx.register_platform_handler(platform, factory) — the generic surface for
plugins to wire native handlers into any platform adapter at connect()
time. Factories receive (native, adapter): the platform's client/app
object (PTB Application, discord.py Bot, slack_bolt AsyncApp, Teams App,
DingTalkStreamClient, aiohttp web.Application) or None for adapters with
no separate native object.
- BasePlatformAdapter._wire_plugin_handlers(native): shared, isolated
invocation helper — a raising plugin cannot block a platform connect.
- All 27 connectable adapters call it: telegram/slack/teams/line/
api_server/msgraph_webhook wire before their dispatch tables freeze;
the rest hook at connect success.
- register_telegram_handler and get_telegram_handler_factories retained
as thin back-compat aliases over the telegram bucket.
- Source-invariant test guarantees every adapter with connect() keeps
calling the hook.
Teknium