Breaks the two import cycles that forced Protocol stand-ins in the F821 sweep, so the two
sites now name the real types.
gateway/platforms/event.py (new leaf): MessageType, ProcessingOutcome, MessageEvent moved
out of base.py verbatim. Their only dependency is gateway.session.SessionSource; base.py
imported helpers.py at module level, so helpers could not name MessageEvent. Now
TextBatchAggregator is typed by the real MessageEvent. 249 importers repointed
(`from gateway.platforms.base import` -> `.event`, preserving each import's layout);
gateway.platforms.__init__ re-exports from .event. The three revert-scheduled PLUGIN-COMPAT
pointers that named these symbols (gateway.slash_commands → MessageType, dingtalk → MessageType,
photon → ProcessingOutcome) and their COMPAT_MANIFEST rows now target gateway.platforms.event.
Docs updated: ADDING_A_PLATFORM.md, adding-platform-adapters.md (en + zh-Hans).
tools/mcp_tool_sampling.py: ElicitationHandler no longer holds a back-reference to its
MCPServerTask (mcp_tool imports sampling, so the task type cannot be named there). It only
ever read owner._pending_call_context, so it takes `call_context: Callable[[], Context | None]`
and MCPServerTask passes `lambda: self._pending_call_context`. The consent call is one
`functools.partial`, run directly or inside the captured Context.
ty on the 11 touched production files vs origin/main: 0 new diagnostics, 14 resolved.
(The one `source: SessionSource = None` diagnostic moves with the class; typing it Optional
exposes ~60 unguarded call sites — separate follow-up.)
Tests: tests/gateway + tests/plugins + tests/tools + touched files, 18,235 passed; the 31
failures reproduce identically on origin/main (macOS /private/tmp, systemd socket,
long-path fixtures, live-service tests).
GitLab (lib/gitlab/web_hooks.rb, app/services/web_hook_service.rb) sends
webhook-id and webhook-timestamp on EVERY delivery and adds webhook-signature
only when a signing token is configured. Selecting the HMAC path on any
webhook-* header would 401 every legacy X-Gitlab-Token install the moment it
upgrades to GitLab 19, so only the signature header selects the path; id and
timestamp then travel with it and the validator still fails closed when either
is missing. svix-* keeps its existing any-header selection.
Tests trimmed to the invariant bar: one parameterized contract (whsec_ and raw
secrets accept; wrong secret, tampered body and stale timestamp reject) plus the
GitLab legacy-token coexistence contract. evals/webhook_auth/standard_webhooks_ab.py
drives a real aiohttp WebhookAdapter on a dedicated loopback port with real
signed requests for before/after evidence.
Related: #47849 (HwangJohn, cherry-picked here), #92024 (earlier salvage of
#47849), #102080 and #103167 (same alias fix, same target).
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
A `/p/<profile>/webhooks/<route>` request resolved the profile from the URL
but ran the route script, prompt render and `skills:` lookup with no
profile scope — the runner only enters `_profile_runtime_scope` later,
around `handle_message` — so routed webhooks loaded the launch (default)
profile's skills and logged "Skill not found" for the routed profile's own.
- gateway/platforms/webhook.py: add `_profile_scope(profile)` (nullcontext
when no prefix was resolved; `_profile_runtime_scope(get_profile_dir(p))`
otherwise, same helper the runner uses) and wrap the script / render /
skill-injection block in it. Bare routes are unchanged.
- agent/skill_commands.py: `scan_skill_commands` scanned the import-time
`SKILLS_DIR` (frozen to the launch home), so even a correctly scoped call
listed default's skills; the #88023 home-keyed cache alone could not fix
that. Use the call-time `_skills_dir()` there and at the two other
SKILLS_DIR-relative sites in the module.
- agent/skill_utils.py: `normalize_skill_lookup_name` used the same frozen
root, so a routed profile's absolute skill_dir was rejected by
`skill_view` ("must be a relative path within the skills directory").
Resolve against `_skills_dir()` — the root `skill_view` itself enforces.
Fixes#67277
Co-authored-by: Juani Lezcano <tky.juani@gmail.com>
Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
Pattern-A architectural fix: blocking calls inside async functions freeze
the gateway/uvicorn event loop for every adapter, timer, and health check.
Known incidents: 17-minute getaddrinfo freeze (#91912 class), 10s restart
freeze in start_gateway (#36163).
Fixes at the four unguarded core sites:
- gateway/platforms/webhook.py: `gh pr comment` subprocess (30s timeout)
now runs via asyncio.to_thread — a webhook delivery no longer freezes
every other platform for the duration of a network call.
- gateway/run.py start_gateway --replace: two time.sleep() waits (10s +
5s worst case) become await asyncio.sleep() (re-lands #36163 at current
line numbers, credit AhmetArif0).
- gateway/slash_commands.py /save: session render + file write move off
the loop (scales with transcript size).
- hermes_cli/web_server.py voice TTS: multi-MB audio file read + unlink
move off the loop.
Prevention gate so the bug class cannot re-enter:
- pyproject.toml [tool.ruff.lint] select gains ASYNC210/220/221/251
(blocking HTTP / Popen / subprocess.run / time.sleep in async def).
These run in the existing blocking `ruff check .` CI job.
- Frozen ratchet baseline in per-file-ignores for the remaining legacy
sites (detached restart watchers; router sweep in flight via #84376;
two platform adapters), each documented for burn-down. New files or
new violations fail CI immediately.
- tests/** keeps the relaxation (deliberate sleeps in fixtures).
Verification:
- ruff check . green on this branch; sabotage file with time.sleep +
subprocess.run in async def fails the gate with 2 errors.
- New behavioral test test_webhook_offloop_delivery.py asserts loop
liveness DURING delivery (ticker coroutine): 1 tick on the old
blocking code (fails), 21 ticks off-loop (passes).
- 50 webhook/replace gateway tests + 26 save/export tests pass.
Co-authored-by: AhmetArif0 <147827411+AhmetArif0@users.noreply.github.com>
ctx.register_platform_handler(platform, factory) — the generic surface for
plugins to wire native handlers into any platform adapter at connect()
time. Factories receive (native, adapter): the platform's client/app
object (PTB Application, discord.py Bot, slack_bolt AsyncApp, Teams App,
DingTalkStreamClient, aiohttp web.Application) or None for adapters with
no separate native object.
- BasePlatformAdapter._wire_plugin_handlers(native): shared, isolated
invocation helper — a raising plugin cannot block a platform connect.
- All 27 connectable adapters call it: telegram/slack/teams/line/
api_server/msgraph_webhook wire before their dispatch tables freeze;
the rest hook at connect success.
- register_telegram_handler and get_telegram_handler_factories retained
as thin back-compat aliases over the telegram bucket.
- Source-invariant test guarantees every adapter with connect() keeps
calling the hook.
A /p/<profile>/ URL prefix on a gateway with multiplex_profiles off was
silently ignored: the request was handled as the gateway-owning profile,
so /p/lokaj/v1/toolsets reported the OWNER's platform_toolsets (and every
other profile-owned config read — skills, capabilities, model options,
agent-run toolset resolution — resolved from the owner too). That is the
exact repro in #91583 defect 2: enabling computer_use with
'hermes -p lokaj tools enable computer_use --platform api_server' showed
enabled in lokaj's config while /p/lokaj/v1/toolsets stayed false, and
enabling it on the owner profile flipped it true.
Per-profile capability isolation is the intended design (ruling on
a different profile's config. Multiplexed gateways were already correct —
the profile-prefix middleware enters _profile_runtime_scope and every
canonical config loader honors the HERMES_HOME override contextvar
(verified empirically for load_config, get_config_path and
_load_gateway_config) — the leak was only the non-multiplex fallthrough.
Fix at the one seam both adapters share: _resolve_request_profile now
rejects (404) a prefix naming any profile other than the one the gateway
actually serves. A self-referential prefix (/p/default/ on the default
gateway, /p/lokaj/ on a gateway launched for lokaj) still falls through
so existing well-formed clients keep working. Same change in the webhook
adapter, which had the identical fallthrough. New shared helper
hermes_cli.profiles.profile_matches_home does the home comparison,
fail-closed.
Tests: tests/gateway/test_multiplex_toolsets_profile_isolation.py —
E2E-style with two real profile homes + config.yamls under a temp
HERMES_HOME, real aiohttp routing through the profile-prefix middleware:
per-profile /p/<x>/v1/toolsets isolation for both owner and secondary
(the #91583 repro asserts computer_use true under /p/lokaj only),
cross-profile key rejection, and the fail-closed non-multiplex prefix
for both adapters. Sabotage-verified: reverting the adapter change fails
the 3 fail-closed tests.
Fixes#91583 (defect 2). Repro and live validation by @kubaboski.
Webhook agent runs default to the constrained hermes-webhook toolset
(web/vision/clarify) because payloads can carry untrusted third-party
content. That default is right for public webhooks but wrong for trusted
local pushes (e.g. an OOM monitor daemon that needs the agent to run
ps/free/py-spy): the only workaround was widening platform_toolsets.webhook,
which elevates EVERY webhook route at once.
This adds a 'toolsets' key on individual webhook route configs (static
routes in config.yaml and dynamic subscriptions in
webhook_subscriptions.json) that replaces the platform-level resolution
for that route only:
- BasePlatformAdapter.toolsets_for_source(): per-source override hook,
default None (no behavior change for any other platform).
- WebhookAdapter.toolsets_for_source(): maps the session chat_id
(webhook:{route}:{delivery_id}) back to its route config and returns
the route's toolsets list.
- GatewayRunner._resolve_enabled_toolsets_for_source(): shared resolver
used by both agent-run call sites; validates the override through the
SAME _get_platform_tools path as platform config, so unknown names and
platform-restricted toolsets (e.g. discord_admin) are dropped rather
than trusted.
Deliberately NOT exposed via 'hermes webhook subscribe': granting elevated
tools is a manual config edit only, so an agent-created subscription
cannot self-grant terminal at runtime.
Cache-safe: the toolset list is resolved before agent construction and is
constant for a route, so the per-session agent signature and frozen system
prompt are unaffected mid-conversation.
Follow-up to the salvaged #71756: instead of webhook importing cron's
private _is_cron_silence_response, the loose autonomous-lane matcher now
lives in gateway/response_filters.py as is_autonomous_silence_response,
sharing LIVE_GATEWAY_SILENT_MARKERS with the interactive exact-marker
rule so the marker sets can never drift. Cron and webhook both delegate
to it. Interactive gateway behavior unchanged.
A webhook route that answered `[SILENT]` still delivered, whenever the model
added a sentence saying why it was staying quiet:
[SILENT]
The new inbound was the same email quoted back a second time, on a ticket
we already answered. Nothing new to reply to, so I closed it.
Webhook subscription prompts tell the agent to answer `[SILENT]` on a tick that
produced no story — a duplicate inbound, a stand-down because a sibling lane
already replied, a routine close. Nobody is waiting on the other end of a
webhook, so a "nothing happened" message has no reader.
Delivery went through the live gateway's `is_intentional_silence_response`,
which requires the response to be EXACTLY a marker. That rule is right for an
interactive chat: swallowing a real answer because it opens with a marker is
much worse than showing a stray marker. It is the wrong trade for an autonomous
lane, where a leaked non-story is a pointless notification on every tick and
models reliably append the explanation that flips the check back to "deliver".
Cron already resolved this the other way — `cron/scheduler.py` treats a marker
on its own first or last line as silence — so the two autonomous lanes
disagreed while the interactive path was fine.
Suppress in `WebhookAdapter.send`, before the deliver-type switch, so every
route (log, github_comment, cross-platform) behaves the same. Reuses cron's
`_is_cron_silence_response` rather than restating the rule, so the two lanes
cannot drift; prose that merely mentions a marker mid-sentence still delivers.
The interactive gateway path is untouched.
Tests: six cases in tests/gateway/test_webhook_adapter.py — bare marker,
marker + trailing prose (the reported shape), marker on the last line, a real
report, a report quoting a marker mid-sentence, and a `log` route. Verified
red-first: with the suppression removed the three silence cases fail
("Expected send to not have been awaited") while the three delivery cases still
pass, so the tests assert the fix rather than the framework.
On Windows with Chinese locale (GBK), subprocess.run(text=True) without
explicit encoding causes UnicodeDecodeError crashes. This fix adds
encoding='utf-8', errors='replace' to all subprocess.run() and
subprocess.Popen() calls that use text=True across 76 non-test Python files.
Fixes#53428 (master tracker for Windows GBK locale crash).
Note: credential_pool.py and electron changes excluded per reviewer request —
those will be submitted as separate focused PRs.
Unifies the two gateway subsystems that were fighting each other: the
'never lose a session' recovery machinery (#54878 stale-route self-heal,
find_latest_gateway_session_for_peer reopening agent_close/ws_orphan_reap
rows) and the session reset/expiry machinery (expiry watcher, /new,
/resume, resume_pending freshness gate).
The unified contract:
- INTENTIONAL boundaries (expiry finalization, auto-reset, /new,
/resume switch) are recorded durably via promote_to_session_reset(),
which upgrades accidental recoverable end_reasons (agent_close,
ws_orphan_reap) to the explicit boundary while preserving other
explicit reasons (compression, etc.). Recovery then correctly refuses
to resurrect them.
- ACCIDENTAL ends (crash, cleanup bug, mistaken reaper) stay
recoverable — genuine crash recovery is untouched.
On top of the cherry-picked contributor commits:
- promote_to_session_reset widened to ws_orphan_reap + parameterized
reason so auto-reset paths stay auditable (idle/daily/suspended/
resume_pending_expired) (#61220, #61993, #63539)
- get_or_create_session auto-reset, reset_session (/new), and
switch_session (/resume) all write through the promote path — the
first-reason-wins end_session no-op could previously leave a reset
session resurrectable behind a stale agent_close row (#61993)
- resume_pending freshness gate now honors session_reset.mode=none:
explicit opt-out of automatic resets also opts out of the zombie
gate (#61052)
- resume recovery note extracted to build_resume_recovery_note() and
made adapter-aware via a new interactive_resume capability flag:
webhook/api_server auto-resume turns now CONTINUE the interrupted
task instead of emitting an unanswerable 'session restored'
acknowledgement that abandoned the work (#57056)
- tests updated to call the real note builder instead of mirroring it
E2E-validated against a real SessionDB + SessionStore in a temp
HERMES_HOME: expiry->agent_close->no-resurrection, /new promote,
crash recovery preserved, mode=none opt-out, routing-table flag sync.
_validate_signature backs the public webhook receiver. It compared each
attacker-supplied signature/token header (GitHub X-Hub-Signature-256,
GitLab X-Gitlab-Token, generic X-Webhook-Signature / -V2, and the Svix v1
header) against a computed hex/base64 digest with hmac.compare_digest on
two str values. compare_digest raises TypeError on a str containing
non-ASCII characters, and the header is raw client input on an
unauthenticated endpoint — so any internet client could POST a single
non-ASCII byte in the signature header and raise out of the handler,
returning a 500 instead of a clean 401. Fail-closed, but an on-demand
crash of the request path.
Route all five comparisons through a small _hmac_str_equal() helper that
encodes both sides to UTF-8 bytes before the constant-time compare
(compare_digest has no ASCII restriction on bytes). Semantics are
unchanged for valid signatures; a hostile non-ASCII header now fails
closed with a rejection instead of raising.
Adds regression tests: non-ASCII GitHub/GitLab/generic/V2 signature
headers return False (no raise), and a non-ASCII configured secret still
matches its exact token value.
Also maps drexux0@gmail.com in scripts/release.py AUTHOR_MAP.
Secondary profiles under gateway multiplex keep tokens/allowlists in
profile secret_scope, not process os.environ. Auth and Slack were still
reading os.getenv, so Slack on a secondary profile failed allowlist and
socket mode. Webhook deliver also only looked at default adapters.
- Prefer get_secret for allowlists / allow-all flags (authz_mixin)
- Slack app token + allowlist via secret_scope with getenv fallback
- Wrap secondary profile message handlers in _profile_runtime_scope
before auth runs
- Resolve home-channel env from secret_scope / PlatformConfig
- Webhook deliver falls back to _profile_adapters for target platform
- Template key event_type for webhook prompts
On Linux, SO_REUSEADDR only allows rebinding past TIME_WAIT (a second
live listener would need SO_REUSEPORT, which we never set), so
disabling it bought no protection there while making a quick gateway
restart fail to rebind for up to ~60s. Keep the BSD silent-split guard
on darwin, default semantics elsewhere.
E2E verified: dual-stack v4+v6 bind on one port, immediate rebind
after disconnect, and live-listener conflict still rejected.
Disable address reuse so an existing family-specific listener cannot silently split traffic with the webhook server. Normalize wildcard bind hosts for local CLI URLs and align setup documentation with the dual-stack default.
The webhook adapter defaulted to host='0.0.0.0' — IPv4 only. On Fly.io
hosted agents the edge router (hermes-agent-router) reverse-proxies public
webhook traffic to <app>.internal:8644 over 6PN, Fly's private network,
which is IPv6-only (.internal resolves to an fdaa:… address). An IPv4-only
listener is unreachable there, so public webhook POSTs to
https://<agent>.agents.nousresearch.com/webhooks/<route> never landed on
the adapter — the router's dial was refused.
Fix: DEFAULT_HOST = None, which makes aiohttp/asyncio create_server bind
BOTH address families. '::' is NOT a valid substitute: on hosts where the
kernel sets bindv6only=1 (verified on Fly machines) it yields an IPv6-only
socket, breaking the IPv4 loopback /health check and the AF_INET
port-conflict probe in connect(). None binds per-family regardless of the
sysctl. An explicit empty-string/null host in config now also normalises to
None (dual-stack) rather than an invalid host=''. Users can still pin a
specific host via platforms.webhook.extra.host.
Validated live on a Fly staging agent: with this default and no config
override, the adapter binds both v4 and v6 (127.0.0.1:8644 and [::1]:8644
both answer), and a public signed webhook POST through the router returns
202 (valid sig) / 401 (bad sig) instead of the router's 502.
Tests: new TestDualStackBind asserts the None default, config resolution
(missing/empty→None, pinned preserved), and a real dual-stack bind opens
both AF_INET and AF_INET6 listeners. Red-proof: these fail on the old
'0.0.0.0' default.
- run_route_script shells out with subprocess.run (up to 30s timeout); wrap
the call in asyncio.to_thread so a slow script can't stall every other
webhook and gateway task on the loop.
- scripts/release.py: map grace@weeb.onl -> evelynburger for the salvaged
contributor commit.
- warn once per route instead of on every request (busy senders would
spam the log)
- document X-Webhook-Signature-V2 / X-Webhook-Timestamp in the webhooks
user guide
Follow-ups for salvaged #58461.
The webhook adapter enforced max_body_bytes only via the Content-Length
header; a Transfer-Encoding: chunked request (content_length=None) or a
spoofed small Content-Length bypassed the cap entirely and read the full
body (bounded only by aiohttp's implicit 1 MiB default, above any
operator-configured smaller limit).
- web.Application(client_max_size=max_body_bytes): aiohttp enforces the
cap on every read path, chunked included
- catch HTTPRequestEntityTooLarge -> 413 (was swallowed into generic 400)
- post-read length re-check as defense in depth
- chunked-upload regression test
Manual port of PR #3955 by @Gutslabs onto current main (handler had
been restructured since); authorship preserved.
The merged webhook session-close fix (#57370, salvaging #57322) wrapped
handle_message in a try/finally — but BasePlatformAdapter.handle_message
is fire-and-forget: it spawns _process_message_background and returns
before the agent run starts. The finally-close therefore ran BEFORE
get_or_create_session created the session row, found no session_id, and
silently no-op'd — the ghost-session leak persisted on the real path.
(The shipped test masked this by stubbing handle_message with a fake
that created the row synchronously.)
Move the close to an on_processing_complete override — the lifecycle
hook the base class fires at the TRUE end of the run, on the success,
failure, and cancellation paths alike. Empirically verified through the
real fire-and-forget pipeline: before, ended_at stayed NULL; after,
ended_at is set with end_reason=webhook_complete and the row is
prunable.
Tests now stub only the runner-side _message_handler (the seam the live
gateway injects) so handle_message / _process_message_background /
on_processing_complete all run for real; adds an AsyncSessionDB-facade
coverage test for the coroutine-await branch.
Replace the webhook delivery-close path's direct reach into private
SessionStore._entries (which also bypassed the store lock) with a public,
lock-held peek_session_id(session_key) accessor. Mirrors the existing
lookup_by_session_id inverse helper. Keeps a getattr fallback for older
stores / test doubles. Adds a unit test for the accessor.
Webhook deliveries created a unique one-shot session (delivery_id baked into
the session key at gateway/platforms/webhook.py:668) but the adapter fired
handle_message via asyncio.create_task WITHOUT ever ending the session
(webhook.py:713, pre-fix). Nothing else closes it: the gateway caches/expires
the agent per session_key but never calls end_session for the webhook path,
and _end_session_on_close teardown doesn't run for these fire-and-forget tasks.
SessionDB.prune_sessions (hermes_state.py:4965) only deletes rows WHERE
ended_at IS NOT NULL. So every webhook session stayed with ended_at NULL ->
unprunable -> unbounded state.db growth. This was the primary driver of the
SQLite lock-contention gateway outage.
Fix: wrap the delivery in _run_delivery_and_close, which awaits
handle_message and then (in finally, so failures still reap) calls
_end_webhook_session -> SessionDB.end_session(session_id, 'webhook_complete').
This mirrors how cron closes its session with 'cron_complete'
(cron/scheduler.py:3065). end_session is first-reason-wins and no-ops on an
already-ended row, so it never clobbers a compression/agent_close reason.
Adds tests/gateway/test_webhook_session_close.py asserting the invariant
(a completed webhook session has ended_at set + is prunable), including the
error-path case, against a real SessionStore + SessionDB.
Two of the three fixes from PR #6660 (the cli.py reopen_session change is
moot — that raw _conn.execute reopen block no longer exists on main).
- gateway/run.py: stop sending raw type(e).__name__ and str(e)[:300] to
end users on chat platforms. Exception text from LLM providers can leak
API URLs, file paths, and partial credentials. Return a generic message;
keep curated status hints for known HTTP codes; full detail stays in logs.
- gateway/platforms/webhook.py: validate pr_number (positive int) and repo
(owner/name regex) before passing to the 'gh pr comment' subprocess.
Payload-controlled values could otherwise inject gh flags (--help, a
different --repo). List-form subprocess means this is arg injection, not
shell injection, but validation is still correct.
Co-authored-by: aaronagent <1115117931@qq.com>
Follow-up to #53791 addressing review feedback: the footgun checker treated
capture_output=/stdout=/stderr=/check_output as proof a subprocess can't pop a
Windows console. That invariant is false — stream redirection controls where a
child's output goes, not whether a console is allocated. From a console-less
parent (Desktop/Electron, pythonw.exe, detached gateway/cron) a console-subsystem
child still flashes a window even when fully captured.
- check-windows-footguns.py: capture/redirect/check_output is no longer a blanket
safe-pass. Added _WINDOWS_FLASHING_PROGRAMS (git/gh/npm/node/python/uv/ffmpeg/
docker/powershell/…); calls to those are flagged even when captured. Non-flashing
programs keep the capture exemption (no 271-site noise). _subprocess_compat.run/
popen calls are inherently safe (wrapper injects CREATE_NO_WINDOW).
- Routed the 35 genuine flashing git/gh/npm/uv/ffmpeg/docker spawns through the
_subprocess_compat.run/popen chokepoint (Brooklyn's wrapper from #53810) — the
durable fix, not per-site annotations. cmd.exe /c start stays # ok (intentional).
- Updated tests + CONTRIBUTING.md rule #17 to the corrected invariant.
After a prolonged outage the in-process network-error ladder escalates to
fatal and GatewayRunner._platform_reconnect_watcher rebuilds a fresh adapter
that reconnects through the bootstrap path. That path called
start_polling(drop_pending_updates=True), discarding every update Telegram
queued during the outage — all messages sent while the bot was down were
silently lost. The in-process ladder and 409-conflict handler already passed
drop_pending_updates=False; only bootstrap did not distinguish a cold first
boot from a reconnect.
Thread an is_reconnect signal from the watcher through
_connect_adapter_with_timeout into adapter.connect(). The base
BasePlatformAdapter.connect() gains a keyword-only is_reconnect=False so every
adapter inherits a tolerant signature (no per-platform breakage when the
runner forwards the kwarg). Telegram translates is_reconnect into
drop_pending_updates=not is_reconnect on both the polling and webhook bootstrap
calls. Cold boot still drops the stale queue; a watcher reconnect preserves it.
Fixes#46621.
Co-authored-by: annguyenNous <annguyen@nousresearch.com>
Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
Co-authored-by: Kewe63 <Kewe63@users.noreply.github.com>
kshitijk4poor