Raw yaml.safe_load of config.yaml is only legal in owner modules; the
write-back round-trip uses the sanctioned raw primitive. Behavior-identical
(same file, same parse, {} on missing).
Follow-ups on the #101406 salvage (#101391):
- The issue's measured incident was a healthy connect followed by silent
polling loss; nothing republished for 11 h. `_schedule_polling_recovery`
(the single entry to the recovery ladder) now publishes "retrying" while
the adapter is running; `_record_polling_progress` already flips it back.
- `BasePlatformAdapter.send_path_degraded` property (default False) replaces
the runner's `getattr(adapter, "_send_path_degraded")` reach into a
plugin-private attribute; Telegram overrides it. `_mark_connected()` reads
the property instead of taking a `degraded=` kwarg, and `_mark_degraded()`
+ `DEGRADED_STATUS_MESSAGE` replace three copies of the same write/string.
- The startup connect stamp (sibling of the reconnect stamp) honours the
same flag.
- Tests: mid-session death publishes retrying; pre-connect death does not;
progress never flips while not running / fatal; runner reconnect stamp
honours the adapter flag.
connect() intentionally returns True when Telegram polling starts in
degraded mode (or a reconnect's require_progress=False skips the strict
readiness gate), so background recovery can retry without failing
gateway startup. But _mark_connected() published platform_state
"connected" unconditionally, and the reconnect watcher in gateway/run.py
stamped "connected" again right after -- so gateway_state.json was
indistinguishable from a healthy adapter for as long as recovery took
(observed ~11h on one seat).
_send_path_degraded already tracks exactly this (set at polling
generation start, cleared on the first confirmed getUpdates round-trip),
so:
- BasePlatformAdapter._mark_connected() takes a `degraded` flag and
publishes "retrying" instead of "connected" when set.
- TelegramAdapter passes its current _send_path_degraded into
_mark_connected().
- The gateway/run.py reconnect watcher checks the same flag before
overwriting the adapter's own status write.
- _record_polling_progress() republishes "connected" the moment
polling actually proves progress, so the state does not stay wedged
at "retrying" until the next disconnect.
Fixes#101391
tests/gateway/test_feishu_bot_admission.py builds FeishuAdapter via
object.__new__ and installs dedup state by hand (install_dedup_state), so
the asyncio.Lock added in __init__ was missing -> AttributeError in CI.
Resolve the lock through a getattr-with-create helper and add it to the
shared test helper.
Offloading the fsync to a worker thread (previous commit) lets two inbound
messages have two persists in flight. Without ordering, a slow first flush
lands after the second and the disk file loses the newer token/id/seen-id
(in-memory state stayed correct, so the loss only surfaced after restart).
The loop-side dict could also be mutated while the worker iterated it.
- weixin ContextTokenStore / discord non-conversational tracker: snapshot
the payload on the loop, hand it to the worker, serialize with a per-store
asyncio.Lock.
- feishu dedup: same asyncio.Lock around the offloaded persist (its snapshot
was already taken under _dedup_lock; the write was not).
- Regression tests: two concurrent persists with a slow first write must
leave the union on disk. All three fail on the previous commit.
atomic_json_write() calls os.fsync(), which blocks until the write
reaches stable storage. #83906 offloaded this in channel_directory.py,
gateway/run.py, and gateway/slash_commands.py, but the same blocking
pattern was still present on three platform-adapter message hot paths
that were never touched by that pass:
- plugins/platforms/feishu/adapter.py: _is_duplicate() persists the
dedup cache on every new inbound message (shared by both websocket
and webhook transports). The in-memory bookkeeping stays under
_dedup_lock; the fsync now runs off it via asyncio.to_thread so
concurrent callers on the ws-thread loop aren't blocked on disk I/O.
- gateway/platforms/weixin.py: ContextTokenStore.set() persists the
context-token cache on every inbound message carrying a
context_token, called from the async poll-loop handler
_process_message.
- plugins/platforms/discord/adapter.py: mark_many() persists the
non-conversational message-id tracker, called from both send() and
send_update_prompt() when a message is tagged nonconversational.
Each fix mirrors the existing asyncio.to_thread(atomic_json_write, ...)
pattern from #83906. Regression tests assert the persist call runs on
a different thread than the event loop, mirroring
test_directory_write_runs_off_event_loop_thread in
test_channel_directory.py.
Findings from the efficiency review pass on the two salvages, applied as one
small follow-up:
- copilot_auth: check the negative cache BEFORE taking the per-fingerprint
exchange lock. During the 60 s post-failure window, dashboard polls now
raise immediately instead of parking an executor thread behind the
in-flight holder (up to ~50 s) to learn the same answer. Test hangs
without the check (timeout 124), passes with it.
- buzz _localize_inbound_media: download_path.read_bytes() was still
evaluated on the loop as the argument to the offloaded cache call — up to
the 128 MiB inbound cap. Read off the loop too.
- test_list_credential_pool_keeps_loop_responsive: 0.5 s block / 0.25 s
threshold (2x margin) so runner descheduling cannot false-fail it while a
real regression still trips it.
The previous commit removed cache_media_bytes from teams/adapter.py's import
block but left one bare call at the Bot-Framework image path — a NameError
that the surrounding 'except Exception' swallowed, so every BF image
attachment was silently dropped (ruff F821). Route it through
cache_media_bytes_async like the file's two other sites.
Same bug class at the sites the sweep did not reach:
- buzz _download_attachment / _localize_inbound_media: cache_media_bytes
called from async methods → cache_media_bytes_async.
- photon _dispatch_inbound: _normalize_binary_payload (base64 decode +
cache write of possibly multi-MB payloads) ran on the loop → closure made
async, helper offloaded via asyncio.to_thread.
Tests: cover cache_media_bytes_async (thread-id + kwarg forwarding);
teams/buzz tests re-pointed at the async seam.
An external-process provider is an agent CLI Hermes drives over stdio rather
than an HTTP endpoint. Three things about it were spelled out for one vendor,
and each was a hard stop for any other:
* ``resolve_provider()`` gates on ``PROVIDER_REGISTRY``. Its auto-extend from
``providers/`` covered api-key providers only, so an external-process profile
never entered it and ``hermes -m <that provider>`` died with "Unknown
provider" before a client was ever built.
* ``resolve_runtime_provider()`` keyed the external-process branch on the
literal ``"copilot-acp"``, so anything else silently fell through to the
OpenRouter default instead of its own runtime.
* ``resolve_external_process_provider_credentials()`` hardcoded the binary
(``copilot``), the argv (``--acp --stdio``), the env var names and the
placeholder api_key — so a third-party provider would have been handed
another vendor's CLI.
Now the profile carries what only the provider knows — ``process_command``,
``process_args``, ``process_command_env_vars``, ``process_args_env_var`` — and
the three core paths key on ``auth_type == "external_process"`` instead of a
name. copilot-acp's values move into its profile verbatim, so
``HERMES_COPILOT_ACP_COMMAND`` / ``COPILOT_CLI_PATH`` /
``HERMES_COPILOT_ACP_ARGS`` and its ``copilot-acp`` api_key placeholder behave
exactly as before; the new tests assert that alongside the out-of-tree case at
every step.
The error for a missing binary now names the provider and its own env override
instead of telling every user to install GitHub Copilot CLI.
Co-Authored-By: Junie <junie@jetbrains.com>
``create_openai_client`` was a hardcoded if-ladder: copilot-acp builds an ACP
stdio shim, gemini builds a native client, everything else gets an
``openai.OpenAI``. There was no extension point, so a provider whose wire
protocol is not OpenAI-over-HTTP could only be added by editing this function —
which is exactly why an ACP provider cannot ship outside this tree today, even
though ``providers/__init__.py`` has discovered out-of-tree profiles from
``~/.hermes/plugins/model-providers/`` and pip entry points for a while.
``ProviderProfile.create_client(**client_kwargs)`` closes that gap. It returns
``None`` by default, so every provider that wants the standard client is
unaffected and the existing ladder still runs as the fallback. copilot-acp is
migrated onto it — its hardcoded branch is gone and its profile supplies the
client in three lines, which is the same three lines an external package writes.
Resolution goes by provider name first, then by ``base_url`` prefix, so a
runtime configured only by URL still reaches its profile — matching what the
replaced ``startswith("acp://copilot")`` branch did. A profile that raises is
logged and skipped: a third-party plugin can fail to provide a client, but it
cannot take the turn down.
Also replaces the two ``isinstance`` checks in ``agent/auxiliary_client.py``
that mean "this client is complete, do not wrap it" with capability flags the
client class declares — ``HERMES_SKIP_TRANSPORT_WRAP`` and
``HERMES_SKIP_ASYNC_WRAP``, mirroring ``SUPPORTS_HERMES_TOOL_CALLS`` in
``background_review.py``. Two in-tree consumers (the ACP shim and the Gemini
native client), an out-of-tree client is covered by the same declaration, and
the hot path no longer imports those modules just to type-test.
Co-Authored-By: Junie <junie@jetbrains.com>
Slots above gemini-3.7-flash (kept) in OPENROUTER_MODELS and
_PROVIDER_MODELS["nous"]; openrouter plugin fallback_models bumped
3.7 -> 3.8; model-catalog.json regenerated.
Verified live with test completions on both Nous Portal and OpenRouter
(model echo + billed). Same 1,048,576 window / 65,536 output / pricing
as 3.7-flash, so provider-agnostic metadata resolves via the existing
gemini entries and both routes bill live (official_models_api) — no
pricing snapshot needed.
Scoped to the two named providers: vertex/gemini/kilocode/gmi curated
lists, setup.py samples, and aux defaults untouched.
The multiplex gateway imports plugins/platforms/matrix/adapter.py once, so
the module-level _STORE_DIR/_CRYPTO_DB_PATH resolved against the root
HERMES_HOME for every profile: all bots' Olm identities landed in one
crypto.db and inbound E2EE failed with "no session found" (#89168).
connect() runs inside _profile_runtime_scope, so resolve the store dir
there via get_hermes_dir (honors the context-local HERMES_HOME) and cache
it on the instance -- diagnostics and error-log paths read outside the
scope then still report the store actually in use. Mirrors the
pairing-store fix (a6397c379).
Salvage of #89169 (per-call resolvers collapsed into one cached resolve;
dead `_CRYPTO_DB_PATH = None` alias dropped -- no external importers).
Also routes the last raw MATRIX_HOMESERVER read in check_matrix_requirements
through _startup_env_secret like its token/password neighbours (#69943).
Fixes#89168
Co-authored-by: Michael Short <18595461+mjshorty@users.noreply.github.com>
Same class as #100627's WECOM_BOT_ID: the one remaining raw os.getenv in
WeComAdapter.__init__ let a secondary multiplex profile pick up the
default profile's bridged websocket URL. Route it through
_get_scoped_secret; folded into the existing scoped-miss test.
SimplexAdapter.__init__ (auto_accept, group_allowed), the registry gates
check_requirements/validate_config/is_connected, _env_enablement and
_standalone_send all read SIMPLEX_* via raw os.getenv. Under
gateway.multiplex_profiles those paths run inside a secondary profile's
scope where os.environ holds the DEFAULT profile's YAML-to-env bridge
output -- so a secondary profile that never configured SimpleX was
auto-enabled on the default's daemon URL and inherited its group
allowlist / auto-accept setting.
Route every read through the module-local `_get_scoped_secret` wrapper
(get_secret; UnscopedSecretError -> os.getenv for the default profile,
which constructs unscoped) -- the same helper the IRC/ntfy/Photon/
Mattermost siblings use. Unlike the extra-only `_scoped_platform_setting`
shape proposed in #100241, this honors BOTH the secondary profile's own
.env (the scope) and its config.yaml extra, and needs no config.yaml
re-read in check_requirements.
Rewrite of #100241.
Co-authored-by: nftpoetrist <264138787+nftpoetrist@users.noreply.github.com>
WeComAdapter.__init__ read WECOM_BOT_ID via a raw os.getenv() call, while
the immediately adjacent line for WECOM_SECRET already used the module's
_get_scoped_secret() helper. Under gateway.multiplex_profiles, a secondary
profile's adapter is constructed inside a scoped context where os.environ
still holds the DEFAULT profile's env-bridge output -- so a secondary
profile's bot would silently connect using the default profile's bot_id
while (correctly) using its own secret, or vice versa on a scope miss.
Switch the bot_id read to _get_scoped_secret(), matching the sibling
_secret/_dm_policy/_group_policy/allow_from reads in the same __init__
that were already migrated in #76664/#93545. _standalone_send's
out-of-process fallback branch constructs a fresh WeComAdapter(pconfig)
and therefore inherits this fix automatically -- no separate change
needed there.
Adds two regression tests to the existing TestWeComAdapterAuthzScope
class (already covering dm_policy/allow_from scoping per #93522),
mirroring its established fixture/assertion style. Mutation-verified:
both fail against the pre-fix code (asserting the default profile's
bot_id leaks into a secondary profile's scope) and pass with the fix.