Salvage of #48637 (Fixes#48628). On a NixOS-style install the venv's
site-packages lives in the read-only store, so ensure()'s
uv -> pip -> ensurepip ladder spends ~15s bootstrapping ensurepip only
to fail against a target it can never write. Fail fast with an
actionable message pointing at the system package manager.
Retargeted onto current main (the PR's base predates the durable-target
subsystem by ~8.1K commits) with two corrections to the original:
- Gate on _lazy_install_target() is None. The container deployment sets
HERMES_MANAGED=true AND HERMES_LAZY_INSTALL_TARGET (a writable
volume); the original guard would have blocked installs that path
legitimately satisfies, breaking the NixOS-container mode.
- Reason string starts with 'unsupported ' because
refresh_active_features classifies FeatureUnavailable by that prefix;
the original wording made 'hermes update' report a hard failure
instead of a skip.
Placed after _unsupported_feature_reason so a platform-specific reason
(more actionable) wins, and so ensure() agrees with
refresh_active_features, which pre-checks that same function.
Extracted from #54314 (@flag0x369), re-derived onto current main: macOS
ships bash 3.2 as /bin/bash, which lacks $BASHPID entirely — the
variable expands to empty string, collapsing every concurrent writer's
'unique' temp path onto the same file (torn snapshot writes under
concurrency). mktemp allocates per-writer unique paths portably.
Live-verified: /bin/bash -c 'echo $BASHPID' prints empty on this box.
The workdir allowlist regex was ASCII-only, so perfectly normal
non-ASCII workdirs (Chinese Obsidian vault paths, accented dirnames)
were rejected with 'disallowed character'. Replace the regex with a
per-character check that accepts Unicode letters/digits (str.isalnum)
plus the same safe ASCII punctuation set, while still rejecting shell
metacharacters, control characters (newlines/tabs), and NUL.
Salvaged from PR #54314.
Co-authored-by: kshitij <82637225+kshitijk4poor@users.noreply.github.com>
Recalled session messages can carry raw ANSI escape sequences (e.g.
archived terminal output), which then re-enter the model's context.
Strip them in _shape_message before content is truncated/returned,
reusing tools.ansi_strip.strip_ansi.
Re-applied onto current main (the original hunk predates the
max_content_len truncation in _shape_message; stripping happens on the
raw content before truncation so escape bytes never count against the
budget). Extracted from #40276.
The picker path fetches the Copilot /models catalog multiple times per
process (list_authenticated_providers -> provider_model_ids ->
_fetch_github_models, plus get_copilot_model_context / normalize
helpers). Cache the filtered catalog at module level with a short TTL
so repeated picker opens do not pay a TLS handshake each time.
Fold-fixes on top of the original patch:
- key the cache by api_key so a mid-process credential swap never
serves the previous account's catalog
- use time.monotonic() so wall-clock adjustments cannot extend the TTL
- deep-copy on store/serve so callers cannot mutate cached entries
- tests updated to patch _urlopen_model_catalog_request (main routes
catalog fetches through open_credentialed_url now), plus TTL-expiry
and credential-change coverage
Extracted from #40276.
Two more call-shape-pinning tests (cold tile resume, default-profile
resume) assert session.resume's exact params; the delegate passes
omit_messages: true like every other Desktop resume call site.
Swept all 5 desktop test files that reference session.resume/activate:
380 of 381 files green (the one failure is a pre-existing locale-
dependent number-grouping test that fails identically on clean main).
Salvage of #69926: omit_messages support ported from the PR's
tui_gateway/server.py base onto the post-split methods_session.py
layout. When a Desktop client passes omit_messages=true on
session.resume / session.activate, the RPC returns messages: [] with
messages_omitted: true and an accurate message_count, skipping the
potentially multi-megabyte compression-lineage serialization over the
WebSocket; Desktop hydrates the transcript via the authenticated REST
route in parallel.
The PR's bundled cron-outputs endpoint and codex quiet-timeout bump
were dropped from this salvage as unrelated (invited back separately).
mergePetInfoMeta now returns the same object reference when all fields
match, and callers skip setPetInfo on reference equality. Without this,
every 15s poll and window-focus refetch fired a nanostores set with a
new-allocated object, triggering a React re-render of FloatingPet even
when nothing changed — a regression from the old samePetRevision guard
which returned without calling setPetInfo.
mark_job_run popped a finite one-shot from jobs.json the moment its
repeat limit was reached and returned early — discarding the
last_status / last_error / last_delivery_error it had just written.
Every finished one-shot vanished from `cronjob action=list` with no
inspectable record, and a delivery failure (agent succeeded, platform
send failed) was silently thrown away with it.
Changes:
- mark_job_run now retires a limit-reached one-shot as a terminal
record (state="completed", enabled=False, next_run_at=None) —
mirroring the existing next_run_at-is-None terminal branch — so the
final status and any delivery error persist and surface in the
cronjob tool's list output (which already emits last_delivery_error
and defaults to include_disabled=True).
- claim_dispatch's stale-job cleanup marks already-ran jobs completed
instead of popping them; genuinely wedged claims (last_run_at never
written) are still removed with the operator-visible diagnostic.
- Retention sweep in the due scan prunes completed one-shot records
older than cron.completed_retention_days (default 7; non-positive
disables) so jobs.json cannot grow unboundedly. Recurring jobs and
non-terminal one-shots are never candidates.
Tests: completion retains record + delivery error, list surfaces it,
completed jobs never re-dispatch, sweep prunes old / keeps recent /
ignores recurring / honors the disable knob; recurring lifecycle
unchanged.
A raw newline in the _CMDPOS start-position class made ANY multi-line
quoted argument look like a command boundary, so hermes send message
bodies, multi-line git commit -m messages, and heredoc text that merely
mentioned dangerous command names tripped the unconditional hardline
blocklist and could not run at all.
Mask newlines inside single/double quotes (detection-only, mirroring the
quote tracking in _iter_shell_command_starts) before building detection
variants. Real threats keep blocking: unquoted newlines stay command
separators, command substitutions inside quotes still anchor, and
_mark_command_starts still re-inserts newlines at genuine quote-aware
command starts. Masking runs on the RAW command before normalization,
which strips escapes and would otherwise corrupt quote state.
Regression tests cover both directions: multi-line quoted data passes
(hermes send, git commit -m, heredocs); bare/chained/substituted
shutdown-class and rm-floor commands still block.
Round-2 A/B (gpt-4o-mini, 6 reps) showed two passages could not survive
paraphrase: the DO-NOT-USE list needs the arrow-list shape with the
'no reasoning needed' qualifier (prose form regressed mechanical-work
routing 6/6->1/6), and the self-report rule needs the concrete
'claiming uploaded successfully may be wrong' framing (without it,
side-effect verification regressed 6/6->2/6). With both restored:
30/42 vs 30/42 on gpt-4o-mini and intent-parity on claude-haiku-4.5.
Final size: 1,900 chars (from 3,963).
The top-level delegate_task description repeated content the model already
receives through parameter descriptions: the concurrency limit (tasks param),
the full nesting clause (role param), context-passing guidance (goal/context
params), and background semantics (background param). Every API call paid for
the duplication (~4,000 chars).
The description now carries only what exists nowhere else in the schema:
use/don't-use routing (execute_code, cronjob), the no-poll rule, the
non-durability warning, the self-report verification contract with concrete
verbs, the language-passing example, the leaf blocked-tool list, and model
inheritance. 3,963 -> 1,704 chars (~570 tokens saved per API call), and the
top-level text is now static (dynamic limits flow only through the two param
descriptions, which are already rebuilt per get_definitions() call).
A/B benchmark across 4 models (gpt-4o, gpt-4o-mini, claude-haiku-4.5,
llama-3.3-70b) showed the naive compaction in PR #72813 regressed weaker
models on exactly the passages it cut (side-effect verification 8/8->0/8 on
gpt-4o-mini; language passing 3/3->0/3 on haiku-4.5). This version keeps
those benchmark-sensitive hooks verbatim.
Tests pin the contracts at keyword level (not prose-literal) plus a size
ceiling, and verify dynamic limits still reach the model via the tasks/role
param descriptions.
Refs #72737, supersedes the delegate_task half of PR #72813.
Two consumer tests assumed a send completes synchronously within the
handler turn: the continuation-drain test polled on handler-call count
then immediately asserted on adapter.sent, and the split-brain heal
test drained with bare zero-delay yields. With ledger calls hopping to
worker threads around each send, the reply can land microseconds after
those checks. Poll for the actual sends with a bounded 2s window —
same invariants, scheduling-robust.
CI slices failed the offload tests with 0.5s witness timeouts: on a
loaded shared runner the event loop thread can take >0.5s to get
scheduled even when NOT blocked, making the probe report a false
positive. A genuinely blocked loop can never set the progress event at
any timeout (the witness coroutine can't run at all), so 5s only
absorbs scheduler flake without weakening the invariant. Mutation
re-verified: reverting the offload still fails all 4 tests.
The sweep-path test parametrizes over _runner/_adapter, which live on
TestGatewayRedeliverySweep; main later added
TestUnconnectedPlatformKeepsItsBudget at the cherry-pick anchor point and
the test landed in that class, where the helpers don't exist
(AttributeError x2). Placement-only move.
The original PR #77274 used positional slicing (current_history[len(history):])
to detect the model-switch-only mutation. But _append_model_switch_marker
strips prior markers in-place before appending the new one, so when a prior
marker existed (every switch after the first in a session), the net length
delta is zero and the slice produces an empty list — the merge path is dead
code for the common case.
Replace with a content-based diff: strip markers from both the turn-start
snapshot and the current history, then check that the non-marker content is
identical. This correctly handles the strip-and-replace behavior.
Also guard against auto-compression making result["messages"] shorter than
the turn-start history — use the full result as the base when that happens.
Added test covering both no-prior-marker and prior-marker cases.
When a model switch occurs mid-turn, `_append_model_switch_marker()`
appends a marker to session history and increments `history_version`.
The turn completion guard then sees `current_version != history_version`
and discards all agent output — producing empty assistant messages in
the session DB.
Detect when the only history mutation during the turn was one or more
model-switch markers. In that case, merge the agent's new messages
into the current history (which now contains the marker) instead of
discarding them. Genuine desyncs (undo/compress/retry) still surface
the warning as before.
Fixes#76870
The universal sync fallback in stream_tts_to_speaker ran strictly serially
per sentence — synthesize, play, and only then start synthesizing the next
sentence — so every sentence boundary added a full synthesis-time of dead
air. Chunked streamers (elevenlabs/openai/gemini/xai) already avoid this;
every other provider (edge, piper, plugin providers) paid it on each reply
in voice mode and the wake-word loop.
_SyncSentencePipeline overlaps the two: one single-threaded synthesis
worker (sentences stay FIFO; providers never see concurrent calls from
this loop — same effective concurrency as before) feeds one playback
worker through a small bounded queue, so sentence n+1 synthesizes while
sentence n plays. Lookahead is bounded (backpressure + at most a couple of
temp files), stop_event short-circuits both stages, synthesis failures are
isolated per sentence, temp files are always unlinked, and the finally
block flushes the pipeline BEFORE tts_done_event fires so continuous voice
mode never reopens the mic over its own voice. synthesize/play are
resolved late so existing monkeypatch-based tests work unchanged.
Measured with a real local model provider (OmniVoice plugin, Apple
Silicon), same 3-sentence reply, playback simulated at the produced clips'
true durations, best-of-2 interleaved runs under identical load:
serial pipelined
time to first word 10.8s 4.4s
mid-reply dead air 11.2s 1.8s (second gap: 0.03s)
full reply wall 33.2s 17.0s
Tests: 4 new (timestamp-proven overlap, order + per-sentence failure
isolation, stop skips queued playback, temp-file hygiene); the existing
sync-fallback and display-callback tests pass unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wrap the masked-link destination in angle brackets so Discord does not
unfurl an OG-preview embed under every tool progress bubble. quote()
percent-encodes any <> inside the URL itself, so the wrapper cannot be
broken out of.
On Linux, /usr/bin/python3 is >1MB, so the size check fired before
the NUL check could run — the binary was returned as unsafe=True
(blocked) instead of (None, False) (skip). Reorder: read the bounded
chunk first, check for NUL bytes (binary → skip), then check size
(oversized text → fail closed).
The gateway lifecycle guard (cron/lifecycle_guard.py) applied shell-style
tokenization and script-reference resolution to non-shell content, with two
regressions:
#77131 - every .py cron script using pathlib division was hard-blocked:
Path.home() / ".hermes" / ".env" tokenizes the bare "/" operator as an
executable path, which resolves to the filesystem root; the regular-file
check then fails closed as unsafe. Since Python runs under the
interpreter, never through a POSIX shell, the shell-script reference walk
is a false-positive generator on Python sources. check_gateway_lifecycle
now skips the walk for *.py scripts (the direct command regex still scans
the full text), and _iter_referenced_shell_scripts skips pure-separator
tokens.
#76762 - terminal commands invoking a binary by absolute path (e.g.
/usr/bin/python3) crashed the guard with ValueError: embedded null byte:
the walk read the binary's bytes, decoded them as text, and re-tokenized
machine code; the recursion then hit Path.resolve() on a NUL-bearing
path while only OSError was caught. _read_referenced_script now skips
NUL-containing files (binaries are not referenced shell scripts) and
resolve() tolerates ValueError.
Shell scripts (.sh/.bash/.zsh) keep the full deep scan; literal lifecycle
commands in .py scripts are still blocked by the direct regex. New tests
cover all four behaviors.
startSocket() awaits useMultiFileAuthState() and fetchLatestBaileysVersion()
before it creates a socket or registers event handlers, and the close handler
re-entered it via a bare setTimeout(startSocket, ...). That leaves two
unrecoverable failure modes on a reconnect:
- a rejection is an unhandled promise rejection (fatal on modern Node)
- a hang leaves the bridge permanently disconnected with nothing left to
retry, while its HTTP server keeps answering 503 to the gateway
The second mode was observed in the field: fetchLatestBaileysVersion() is a
plain fetch to raw.githubusercontent.com with no AbortSignal, and after a
stream:error 503 disconnect the bridge logged 'Reconnecting in 3s...' once
and then sat silent and disconnected for 27+ hours until manually restarted.
Fix, as two pure helpers in bridge_helpers.js (keeping bridge.js side-effect
free to test):
- createReconnectScheduler(): every (re)connect entry point now catches a
failed startSocket() and reschedules it instead of dying or going silent
- createVersionResolver(): bounds the version fetch with a 15s timeout and
falls back to the last known-good version (or the Baileys default before
first success) instead of pending forever
Follow-up fixes for the combined Matrix crypto salvage (#71073,
#71543, #71547):
1. Construct _pickle_key from client.device_id (resolved from whoami)
instead of self._device_id (the configured value). Without this, when
#71543 makes the token's real device win over a stale
MATRIX_DEVICE_ID, the pickle key is built from the stale value and
the Olm account is stored under a key that can never be looked up
again — perpetuating the same decryption failure the PRs aim to fix.
2. Skip _migrate_legacy_crypto_pickle when the store was just deleted
by _reset_crypto_store_if_device_changed — there is no account to
migrate. Also check the migration return value and log a warning on
failure instead of proceeding to olm.load() which fails with a
cryptic BAD_ACCOUNT_KEY.
3. Add a local dict cache (_enc_info_cache) to _CryptoStateStore so the
homeserver fallback in get_encryption_info() does not make a network
round-trip on every is_encrypted() call. MemoryStateStore does not
implement set_encryption_info, so the existing cache-back is a no-op.
4. Log homeserver encryption-info query failures at DEBUG level instead
of silently returning None (which would cause OlmMachine to treat an
encrypted room as unencrypted).
5. Update _CryptoStateStore docstring to mention the homeserver fallback.
The account was written under the new pickle key before sessions were
re-pickled. The account is effectively the migration's commit marker —
once it reads under the current key, the fast path short-circuits every
later startup — so a sweep that errored or was interrupted left the
remaining legacy-key sessions stranded permanently with no retry.
Sweep first, commit the account last, and return False on sweep failure
so the migration is retried on the next start.
Also corrects the unreadable-row log: it claimed rows were being dropped
while no DELETE was ever issued. Such rows are left in place (already
unusable; deleting crypto material on a guess is not worth it) and the
message now says so.
Adds session-sweep coverage, which was previously absent: rows rewritten
under the current key, rows already current left alone, unreadable rows
left in place, and a failed sweep that leaves the account uncommitted.
The existing migration test now fakes the olm C-extension so the suite
no longer requires libolm.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Olm account pickle key is derived from the account ID plus the
configured device ID (acct:device_id). If the crypto store's account was
created before MATRIX_DEVICE_ID was set — e.g. the very first password
login, where the device ID is only known after connecting — it gets
pickled under "<acct>:default". Setting MATRIX_DEVICE_ID afterwards (a
reasonable thing to do once you know the device ID you want to pin)
changes the derived pickle key, and every subsequent unpickle attempt
fails with BAD_ACCOUNT_KEY. In optional-E2EE mode that failure is
swallowed and encryption silently stays disabled instead of surfacing an
actionable error.
_migrate_legacy_crypto_pickle() detects the BAD_ACCOUNT_KEY failure,
tries the known legacy pickle keys, and re-pickles the account (plus
every stored olm/megolm session — sessions share the same pickle key, so
migrating only the account would leave them unreadable on the next
decrypt and silently break key sharing with peers) under the current
key. It only reports failure when no known key can unpickle the
account, with a log message pointing at what changed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
connect() resolved client.device_id as `self._device_id or resolved_device_id`,
so a configured MATRIX_DEVICE_ID masked the live whoami() device. With
persisted A, configured A and a rotated token reporting B, the reset
compared A to A and never fired — exactly the token-rotation case this PR
claims to handle.
An access token is bound to one device and the homeserver only accepts key
uploads for that device, so a configured value naming a different one
cannot work. The live whoami() device now wins on conflict and logs an
error naming both. The configured value is still preferred when whoami()
reports no device.
Adds a connect()-level regression for persisted A + configured A +
whoami B, and corrects test_connect_uses_configured_device_id_over_whoami,
whose stated premise this inverts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The crypto store is keyed by Matrix user ID, not device ID, so swapping in
a new access token (which mints a new device_id) silently inherits the
previous device's Olm account. That account's identity keys can never be
published under the new device ID, and the pickle key embeds the old
device ID anyway — the result is stale-key mismatches and cross-signing
signatures the homeserver refuses to replace, degrading E2EE in ways that
are hard to diagnose (peers silently withhold room keys).
_reset_crypto_store_if_device_changed() compares the store's persisted
device ID against the live one at connect time and wipes the store on
mismatch, so a fresh Olm account is generated for the new device instead
of reusing stale key material.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
_CryptoStateStore.get_encryption_info only consulted mautrix's in-memory
MemoryStateStore, which has no record of m.room.encryption for rooms the
bot joined in the past (the raw-sync path never feeds those state events
through set_encryption_info). On a fresh crypto store this returns None
for all previously-joined rooms, so OlmMachine reports them as unencrypted,
never tracks peer devices, and silently drops all inbound messages.
Fix: pass the mautrix Client into _CryptoStateStore so get_encryption_info
can fall back to a live GET /_matrix/client/v3/rooms/{room_id}/state/
m.room.encryption query when the in-memory store returns None. The result
is cached back via set_encryption_info so subsequent lookups (and
OlmMachine device tracking) hit the fast path.
Post-tool compression path passed context_compressor.last_prompt_tokens (0 in the no-usage
fallback) to _compress_context instead of the overhead-aware _real_tokens computed just above
— same tool-blind bug as the overflow handlers (upstream PR #77169 review, teknium1). Also adds
production-path regression tests asserting the 413, context-overflow (two wordings), and
Anthropic long-context recovery handlers pass estimate_request_tokens_rough(..., tools=...)
(sentinel-patched) to _compress_context.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
liuhao1024