The Aug 8 default-on upscaling policy (66ea4e686) chained the Clarity
Upscaler after every sub-2MP generation. Clarity is an SD1.5 creative
tile-diffusion enhancer (creativity 0.35, "masterpiece" prompt prefix) —
it redraws content, which degraded output on 100% of generations for
models like GPT Image 2 and Ideogram whose value is precise text
rendering, CJK, and photorealistic detail.
Policy now: no model upscales by default, on FAL or Krea. The `upscale`
tool param remains as a per-call opt-in (`upscale: true`); explicit
requests still chain Clarity (FAL) / Krea Enhance as before.
- FAL catalog: all 17 default-on entries flipped to upscale=False
- Krea plugin: medium + medium-turbo per-model defaults flipped off
- Tool schema: upscale param described as opt-in with a fidelity warning
- Tests updated: catalog invariant now pins all-off; default-on cases
now assert no upscaler call
- Docs (en + zh) updated to the opt-in policy
pluginSocket (hermes.ts) is documented as "the live twin of pluginRest,
scoped the same way", but it calls window.hermesDesktop.getConnection()
with no profile argument, while pluginRest passes the active profile via
profileScoped(). getConnection's IPC handler (ensureBackend in
electron/main.ts) falls back to the primary profile whenever the profile
argument is empty, so an unscoped call always resolves to the primary
profile's backend regardless of which profile is actually active.
For a plugin used from a non-primary profile (e.g. kanban), this means REST
calls go to the correct pooled backend while the plugin's WebSocket silently
connects to the wrong one — a multi-profile user sees one profile's data
with another profile's live events.
Fix (adapted to the post-#87600 registry-agent store shape during salvage):
resolve the plugin socket's connection through the same (connectionId,
profile) source of truth ensureGatewayProfile/ensureGatewayAgent maintain
for $connection — store/gateway's setActive now pushes the active scope's
registry connection id into the hermes module (setApiRequestConnection,
the no-store-import twin of setApiRequestProfile), and pluginSocket
resolves via getConnectionFor for registry-agent scopes and
getConnection(profile) for the local pool. The plugin socket therefore
follows registry-agent activations too, not just profile switches.
voice-playback.ts's resolveSpeakStreamUrl had the same gap originally, but
main has since fixed it independently (via the getApiRequestProfile()
getter rather than direct store access) — dropped from this PR as
redundant, keeping only the still-open pluginSocket gap.
Co-authored-by: Hermes Agent <hermes@nousresearch.com>
Mirror the strict unit-name shape from the hermes-serve gate (review on
PR #83595) on the gateway side too: the discovery gate and the SIGUSR1
eligibility helper now accept only `hermes-gateway.service` or the
`hermes-gateway-<profile>` family, so a near-prefix unit like
`hermes-gatewayd.service` can neither enter the restart path nor be sent
a SIGUSR1 it does not handle.
Review on #83595 flagged two service-lifecycle gaps in the hermes-serve
restart support:
- The unit-name gate accepted anything starting with "hermes-serve",
which also matched the unrelated hermes-server.service. Require the
exact base unit or the hyphenated profile family instead.
- The fleet-restart loop and _finish_dashboard_update_cleanup() could
both restart the same hermes-serve unit — the loop restarts it
directly, then cleanup's PID scan finds the fresh process and
restarts its owning unit again. Thread the fleet loop's restarted
unit names through to _kill_stale_dashboard_processes() so it skips
units already handled.
hermes update discovered and restarted hermes-gateway* systemd units but
never looked for hermes-serve* — the Desktop app's backend — so it kept
running stale pre-update code until the user restarted it by hand (#83438).
Extend the systemd unit discovery/restart loop to also match hermes-serve*
units. They don't wire SIGUSR1 to a graceful drain (only gateway/run.py
does), so restart eligibility for the graceful path is now gated on unit
name via a small, directly-tested helper; hermes-serve units fall straight
to the existing blunt systemctl restart path, matching the workaround the
issue already documents.
f1ea4a56c ("cover the remaining setup-time .env reads with utf-8-sig",
following 75afc47ba for mem0/hindsight) swept this class; openviking's
_write_env_vars was missed and still reads with strict utf-8.
It copies every existing line through on each update, so the read decides
whether a credential update lands:
BOM'd .env -> the first key never matches, so the old line survives and
the new value is appended as a duplicate. .env loaders keep
the first occurrence, so the update silently does nothing.
cp1252 .env -> UnicodeDecodeError aborts setup outright.
Read exactly like the canonical hermes_cli/config.py save_env_value
(utf-8-sig + errors="replace"). A plain UTF-8 file rewrites byte-identically.
Scope: hermes_cli/memory_setup.py has the same read but is already the
subject of #30281 / #60587, so it is left alone here.
(cherry picked from commit 175c6852c2c255b3219575b5de0b1b70f1f0efcb)
'database disk image is malformed' contains the word 'disk', so
classify_persistence_error bucketed SQLITE_CORRUPT / SQLITE_NOTADB
failures as 'disk' and the turn-completion explainer told users to
free disk space for a structurally damaged state.db (the #77386-family
misdiagnosis, reproduced in the v0.20.0 malformed-DB incident report).
- hermes_state: new 'corrupt' bucket in PERSISTENCE_ERROR_CAUSES,
matched via _DB_CORRUPTION_MARKERS BEFORE the locked/disk buckets
- run_agent: explainer text for 'corrupt' points at hermes doctor and
explicitly says freeing space will not help
- cron explainer-variant suppression picks the new variant up
automatically (it iterates PERSISTENCE_ERROR_CAUSES)
* fix(tui): send atomic CSI u for modified Enter in IDE terminals
VS Code/Cursor/Windsurf terminals bound Shift/Ctrl/Cmd+Enter to the
legacy \\r\n sequence, which Ink's parse-keypress split into a
backslash keypress plus a plain Return — inserting a stray backslash and
submitting instead of adding a newline. Emit Kitty CSI u sequences that
encode the modifier atomically, and migrate keybindings users already
have on disk.
Co-authored-by: yatesjalex <yatesjalex@users.noreply.github.com>
* fix(tui): treat a bare LF as a newline in macOS composer terminals
Terminals that can't send a distinct Shift+Enter collapse a modified
Enter / Ctrl+J down to a bare LF. shouldPreserveCtrlJNewline() already
handles the env-detectable cases (SSH, Windows Terminal, Ghostty, WSL),
but plain macOS terminals (Terminal.app, iTerm2 defaults) do the same and
aren't env-detectable, leaving no keyboard-driven newline there. Fold the
return-key decision into shouldInsertNewlineOnReturn() and accept a bare
LF as a multiline fallback on macOS too, keeping CR as submit everywhere.
Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com>
---------
Co-authored-by: yatesjalex <yatesjalex@users.noreply.github.com>
Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com>
Widen PR #87757 to cover the ZIP path: _update_via_zip() also calls
_finish_dashboard_update_cleanup() but never runs _reload_config_modules,
so the Windows git-broken fallback would still crash with the same
ImportError (cannot import name 'bounded_probe_run' from the stale cached
hermes_cli._subprocess_compat).
- new _reload_process_scan_modules() called inside
_finish_dashboard_update_cleanup itself, so every current and future
call site is covered; reloads dependency-first
(_subprocess_compat, then dashboard_procs)
- reload failures log at warning (a miss surfaces seconds later as an
ImportError in the same process)
- regression tests: reload-before-kill ordering, node-failure skip,
stale-module symbol restoration (the exact #87134 boundary state),
nonfatal reload failure, and the #87757 reload-list contract
hermes update runs in the PRE-pull Python process. After git pull updates
source files on disk, modules already in sys.modules still hold the OLD
code. The existing _reload_config_modules() reloaded only config modules,
but the post-update dashboard cleanup path (_finish_dashboard_update_cleanup
-> _scan_dashboard_processes) imports hermes_cli._subprocess_compat lazily;
a new symbol added there (e.g. bounded_probe_run) is invisible to the
cached module object, causing ImportError during the cleanup step.
Extend the reload list to include hermes_cli._subprocess_compat and
hermes_cli.dashboard_procs so the cleanup uses freshly-pulled code.
Follow-up to #87400: drop the max_iterations and prompt_file knobs from
auxiliary.background_review. The aux model routing (provider/model/
base_url/...) predates #87400 and stays; the enabled switch and the
usage telemetry stay. The fork's iteration budget returns to the
historical hardcoded 16.
Secondary-gateway events were tagged with connectionId (store/gateway
fan-out) but no consumer read it: working/attention tracking, the
pruneSecondaryGateways keep-set, and the profile-scoped event gates
(skin.changed / change-watcher broadcasts / approval-mode reconcile)
all keyed by session id + bare profile name. Every registered source
exposes a 'default' profile (the roster force-unshifts it), so two
connected gateways collided — gateway B's 'default' activity was
attributed to gateway A's 'default', keeping the wrong socket alive
and applying the wrong source's config/skin/cron changes.
Thread connectionId through consumption using the existing composite
backendScopeKey helper:
- session-states records each registry-tagged event's (connectionId,
profile) scope per runtime session; liveSessionScopes() projects the
busy/needs-input ones as composite keys for the gateway keep-set.
- recomputeKeptGateways (use-gateway-boot) seeds the keep-set with
those scopes; pruneSecondaryGateways matches registry-scoped entries
ONLY on their composite key, while local entries keep matching bare
profile names (single-source path unchanged).
- gateway-event's 'from the active profile' gates now compare the
event's composite scope against the active gateway's connection via
the new activeGatewayConnectionId(); untagged local/primary events
behave byte-identically.
Display-only surfaces that already use roster handles are untouched.
ensureRegistryBackend delegated kind==='local' to ensureBackend(), which
follows the v1 connection.json routing table — under a v1 REMOTE global
mode (the migration keeps the mandatory 'local' entry AND makes that
remote the registry primary) the roster's 'This device' rows enumerated
and dialed the REMOTE primary: every profile appeared twice (forcing
-slug handles) and clicking a local agent talked to the remote box.
resolveRegistryLocalRoute() (pure, colocated with the registry helpers)
now decides the local entry's path: delegate to the legacy route only
when v1 is itself local (single-source behavior byte-identical);
otherwise spawn/reuse a forced-local pool child via spawnPoolBackend's
new forceLocal option, pooled under the composite conn:local::<profile>
key so it cannot collide with the v1 remote descriptor cached at the
bare profile key.
Persist fork token usage under session_model_usage task=background_review,
emit a per-fork completion log line, and expose enabled/max_iterations/
prompt_file so operators can see and bound the automatic review cost.
Address review feedback: load auxiliary.background_review once per spawn,
classify completion logs by summarize action prefixes, treat explicit
api_call_count=None as the documented default of 1, and WARNING on the
fail-open enabled-gate path.
The Windows-only test asserted encoding/errors kwargs on a mocked
subprocess.run, but the scan now routes through bounded_probe_run
(#87134), so subprocess.run is never invoked. Assert the probe call's
contract instead (errors='ignore', finite timeout), verify the parsed
PIDs, and add a fail-open case for probe failure. The test no longer
needs a Windows host once the probe is mocked, so the windows_only
gate is dropped.
subprocess.run(capture_output=True, timeout=N) is not hang-safe on
Windows: after the timeout fires, run()'s cleanup kills the direct child
and then joins the pipe reader threads with an UNBOUNDED communicate().
A descendant (conhost.exe under wmic/powershell) holding duplicated pipe
handles keeps the pipes from EOF and the join never returns.
_scan_gateway_pids() runs its wmic / Get-CimInstance Win32_Process scans
exactly that way, and on machines where the full process scan genuinely
exceeds its 10/15s budget (cold WMI on first boot, ARM VMs, heavy
Update/AV activity) hermes update wedged forever inside
_pause_windows_gateways_for_update() before printing a single line —
observed live on a fresh Windows 11 ARM64 VM with a faulthandler stack
pinning the main thread in subprocess._communicate and only a conhost.exe
child surviving. The single-flight update lock then blocks retries until
the wedged process is killed by hand.
This is the same deadlock class bounded_git_probe already fixed for git
probes (#68609 / #66037). Generalize that proven pattern into a shared
bounded_probe_run() — explicit communicate(timeout), kill_process_tree on
failure, bounded 1s drain, then abandon the daemonic readers — and
migrate the whole call-site class onto it:
- hermes_cli/gateway.py _scan_gateway_pids (the site that hung; reached
from hermes update, cron, gateway restart/status, dashboard)
- hermes_cli/dashboard_procs.py wmic scan (same shape, reached on update)
- hermes_cli/claw.py tasklist + PowerShell probes (same shape; its
try/except cannot catch a hang because a hang raises nothing)
- bounded_git_probe now delegates to bounded_probe_run (identical
contract, one copy of the cleanup logic)
Unlike bounded_git_probe, bounded_probe_run returns the CompletedProcess
(or None) rather than collapsing to stdout, because the gateway scan
branches on returncode to trip its wmic -> powershell fallback.
Tests: tests/hermes_cli/test_bounded_probe_run.py covers success,
nonzero-exit passthrough, spawn failure, bounded timeout (fails against
the old unbounded semantics — verified by sabotage), errors= decoding,
DEVNULL stdin, POSIX process-group placement, and the bounded_git_probe
delegation contract. Existing test_git_probe_tree_kill.py passes
unchanged against the delegated implementation.
Closes#87134
Salvage hardening on top of #87308 (thanks @Dudeman456):
- Tri-state verdict: inconclusive probes (binary missing, --help
failed/timed out) return None and fall through to the normal spawn
path, preserving the established 'Could not start Copilot ACP
command' error instead of masking it. This also fixes the two
test_copilot_acp_client HOME-env regressions that went red on the
PR: their mocked-Popen path was intercepted by the new unmocked
subprocess.run probe.
- Cache definitive verdicts per binary path so CLIs that DO support
--acp pay the ~50ms --help cost once per process, not per prompt.
- Skip the probe entirely when custom ACP args don't include --acp.
- Fix the help-text regex: the old pattern never matched '[--acp]'
(leading '[' is neither start-of-string nor whitespace) and \b
after 'p' matched '--acpfoo'.
- Hermeticity: stub subprocess.run in the two HOME-env tests; add 6
probe-specific tests (fast-fail, fall-through, caching, skip).
CopilotACPClient unconditionally passes [self._acp_command] +
self._acp_args (default ['--acp', '--stdio']) to subprocess.Popen.
When the resolved CLI doesn't accept --acp (e.g. Claude Code
v2.1.233, where 'claude --acp --stdio' exits 1 with
'error: unknown option') the subprocess dies in ~250ms with the
error on stderr, but the parent ACP loop has no fast-fail for this
shape and waits the full child_timeout_seconds (default 600s,
observed 109s+ before user interruption) for stdout that never
arrives.
Add _acp_supported() that probes the CLI's --help output for the
--acp flag in ~50ms, then call it at the top of _run_prompt before
any spawn happens. When the probe fails, raise a RuntimeError that
names the unsupported flag, lists the expected fix (install
@github/copilot late 2025+, or set HERMES_COPILOT_ACP_*), and
returns control to the caller in ~280ms instead of hanging the
delegate_task parent for hundreds of seconds.
Measured locally against Claude Code v2.1.233:
- Before: delegate_task acp_command=claude hangs 109s+ then
returns tokens={input:0, output:0}.
- After: delegate_task acp_command=claude raises RuntimeError
in 280ms with a clear actionable message.
This does NOT change behavior for supported CLIs (the new
@github/copilot ships with --acp) — the probe returns True and
the spawn proceeds unchanged.
Refs the bundled claude-review-delegate skill which already
documents this class of transport-mismatch pitfall for users
who call 'claude -p' directly; this fix closes the same gap for
the delegate_task MCP path.
Response to AI review (Enough1122) on #86812:
1. `_create_titled_session`: log the underlying exception before returning
None so programmatic callers aren't left with an undebuggable "could
not be created" — failures (DB lock, I/O, import) now land in errors.log
via logger.exception.
2. Drop the source-reading `TestSourceGuard` — it violated the repo's
"never read source code in tests" rule and was a change-detector
(passed even if behavior regressed). The stderr routing is already
covered by the real-path `test_missing_session_fails_on_stderr`.
3. Bare `-c` + `--create-if-missing` now prints a stderr note explaining
the flag needs a session name, instead of silently ignoring it — makes
the no-op self-evident to programmatic callers.
Tests: 6 targeted + 8 adjacent, all passing.
`hermes chat -c "<title>" -q "<text>"` silently no-oped when no session
matched the title under quiet/programmatic use: the not-found message was
written to stdout (the channel quiet callers parse as the final response),
so a background send to a not-yet-existing named session vanished with no
error. Surfaces via Hermes-Bot-Mode bot-to-bot handoffs (#86794).
- not-found message now goes to stderr (exit 1 unchanged), so programmatic
callers always see it even with -Q/--quiet
- new --create-if-missing: with `-c <title>` and no matching session, create
a fresh session carrying the title and proceed — the deterministic
"send to this named thread, making it if needed" primitive plugins asked for
- extract the -c resolution block into _resolve_continue_arg for testability
Tests: flag parsing, titled-session creation, stderr routing, source guard.
Streamed replies return None so the adapter does not send twice.
The /loop hook then saw empty text and never ran, so
awaiting_response stayed true and later ticks never fired.
Stash the delivered text on the event and use it for the post-turn
hooks. /goal uses the same path.
Tests: tests/gateway/test_loop_command.py
The stale-lock removal tests asserted the sweep result while leaving
_git_proc_running() live: on CI the parallel per-file runner almost
always has a real git subprocess in flight, pgrep -x git hits, and
clear_stale_git_locks correctly refuses to sweep — failing the tests
for reasons unrelated to the code under test (surfaced on PR #86918,
which doesn't touch gitlock at all).
Monkeypatch the guard to False in the sweep tests and add an explicit
test pinning the guard's block-while-git-running behavior.
The previous except Exception silently fell back to os.getenv if the
_scoped_key_env import ever failed — under multiplex that is exactly the
fail-open this PR removes (secondary profiles would regress to 'No LLM
provider configured' with zero trace). Catch only ImportError, log a
WARNING naming the consequence, and let any other failure propagate.
Also replaces the lambda fallback with a named nested function.
Three regression tests: scoped DEEPSEEK_API_KEY is visible to auto
detection under multiplex (the #86917 failure); unscoped paths keep the
os.environ read; explicit config provider still wins.
resolve_provider's auto path read provider API keys with bare
os.getenv — under multiplex a secondary profile's keys live only in its
secret scope, so auto-detection found nothing and every secondary
profile with model.provider: auto failed with 'No LLM provider
configured' at agent init (reproduced on a live 7-profile gateway).
Route both env-key reads (the OPENAI/OPENROUTER tier and the
PROVIDER_REGISTRY loop) through _scoped_key_env, the scope-aware helper
auxiliary_client already uses: secret scope wins under multiplex,
UnscopedSecretError falls back to os.environ (default-profile/CLI
paths unchanged). Same bug class as #86905.
Verified in a gateway-accurate simulation (hermes_home_override +
profile scope): resolve_provider('auto') now returns the secondary
profile's own provider (deepseek) instead of erroring.
Lesson from the feishu DM multiplex investigation: under multiplex,
os.environ holds the default profile's values, so any profile-level env
config (credentials AND authorization) must be read scope-aware, and a
scoped miss with a scope installed must fail closed instead of borrowing
from os.environ. The _get_scoped_secret wrapper is copy-pasted across
~15 platform adapters — new adapters and edits to existing ones must
keep the fail-closed semantics.
"\x00" in script_path raises TypeError when a caller passes a non-str
(e.g. a pathlib.Path, which is not iterable) — the guard itself would
crash the scheduler. All current call sites pass plain str, but the
guard must be crash-proof: str() first, then check. Adds a regression
test running a real script through _run_job_script with a Path argument,
which fails with TypeError on the pre-fix guard.
_run_job_script wrapped only expanduser() in its ingestion try/except.
On Linux an unexpandable NUL-bearing value raises inside that call, so it
landed on the clean fail-with-report path; on Windows expanduser() never
expands '~user' (and so never raises), and the NUL surfaces later as an
uncaught ValueError from resolve()/exists() — crashing the scheduler.
Align with cron.lifecycle_guard._expand_candidate_path, which already
documents this as the whole-class fix (the per-syscall catching produced
#76762, #77703, #77780, #78256): reject '\\x00' eagerly at the ingestion
boundary so both platforms fail identically.
The Bedrock API-key flow stored the bearer token in OPENAI_API_KEY and set
a bare `provider: custom`. Since #28660 that variable is only honoured for
openai.com hosts, so for bedrock-mantle.*.api.aws the token was dropped and
requests went out with api_key="no-key-required", a 401 on every call.
Write a named `providers.bedrock-mantle` entry with
key_env: AWS_BEARER_TOKEN_BEDROCK instead. The named-provider branch in
runtime_provider.py resolves key_env; the bare-custom branch cannot.
Fixes authentication only. Per-model mantle route selection is separate.
Appending --daemonized after ORIGINAL_ARGS put it past the `--`
relaunch-args separator on Linux, so it was absorbed into
RELAUNCH_ARGS instead of being parsed as a flag. HANDOFF_DAEMONIZED
never got set, so the one-shot self-detach block re-fired on every
re-exec -- an unbounded self-exec loop (thousands of iterations/sec,
100%+ CPU, argv growing until execve fails with E2BIG) whenever
relaunch args were present, which is the normal invocation shape on
Linux.
Fixes#86957
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Teknium