opencode-free broke two provider-surface contract tests: 'api_key
providers must expose a credential env var' and 'GUI ⊇ hermes model
universe'. Both premises assume a credential exists. Add a keyless flag
to HermesOverlay + ProviderDescriptor (same derived-exemption pattern as
virtual providers) so any future anonymous provider is covered without
hardcoded slugs. Nothing to configure = no Providers-tab card, by design;
the model picker remains the selection surface.
Reworks the salvaged OpenCode Free provider to match the tier's real
auth contract (verified live 2026-08-21): the Zen relay serves free
models ANONYMOUSLY and 401s any unrecognized bearer, so the provider now
declares no credentials at all and routes every model through the shared
keyless machinery from the Ox Alpha fix (empty Authorization default
header overriding the SDK bearer).
On top of the salvaged base:
- auth.py: no api_key_env_vars; drop the keyed-auth special case
- runtime_provider.py: restore the plain fail-closed path (opencode-free
never reaches it — the keyless runtime resolves first)
- models.py: opencode-free joins the opencode family (prefix stripping,
Zen endpoint routing incl. muse->responses); keyless predicate extended
with unsuffixed free slugs (big-pickle); free runtime pins EVERY
opencode-free model keyless; curated catalog replaces the models.dev
cost==0 filter (it lags reality: deepseek-v4-flash-free stayed 'free'
there after its promo ended and the relay began 401ing it — delisted)
- agent_runtime_helpers.py: replace the httpx transport-sharing auth-strip
wrapper with the shared header policy (no proxy-mount loss)
- model_setup_flows.py: skip the API-key prompt for opencode-free
- plugin profile: keyless headers, no env vars
- .env.example + providers.md: keyless docs (no OPENCODE_FREE_API_KEY)
- tests rewritten to the keyless contract, incl. catalog-membership
invariant (every curated model must satisfy the keyless predicate)
E2E: full AIAgent turns with zero keys complete on x-preview-f-free via
provider opencode-free and alias 'free', incl. a real terminal tool
round-trip; muse routes to /v1/responses; picker lists 8 keyless models.
Adds an OpenCode Free provider plugin. Free model discovery uses models.dev
(cost.input == 0 AND status != "deprecated"), matching opencode CLI's exact
filter logic.
The free tier requires a real account API key and throttles third-party
clients by User-Agent:
- With OPENCODE_FREE_API_KEY configured, the key is sent as a Bearer token
and requests identify as "opencode/latest".
- Without a key, the keyless fallback strips the SDK's always-injected empty
Authorization header and still sends the opencode User-Agent.
- The credential resolver no longer blanks OPENCODE_FREE_API_KEY
unconditionally (the stale keyless-tier assumption), and credential-pool
exhaustion no longer surfaces the misleading "Set OPENCODE_FREE_API_KEY"
message.
Co-authored-by: Jean-François <jfm@laposte.net>
Signed-off-by: Rudraksh Chahal <131520192+rudrakshchahal@users.noreply.github.com>
It exercises build_prompt_cache_plan's direct_native_tool_cache fallback,
not repeated apply on pre-decorated input, so it belongs with the other
plan-layout tests rather than in TestApplyIdempotency.
Follow-up to the #90972 salvage:
- strip loop: copy.deepcopy(msg) -> dict(msg). strip_anthropic_cache_control
is copy-on-write on content parts by contract (pops the top-level key,
rebuilds content lists/part dicts fresh), so a shallow top-level copy
preserves the caller-non-mutation guarantee — verified for all four
marker shapes — and removes the redundant second deepcopy the re-mark
path paid on already-decorated input. Docstring updated to match.
- tests: moved the surviving idempotency tests into
tests/agent/test_prompt_caching.py (where this module's tests live) as
TestApplyIdempotency; dropped the three tests that duplicated existing
coverage (dynamic_tool_accounting ~= TestPromptCachePlan::
test_copies_sections_and_keeps_canonical_tools_plain which already
asserts == 4; can_carry_marker_envelope_vs_native ~= TestCanCarryMarker;
never_exceeds_four_markers subsumed by the idempotency test).
- exact-count assertions per review: idempotency fixture pins == 4,
no-tools fallback pins == 3 (marker loss can no longer masquerade as
safety); added the one new _can_carry_marker assertion (native=True
empty assistant) to TestCanCarryMarker.
- new part-level stale-marker mutation guard (the other detection branch,
where part-dict aliasing is the risk); fails on pre-fix base with
marker accumulation (9 > 4), passes with the fix.
apply_anthropic_cache_control never stripped pre-existing cache_control
markers before placing new ones, so calling it twice (or handing it
messages a prior call already marked) accumulated markers past
Anthropic's 4-breakpoint limit and produced HTTP 400
'cache_control can only be specified up to 4 times'.
Strip any pre-existing markers from per-message copies before marking,
mirroring the strip-then-mark pattern build_prompt_cache_plan already
uses. Only messages that already carry a marker pay the copy cost; the
copy-on-write contract (caller-owned messages are never mutated) is
preserved. Repeated calls now converge to byte-identical output.
Salvaged from #90972 by @JoaoMarcos44 (net diff of the PR's commit
stack, intermediate reverts collapsed).
Related: #90971
- Replace ps -A eww with ps -Aww: the BSD e flag is illegal
on macOS/BSD ps, making the fallback silently return [] on every macOS
machine. The matcher only needs argv (not env vars), so e is
unnecessary. -ww keeps unlimited-width output on both BSD and
procps ps.
- Add all_profiles parameter to _get_service_pids(). When True
on macOS, enumerate every ai.hermes.gateway* launchd agent across
profiles via bare launchctl list instead of only the current
profile's label. This prevents the update sweep from misclassifying
sibling-profile launchd gateways as manual processes (#73626).
- Thread all_profiles through find_gateway_pids() to
_get_service_pids().
- Update two _get_service_pids() call sites in update_cmd.py to
pass all_profiles=True so the update fleet sweep excludes every
service-managed gateway across all profiles.
- Add TestPsFallbackBsdCompat: verifies ps argv uses -Aww
not -A eww, and that pid=,command= output columns are present.
- Add TestGetServicePidsAllProfiles: verifies default scope uses
launchctl list <label>, all_profiles uses bare launchctl list
with prefix filtering, handles empty/broken output gracefully, and
preserves systemd behavior.
Tranquil-Flow
The Zen relay serves *-free models (x-preview-f-free / Ox Alpha) ONLY
anonymously: any Authorization bearer it doesn't recognize is a 401
'Invalid API key' — including our no-key-required placeholder and valid
OpenCode GO subscription keys. The Go relay doesn't serve the free tier
at all ('Model x is not supported'). So the free model failed for every
Hermes user: keyless setups got the placeholder bearer, and OpenCode
subscribers sent a Go key to a relay that rejects it.
Fix (class-wide for all 8 current *-free Zen slugs, not just Ox Alpha):
- hermes_cli/models.py: is_opencode_zen_free_model / opencode_zen_free_runtime
/ opencode_zen_free_headers — one shared policy: free slugs pin to the
Zen relay with a keyless placeholder and an empty Authorization header
that overrides the OpenAI SDK's 'Bearer <key>'.
- runtime_provider.py: free slugs route through the keyless runtime before
the credential-pool/explicit/api_key paths (no key required; Go
selections heal to Zen). Paid models still fail closed without a key.
- agent_init.py + auxiliary_client.py: the placeholder key swaps in the
empty-Authorization headers at both client-build chokepoints.
Verified live (2026-08-21): anonymous chat/completions 200 incl. tools,
streaming, parallel; bad bearer 401; full E2E AIAgent turn with a real
terminal tool round-trip completes keyless under both opencode-zen and
opencode-go providers. Sabotage run: routing tests fail without the fix.
* fix(gateway): persist prompt.submit truncation to the session's own profile DB
`_get_db()` returns the LAUNCH profile's SessionDB handle. App-global
remote mode gives a session its own profile (`session["profile_home"]`)
whose transcript lives in that profile's `state.db`, so a write keyed on
`session_key` that goes through `_get_db()` addresses the wrong database.
In the `prompt.submit` truncate branch that has two consequences. The
edit/resend never sticks — `session.resume` reopens the profile db and
resurrects the undone turns — and when the launch profile happens to hold
a row under the same session id, the truncated transcript is inserted
into a profile the session does not belong to.
It also silently voids the branch's own fail-closed contract. The handler
persists before it rewrites `session["history"]` precisely so that a
failed write refuses the turn and leaves memory and DB aligned; that only
holds if the handle it checks is the one that owns the row.
`_session_db(session)` is the profile-aware resolver that already exists
for this: the profile's `state.db` when `profile_home` is set, otherwise
the shared launch handle. Non-profile sessions are unaffected —
`_session_db` borrows the same shared handle and leaves it open.
`active_only=True` and `archive_dropped=True` are carried through
unchanged; only the handle the call is made against changes.
* fix(gateway): resolve the /undo command against the session's own profile DB
`command.dispatch`'s `/undo` branch opened the launch profile's handle via
`_get_db()`, but every read and write under it is scoped by session id:
`list_recent_user_messages`, `rewind_to_message` and the
`get_messages_as_conversation` reload all key on `session_key`.
For a session with its own profile (`session["profile_home"]`) the rows
live in that profile's `state.db`, so against the launch handle
`list_recent_user_messages` returns nothing and the command fails closed
with `4018 "no user messages to undo"` — for the entire session, on every
invocation, even though the transcript is right there in the profile db.
Route the whole branch through `_session_db(session)`, which yields the db
that owns the session's row and closes a profile handle on exit. Sessions
without a profile keep borrowing the shared launch handle exactly as
before, so this is behaviourally identical for them.
* fix(gateway): read /history and /context from the session's own profile DB
`_format_live_history_output` and `_format_live_context_output` rebuild the
transcript from the database rather than from `session["history"]`, because
the in-memory list is empty for a session this process did not run itself.
Both reads are scoped by session id but were issued against `_get_db()`,
the launch profile's handle.
A session with its own profile (`session["profile_home"]`) keeps its rows
in that profile's `state.db`, so both reads come back empty and the
commands under-report: `/history` renders "No conversation history yet."
and `/context` falls back to the empty in-memory list and reports a
conversation of zero messages. Both swallow their exceptions, so there is
no error either — just a wrong answer about the user's own transcript.
Resolve both through `_session_db(session)`, the profile-aware resolver
used by the rest of the session-scoped paths.
* test(gateway): cover session-scoped transcript ops against a profile DB
Regression coverage for the three session-scoped sites that resolved
against the launch profile's handle instead of the db owning the
session's row. Each test drives the real JSON-RPC entry point with a
session carrying `profile_home`, seeds the transcript into the profile's
own `state.db`, and asserts against both databases.
Per site, with the production change reverted to its pre-fix form:
- `prompt.submit` truncation — `test_truncation_persists_to_the_profile_db`
and `test_truncation_does_not_copy_rows_into_the_launch_profile` fail.
The second seeds a row under the same session id in the launch db so the
foreign write succeeds instead of failing a key check, which is the case
that copies a transcript into a profile it does not belong to.
- `/undo` — `test_undo_rewinds_the_profile_transcript` fails with
`4018 "no user messages to undo"`.
- `/history` and `/context` — `test_history_reads_the_profile_transcript`
and `test_context_reads_the_profile_transcript` fail, reporting an empty
conversation.
`test_undo_still_uses_the_shared_handle_without_a_profile` and
`test_truncation_without_a_profile_uses_the_shared_handle` pin the
unchanged path: with no `profile_home` the resolver must borrow the shared
launch handle and leave it open. Both stay green in every direction, so a
future change cannot satisfy the profile cases by abandoning the shared
one.
* fix(desktop): aim truncations by durable id alone on tail-only transcripts
The cold-open transcript is a newest-first prefetch page
(LATEST_SESSION_MESSAGES_LIMIT = 120) with the resume RPC sent
omit_messages — older rows only arrive via "Show earlier" backfill.
planEdit/planReload/planRestore still counted truncate ordinals over
that windowed list, so every edit/reload/restore in a session longer
than the prefetch page sent a window-relative ordinal alongside the
durable row/message id. The gateway's #82959 cross-check resolved the
durable id to its full-history ordinal, read the offset as drift, and
refused with 4030 — making the Edit affordance permanently dead in
long sessions.
When the transcript may be tail-only (the transcript-tail
bookkeeping's possiblyTruncated), drop the client ordinal and address
the truncation by durable id alone — the same rule runRewindSubmit
already applies to content-resolved row ids (#87059). The ordinal
tripwire stays on whenever the transcript is complete.
Closes#88082
* fix(desktop): drop client rewind ordinal whenever a durable id is present
#88092 gated the drop on tail-only prefetch. After in-place compact the
live scrollback is treated as complete, so Restore still sent a
display-lineage ordinal next to a resolved row id and the gateway
refused with 4030 (#89244). prefix_user_count is structurally 0 on
in-place because get_ancestor_display_prefix is cross-session.
Same choke point: if a durable truncate_before_row_id or a real
truncate_before_message_id is present, omit the client ordinal.
confirm_empty_truncate is still carried from a caller ordinal of 0.
Unknown ids still fail closed at 4018.
Closes#89244
---------
Co-authored-by: briandevans <252620095+briandevans@users.noreply.github.com>
Co-authored-by: zengzheqing <yuntianqing@yahoo.com>
* fix(docker): stage2 API_SERVER_KEY bootstrap no longer depends on .env existing (OOF-285)
Fleet sweep found 144/351 started hosted instances (41%) on v2026.8.13+
with no API_SERVER_KEY: the loopback gateway api_server (which serves
/api/cron/fire on :8642) never started, so every scheduled cron fire was
silently lost until the NAS retry budget exhausted.
Root cause chain:
- .dockerignore excludes .env.example (image-size optimization), so
/opt/hermes/.env.example does not exist in shipped images
- stage2's first-boot seed `seed_one ".env" ".env.example"` is a silent
no-op when the source is missing -> fresh volumes never get a .env
- the API_SERVER_KEY generation added in #84339 was gated on
`[ -f "$HERMES_HOME/.env" ]` -> never ran on those instances
Fixes:
- stage2-hook.sh: keygen now creates an owner-only .env when missing
instead of requiring it to exist; still append-only w.r.t. operator
keys, still refuses symlinked paths
- .dockerignore: re-include .env.example (negation after the .env.*
exclusion) so the first-boot template seed works again
- tests: new tests/tools/test_stage2_hook_api_server_keygen.py covers
create-when-missing, append-without-clobber, operator-key preservation,
symlink refusal, and a .dockerignore contract test for .env.example
* fix(docker): container-provided API_SERVER_KEY wins over stage2 keygen (review)
The bootstrap generated a key whenever .env lacked one, without checking
the inherited container environment. That broke the documented
`docker run -e API_SERVER_KEY=...` flow: Hermes loads $HERMES_HOME/.env
with override=True (hermes_cli/env_loader.py), so the generated key
silently shadowed the operator's env key and 401'd existing clients.
- stage2-hook.sh: skip generation when API_SERVER_KEY is present in the
container environment; if BOTH the env and .env carry keys, warn that
the .env value wins at runtime and touch nothing
- tests: regression tests for the env-provided path (skip + no .env
write; env+file conflict warns without clobbering); sandbox runner now
pins/unsets API_SERVER_KEY explicitly so results don't depend on the
host environment
* fix(docker): drop stale empty API_SERVER_KEY= line when container env provides the key
A leftover empty 'API_SERVER_KEY=' assignment in .env clobbers a
container-provided key at runtime (.env loads with override=True and
python-dotenv sets the empty string), so the api_server startup guard
fails and every scheduled cron fire is silently lost — the exact
symptom class this PR fixes, reintroduced in the env-key branch.
Remove the stale empty line (behind the existing symlink guard) before
skipping generation, so the operator's env key actually wins. Addresses
the IMPORTANT finding both reviewers converged on.
Test: env-key + stale-empty-line combination now covered; strict
removal assertion gated on GNU sed (BSD sed on macOS dev hosts skips
the -i invocation, same caveat as the append test).
* fix(docker): warn at boot when a container-provided API_SERVER_KEY is too weak to start the api_server
The startup guard refuses keys under 16 chars. Now that a
container-provided key suppresses stage2 generation, a weak
`docker run -e API_SERVER_KEY=...` value means the api_server stays
down (cron fires unavailable) instead of clients getting 401s against
a generated key. Say so in the boot log, where the operator will look.
* fix(docker): create .env under umask 077 instead of touch+chmod
touch created the file with the inherited umask (typically 0644), then
a silenced chmod tightened it to 0600 — a brief group/world-readable
window, and no warning if the chmod failed. Creating under umask 077
makes the file owner-only from the first instant with no dependence on
a second command succeeding. Covered by the existing 0600 mode
assertion in test_keygen_creates_env_when_missing.
* fix(docker): guard the API_SERVER_KEY append so a read-only .env degrades to a warning, not a failed boot
stage2 runs under set -eu; the unguarded printf append meant a keyless
.env on a read-only volume (or full disk) aborted the whole cont-init
phase and the container boot. Guard it and emit the same loud warning
the create-failure path uses.
Test harness now runs the extracted block under set -eu to match
production (it ran set -u only, so it could not see this defect class);
new read-only regression test verified RED against the unguarded
append via mutation.
* fix(docker): only warn about a weak container API_SERVER_KEY when it is actually the effective key
The <16-chars warning fired before the .env inspection, so a weak
container key alongside a strong .env key produced a false boot-log
claim that the api_server 'will refuse to start' — immediately followed
by the both-keys warning saying the .env value wins, and the server in
fact starts. Move the check into the branch where the env key really is
the effective key on this boot (round-2 review finding, verified by
execution against python-dotenv last-wins semantics).
---------
Co-authored-by: Ben Barclay <ben@nousresearch.com>
Phase 1 of the fleet-update reliability plan (#91277): the updater now
proves its outcome instead of assuming it.
- hermes_cli/build_info.py: get_code_identity() — process-cached code
identity (git sha for source installs, baked .hermes_build_sha for
Docker images, pyproject version).
- gateway/status.py: every runtime-status write stamps the writer's
code_sha/code_version into gateway_state.json, so a running gateway's
actual code generation is observable from disk.
- hermes_cli/update_receipt.py (new): machine-readable receipt of each
update run (steps, skips with reasons, gateway restart outcome, fleet
snapshot) under ~/.hermes/logs/update_receipts/ with a latest.json
pointer for the dashboard/desktop; plus collect_fleet_versions() /
print_fleet_version_matrix() comparing every live profile gateway
against the freshly updated checkout.
- hermes_cli/update_cmd.py: wires receipt begin/steps/finalize into the
git, ZIP, and hard-failure paths; after the restart phase, prints the
fleet version matrix and escalates provably-stale gateways into the
existing gateway_fleet_restart_incomplete exit-1 contract. Pre-stamp
gateways report 'unknown' and never fail the update (no false
positives during rollout).
Silent-failure classes made visible: #88848, #74973, #85753, #81193.
Mixed-version fleet classes made loud: #88654, #69754, #77553, #56717.
sol-reviewer round-4 IMPORTANT (reproduced by execution): the enroll
warning read multiplex_profiles via load_gateway_config() under the
SECONDARY profile's HERMES_HOME, but the flag normally lives in the
DEFAULT root's config.yaml — so the warning never fired in the real
topology, preserving the round-3 defect it claimed to fix.
The topology decision now mirrors the multiplexer-conflict guard in
hermes_cli/gateway.py: secondary detection is the resolved-path
relationship to <default_root>/profiles/ (not a directory-name
heuristic — also fixes the round-4 MINOR false positive on unrelated
dirs named 'profiles'), and the multiplex flag comes from the
GATEWAY_MULTIPLEX_PROFILES env override or a raw read of the default
root's config.yaml. The raw read also avoids running the full
enablement pass (round-4 MINOR: load_gateway_config() emitted the
relay-exclusive sweep's own warnings into enroll output).
The warning now replaces the generic 'restart to pick up the new env'
line instead of following it (round-4 NIT: the two messages were
contradictory), and the helper returns whether it fired.
New test file pins all six topology cases, including the exact
false-negative reproduction (flag in default root only) and env
override in both directions.
sol-reviewer round-3 findings: the regression tests stopped at the
config boundary, so config/registration agreement was only manually
verified. New TestConfigRegistrationAgreementUnderMultiplexScope
exercises both sides under an active profile scope: a process-env
stamp yields Platform.RELAY enabled AND relay_url()/
register_relay_adapter() agreement AND a constructed RelayAdapter with
a live transport; a profile-only .env stamp is inert on BOTH sides (no
half-enabled state). Also narrows the test_config docstring that
overclaimed profile .env stamps are unsupported — the launch profile's
.env still activates relay via load_hermes_dotenv's os.environ export;
only an isolated multiplex scope is never consulted.
sol-reviewer round-2 IMPORTANT: relay env vars had no scope
classification, so the two readers disagreed under a multiplexed
profile scope — gateway/config.py (scope-aware getenv) dropped a
process-env GATEWAY_RELAY_URL during the scoped runner reload while
gateway/relay's relay_url()/register_relay_adapter()/self-provision
(direct os.environ reads) still saw it. Result: adapter registered but
Platform.RELAY absent from config, so the connect loop never dialed
and direct adapters stayed up. The inverse split (profile-only stamp:
config enables RELAY, registration finds no URL) was equally dead.
GATEWAY_RELAY_* ROUTING stamps (URL, ENDPOINT, ALLOW_DIRECT_PLATFORMS,
PLATFORMS, BOT_IDS, ROUTE_KEYS, INSTANCE_ID, WAKE_URL, DISPLAY_NAME)
are now in _GLOBAL_ENV_EXACT: deployment config read from os.environ
under any scope, exactly like the API_SERVER listener settings
(#69379), so every reader resolves the same value. Relay AUTH material
(SECRET, ID, DELIVERY_KEY, IDP_*) is deliberately NOT global — it
stays profile-scoped with the fail-closed multiplex guard, mirroring
the non-secret/secret line the terminal env blocklist already draws
(tools/environments/local.py).
The round-1 multiplex regression test asserted the now-rejected
semantic (profile-scoped stamps win); it is inverted to pin the
global-stamp contract: a process-env stamp survives the profile scope
(sweep runs, matching registration), and a profile-only .env stamp
does NOT activate relay.
The named-custom-provider runtime path returned a static api_mode, so a
providers: entry like opencode-go-bridge -> https://opencode.ai/zen/go/v1
sent responses-only models (grok-4.5, gpt-5.6-luna) to /chat/completions
and got HTTP 503 (#85589 repro). Now: when the provider name is in the
OpenCode family or the base_url is hosted on opencode.ai, derive api_mode
from the effective model and run the symmetric /v1 normalization — unless
the user declared an explicit transport, which stays authoritative.
5 new regression tests against a real temp HERMES_HOME config.
Builds on @Lesnak1's #85619 (issue #85589):
- New opencode_provider_family() single-owner predicate in
hermes_cli/models.py — resolves built-in AND custom family providers
(opencode-go-bridge, OpenCode-Zen-Custom, ...) case-insensitively.
Migrated all 8 inlined family checks (models.py x3, runtime_provider.py
x4 from the salvaged commits) plus 4 sibling sites the PR missed:
cli.py api_mode sync, agent_runtime_helpers.py double-/v1 guard,
model_normalize.py flat-namespace strip, model_switch.py base_url
normalization.
- Responses transport: alias OpenCode-reserved function names
(web_search, search_files -> hermes_*) on the wire and map them back on
dispatch — same pattern as the xAI web_search collision fix. Matches
family providers and any base_url on opencode.ai. Fixes the HTTP 400
'custom function name X is reserved' half of #85589.
- Tests: custom-provider routing assertions + 5 new transport alias tests.
Route the model-supplied target through _bound_error_text so a huge
bogus target can't bloat context, and restore the "Use 'memory' or
'user'" hint. Follow-up to HexLab98's review note on the salvage.
Normalize malformed memory config during initialization and bind per-target write permissions to the session MemoryStore so direct and staged writes cannot update a disabled built-in store.
Reuse the built-in store predicate during agent initialization and evaluate the config-backed memory tool check immediately after edits instead of applying the generic external-probe TTL.
A single urlopen(timeout=5) TimeoutError from the PS runspace listener
failed the test on a loaded runner (run 32440286339) even though the
listener recovered moments later — a transient stall is not the hang
this test guards. /progress sampling now retries until a deadline
(only a persistently unresponsive listener fails), the self-test hold
grows 10s -> 30s so retry time cannot push sampling past the held
stage, and the exit wait gets matching headroom.
The CLI (hermes cron create/edit) routes through cronjob(); removing the
parameter outright broke that lane (CI slices 6/9). The parameter is back
on the function, but CRONJOB_SCHEMA and the registry handler still omit
it — same pattern as the intentional model/provider/base_url omission.
New test proves a hallucinated reasoning_effort arg through the model
dispatch is dropped.
Standing policy: models do not make model-configuration decisions (the
only exception is user-defined profile selection in Bot Mode/kanban).
The per-job reasoning pin stays fully functional via
`hermes cron create/edit --reasoning-effort` and the job store; the
cronjob tool still SURFACES the pin in listings but cannot set it.
A schema-absence test pins the policy.
A cron job can now pin its own reasoning (thinking) effort, independent
of the global agent.reasoning_effort and per-model reasoning_overrides.
Heavy scheduled analyses can run at high while cheap recurring jobs run
at minimal, without touching the fleet-wide default.
- cron/jobs.py: new optional job field, validated at the storage choke
point against the canonical grammar via the shared
hermes_constants.parse_reasoning_effort (spelling-only; capability
clamping stays owned by the provider transports at send time, same as
config-set effort). Empty string clears on update; invalid values
raise ValueError before anything persists. Not a drift-guard axis.
- cron/scheduler.py: _resolve_job_reasoning_config resolves per-job pin
> agent.reasoning_overrides > agent.reasoning_effort at fire time,
after the auth-fallback model swap (the pin is model-independent by
design). A stored value that no longer parses warns and falls back to
config resolution instead of killing the tick.
- tools/cronjob_tools.py: reasoning_effort on BOTH mutation verbs
(create and update), conditional key in _format_job, schema documents
grammar/precedence/transport clamping/clear semantics. Agent-settable,
unlike model/provider pins: it cannot redirect spend to a different
model.
- hermes cron create/edit --reasoning-effort (empty string clears).
- Docs: cron feature page tip + CLI reference rows.
Tests: tests/cron/test_cron_reasoning_effort.py (32) — store contract,
scheduler precedence incl. byte-identical absent-field behavior and
garbage fallback, tool create/update/clear/error paths, schema surface.
Two review nits on the clarify ambiguity fix:
- _approval_send_outcome swallowed the failure detail the old inline
callers logged (scheduling exception text / SendResult error). Both
the approval and clarify lanes now share one warning with the detail,
logged in the classifier itself.
- The clarify tests pinned the disposition helper but nothing proved
the ambiguous branch actually reaches the bounded wait. The
send-then-wait sequence is extracted to _clarify_send_then_wait (the
callback closure now just binds context onto it) and the suite gains
caller-path tests: ambiguous/sent -> wait_for_response with the
generated clarify_id and configured timeout; definitive failure ->
sentinel without waiting; no-response timeout sentinel preserved;
plus caplog assertions that failed sends log their detail.
Relay + gateway sweep 289/289.
The clarify caller treated a send-scheduling timeout as a definitive
failure: clear_session() + '[clarify prompt could not be delivered]'.
Same physics as the approval card fixed earlier in this PR — the card
may well have posted with a late connector ack — so the teardown ran
out from under a rendered clarify card and the user's answer resolved
nothing.
New _clarify_send_disposition() routes the outcome through
_approval_send_outcome: only a DEFINITIVE failure (error result,
non-timeout exception, no future) clears the registration and aborts;
ambiguous logs a warning and falls through to wait_for_response, whose
existing bounded wait already handles the truly-lost-card case. This
makes the boundary rule stated in the ambiguity test docstring hold
for the clarify lane, not just approvals.
Tests: 5 disposition tests mirroring the approval suite, including
clear_session-not-called on timeout. Mutation-verified: folding
ambiguous into the failed branch sends
test_timeout_keeps_registration_armed_and_proceeds_to_wait red.
Relay + gateway sweep 283/283.
test_expired_own_prompt_notifies_instead_of_unknown_command asserted the
expiry notice synchronously after _consume_prompt_response returned. The
notice now rides a background task (read-loop self-deadlock fix: awaiting
a send from the prompt_response handler blocks the very read loop that
resolves the send's result future), so the test yields one tick before
asserting egress. Behavior contract unchanged: exactly one notice, no
chat dispatch.
Round 2 of the approval-turn stuck-stream hunt. Round 1 (interim-marked
acks) fixed the draft-hijack-by-matching path — live logs confirm the
absorption fallback no longer fires — but the freeze persisted because
of a second, deeper defect on the same codepath:
_consume_prompt_response executes ON the transport read loop (inbound
frame -> _handle_frame -> _inbound handler). The handler awaited
self.send() for its '✅ Approved once' ack — but send() blocks on an
outbound_result future that ONLY the read loop can resolve, and the
read loop is blocked inside this very handler. Guaranteed self-deadlock
for the full outbound timeout (30s) on EVERY button tap. While wedged,
everything on the transport starved: draft appends (the frozen stream
right after approving), sibling approval-card sends (timed out into
'possibly-delivered' — the observed double-approval ambiguity), and the
turn's seal (timed out ambiguous -> plain-send fallback -> duplicate
final). Log signature was the tell: card-send timeout at tap time, no
absorption INFO, no seal-failed WARNING, no suppression line.
Fix: _send_lifecycle_ack() — acks ride a background task with strong
ref retention; the handler returns immediately and the read loop keeps
consuming, so the ack's own result frame resolves normally. Applied to
all six lifecycle sends (approval ack, slash-confirm ack + result text,
clarify acks, expiry notice). Acks are cosmetic by contract; failure
logs at debug and never breaks the reader.
Tests: new deadlock-shape test (gated transport send; handler must
return within 1s and the ack must still egress afterwards — RED on the
awaited version via TimeoutError at the exact deadlock), prior 3 tests
green with a yield for the background task. Targeted sweep 203/203.
Live finding (rc.4 staging, 100% reproducible on approval turns): after
resolving an exec-approval prompt_response, the adapter sends a short
ack ('Approved once'). _prompt_reply_metadata carried only placement
metadata (thread_id) — no per-turn identity, no interim marker — so
send()'s single-open-stream fallback (review B2) matched the approval
turn's OWN live draft and sealed it with the ack text. From there,
silently: every later append died on the post-seal tombstone (built for
millisecond stragglers, deliberately quiet), freezing the visible draft
mid-word; the turn-final found no open draft and fell through to a
plain send — the duplicate 'fallback' message. No suppression line, no
seal-failed warning: the log signature was pure absence.
Fix: _prompt_reply_metadata stamps _interim_send=True, which send()
already honors by bypassing draft matching. One source covers the whole
lifecycle class (approval ack, slash-confirm ack, prompt-expired
notice — all six call sites route through it).
Observability (the quiet parts, out loud):
- single-open-stream absorption now logs at INFO with the absorbed key;
- the FIRST post-seal tombstone swallow per draft key logs at WARNING
(bounded FIFO dedup) — one swallow is the normal straggler race, a
burst means a live stream was sealed mid-flight by someone else.
Tests (RED-first: both ack tests failed on the unfixed adapter at the
'draft still armed' assertion): approval ack leaves the open draft
armed and egresses as a plain send op; expiry notice same; regression
control pins the B2 contract — a real identity-less turn-final still
absorbs into its single open stream.
Three live findings from rc.4 staging, all on the relay-fronted Slack
path, all with the failure observed in live logs before the fix:
1. Approval-send timeout is AMBIGUOUS, not failed (no re-ask).
send_exec_approval through the connector can time out with the card
already rendered — the connector may ack after the deadline (slow
platform API call, transient backpressure, event-loop stall) — and
the timeout-as-failure path re-sent and produced duplicate cards.
The outcome is now tri-state: sent / failed / ambiguous. Ambiguous =
no re-send, no text fallback; the prompt registration stays armed so
a late tap still resolves. Only a definite send error falls back to
text.
2. pending_approval tool results forbid re-issuing the command.
With one card correctly armed, the agent could still mint a SECOND
card by re-running a rephrased variant of the gated command after
reading the pending_approval tool result (observed live: same
command re-issued in a different form, two cards). The tool message
now instructs: do not re-run/rephrase; wait or report pending.
Applied to both the terminal and execute_code arms.
3. Draft interim AND seal frames carry format_hints.
format_hints are stamped on send, edit, and send_for_platform, but
both draft-frame builders (send_draft interim + _seal_open_draft
seal) shipped bare metadata. A streamed final therefore arrived at
the connector hintless and sealed as a plain code block while
non-streamed sends rendered native markdown blocks (observed live:
language-tagged block on send/edit, downgrade on streamed seal).
Both sites now stamp _with_format_hints_for_chat
(destination-resolved, same pattern as the existing lanes).
Verified live after the fix against the platform's stored message
payload: rich_text_preformatted with language field on a streamed
seal.
Tests: tri-state outcome unit tests (5), draft/seal hint stamping + knobs-
off regression control (2, RED-first), existing format-hints suite intact
(14/14). Mutation-verified: reverting the adapter hunk sends
test_draft_interim_and_seal_frames_carry_hints red; restore -> green.
Boundary sweep (text egress lanes crossing the frame contract): send ✓
(pre-existing) edit ✓ (pre-existing) send_for_platform ✓ (pre-existing)
draft-interim ✓ (this PR) draft-seal ✓ (this PR); task_card lane carries
no text content — exempt.
sol-reviewer findings: multiplex secret-scope regression test (global
stamp must not leak into a scoped profile; scoped stamp must trigger
the sweep), WARNING-vs-INFO log-level assertions, and a check that
_enabled_explicit never survives config load.
- Add x-preview-f-free (Ox Alpha: free, 1M context, ZDR) plus all newly
listed Zen models (gpt-5.6 sol/terra/luna, claude-opus-5, gemini-3.7/3.6
flash + lite, grok-4.6/4.5, muse-spark-1.2, kimi-k3, qwen3.7-max,
hy3-free, laguna-s-2.1-free, nemotron-3.5-lightning-free,
muse-spark-1.2-contributor-free) and Go models (gpt-5.6-luna, grok-4.5,
glm-5.3, qwen3.8-max, hy3, hy3-preview, muse-spark-1.2-contributor).
- Drop delisted north-mini-code-free from Zen.
- Route grok-* on Zen and Go through /v1/responses per the published
endpoint tables (grok-4.6/4.5/build-0.1 on Zen, grok-4.5 on Go).
- 1M context fallback for x-preview-f (Ox Alpha).
- Refresh hermes setup provider samples for both providers.
Catalogs verified against live GET /zen/v1/models and /zen/go/v1/models
plus https://opencode.ai/docs/zen/ and /docs/go/ endpoint tables (2026-08-20).
OpenCode Go and Zen serve muse-spark* only on /v1/responses.
Hermes was sending /chat/completions, which returns HTTP 503
with an empty assistant message. Match the published endpoint
table and the existing gpt-* routing.
- Route muse-spark* to codex_responses on opencode-go and opencode-zen
- Add regression assertions next to the gpt-5.6-luna cases
Deployments that intentionally mix connector-fronted and direct ingress
can set GATEWAY_RELAY_ALLOW_DIRECT_PLATFORMS=true to keep directly-
connected messaging adapters enabled beside the relay. Unset, the
GATEWAY_RELAY_URL env stamp keeps its exclusive behavior. Like the
trigger, the opt-out is a deploy-stamp env var, not config.yaml.
A GATEWAY_RELAY_URL set in the process environment marks a
connector-fronted deployment where the connector owns every platform
connection. A directly-connected messaging adapter in the same process
is a second, unmanaged ingress path: it causes duplicate deliveries and
split sessions, and its live socket disarms scale-to-zero.
At the end of _apply_env_overrides, after all enablement passes, the
env stamp now disables every other enabled messaging platform:
- Explicitly-enabled platforms (config.yaml enabled: true) are disabled
with a WARNING that names the platform.
- Credential-auto-enabled platforms are disabled with an INFO line.
- Non-messaging surfaces (local, api_server, webhook) are untouched --
the same exclusion set as the scale-to-zero arm gate.
- gateway.relay_url in config.yaml alone (no env stamp) keeps the old
additive behavior: relay runs beside direct adapters.
With the gateway owning the suspend, the idle predicate covers every
work source (agent turns, cron jobs, API-server runs, background work,
fail-awake on unreadable sources) and the relay drains + flips before
the freeze, so a long timeout no longer buys safety — real work always
blocks the suspend and resume is sub-second. 5 idle minutes just bills
idle RAM. Per-instance override stays config.yaml
gateway.scale_to_zero.idle_timeout_minutes (D2).
New behavior-contract test: invalid config values degrade to the module
default (whatever it is), never zero/negative — asserts the RELATION,
not the literal, per the no-change-detector-tests rule.
Address sol-reviewer findings:
- The shared shutdown-drain counters swallow exceptions to 0 — fine for
a drain, unsafe for a suspend predicate (a transient read failure made
live work look idle, reopening the mid-job freeze). The suspend path
now reads both sources itself and treats an unreadable source as work
(sentinel 1, fail-awake) with a debug log. A MISSING api_server
adapter remains a normal not-work state.
- is_idle()'s parameter renamed running_agent_count -> active_work_count:
it receives the broad aggregate, and the old name invited future
callers to pass only agents again.
- New failure-path tests: unreadable cron source and unreadable API
source each hold the machine awake (both fail against the fail-open
shape); missing adapter stays idle-capable.
The config key is declared in DEFAULT_CONFIG and shown in the auxiliary
config UI, but the judge path never read it — a user raising the timeout
for a slow-but-healthy endpoint got the same 30s cap, and the loop
auto-paused on transport failures advising a provider/key check. Mirror
the _goal_judge_max_tokens reader and resolve the timeout at call time;
explicit timeout= arguments still win.
Fixes#91022
`uv pip install -e .` never audits an editable target. It reinstalls on every
invocation and rewrites the console-script shims each time, which is the only
reason `hermes update` has to quarantine the running `hermes.exe` on Windows —
and a quarantine that loses its race is the whole `os error 32` family.
Gate the reinstall on whether the pull actually touched a file that defines the
install. It's safe to skip because the editable finder is pinned to a static
module list (`py-modules` + `packages.find.include`), so the one source-only
change that could stale it — a new top-level module or package — cannot land
without a `pyproject.toml` diff. Dependencies and `[project.scripts]` live
there too, and new submodules inside an already-mapped package resolve through
the real directory.
The predicate fails closed: no pre-pull SHA, an unresolvable one, or a failed
`git diff` all reinstall as before. On the skip path the two verifiers that
normally run inside the install run directly, so a wrong skip self-heals into a
real install rather than leaving an unchecked venv.
This is the pattern the file already uses everywhere else — `_tui_need_npm_install`
diffs node_modules against package-lock.json, and the desktop build is gated on a
content hash so `hermes update` "will skip if nothing actually changed". The
Python editable install was the one path with no such gate.
Teknium