Commit Graph

12854 Commits

Author SHA1 Message Date
Ben Barclay 613849c190 fix(telemetry): close the consent window on the config transition
Fourth independent review. Two more consent leaks, both reproduced through
the real relay entry point before and after the fix. Both are failures of
my own round-3 fix, which recorded revocation in the wrong place.

BLOCKER 1 - revoking while idle recorded nothing. _record_revocation lived
inside send_pending's loop, but _send_exported_packages returns early when
send is false, before a sender is ever constructed. The dominant case is a
user turning sending off while no pass is running, so the loop that was
meant to observe the revocation could never run. Reproduced: 6 periods
collected during a refused window were transmitted on re-enable.

The window now closes on the observed config EDGE, before the early return.
Last-seen send state is persisted because each hook fires in a fresh
process, so a true->false transition is only visible by comparison. The
rising edge also opens the window explicitly: the sender only runs when
there is something to send, so a user who opts in and out before any
package exists would otherwise have no window for record_revoked to close.

BLOCKER 2 - turning COLLECTION off never recorded revocation. The
not-enabled branch in setup.py force-set send=false and returned without
calling _record_send_consent_change, so `hermes tools` -> disable shared
metrics silently dropped consent while leaving the window open. Same
retroactive release on re-enable. Both consent surfaces now record, and
setup keeps the relay's edge detector in step.

Also, from the same review's mutation sweep:
- the scheme check is now pinned as an allowlist. Replacing the http test
  with `if True` survived the entire suite, because every non-http case
  targeted a REMOTE host where the loopback branch rejects anyway. Only a
  non-http scheme on loopback distinguishes the two. Shipped behaviour was
  already correct; nothing guarded it.
- A.3 no longer claims rotation bounds long-term linkability outright.
  Measured against 11 real packages: resource is a stable low-entropy
  tuple and periods are contiguous across a rotation, so for a RARE
  configuration those can bridge windows. The honest claim is that
  rotation raises the cost, not that it makes correlation impossible.

Two mutants are documented as unkillable rather than papered over with
tests that only appear to cover them: the _defer clamp is unreachable from
any current caller, and widening the falling-edge check to an
unconditional else is behaviourally equivalent because record_revoked is
idempotent and no-ops without an open window.

An earlier version of the anti-spurious-revocation test could not fail
either - it used a never-consented store, where record_revoked no-ops
regardless. Rewritten to opt in, revoke, re-enable, and then assert that a
steady enabled state does not re-close the reopened window.

259 tests pass. Staging E2E re-run: both packages 202.
2026-08-27 09:47:47 +10:00
Ben Barclay 8ddff33e4c fix(telemetry): head-of-line starvation and consent-revocation leak
Third independent review. Both blockers reproduced against a real store
before and after the fix.

BLOCKER 1 — head-of-line starvation. The claim query is LIMIT 1, and a
package already handled this pass was rejected AFTER the fetch, so
_claim_next returned None and send_pending read that as 'queue empty'.
Any row that sorts first and becomes eligible again mid-pass therefore
terminated the pass. This is reachable normally: a 429 with a short
Retry-After, or a pass outliving the 15-minute failure backoff (a legal
pass runs ~1900s). Measured: 10 of 19 healthy packages silently dropped.
The seen-set is now excluded IN SQL, so None genuinely means no eligible work.
Same scenario now delivers 19 of 19.

BLOCKER 2 — revoking consent leaked once it was re-granted. opt_in_period
was write-once, so packages collected while the user had send: false
still had period_start >= the ORIGINAL opt-in day; re-enabling released
the whole refused window. Reproduced: 5 packages from a 5-day opted-out
window transmitted on re-enable. Turning sending off now closes the
consent window, and the next enabled pass opens a new one from that day.
Recorded both in the setup wizard and in the sender itself, because
config.yaml can be hand-edited where the wizard never sees it.

Also: a send_attempts ceiling (a poisoned head row burned ~160 requests
over 30 days, unbounded), _defer clamps to >= 1s so it cannot write a
past deadline, and the dead skipped_not_due field is removed.

Test-quality fixes, since vacuous tests have been the recurring problem:
- the lease test asserted only 'in the future', passing for a 1s lease;
  it now requires the lease to outlast one package's worst legal case
- test_shutdown_joins_the_send_thread grepped getsource for a method
  name — a change-detector AGENTS.md rejects — and is now behavioural
- gzip determinism was unguarded: both retries in one pass compress in
  the same second, so removing mtime=0 was caught by nothing. Now
  compares output across a real second boundary.

All five new regressions are mutation-verified: reintroducing each bug
fails its test. The first attempt-ceiling test SURVIVED its mutation
(the seeded row was excluded by another predicate) and was rewritten to
drive the real loop.

251 tests pass. Staging E2E re-run: both packages 202.
2026-08-27 09:09:47 +10:00
Ben Barclay d0a7144ba1 fix(telemetry): per-row claiming, mid-pass consent re-check, narrower 4xx
Second independent review found the lease fix incomplete. Reproduced
each finding before fixing.

BLOCKER — the batch lease expired mid-pass. _claim took up to 20 rows
under ONE shared lease, but a single package can legally consume ~96s
(three 30s timeouts plus 1s+5s backoff), so a full batch runs ~1900s
against a 180s lease. Later rows' leases expired while this pass still
held them, and another process re-sent them. Reproduced: 192s elapsed,
pkg-2 POSTed twice.

Packages are now claimed ONE AT A TIME, immediately before being sent,
so a lease only has to cover the package actually in flight. Verified:
same scenario now sends each package exactly once.

HIGH — revoking consent did not stop a running pass. The runtime read
send consent once before starting the thread, so a pass could keep
transmitting for minutes after a user set send: false, contradicting
the documented promise that it 'stops transmission immediately'.
Consent is now re-read before every package and fails CLOSED if it
cannot be established.

MEDIUM — all non-429 4xx were treated as permanent, discarding data.
403 is the ingest service's own origin guard: a Transform Rule or edge
misconfiguration would have permanently dropped every package sent
during the incident. Only 400 (malformed envelope) and 413 (over the
1 MiB cap) are terminal now; everything else retries.

MEDIUM — valid JSON that is not an object blocked the whole queue.
json.loads('["a"]') succeeds, then .get() raised AttributeError inside
the claim transaction, rolling it back and starving every healthy
package behind it. Payload shape and install_id are now validated, and
an unusable row is rejected individually.

LOW — the clock-rollback comment and test name claimed the opposite of
the code. The behaviour is right (a future issued_at means the recorded
age is untrustworthy, so reissue); the wording is now honest about it.

LOW — removed the stale HERMES_TELEMETRY_ENDPOINT reference left in
config_defaults after the override was deleted.

247 tests pass (was 234). Staging E2E re-run: both packages 202.
2026-08-26 17:10:43 +10:00
Ben Barclay 49757d5e39 fix(telemetry): address review findings on the shared-metrics sender
Independent review found the claim mechanism did not work. Reproduced
against the real store: two senders POSTed the same package.

The claim wrote next_attempt_at = now, but selection requires
next_attempt_at <= now, so a concurrent pass matched the same row
immediately. It now writes a LEASE INTO THE FUTURE
(_CLAIM_LEASE_SECONDS), which is what actually excludes another pass,
and expires by itself if a process dies mid-send. _mark is additionally
guarded on send_state so a straggler whose lease lapsed cannot
overwrite a completed send back to pending.

The old concurrency test could not fail: it raised AssertionError from
inside a transport, and _send_one catches every exception as a
retryable transport error. It now records what the second pass saw.

Also from review:

- shutdown() never joined the send thread; the join was only wired into
  deactivate(). A short-lived CLI therefore killed an in-flight send at
  exit, on the only cadence this feature has.
- Removed HERMES_TELEMETRY_ENDPOINT. AGENTS.md reserves HERMES_* for
  secrets, and a behavioural override here was a consent hazard: an
  inherited variable could silently redirect telemetry a user agreed to
  send to Nous. The staging E2E writes the endpoint into its throwaway
  profile instead, which also exercises the real config path.
- Added the  shared-metrics toggle that AGENTS.md requires
  as the third opt-in surface, delegating to the setup prompt so the
  consent rules stay in one place.
- Non-429 4xx (401/403/404/413/422) are now permanent. Only 400 was,
  so a wrong path or oversized body retried every 15 minutes for 30
  days until retention pruned it.
- The opt-in day is stamped when the user consents, not on the first
  send pass, which silently dropped the opt-in day whenever the next
  export crossed midnight UTC.
- gzip now uses mtime=0. The embedded timestamp made two sends of one
  package differ on the wire, so the 'byte-identical retry' E2E was
  comparing parsed bodies and could not have caught it. It now compares
  raw request bytes.
- Reconciled the three stale claims in relay-shared-metrics.md that
  said no remote-delivery path exists.

233 tests pass (was 213). Staging E2E re-run through the config path:
both packages 202, and the service logged both objects written to S3.
2026-08-26 16:31:52 +10:00
Ben Barclay 6fdf6f4d4a feat(telemetry): run the send pass off the export hook
Step 6 of the shared-metrics exporter, plus a loopback E2E.

_export now triggers an opt-in send pass on a daemon thread. The hook
runs on finish_task — the user's interactive path — so a 30s network
timeout there would be felt directly; the thread keeps that latency off
the caller. A test asserts _export returns in under a second while a
send is deliberately blocked.

At most one pass is in flight per process: a queued second pass would
add nothing, because the next hook fire picks up whatever is still
pending. Shutdown joins the thread for at most two seconds, then lets
it go — the packages remain in SQLite and go out on the next run, so
blocking a user's exit on a slow network is the wrong trade.

Sending is resolved per pass from the profile's own config, so turning
it off takes effect at the next hook fire without a restart.

E2E (tests/hermes_cli/test_shared_metrics_sender_e2e.py): the real
sender against a real HTTPServer on loopback — actual urllib, gzip,
headers and sockets rather than an injected fake. Covers delivery and
sent-state, 400/429/5xx handling, a retry sending byte-identical
bytes, gzip shrinking a realistic 120-metric package and the server
parsing it back, install_id never crossing the wire, the outbox file
staying untouched, and a dead server deferring without raising.

Wiring tests: 12, all negative-space properties — no send without
opt-in, no blocking, no pile-up, no crash propagation.
2026-08-26 15:49:25 +10:00
Ben Barclay 00c75cea33 feat(telemetry): send exported packages to the ingest service
Steps 4, 5 and 7 of the shared-metrics exporter: the send logic, the
consent gate, and backoff plus multi-process claiming. These arrive
together because the sender is not correct without all three.

Contract handling: 202 marks sent; 400 is permanent and never retried;
429 honours Retry-After (clamped to a day so a bogus value cannot park a
package); 5xx, timeouts and transport errors retry three times in-process
with 1s/5s/25s full-jitter backoff, then defer to a later pass.

Consent is gated on the package's PERIOD, not its creation time. A period
is split across packages created on different days, so a created-at gate
would send a period's tail while dropping its head and silently
undercount the opt-in day — data that looks complete and is wrong. The
opt-in day is recorded once and never moves, so toggling sending off and
on does not re-open the pre-consent backlog.

Rows are claimed in a write transaction, which is what stops two Hermes
processes sharing one database from sending the same package twice.
next_attempt_at persists backoff across restarts, so a hard-down service
is not retried on every task completion.

The body is recomputed from payload_json rather than stored a second
time: json.dumps is deterministic here (verified against the real outbox
— 11 of 11 files reproduce byte-for-byte), and the only mutable input,
the derived identity, is frozen on the row at first attempt. That keeps
retries byte-identical across a salt rotation for ~36 bytes instead of a
duplicate ~11 KB payload.

The outbox directory is never written to or deleted from. A 202 updates
SQLite only, because those files are the user's 30-day local history and
retention already owns their lifecycle.

Tests: 33. Two of them caught real defects in this commit — an
unreadable row aborted the claim transaction and blocked every package
behind it, and the compression assertions were passing through an
injected fake that bypassed the code under test.
2026-08-26 15:45:02 +10:00
Ben Barclay 7ffd454df6 feat(telemetry): derive the transmitted install identity via keyed HMAC
Step 3 of the shared-metrics exporter.

The shared-metrics doc commits that a remote exporter 'must not reuse
the persistent local identifier by default'. install_id is therefore
never transmitted: each package carries
HMAC-SHA256(local-only rotation salt, install_id) instead.

Within a 30-day rotation window the value is stable, so distinct
installs remain countable — the first question the data has to answer.
Across windows it changes, bounding long-term linkability. The
derivation is one-way, so the service cannot recover install_id.

The salt lives in telemetry_state next to install_id, so removing the
shared-metrics directory resets both together and the documented reset
behaviour keeps working with no second cleanup path.

Rotation is deliberately not a bare 'age > interval' check: a clock
that jumps backwards must not read as an expired salt, and an
unparseable issued-at reissues instead of raising.

substitute_install_id replaces exactly one field and copies rather than
mutating, so payload schema evolution stays a sender-side concern.

Tests: 19, including that install_id never survives substitution, that
no other field changes, and — the property that keeps retries
contract-compliant — that a package rebuilt from a FROZEN derived id is
byte-stable across a salt rotation while a fresh derivation is not.
2026-08-26 15:41:18 +10:00
Ben Barclay e5180ab3df feat(telemetry): add opt-in send config and send-state columns
Step 1+2 of the shared-metrics exporter.

Config: telemetry.shared_metrics.send (default false) and .endpoint
(default production), resolved by a new shared_metrics_send_config
module. Precedence is HERMES_TELEMETRY_ENDPOINT > config > default; the
env var exists so the live staging E2E never has to mutate a user's
config. send requires enabled and never implies it — that combination
is a misconfiguration the user believes is working, so it logs an ERROR
once per process rather than silently doing nothing. Plaintext
endpoints are refused unless the host is loopback, so a typo cannot
send telemetry in clear text.

Per AGENTS.md, outbound telemetry needs a user-facing opt-in, so
setup_telemetry now prompts for sending as a second, separate question
and force-disables send when collection is turned off.

Storage: six additive nullable columns on package_outbox for send
bookkeeping. The store schema version deliberately does NOT move —
_ensure_schema_in_transaction raises on any version it does not
recognise and has no forward-compatibility branch, so bumping it would
hard-fail an older Hermes, a second profile on an older build, or a
rollback, against the same file. Old readers select named columns and
never SELECT *, so the additions are invisible to them.

Also corrects the two places that promised telemetry is never uploaded
(config_defaults comment and cli-config.yaml.example); leaving them
would make them false privacy statements once sending ships.

Tests: 26 covering config precedence, the enabled/send relationship,
transport safety, fresh-database creation, upgrade from a pre-send
database (rows preserved, version pinned, idempotent), and that the
shipped export query still runs. Mutation-checked: bumping the schema
version fails 5 of them.
2026-08-26 15:39:47 +10:00
webtecnica cc5ff96f9e fix(macos): stable TCC anchor for uv-managed python interpreter (#85345) 2026-08-25 22:10:12 -07:00
Teknium a329346f4f test: point drain-timeout assertion at _CUA_INSTALLER_DRAIN_GRACE
The #87196/#87720 conflict resolution kept the bounded-drain helper and
its constant; the windows_only kill-tree test still asserted the dropped
_CUA_INSTALLER_REAP_TIMEOUT name. Same 2-communicate contract, surviving
constant.
2026-08-25 21:59:52 -07:00
Teknium 23c3f5086e feat(update): unattended-safe cua-driver refresh with fail-fast preflights
Re-enables the routine confirmed-upgrade path on Windows that #95008
deferred wholesale, now that every unattended-hostile surface is closed:

- stdin=DEVNULL (salvaged #79871): upstream's Read-Host consent prompt
  can't block a hidden console.
- Bounded post-kill drain (salvaged #87720): a kill-surviving descendant
  holding the stdout pipe can't strand the update past its ceiling.
- 120s background ceiling (salvaged #87196): safe now that a legitimate
  600s lock wait can't occur on this path.
- NEW lock preflight: upstream's install lock held by a live process ->
  skip in ~0s instead of eating its 600s stale-lock window (the actual
  11-minute hang observed 2026-08-25; UAC was a red herring — base
  install is no-admin by upstream design).
- NEW 5s network preflight: github.com unreachable -> skip immediately.
- Windows unattended runs pass -NoAutoStart, skipping the ONLY
  install.ps1 branch that self-elevates (autostart task re-registration).
- Timeout diagnosability: partial installer output is logged on kill so
  the next hang names its stage instead of dying silently.

Contract repairs and fresh installs stay interactive-only (SmartScreen /
first-time elevation legitimately need a human).
2026-08-25 21:59:52 -07:00
blunkjamie-dev 105cea64f7 fix(update): bound optional cua-driver refresh 2026-08-25 21:59:52 -07:00
Jack Lau 65fada0945 fix(update): bound the cua-driver installer drain after a failed kill
On Windows, `hermes update` can hang past its own 660s cua-driver timeout
until the user kills an orphaned PowerShell by hand. The timeout ceiling is
not the problem; the code that runs after it is.

`_run_cua_driver_installer` handles `TimeoutExpired` by killing the process
tree and then draining the pipes with a bare `proc.communicate()`. The kill
is best-effort by construction: every `psutil.Error` in `_kill_installer_tree`
is logged at debug level and stepped over, on the reasoning that a partly
killed tree beats none. That is the right call, but it means the drain has to
survive a partial kill, and an unbounded drain does not.

The concrete case is the one reported. `install.ps1` self-elevates through
`Start-Process -Verb RunAs`, so the descendant runs at High integrity and a
medium-integrity `child.kill()` raises `AccessDenied`. The per-child handler
logs it and continues. That survivor is still holding the `stdout=PIPE` write
handle it inherited, so the following `communicate()` waits for an EOF that
arrives only when somebody kills that process manually. A bounded 660s wait
becomes an unbounded one, after the warning has already printed.

Bound the drain instead. A kill that landed closes the pipe immediately, so
this costs nothing on the normal path; a kill that did not costs 15s rather
than forever. The original `TimeoutExpired` is re-raised either way, so the
existing manual re-run hint still prints and the update unwinds. Losing the
tail of a timed-out installer's log is the cheaper half of that trade, and it
is only lost in the case where the run already failed.

The drain deliberately does not close the pipe handles. `communicate()`'s
reader threads are still blocked on them and closing underneath them races;
they are daemon threads, so abandoning them does not hold the interpreter
open.

Both timeout handlers (streaming and captured) now go through one helper.
The streaming child inherits the console rather than a pipe, so it is much
harder to stall there, but the two branches should not drift on a rule this
small.

Tests: 5, in a new `TestInstallerTimeoutDrainIsBounded`. Two fail without the
fix, including the reported scenario end to end (a child kill refused with
`psutil.AccessDenied`, asserting the drain still carries a deadline). The
deadline is asserted as a kwarg rather than by timing, because a test that
proved the hang by hanging would be the same defect wearing a test's name.

Scope note: this does not touch the `stdin` inheritance that lets
`install.ps1`'s `Read-Host` block in the first place. That is #79684 and open
PR #79871 already carries the one-line `stdin=DEVNULL` fix; the two are
independent and neither subsumes the other, since `DEVNULL` cannot unblock a
UAC elevation dialog.

Fixes #87703
2026-08-25 21:59:52 -07:00
Kudakwashe Paradzayi 76faeb969b test: require Bot Chat preview under a live writer
The hang bound alone would pass if read-only open degraded to None.
2026-08-25 21:59:38 -07:00
Kudakwashe Paradzayi 458f26c0dc fix(desktop): stop the Bots roster hanging on a live profile
profiles.list opened every profile state.db as a writable SessionDB,
which waits out write-lock patience while that profile's backend is
mid-turn. The desktop RPC timed out and Bot Mode's infinite React
Query retry kept the sidebar on a spinner.

Inspect those DBs read-only and bound roster retries so names still
paint.
2026-08-25 21:59:38 -07:00
Teknium 6cb3a26353 fix(doctor): classify certificate-anchored DRs as stable + keep explicit env_map hermetic
Two follow-ups on top of the #86391 salvage:
- check_macos_tcc_grants: a certificate-anchored DR (hermes desktop
  --setup-tcc-identity, or a notarized release) now reports as stable in its
  own class instead of falling into the identifier-pinned message; the
  identifier-pinned message points at --setup-tcc-identity for the strongest
  anchor.
- collect_relay_plugin_cutover_findings: only merge process-level env vars
  when env_map is None (run_doctor's live path). An explicit env_map is a
  complete environment description — merging os.environ on top made
  report_deprecated_config_and_env non-hermetic on boxes exporting legacy
  relay vars (10 findings vs the expected 2 in
  test_report_does_not_count_as_blocking_issue).
2026-08-25 21:59:36 -07:00
David Metcalfe 2d37ed056e fix(macos): harden TCC check against codesign timeouts, clarify scope
Review feedback (AI review on #86391):
- guard _macos_desktop_dr subprocess.run against TimeoutExpired/FileNotFoundError
  so a hanging codesign degrades to the unreadable-DR warning, never crashing
  the doctor run (matches the file's existing subprocess guard pattern)
- select the desktop bundle by newest-mtime across release/mac-*/Hermes.app,
  matching _desktop_packaged_executable, instead of a fixed arch order
- note the cdhash-match proxy assumption at the classification site
- document why /Applications/Hermes.app (Hermes-Setup launcher,
  com.nousresearch.hermes.setup, certificate-anchored) is deliberately not probed
- extend the repair hint to cover per-service resets
- regression tests: codesign timeout and missing-codesign paths
2026-08-25 21:59:36 -07:00
David Metcalfe 8665b1e4db fix(macos): guard empty DR in TCC check, tighten platform-guard test
GPT-OSS review: an empty codesign output would fall through to the
'stable identity' branch and false-positive. Guard with  and
cover the empty-string case. Flash review: the non-macOS silence test
mocked the bundle to None, so it never exercised the platform guard;
mock a real path instead.
2026-08-25 21:59:36 -07:00
David Metcalfe 36c1755065 fix(macos): detect stale TCC grants and guide one-time re-grant
TCC keys permission grants to the app's code-signing requirement. Grants
made to pre-#73681 builds carry a cdhash-pinned requirement that no
longer matches the rebuilt bundle, so macOS re-prompts on every capture
even though the System Settings toggle shows ON — and the modern prompt
has no Allow button, so users cannot complete the one-time re-grant.

- hermes doctor: new check_macos_tcc_grants() reports the desktop
  bundle's DR class (cdhash-pinned → grants reset on every update;
  identifier-pinned → stable) and prints the exact stale-grant repair
  (tccutil reset, toggle ON, fully quit & relaunch).
- hermes update: after a successful update on macOS with a desktop app
  installed, print the one-line stale-grant guidance.
- docs: desktop.md no longer claims grants persist 'out of the box';
  documents the one-time re-grant for pre-fix grants.

Closes #86385
2026-08-25 21:59:36 -07:00
Teknium afee35700e fix(delegation): suppress subagent-owned process notifications in parent chat by default
Background processes started by subagents (task_id sa-*) route their
notify_on_complete / watch_pattern notifications to the parent
conversation (b95ec1cb5) because anything outliving the child needs a
durable consumer. In practice these 'npm ci finished' walls are noise
mid-conversation — the child's consolidated delegation result is the
deliverable.

- New config key delegation.surface_child_process_notifications
  (default false = suppress). Flag true restores the previous behavior
  exactly (delivery with subagent attribution line).
- drain_notifications drops (never requeues) completion/watch_match/
  watch_disabled events whose task_id starts with 'sa-' when the flag
  is false, logging at debug with session_id+task_id for diagnosis.
  Requeueing would pin them forever — children never drain notifies.
- async_delegation events are NEVER suppressed (they ARE the result).
- watch_disabled emitters now carry task_id so sa- sessions' safety
  events follow the same suppression as their other events.
- Config read errors fall back to the default (suppress) and never
  crash the drain loop.
- Docs: delegation.md + configuration.md.
2026-08-25 21:59:04 -07:00
Teknium f751a8c546 fix(update): also defer the missing-binary CUA install on Windows
Follow-up to the salvaged #94296: the two guards covered the repair and
confirmed-update branches, but when cua-driver is enabled yet not
installed at all, control still reached _run_cua_driver_installer() and
an automatic 'hermes update' would launch the interactive install.ps1
anyway. Add the same defer before the installer run, keep POSIX
behavior unchanged, and give the confirmed-update message a natural
fallback when latest_version is unknown.
2026-08-25 16:53:01 -07:00
Royalaid 0c23bf19af fix(update): defer interactive CUA installs on Windows 2026-08-25 16:53:01 -07:00
Teknium a935154378 test(tool-search): make salvage-seam tests order- and stem-robust
Two interaction seams between the #92693 salvage (merged as #95050) and
this branch: the source-label indexing test now compares in token space
(the stemmer shortens 'catalogsource' to 'catalogsourc'), and the
unregistered-core-name describe test forces the unregistered condition
via monkeypatch instead of depending on which sibling test file imported
model_tools first.
2026-08-25 16:39:12 -07:00
alt-glitch 00305aec61 test(tool-search): kill the shared-stemmer mutant with cache-missing input
The parallel determinism test warms _stem's lru_cache after ~11 distinct
stems, so almost no iterations reach the underlying stemmer and a shared
(non-thread-local) instance survives it. New test bypasses the cache with
per-iteration unique tokens via _stem.__wrapped__, so thousands of stems
run concurrently: a shared stemmer's mutable parse state fails it within
2,000 calls (verified — the mutant dies 8/8 runs; healthy runs stay green).
2026-08-25 16:39:12 -07:00
alt-glitch 6d37f2b78a test(tool-search): exercise stemmer in parallel 2026-08-25 16:39:12 -07:00
alt-glitch 3b065745ca refactor(tool-search): keep batch caps internal 2026-08-25 16:39:12 -07:00
alt-glitch e35a7bddee perf(tool-search): cache stems and bound result metadata 2026-08-25 16:39:12 -07:00
alt-glitch b09f617722 fix(tool-describe): separate missing and direct names 2026-08-25 16:39:12 -07:00
alt-glitch 1e86a263ed fix(tool-search): preserve exact and per-query ranking semantics 2026-08-25 16:39:12 -07:00
alt-glitch e455e4afd0 feat(tool-search): multi-query search, batched describe, Snowball stemming
tool_search now takes queries: string[] (searched independently against
the same catalog, limit applies per query, default 5 / max 25) and
returns the split shape: per-query groups carry tool names only, one
shared tools map holds each matched tool's source, description (400-char
cap) and required parameter names once. When some queries miss, a single
top-level available_sources + hint block replaces the old per-response
fallback.

tool_describe now takes names: string[] and returns a map keyed by name;
unknown names collect in not_found (with the refresh hint) and
non-deferrable names keep their per-name spelling-check error in errors,
so one bad name no longer fails the whole call. Duplicates dedupe
silently.

The shared tokenizer now applies Snowball stemming (english, exact-pinned
snowballstemmer) at both index and query time, closing the measured
plural/singular miss where 'issues' failed to return create_issue. The
inline BM25 is unchanged. Stemmer instances are thread-local (they carry
mutable parse state and bridge dispatch can run on parallel tool-call
threads).

New config knobs under tools.tool_search: max_queries / max_describe_names
(default 10 each, floor 1, no upper clamp) bound the per-call array
inputs; over-cap calls error so the model repairs in one round-trip.

No backward compatibility with the single query/name shapes, by decision.
scripts/analyze_livetest.py renders both shapes since transcripts on disk
may predate this change.
2026-08-25 16:39:12 -07:00
Teknium 28f2ea86e2 fix(desktop): make --setup-tcc-identity produce a VALID signing identity on modern macOS
Fixes the two live E2E blockers @ctaylor86 found on PR #77189 (macOS 26.3.1,
OpenSSL 3.6.3):
- retry the PKCS#12 export with -legacy when security import rejects the
  OpenSSL 3 default format ('MAC verification failed during PKCS12 import')
- trust the self-signed root for the codeSign policy (security
  add-trusted-cert -r trustRoot -p codeSign) — an imported-but-untrusted cert
  is invisible to find-identity -v and unusable by codesign
- gate success on find-identity -v -p codesigning (postcondition), and use
  the same -v probe for idempotency so an untrusted leftover cert is repaired
  instead of reported as done

Tests rewritten as stateful fakes (valid only after import+trust), plus new
coverage for the -legacy retry, trust failure, postcondition gate, and the
untrusted-cert repair path; sabotage-verified (reverting to the name-in-output
probe fails 4 tests). Docs: manual fallback now includes the Trust step.
2026-08-25 16:24:19 -07:00
CriptoGus eeb391d0b1 feat(desktop): add --setup-tcc-identity to keep macOS TCC grants across rebuilds
Adds a one-shot `hermes desktop --setup-tcc-identity` command that creates a
self-signed code-signing certificate in the login keychain (openssl +
security import), grants codesign access to it, writes
desktop.macos_signing_identity to config.yaml, and re-signs the packaged app
with a certificate-anchored Designated Requirement.

macOS persists permission grants (Full Disk Access, Accessibility, Files and
Folders, microphone) against the app's code-signing identity, not its path.
The default identifier-pinned ad-hoc signature is stable across rebuilds, but
a certificate-anchored identity is the strongest guarantee — the same
mechanism yabai/skhd rely on. Previously users had to create the certificate
manually in Keychain Access; this command automates the whole flow and is
idempotent (re-run after updates).

Docs: desktop.md TCC section now leads with the command, keeps manual steps.
Tests: 4 new — fresh cert creation path, idempotent reuse, non-macOS no-op,
cmd_gui early-exit before build.
2026-08-25 16:24:19 -07:00
alt-glitch 62b2d78025 fix(tool-search): bridge batch barrier, listing truncation, source indexing (salvage #92693, part 1)
Four fixes to the tool-search deferral layer, split from PR #92693 (the
availability-cache staleness fix ships separately):

1. The parallel batch planner now peels the tool_call bridge wrapper and
   decides admission on the underlying tool — supports_parallel_tool_calls
   works again when deferral is active. Unparseable wrappers stay
   sequential barriers; bridged calls get exactly the admission the same
   call gets direct. tool_search/tool_describe lookups batch concurrently.
2. _short_desc no longer truncates listing lines at 'e.g.', hostnames, or
   version strings — a sentence terminator must be followed by whitespace.
3. BM25 indexes the source label (e.g. 'linear' for mcp-linear), so
   service-name queries reach tools whose own name omits the service; the
   dead 'mcp' prefix token is stripped.
4. Substring-fallback docstring corrected (token misses, not zero-IDF).

Salvaged from #92693 by @alt-glitch with authorship preserved.
2026-08-25 15:24:15 -07:00
Krzysztof Radzikowski 36f1423411 fix(update): gateway-only concurrent instances no longer abort hermes update (#37039)
On Windows, the pre-update concurrent-instance gate aborted with exit 2
whenever ANY other process held the venv hermes.exe shim — including the
gateway itself, which _pause_windows_gateways_for_update() stops moments
later and the post-update restart phase brings back. Users with a running
gateway were forced into a manual taskkill dance before every update.

The gate now filters gateway runtimes out of the abort list and proceeds
when nothing else is concurrent. Classification delegates to
_is_pausable_gateway -> gateway.status.looks_like_gateway_command_line
(the canonical shlex-tokenized, profile-selector-aware matcher shared by
the Desktop preflight exemption and the venv-holder guard fallback), so
the gate's exemption and the pause machinery cannot drift apart. Anything
not positively identified as a gateway — REPLs, dashboard, Desktop
backend children, gateway MANAGEMENT commands like 'gateway status',
unreadable cmdlines — still aborts exactly as before, and the abort
message now lists only the PIDs that are actually the user's problem.

Surgical reapply of PR #37039 by @damadorPL onto current main (the gate
moved from hermes_cli/main.py to hermes_cli/update_cmd.py in the main.py
decomposition); his substring classifier was replaced with the canonical
matcher, which also fixes the 'hermes gateway status' misclassification
flagged in review.

Co-authored-by: Hermes <hermes@nousresearch.com>
2026-08-25 14:48:51 -07:00
Gille 7c5c994397 fix(teams): request supported transcript content format 2026-08-26 02:54:35 +05:30
Jeff Mettel bee489d563 fix(update): restart a booted-out launchd gateway instead of silently skipping it (#74973, salvage #75021)
Port of @jeff-mettel's fix onto the post-#91378/#92902 fleet-restart
shape. The current-profile restart was gated on `launchctl list <label>`
exiting 0 - a booted-out job (plist present, definition deregistered:
crashed helper, manual bootout, failed prior update) fails that check,
so the branch silently skipped: no restart, no message, KeepAlive unable
to revive a definition launchd no longer knows, update printing
'Update complete!' with the gateway down. `launchctl list` is also
session-scoped and unreliable as a loaded/unloaded classifier.

- _restart_launchd_gateway_after_update() (his extraction, adapted):
  plist-exists is the ONLY gate; launchd_restart() owns the
  bootout/bootstrap/kickstart ladder for every plist-present state;
  every failure path is loud and names the manual recovery command.
  The gate-error 'except: pass' (the second silent variant) now counts
  the label failed and tells the operator.
- Success still requires the #92902 supervision verify (fresh
  supervised PID), composing his fix with the returned-is-not-supervised
  guard.
- His regression suite adapted to the (restarted, failed) contract; the
  old 'unregistered -> left alone' pinning test FLIPPED - it pinned the
  bug.

A/B: his suite + the flipped test red on merge-base product code
(silent skip live), green at head. No macOS CI lane exists; field
evidence is #74973's reproductions plus the launchctl print output
shapes pinned in the suite.
2026-08-25 14:10:10 -07:00
Teknium 9de5460c12 feat(cron): acked failure signatures stop re-pinging — durable incidents + ack CLI (salvage #94692) (#95017)
* feat(cron): durable failure incidents with signature dedup and ack

Introduce a durable cron incident store (cron_incidents in the shared
cron/executions.db) that groups "same job + same error signature" across
runs, so a known recurring failure stops re-pinging the operator every run
once it has been acknowledged.

- cron/incidents.py: lazily-created incident table (detected -> alerted ->
  reviewed -> closed lifecycle; closed is per-signature terminal), sha256
  signature dedup over job_id + normalized error, redacted/truncated error
  storage, failure-type classification, and ack/list/get/count helpers.
- cron/scheduler.py: record an incident on the failure delivery path and
  suppress the per-run failure ping when the exact signature is acked (both
  the normal failure path and the processing-raised retry path). Best-effort:
  an incident-store error never breaks the cron run or delivery. Streak nudge,
  alert-once markers, and delivery-error behavior are untouched.
- hermes_cli: add `hermes cron incidents [--state ...]` and
  `hermes cron incidents ack <id>`.
- tests/cron/test_cron_incidents.py: dedup, lifecycle, redaction,
  classification, lazy-schema, scheduler gating, and CLI coverage.

Non-goals deferred to later slices: Discord buttons/review view, HMAC action
tokens, owner-agent review launch, approval-gated fixes, incident playbooks.

* refactor(cron): tighten incident lifecycle, wire alerted state and suppressed_acked outcome

Follow-ups on top of the salvaged #94692:
- Drop the dead 'reviewed' state and the SQLite CHECK (state validity
  lives in INCIDENT_STATES so future slices can add states without a
  table rebuild); lifecycle is detected -> alerted -> closed.
- Actually mark incidents 'alerted' after a failure ping reaches
  delivery, on both the normal and exception delivery paths.
- Record ack-suppressed runs with a distinct 'suppressed_acked'
  delivery outcome (registered in cron_health monitoring) instead of
  the ambiguous generic 'suppressed'.
- Drift-skip alerts explicitly bypass the ack gate (they carry the
  remediation command and alert once via drift_alerted already).
- Docs: failure-incidents section in the cron guide.
- Tests for the alerted transition + never-resurrect-closed.

---------

Co-authored-by: Laura López Real <113060513+laulopezreal@users.noreply.github.com>
2026-08-25 14:02:40 -07:00
Jeffrey Quesnelle 5b82658b3c Merge pull request #90129 from rroverin/fix/nvidia-nim-tool-message-name-field
fix(providers): strip name/tool_name from NVIDIA NIM tool messages
2026-08-25 16:39:36 -04:00
Teknium 4032a15ad0 refactor(prompt): remove the ~1.2K-token Nous Subscription block from the system prompt (#95005)
* refactor(prompt): remove the Nous Subscription block from the system prompt (~1.2K tokens/call)

* chore: retrigger CI (zero-job dispatch failure, auto-heal)
2026-08-25 12:59:29 -07:00
andyst-dev def7bdc638 fix(cli): don't re-run npm install on every TUI launch with npm>=10 reduced hidden lockfile
_tui_need_npm_install compared every field of the root package-lock.json
against node_modules/.package-lock.json. npm>=10/11 writes a reduced hidden
lockfile that omits declarative fields (version/dependencies/dev) and adds
extraneous, so nearly every package looked 'changed'; workspace link entries
("link": true, paths outside node_modules/) are never materialized by the
partial --workspace install. Both made the check return True forever, so
hermes --tui re-ran npm install (and dirtied package-lock.json) on every
launch (#84617).

Compare only the keys both sides record with non-null values (resolved,
integrity, ...), ignore workspace link entries and non-node_modules paths in
the missing-entry check, and treat extraneous as an npm runtime annotation.
Real skew (lockfile bumped while node_modules is behind) is still detected.
2026-08-25 12:09:08 -07:00
PRATHAMESH75 a96bad8e71 fix(cli): scope TUI npm-install closure to all selected workspaces
On Termux the launch install also selects ui-tui's child packages/*
workspaces (include_child_workspaces=True), so npm installs each child's
devDependencies. The freshness closure only followed devDependencies for
the ui-tui workspace itself, so a devDependency unique to a selected child
was dropped from the closure and a genuine missing package slipped past
_tui_need_npm_install.

Derive the closure from every workspace the install path selects, following
devDependencies for each. _npm_lock_workspace_closure now accepts the set of
selected workspace keys (dev-included roots); _tui_selected_workspace_keys
mirrors _make_tui_argv (ui-tui, plus child packages/* on Termux). Adds a
child-workspace-devDependency regression test (installs on Termux, ignored
off Termux) plus a closure-level dev-scope test.
2026-08-25 12:09:08 -07:00
PRATHAMESH75 0c47cd5260 fix(cli): scope TUI npm-install check to the ui-tui workspace closure
_tui_need_npm_install compared the full multi-workspace root
package-lock.json against the hidden .package-lock.json, but the launch
install is scoped with npm install --workspace ui-tui and only writes the
ui-tui dependency closure. Every dep belonging solely to another workspace
(apps/desktop, web, ...) was therefore reported as missing, so the check
returned True and printed "Installing TUI dependencies..." on every launch.

Restrict the comparison to the ui-tui workspace's dependency closure,
computed from the root lock's packages map (following npm's node-resolution
walk and workspace symlinks). Standalone / own-lockfile layouts and any
case where the workspace can't be located fall back to the full comparison,
so drift on a genuine ui-tui dependency is still detected.

Fixes #66978
2026-08-25 12:09:08 -07:00
ethernet cbd8de8ad6 fix(gateway): bind session cwd for the live system-prompt rebuild
The function _persist_live_session_system_prompt runs on the RPC
dispatcher thread. A model switch calls it. On that thread the
_SESSION_CWD contextvar is not set. Because of this,
resolve_agent_cwd() falls back to the process TERMINAL_CWD value.
The desktop pins TERMINAL_CWD to the home directory. The rebuilt
prompt then records the home directory as the working directory.

The wrong prompt persists to the session database. Later turns
restore the stored bytes without change, because the turn prologue
rebuilds the prompt only when the cache is empty. The wrong line
never heals. The terminal tool is not affected, because it reads
the per-session cwd record.

The desktop composer sends a model switch before the first turn
when its model differs from the config default. Because of this,
a new project session can show the wrong working directory for
its full life.

Fix: bind the session context around the rebuild, in the same way
the function already binds the profile home for issue #50233. The
test drives the real function from a bare thread and asserts the
persisted prompt carries the session cwd.
2026-08-25 09:47:36 -07:00
Teknium ba9fc55e16 feat(web): cache_exempt_hosts — always-live fetches for staging/tunnel sites
Sites under active development but tested over the public internet
(Vercel previews, ngrok tunnels, staging domains) are public DNS, so
the local-dev never-cache rule can't catch them. web.cache_exempt_hosts
lists hosts whose pages are always fetched live: exact, "*.wildcard",
or domain-suffix matching (label-boundary aware — mysite.dev covers
preview.mysite.dev but never evilmysite.dev). Checked on both store
and lookup, so adding an exemption takes effect immediately even for
entries cached before the config change.
2026-08-25 04:21:45 -07:00
Teknium f0381ee4ae fix(web): never cache local development URLs in the extract cache
Dev servers, hot-reload builds, and chat-GUI artifact previews live on
localhost/private addresses and change on every save — a 20-minute
cached copy would show a stale build exactly when freshness is the
point of fetching. The extract cache now declines loopback, private,
link-local, *.local, *.localhost, and single-label LAN hostnames on
both put and get. Hostname heuristics only (no DNS) — this is a
freshness carveout, not a security boundary; SSRF enforcement is
unchanged in tools/url_safety.py.

Public URLs keep the full TTL.
2026-08-25 04:21:45 -07:00
Teknium 8adef09be8 fix(web): extract cache serves only after policy + provider gates; rescue and format/provider isolation
Review fixes for #94618 (all three blockers reproduced by the reviewer
through the real web_extract_tool):

1. Cache lookup moved AFTER provider resolution and strict-selection
   validation, and gated per-URL on the website blocklist policy — a
   blocklist-blocked or misconfigured-backend call now behaves exactly
   as it would without a cache instead of serving cached content.
2. Rescue-served extract batches are never cached (mirrors the search
   memo's exclusion), keeping one-shot rescue one-shot.
3. Cache entries now get dedicated per-(url, format, provider) files
   instead of sharing the URL-keyed truncate-store file — html and
   markdown (or two backends') copies of one URL no longer overwrite
   each other, and switching extract backends within the TTL never
   serves the old backend's rendering.

Also from review: per-process index tmp filename (cross-process writers
can no longer truncate each other mid-write) and held flight locks are
never evicted from the bounded lock table (eviction could have allowed
a duplicate paid request).

New regression tests for formats/provider keying; E2E harness extended
with policy-block, strict-selection, rescue-two-call, and dual-format
scenarios — 6/6 pass; original 13/13 still pass.
2026-08-25 04:21:45 -07:00
Teknium 04603fc040 feat(web): TTL result caching for web_search + web_extract
Repeat searches (same normalized query + provider) within a 20-minute
TTL are served from an in-process memo, and concurrent identical
queries are single-flighted so a parallel subagent fan-out pays for
one vendor request instead of N. Requested limits bucket up to
10/20/50/100 so near-identical requests share an entry; callers get
their requested count sliced from the bucket.

Repeat extracts of the same URL are served from the existing
cache/web full-text store (previously written for read_file paging
but never read back), via a small JSON sidecar index. Disk-backed, so
CLI, gateway, cron, and subagents share it. Cached extracts re-run
the normal truncate pipeline, so per-call char_limit still works.

Both caches sit after every safety gate (secret-URL, SSRF, policy,
provider resolution) and directly around the paid vendor call — hits
skip only the network request. Only successful responses cache;
rescue-served responses are never cached (one-shot rescue must stay
one-shot). Config: web.cache_enabled (default on),
web.cache_ttl_minutes (default 20, clamped 1-1440).

Idea credit: query coalescing + num-bucketing pattern observed in
Apodex FrontierAgent (Apache-2.0).
2026-08-25 04:21:45 -07:00
Jan-Stefan Janetzky 965689d13a fix(mcp-catalog): failed probe keeps the prior tool filter instead of wiping it
The probe-fail path ignored prior state entirely: with no manifest
default_enabled it wrote include=None, which pops the whole tools
block. For exclude-mode manifests default_enabled is necessarily unset,
and for the 30+ OAuth entries the entry rewrite precedes first auth —
so the common reinstall-while-unreachable case removed the curated
excludes and enabled every tool on next connect. The fallback order is
now: prior include > prior exclude > manifest default > no filter.
2026-08-25 04:21:37 -07:00
Jan-Stefan Janetzky fb1ec36a4b fix(mcp): treat tools.include: [] as an explicit empty whitelist
_normalize_name_filter([]) returns an empty set, which is falsy, so
_should_register fell through to "no filter" and registered every tool
— the exact opposite of what _apply_tool_selection wrote when the user
unchecked everything in the install checklist ("contributes nothing
until reconfigured"). Whitelist mode is now keyed on the include key
holding a valid filter shape (str/list/tuple/set) rather than on set
truthiness, at both the live-discovery and cached-manifest sites.
Invalid include values keep the old warn-and-ignore behaviour.
2026-08-25 04:21:37 -07:00
Jan-Stefan Janetzky 164e25a936 test(mcp-catalog): actually exercise the exfil-shaped-manifest rejection
The test wrote the evil manifest and imported install_entry but never
called it and asserted nothing, so the security gate in
_save_mcp_server/validate_mcp_server_entry was left uncovered. Now it
calls install_entry, expects CatalogError, and verifies the entry was
not persisted. Also drops the hardcoded ~/.hermes/ path from the
fixture args (AGENTS.md tests rule); the egress + exfil-hint shape
still trips both patterns.
2026-08-25 04:21:37 -07:00