Commit Graph

82 Commits

Author SHA1 Message Date
Teknium 23b9ffc4fa fix(integration): restore subprocess stdin=DEVNULL / utf-8 encoding guards and windows-footgun gates dropped by round-3 compaction
Repo scanners (check_subprocess_stdin, check-windows-footguns --all) flagged 21 sites where
the r3 single-line collapses lost stdin=DEVNULL, encoding='utf-8'/errors='replace', the
'# windows-footgun: ok' same-line marker, or the getattr(os, 'geteuid') gate. Each guard is
restored at the call site (real portability/hang fixes, not suppressions).
2026-09-03 02:46:19 -07:00
Teknium bb430e6990 refactor(tools): mcp_oauth inline provider-class builder + _fill_result, fold client-change guard; lazy_deps walrus guards, dist-roots/bytecode-warm folds, docstring compaction; gateway timestamp/token folds 2026-09-03 00:11:01 -07:00
Teknium 6663194b51 refactor(tools): lazy_deps ensure_and_bind/stamp-read folds 2026-09-02 22:53:19 -07:00
Teknium 3b906fc9ab refactor(tools): lazy_deps LAZY_DEPS annotation comments tightened, walrus target guard; mcp_oauth paste/dashboard folds 2026-09-02 22:46:23 -07:00
Teknium 55ba3184b7 refactor(tools): mcp_oauth CIMD eligibility comments inline; lazy_deps install_specs/ensure guard folds 2026-09-02 22:40:32 -07:00
Teknium 37ea7fe36a refactor(tools): mcp_oauth storage path partialmethods; lazy_deps contextlib.suppress ladders; gateway scheme guard 2026-09-02 22:27:21 -07:00
Teknium b39c93e993 refactor(tools): lazy_deps hand-compacted docstrings/comments (LAZY_DEPS table untouched) 2026-09-02 22:22:42 -07:00
Teknium 556f31e496 refactor(tools): mcp_oauth SDK loader -> _sdk_class, fold cached-redirect lookups and callback poll; lazy_deps installer helpers; managed_tool_gateway _clean 2026-09-02 21:55:04 -07:00
Teknium ae67178be1 refactor(tools/infra): split cronjob god-methods into per-action helpers; extract tool_search catalog/validation; compact registry, lazy_deps, tool_backend_helpers, desktop_ui 2026-09-02 14:47:02 -07:00
Teknium b6a3d3f440 refactor(tools): restore stdin=DEVNULL on subprocess probes dropped during compaction
voice_mode WSL playback probe, subagent_worktree._run_git, cua_backend
loginctl/xprop probes lost their explicit stdin= (and one line-wrapped past
the encoding= same-line rule); lazy_deps._run carries it via _SUBPROCESS_KW
so mark it for the guard. Fixes tests/tools/test_subprocess_stdin_guard.py
and tests/scripts/test_footgun_subprocess_encoding.py (green on base).
2026-09-02 14:45:42 -07:00
Teknium ec49ae7c0f refactor(tools): wire file_operations to extracted common/lint/search modules; restore literal security pins in lazy_deps 2026-09-02 14:43:45 -07:00
Teknium d4cec15b47 refactor(tools): first-wave simplification of tools/ (file ops split, lazy_deps, code_exec, approval, browser, delegate, mcp, skills, terminal, voice, media)
Behavior-neutral structural pass over tools/*: god-file extractions into
sibling modules (file_operations_common/lint/search, file_tools_paths/
read_tracking/write, code_execution_env/rpc, tool_search_catalog/names/
validation, tts_command_provider, ...), duplicate helper unification,
if/elif -> dispatch tables, dead-code removal, docstring compaction.
Tool schemas (get_tool_definitions) verified byte-identical to base.
2026-09-02 14:43:45 -07:00
Teknium f298911467 fix(lazy-deps): pass --compile-bytecode on the uv tier and skip metadata dirs in the warm
Follow-up to the #100829 salvage. uv pip install writes no __pycache__ by
default (pip does), so --compile-bytecode covers the whole install including
transitive deps, which the per-spec warm never sees. Also skip *.dist-info /
*.egg-info roots in _installed_dist_roots — they own no importable code.

Live: fresh cpython-3.12.13 venv, real uv install of anthropic==0.87.0 via
_venv_pip_install: main -> 0 pyc, first import 0.468s; after -> 1212 pyc
(546 anthropic), first import 0.205s.

Refs #100461
2026-09-02 00:03:55 -07:00
joaomarcos d380651a9f fix(lazy-deps): byte-compile lazily installed backends at install time
A pip/uv install writes .py sources and no __pycache__ — and reinstalling
the same version still deletes the cache the previous copy had. Nothing in
Hermes compiles them, so the whole compile is paid by whoever imports the
package next. For a lazily installed backend that is the foreground of a
user request, with nothing printed while it runs.

Measured for anthropic==0.87.0 (541 modules) on cpython-3.12.13: the first
import after an install costs 2.2-2.7s against 0.7-1.0s warm, and 10.5s
under concurrent load. N per-profile daemons cold-starting together each
pay it in full, because none of them has written the cache yet.

Compile the freshly installed distributions in _venv_pip_install instead,
on the success path of both the uv and pip tiers. The caller is already
waiting on an installer there and can see why. Package directories are
resolved from each distribution's own file list, so specs whose import
name differs from their package name (python-telegram-bot -> telegram)
are covered. Best-effort: a compile failure never invalidates an install
that succeeded, and sys.dont_write_bytecode is honored.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MhAnkrFktFdmZwf64fUYLE
2026-09-02 00:03:55 -07:00
Teknium a9e72f1b58 refactor(read_file): schema diet + bundle anydoc 0.2.4 + typed NeedsOcrError/hosted-OCR wiring (#97195)
* refactor(read_file): capability-gate the anydoc format list; PDF coverage teaching lives in the response-time warning (426 -> 244/291 tok/call)

* feat(read_file): bundle firecrawl-anydoc 0.2.4 in core, typed NeedsOcrError handling, config-gated hosted OCR with local-OCR-first guidance

* refine(read_file): NEEDS-OCR warning hints at checking for an OCR skill without naming one; hosted_ocr knob unadvertised (maintainer-directed)

* simplify(read_file): drop the anydoc schema gate — bundled core dep makes absence a broken install, not a variant; formats stated unconditionally (263 tok/call)

* feat(read_file): PDF wording upgrades to 'scanned or text' when a trusted hosted-OCR route exists (direct key or explicit config; nous gateway excluded until Parse proxy works)

* simplify(read_file): FIRECRAWL_API_KEY is the ONLY hosted-OCR gate — nous gateway route removed (Parse proxy broken), config true no longer unlocks; false still disables
2026-08-28 08:46:11 -07:00
xxxigm f84f94b494 fix(teams): do not call App() when the SDK was never bound
find_spec("microsoft_teams") can be true from sibling namespace packages
while App is still None, so a failed lazy-install crashed connect with
'NoneType' object is not callable instead of a missing-SDK error.
Probe microsoft_teams.apps via the parent first — a dotted find_spec
raises ModuleNotFoundError on 3.11 when the namespace is absent.
2026-08-25 12:07:33 +05:30
elphamale 11a9dcf567 feat(mcp): migrate to the mcp 2.x SDK
mcp 2.0.0 implements MCP revision 2026-07-28 and makes three breaking
changes Hermes sits on top of: `mcp.server.fastmcp` is gone, every model
field is renamed to snake_case (camelCase survives only as a
serialization alias, which pydantic does not expose to attribute
access), and the SDK's own HTTP stack moved from `httpx` to `httpx2`.

Bump the pin across the dev/mcp/computer-use extras and port the tree:

- `mcp_serve.py` and `agent/transports/hermes_tools_mcp_server.py` move
  from `FastMCP` to `mcp.server.MCPServer`, which has the same
  decorator/add_tool surface. The hermes-tools server already
  synthesised `__signature__` from Hermes' JSON Schema, which is exactly
  what 2.0's `add_tool` reads.
- SDK model reads go through `mcp_field(obj, snake, camel)`, which reads
  both spellings. A single-spelling read fails *silently* on the other
  generation — empty tool schemas, dropped structured content, tool
  results vanishing from sampling conversations — and `mcp` is an
  optional extra users install at their own version.
- `sdk_httpx()` resolves the httpx flavour from the SDK's own transport
  module, so objects handed to `streamable_http_client`, the `sse_client`
  factory, and the OAuth metadata helpers come from the module the
  installed SDK actually imports.
- HTTP support is gated on either streamable-HTTP entry point, not just
  the deprecated alias 2.0 removed.
- OAuth: `OAuthClientProvider` lost its `timeout` argument (the
  configured `oauth.timeout` now bounds the callback waiter's own poll
  loop, where the browser round-trip was always awaited), and
  `callback_handler` must return `AuthorizationCodeResult` rather than a
  tuple. 2.0 also validates the RFC 9207 `iss` parameter, so the
  callback handler and paste fallback capture it.

`mcp`/`mcp-types` 2.0.0 are inside the 14-day `exclude-newer` window, so
two narrow `exclude-newer-package` entries unblock `uv lock`, annotated
for removal on or after 2026-08-11. `httpx2` needs no exemption: 2.7.0 is
already outside the window and satisfies mcp's floor.

Refs #69931

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 23:26:10 -07:00
Tachi b67f202184 fix(update): honor lazy install opt-out during restore 2026-08-14 22:33:44 -07:00
Tachi 979a20052f fix(update): preserve activated extras across runtime rebuilds 2026-08-14 22:33:44 -07:00
Teknium fe8b44dac4 fix(ci): sync lazy_deps SDK pins + update WAL vacuum test contract
The Aug 12 pin bump (91345435a) updated pyproject/uv.lock but not
tools/lazy_deps.py, tripping the #31817 downgrade-guard tests; the
post-VACUUM TRUNCATE fix landed without updating the checkpoint test
that pinned the old no-TRUNCATE rule. Both red on pristine main.
2026-08-12 20:02:07 -07:00
Teknium 45f31de4e9 fix(deps): mirror aiohttp 3.14.3 pin into lazy_deps feature specs
tests/test_project_metadata.py and test_packaging_metadata.py enforce
that lazy_deps.py exact pins match pyproject extras and uv.lock.
2026-08-08 14:06:48 -07:00
fangliquanflq 8de3ddb9ef fix(tools): preserve document extraction boundaries 2026-08-08 05:13:46 -07:00
kshitij 99237a4444 refactor: derive teams install hint via feature_install_command(venv_pip=True)
Fold the remaining simplify-code reuse finding: teams' _install_hint()
duplicated lazy_deps' spec-fetch + quote + join (feature_install_command
already builds pip commands from LAZY_DEPS). Add a venv_pip=True variant
to feature_install_command — sys.executable -m pip targeting, correct in
every install layout and immune to PEP 668 — and shrink the teams helper
to a one-line call.  Also gives matrix and the other platforms a shared
derived hint to adopt later.  New test mutation-checked (fails when
venv_pip returns the uv form).
2026-08-07 13:28:43 +05:30
Teknium b2598b41e1 feat(read_file): widen document extraction to PDF/legacy Office/ODF/RTF/EPUB via optional anydoc
read_file's auto-extraction covered only the stdlib trio (.ipynb/.docx/
.xlsx). firecrawl-anydoc (MIT, Rust core, imports as `anydoc`) converts
Word, PowerPoint, Excel — including legacy .doc/.ppt/.xls — OpenDocument,
RTF, EPUB, and PDF to clean Markdown through one shared document model.

Wiring follows the footprint ladder: no new tool, no hard dependency.
- tools/read_extract.py gains an ANYDOC_EXTENSIONS set that is active
  only when the converter imports; the stdlib extractors remain
  authoritative for their three formats so behavior is identical with
  or without the package.
- tools/lazy_deps.py adds tool.doc_extract (firecrawl-anydoc==0.1.6),
  installed on first read of such a file with prompt=False so read_file
  can never block. Lazy-only for now: the package's first release was
  2026-08-04, inside uv's 14-day exclude-newer quarantine, so the
  mirrored pyproject extra lands after it clears.
- Any anydoc ConvertError maps to ExtractionError, falling back to the
  existing path/binary handling instead of erroring the tool.

Tests: real-binding suite skips cleanly when the wheel is absent
(verified: 15 passed/3 skipped without it, 18 passed with it), plus an
absent-dep contract class that pins the fallback regardless of local
install state.
2026-08-05 17:07:47 -07:00
liuhao1024 f07f47fe7d fix(lazy-deps): skip the install ladder on package-manager installs
Salvage of #48637 (Fixes #48628). On a NixOS-style install the venv's
site-packages lives in the read-only store, so ensure()'s
uv -> pip -> ensurepip ladder spends ~15s bootstrapping ensurepip only
to fail against a target it can never write. Fail fast with an
actionable message pointing at the system package manager.

Retargeted onto current main (the PR's base predates the durable-target
subsystem by ~8.1K commits) with two corrections to the original:

- Gate on _lazy_install_target() is None. The container deployment sets
  HERMES_MANAGED=true AND HERMES_LAZY_INSTALL_TARGET (a writable
  volume); the original guard would have blocked installs that path
  legitimately satisfies, breaking the NixOS-container mode.
- Reason string starts with 'unsupported ' because
  refresh_active_features classifies FeatureUnavailable by that prefix;
  the original wording made 'hermes update' report a hard failure
  instead of a skip.

Placed after _unsupported_feature_reason so a platform-specific reason
(more actionable) wins, and so ensure() agrees with
refresh_active_features, which pre-checks that same function.
2026-08-03 14:09:46 +05:30
ethernet 25d0bcd424 fix(runtime): resolve Hermes-managed Node and uv before bare PATH
Hermes installs runtimes for itself — `uv` at `$HERMES_HOME/bin/uv`, Node
at `$HERMES_HOME/node` — and neither directory is on an arbitrary
process's PATH. Every `shutil.which("node"/"npm"/"npx"/"uv")` in Hermes's
own code therefore has two failure modes: the managed runtime is invisible,
so the caller reports "not installed" or degrades to a slower tier on a
machine that has exactly what it needed; and when a system copy also
exists, the one Hermes does not own wins.

Routed the Hermes-owned call sites through managed-aware resolvers:

- `agent/lsp/install.py`, `hermes_cli/dep_ensure.py`, `hermes_cli/main.py`
  (`_make_tui_argv`), `hermes_cli/tools_config.py` (`_run_post_setup`) now
  use `find_node_executable()`.
- `hermes_cli/tools_config.py::_pip_install` and `hermes_cli/setup.py`'s
  vercel install use `ensure_uv()` (installing uv is in scope during setup,
  and the Windows installer's `uv venv` does not seed pip, so the fallback
  tier is "No module named pip"). `tools/lazy_deps.py` uses `resolve_uv()`
  — a lookup, not a bootstrap, because it runs mid-turn for an optional
  dependency and downloading a runtime as a side effect exceeds what the
  caller asked for.
- `hermes_cli/gateway.py`: extracted `_append_node_dir_for_service()`,
  shared by the systemd unit and launchd plist generators, which appends
  the managed dirs before the PATH-resolved one. A service definition is
  written once and survives reboots, so resolving a system Node that
  happens to lead the installing shell's PATH bakes the wrong interpreter
  in permanently. Managed dirs are profile-scoped, so each profile's unit
  still names its own Node; the existing symlink-parent rule (don't
  `.resolve()`) is preserved verbatim.
- `tools/environments/local.py`: the terminal tool's subshell PATH gains
  the managed dirs, appended alongside the sane entries rather than
  prepended — a tool the user deliberately put on their own PATH still
  wins, and the managed one only fills a gap. This is also what makes the
  bare `which("uv")` in `tools/env_probe.py` correct: that probe reports
  the environment the *model* sees, and the model can only run what is on
  that subshell's PATH.

`scripts/install.ps1`: the persisted User PATH update becomes
`Set-ManagedNodeFirstOnUserPath`, a move-to-front rather than an
add-if-missing. Installs made by an older install.ps1 already have the
managed dir in User PATH — at the tail, behind a system Node — and an
add-if-missing check sees it present and leaves that ordering in place
forever, so the users the bug hurt would never be repaired. Unrelated
entries keep their relative order (empty segments included; a trailing
`;` is legal and the installer's other PATH code preserves them),
duplicates collapse, and it writes only when the string actually changes.

Tests:

- `tests/test_managed_runtime_resolution.py` — AST guard that fails any
  new bare `which()` for a managed runtime, with a short justified
  allow-list and a companion test that fails when an allow-list entry goes
  stale. Reading source is banned by AGENTS.md and this is the documented
  exception: the property is "no call site anywhere spells it this way",
  which no runtime seam can observe.
- `scripts/ci/test_install_ps1_path_migration.ps1` — behavioral, not a
  source regex: it lifts the real `Set-ManagedNodeFirstOnUserPath` out of
  install.ps1's AST and rewrites only the two registry calls into an
  in-memory store, so the shipped split/dedupe/prepend/change-detection
  logic executes for real. Not in the default lane (Linux runners have no
  PowerShell host); runs under `pwsh`. 13/13 assertions pass.
2026-08-01 21:17:51 -04:00
Eugeniusz Gilewski 64dd865912 fix(deps): repair Google transitive security floors (#72108)
Google API and authentication packages permit vulnerable httplib2 and pyasn1
transitives, while the Workspace and Google Chat runtime installers previously
treated any importable version as sufficient. Existing environments could
therefore remain vulnerable after the project dependency pins were repaired.

Carry the fixed versions through the Google and Vertex extras, lazy feature
requirements, lockfile, and both runtime installers. Route the documented
Google Chat installation path through its maintained secure requirements
instead of an unconstrained direct pip command.

Detect stale distributions, install only unsatisfied requirements, and verify
the result before continuing. Behavioral tests cover those repair invariants
without freezing manifests, lockfiles, or complete package sets.

Related #72108
Extracted from #72840
Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
2026-07-31 23:18:38 -07:00
Yorkstone Supplies (sycamoregroupltd) a7c26bbb5c fix(deps): pin patched httplib2 for google extra 2026-07-31 22:28:21 -07:00
ethernet abcd213504 fix(sec): update pillow to 12.3.0
pillow 12.2.0 has 26 known vulnerabilities:

GHSA-45hq-cxwh-f6vc
GHSA-4x4j-2g7c-83w6
GHSA-5x94-69rx-g8h2
GHSA-62p4-gmf7-7g93
GHSA-6r8x-57c9-28j4
GHSA-8v84-f9pq-wr9x
GHSA-9hw9-ch79-4vh6
GHSA-fj7v-r99m-22gq
GHSA-jjj6-mw9f-p565
GHSA-pg7v-jwj7-p798
GHSA-phj9-mv4w-65pm
GHSA-vjc4-5qp5-m44j
GHSA-xj96-63gp-2gmr
PYSEC-2026-2253
PYSEC-2026-2254
PYSEC-2026-2255
PYSEC-2026-2256
PYSEC-2026-2257
PYSEC-2026-3451
PYSEC-2026-3452
PYSEC-2026-3453
PYSEC-2026-3454
PYSEC-2026-3493
PYSEC-2026-3494
PYSEC-2026-3495
PYSEC-2026-3496

they're fixed in >= 12.3.0
2026-07-31 13:42:03 -04:00
ethernet a7efeb0829 fix(sec): update mcp to 1.28.1
mcp 1.26.0 has 3 known vulnerabilities: PYSEC-2026-3481,
PYSEC-2026-3482, PYSEC-2026-3483

they're fixed in >= 1.28.1
2026-07-31 13:42:03 -04:00
Teknium c770515e2b modernize re-added Vercel integrations: SDK 0.7.2, telemetry off, sibling-site wiring
- Bump vercel SDK pin 0.5.7 -> 0.7.2 (pyproject, lazy_deps) and regenerate uv.lock
- Disable the SDK's new default-on telemetry (VERCEL_TELEMETRY_DISABLED=1
  set before import, user-overridable) per the no-opt-out-telemetry policy
- Move _model_flow_ai_gateway into hermes_cli/model_setup_flows.py (god-file
  decomposition landed after the removal)
- Widen post-removal backend sets that vercel_sandbox missed: terminal_tool
  container_backend + _CONTAINER_BACKENDS, file_tools fallback set,
  env_probe._REMOTE_BACKENDS, approval._should_skip_container_guards,
  prompt_builder probe container_config
- Add terminal.vercel_runtime to config_defaults + TERMINAL_CONFIG_ENV_MAP
- Re-add vercel dependency group to nix #full variant (reverts #33773 workaround)
- Update restored tests to current contracts: upload-only credential sync-back
  (bcfc7458fa), registry-derived provider env list, parametrized backend fixture,
  drop tests superseded on main (slack wizard move #41112, nous status format)
2026-07-29 19:48:37 -07:00
Teknium ad12df6ba4 Revert "remove Vercel AI Gateway and Vercel Sandbox (#33067)"
This reverts commit febc4cfec0.
2026-07-29 19:48:37 -07:00
victor-kyriazakos 1773752c8c Merge remote-tracking branch 'origin/main' into feat/gateway-health-diagnostics-monitoring
# Conflicts:
#	uv.lock
2026-07-29 15:37:14 +00:00
Shannon Sands 8bbd77f368 fix: route memory-provider dep installs through lazy_deps durable target
Installing a memory provider (Honcho, mem0, hindsight, ...) from the
dashboard Plugins page failed on hosted deployments with a permission
error: the setup endpoint shelled out to
`uv pip install --python sys.executable`, which targets the sealed
read-only venv under /opt/hermes (immutable hosted image, NS-579/#49113).

The correct mechanism already exists: tools/lazy_deps.py redirects
installs to the writable durable target on the data volume
(HERMES_LAZY_INSTALL_TARGET=/opt/data/lazy-packages) when the venv is
sealed (HERMES_DISABLE_LAZY_INSTALLS=1), appends the target to the END
of sys.path (core venv always wins collisions), and constrains shared
deps to core-venv versions. The dashboard installer simply never used
it.

Fix:
- tools/lazy_deps.py: new public install_specs() — installs arbitrary
  manifest-declared pip specs through the same environment routing as
  ensure(): venv-scoped by default, durable-target on sealed images,
  refused with an actionable reason when gated off (config kill switch
  or sealed venv without a target — never surfaces raw EROFS/EACCES).
  Specs are validated with _spec_is_safe(); post-install it invalidates
  import/metadata caches so availability rechecks in the same process
  see the new packages without a restart. Never raises.
- hermes_cli/web_server.py: _install_memory_provider_pip_dependencies
  now calls install_specs() instead of building its own uv/pip
  subprocess. Blocked installs surface the gate reason in the setup
  results; the response's status block reflects post-install
  availability (stale 'missing deps' state clears immediately).
- hermes_cli/memory_setup.py, plugins/memory/honcho/cli.py,
  plugins/memory/mem0/_setup.py: CLI setup wizards routed through
  install_specs() too — same sealed-venv failure mode, same fix.

No hosted setup path writes to /opt/hermes anymore; provider discovery
and installation now use the same environment (sys.path activation is
shared with the lazy-install bootstrap in hermes_bootstrap).

Tests:
- tests/tools/test_lazy_deps.py: TestInstallSpecs — gating matrix
  (sealed+no-target blocked with immutable-deployment reason, config
  kill switch, sealed+target proceeds), spec-safety rejection before
  any subprocess, venv-scoped vs --target command display, failure
  stderr passthrough, never-raises contract.
- tests/hermes_cli/test_web_server.py: setup endpoint routes pip
  through lazy_deps (regression guard asserts no direct 'pip install'
  subprocess), blocked-reason surfacing, same-response availability
  recheck clears stale missing state.

Fixes NS-605 (Plain T-1111).
2026-07-28 22:40:33 -07:00
Teknium 0cf58de85e Merge remote-tracking branch 'origin/main' into wake-toggle-config
# Conflicts:
#	tests/test_tui_gateway_server.py
#	tui_gateway/server.py
2026-07-28 12:37:35 -07:00
Dennis f50a7c307a fix(stt): preprocess .silk voice notes before transcription
Decode WeChat/QQ SILK v3 voice notes to WAV inside transcribe_audio so
any platform that caches a .silk file gets STT for free (same central-
normalization philosophy as the outbound container repair). pilk is
lazy-installed on first use (stt.silk in tools/lazy_deps.py) instead of
being added to the voice extra.

Fixes the inbound half of #32196.

(cherry picked from commit e5db79369d; reworked to compose with the
provider-scoped upload size cap and to lazy-dep pilk)
2026-07-28 11:53:36 -07:00
Ben Barclay 7d19033d2e fix(wake): run openWakeWord on tflite on macOS ARM64
openWakeWord's ONNX backend returns near-zero scores on Apple Silicon
(dscripka/openWakeWord#336), so "Hey Hermes" never crossed the 0.5
threshold: the listener armed, the microphone worked, and nothing fired.

Bisecting the pipeline puts the fault in exactly one stage — feeding the
same audio through both backends, the melspectrogram front-end is
bit-identical (maxdiff 0.00000) and the wake classifier agrees on
identical features, while the shared embedding model diverges by 45.44.
Cross-feeding confirms it: tflite features scored through the *onnx*
classifier give 0.9948 vs 0.000009 for onnx features. A telling
secondary symptom is that scores fall as input gets louder (0.5x ->
0.00031, 8x -> 0.000066), which is garbage inference rather than a weak
detection.

Selecting tflite in config alone does not fix it. openWakeWord hardcodes
`import tflite_runtime.interpreter` but declares tflite-runtime for
`platform_system == "Linux"` only; on macOS the equivalent wheel is
ai-edge-litert, so that import always fails and model.py silently
downgrades back to onnx. The result is a detector that reports itself
listening and can never fire.

- default the backend per platform (tflite on macOS ARM64, onnx
  elsewhere) instead of hardcoding onnx, and pick the matching bundled
  model artifact
- bridge tflite_runtime -> ai_edge_litert through sys.modules, in-process,
  with no writes to site-packages
- refuse the silent onnx downgrade on macOS ARM64 and report the missing
  runtime through check_wake_word_requirements() so the GUI surfaces an
  actionable hint rather than arming a dead ear
- lazy-install ai-edge-litert via its own feature key, because lazy-dep
  specs cannot carry PEP 508 markers (_spec_is_safe rejects ";")

An explicit `inference_framework` in config still wins, so anyone pinning
a backend keeps it.

Verified on macOS 26.5.2 / M-series: "hey hermes" scores 0.0005 on onnx
and 0.9423 on tflite from the same clip, with cross-phrase controls at
0.0003. Live over-the-air through the real microphone fires 4/4
utterances (peak 0.9532).
2026-07-28 07:59:34 -07:00
Hermes Agent 0ae305ed4e feat(voice): open-vocabulary wake phrases via sherpa-onnx KWS
New "sherpa" wake_word provider: the configured phrase is BPE-tokenized
at runtime against a small streaming zipformer KWS model (~13 MB English,
one-time download cached under HERMES_HOME), so ANY typed phrase works
with zero training — including per-profile phrases like "hey coder".

wake.sherpa lazy-dep group + [wake] extra grow sherpa-onnx/sentencepiece;
requirements probe routes per provider; sensitivity maps onto sherpa
keywords_threshold. E2E-verified on real audio (target phrase fires,
foreign phrase stays silent, reset drops buffered state).
2026-07-28 07:58:16 -07:00
Brooklyn Nicholson 5f43452e91 feat(voice): add "Hey Hermes" wake word to start a hands-free session
Adds an opt-in, on-device hotword listener for the CLI. With
wake_word.enabled (or /wake on), Hermes listens in the background for a
wake phrase; on detection it starts a fresh session, captures one
utterance through the existing voice pipeline, and answers — the
"Hey Siri" pattern.

- tools/wake_word.py: provider-pluggable detector (openWakeWord, free
  local default; Porcupine, premium) over the shared 16 kHz sounddevice
  capture path. Background daemon thread with pause/resume so it yields
  the mic during a voice turn.
- CLI wiring: startup listener (off-thread), on-wake flow, an idle
  watchdog that resumes the detector after each turn, cleanup hook, and
  a /wake [on|off|status] command.
- config.yaml wake_word section; PORCUPINE_ACCESS_KEY as an optional
  secret. Engines lazy-install via the [wake] extra.
- Hands a transcript to the input queue exactly like voice mode, so no
  system-prompt/cache mutation. No new core model tool.
- Tests (mocked, no live audio/network) + feature docs.
2026-07-28 07:56:45 -07:00
Victor Kyriazakos 6a174e9967 Merge origin/main into feat/gateway-health-diagnostics
# Conflicts:
#	cron/executions.py
#	cron/jobs.py
2026-07-28 13:09:17 +00:00
embwl0x a48251c3dc fix(update): refresh cve dependency pins 2026-07-26 19:29:07 -07:00
Vite Balloons 2a55f33483 fix(update): avoid refreshing inactive lazy backends 2026-07-26 19:28:55 -07:00
teknium1 40dc36a842 fix(deps): converge huggingface-hub on one exact version (1.24.0) across lock and lazy pin
hermes update's lazy-refresh pass re-asserts LAZY_DEPS pins whenever the
package is present (active_features() is presence-based). The
tool.trace_upload pin huggingface-hub==1.2.3 sat below transformers'
>=1.5.0,<2 requirement, so every update force-downgraded the shared
package and broke Hindsight local embeddings on daemon startup (#60783).

Keep the exact-pin security posture — no ranges — but move the pin to
1.24.0 (current) and bump uv.lock in lockstep (uv lock --upgrade-package
huggingface-hub: hub 1.4.1->1.24.0, hf-xet 1.3.1->1.5.2, click
8.3.1->8.4.2, drops typer-slim), so the entire tree converges on ONE hub
version. The refresh pass now reports 'current' with zero churn.

Invariant tests (not snapshots): the lazy pin must equal the uv.lock
resolved version, and must sit inside transformers' accepted window.
HfApi surface used by trace upload (whoami/create_repo/upload_file)
verified present with identical kwargs on 1.24.0 in a live venv.
2026-07-26 17:24:31 -07:00
Victor Kyriazakos 505d12f662 refactor(monitoring): scope telemetry substrate to gateway health/diagnostics export
Salvages the event-spine foundation from feat/telemetry-observability
(emitter, typed events, OTLP streaming, redaction — authorship preserved in
the preceding commits) and scopes it to the plane enterprise operators need
today: gateway Service Health Monitoring plus redacted Operational
Diagnostics, exported over OTLP.

Dropped from the salvaged branch, deliberately:
- run/model/tool trajectory capture (plugins/telemetry hooks, tel_spans)
- the local JSONL + state.db tel_* store (monitoring is egress, not storage)
- usage rollups/metrics, /insights integration, bulk export
- hermes telemetry CLI (replaced by hermes monitoring status)

Those planes — shared client usage metrics and enterprise trace telemetry —
are being designed on the NeMo Relay integration with distinct consent,
policy, and export boundaries; this keeps the monitoring plane content-free
and independently enableable.

Renames agent/telemetry -> agent/monitoring, config telemetry.* ->
monitoring.*, and pins the otlp extra at OpenTelemetry 1.39.1 (matching
uv.lock; 1.30.0 conflicts with mistralai>=2.4 on opentelemetry-api).
2026-07-24 18:54:45 +00:00
emozilla 3e28eaccde feat(telemetry): local-first telemetry & observability
Add a built-in telemetry system that records what the agent does — workflows,
model calls, tool calls, errors — to the local machine, powers `/insights`, and
can export to an operator-chosen destination. Default-on locally; nothing leaves
the machine unless the user exports it or opts into the aggregate plane.

Three planes with a hard wall between them:
  - local: full-fidelity observability (real model/provider/tool names), on by
    default, never leaves the machine.
  - aggregate: opt-in metadata, default off. No uploader ships — consent is
    recorded via telemetry.consent_state, and `preview` shows what would be
    produced, computed locally.
  - trajectories: full message content, opt-in, exported only to the operator's
    own destination.

Mechanism:
  - Bundled `telemetry` plugin registers observational lifecycle hooks
    (on_session_start / post_api_request / post_tool_call / on_session_finalize).
    No core call sites are edited; hooks already carry the data.
  - Fire-and-forget emitter: emit() returns in microseconds, never blocks or
    raises into a model/tool call. A daemon thread writes events to an
    append-only JSONL log and the tel_* tables in state.db (its own sqlite
    connection, separate from SessionDB).
  - tel_runs / tel_model_calls / tel_tool_calls live in the declarative
    SCHEMA_SQL and are reconciled automatically; SCHEMA_VERSION 16 -> 17.
  - metrics derives rollups for /usage and /insights; rollup builds per-run
    summaries for `hermes telemetry preview`.

Consent is config, not a parallel command surface. The config file is the root
of trust: set telemetry.consent_state with `hermes config set`, or pin any
telemetry.* key (including allow_aggregate) via managed scope, which overrides
the user's value per key. `hermes telemetry` exposes only what config cannot:
status (report), preview (query), and export.

Export:
  - exporter_bulk writes telemetry (and, when the trajectories plane is enabled,
    session content) to ndjson/json.
  - otlp_exporter streams spans to a configured OpenTelemetry Collector over
    OTLP/HTTP. The SDK is an optional extra (hermes-agent[otlp]), lazily
    installed via tools.lazy_deps on first use.
  - Secrets are always redacted on every export path
    (redact_sensitive_text(force=True)); content export is gated by the
    trajectories plane, and PII scrubbing follows telemetry.content_redaction.
    OTLP auth headers reference environment variable names, never inline values.

No outbound emission to Nous. The aggregate uploader is intentionally not built.
2026-07-24 18:54:45 +00:00
Stoltemberg c89481db5e fix: add explicit UTF-8 encoding to all subprocess text=True calls (#53428)
On Windows with Chinese locale (GBK), subprocess.run(text=True) without
explicit encoding causes UnicodeDecodeError crashes. This fix adds
encoding='utf-8', errors='replace' to all subprocess.run() and
subprocess.Popen() calls that use text=True across 76 non-test Python files.

Fixes #53428 (master tracker for Windows GBK locale crash).

Note: credential_pool.py and electron changes excluded per reviewer request —
those will be submitted as separate focused PRs.
2026-07-24 11:45:57 -07:00
Cal Marley 5c5960d9f9 fix(windows): suppress console window flashes in env probes, lazy installs, and platform.win32_ver()
From windowless processes (the pythonw gateway and the kanban workers it
spawns), three spawn paths flash visible console windows on Windows:

1. tools/env_probe.py::_run() ran its interpreter/pip probes
   (python3 / python / pip / 'python3 -m pip' / PEP-668 check, ~5 per
   worker start) without creationflags — one console flash per probe.

2. tools/lazy_deps.py had four spawn sites with the same defect:
   'uv pip install', the 'pip --version' probe, ensurepip, and the
   pip install fallback.

Both now pass creationflags=windows_hide_flags() (CREATE_NO_WINDOW on
Windows, 0 on POSIX) — stdio capture still works because the child is
hidden, not detached.

3. CPython 3.11's platform.win32_ver() unconditionally calls
   _syscmd_ver(), which runs 'cmd /c ver' via
   subprocess.check_output(shell=True) with no window suppression. Any
   dependency touching platform.uname()/version()/platform() at import
   time flashes one 'cmd' window per windowless process. New helper
   _subprocess_compat.suppress_platform_ver_console() (Windows-only,
   never raises) stubs platform._syscmd_ver so win32_ver() falls back to
   sys.getwindowsversion().platform_version — verified byte-identical
   platform.platform() output on CPython 3.11
   ('Windows-10-10.0.26100-SP0' either way). Called at the top of
   hermes_cli/main.py, right after the hermes_bootstrap guard, before
   heavyweight imports.

Verified on Windows 11 by polling EnumWindows at ~15 ms and attributing
new visible HWNDs to the suspect process tree (conhost child presence is
NOT evidence of a visible window — it appears even with
CREATE_NO_WINDOW). Tests: tests/tools/test_windows_native_support.py,
test_env_probe.py, test_lazy_deps.py, test_lazy_deps_durable_target.py —
153 passed; the 3 failures are pre-existing on upstream/main in a
Windows environment (POSIX-only assertions and NTFS chmod semantics).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 18:20:24 -07:00
Erosika 29e0471708 fix(honcho): update SDK and restore CI coverage 2026-07-16 12:48:48 -07:00
Teknium d6590f8b17 chore(slack): bump slack-bolt to 1.29.0 and slack-sdk to 3.43.0
Slack's June 30 Agent messaging experience changelog lists Bolt Python
1.29.0 / Python SDK 3.43.0 as the Agent View minimums. Bump the
messaging/slack extras and the platform.slack lazy-install pins to
match, and regenerate uv.lock. All adapter API surfaces verified
present against the new versions in a clean venv.
2026-07-14 13:58:36 -07:00
teknium1 651e632b6d fix(feishu): ship Channel signaling SDK support 2026-07-09 20:31:49 -07:00