Commit Graph

3587 Commits

Author SHA1 Message Date
Teknium 04de344f41 refactor(computer_use): compact backend/permissions/vision_routing/__init__ docs; tighten doctor helpers 2026-09-02 15:45:55 -07:00
Teknium e3b79addc6 refactor(computer_use): tool.py — compact comments/docstrings, hoist vision prompt, tighten layout 2026-09-02 15:43:11 -07:00
Teknium 0360b71926 refactor(computer_use): split CuaDriverBackend into capture/input mixins and a driver-resolution module 2026-09-02 15:42:36 -07:00
Teknium 17df4741df refactor(terminal): extract scope-aware TERMINAL_* config readers/cwd sanitizers to terminal_tool_config 2026-09-02 15:42:26 -07:00
Teknium 1f7beadae7 refactor(mcp): dedupe error classification ladders, optional-type import helper, cause-chain walker 2026-09-02 15:41:56 -07:00
Teknium 8e9628e9cf refactor(delegate): progress relay -> _ChildProgressRelay class + event table; split _dispatch_background into collaborators 2026-09-02 15:40:23 -07:00
Teknium 5eaa155067 refactor(mcp): unify image/audio/resource block decode+cache into two helpers 2026-09-02 15:38:58 -07:00
Teknium d6ec8c3018 refactor(mcp): unify live/cache registration into one candidate loop; phase-split sampling and elicitation handlers 2026-09-02 15:38:42 -07:00
Teknium 80affc7ee3 refactor(computer_use): doctor — reuse permissions._child_env, split fallback/report rendering into helpers 2026-09-02 15:37:58 -07:00
Teknium cf9f454cdc refactor(computer_use): tool.py — _reject_unsafe, _pop_session_locked, _cache_file, summary/envelope helpers, dispatch table for _summarize_action 2026-09-02 15:37:47 -07:00
Teknium 282e63f81c refactor(stt): split transcription_tools into common/audio/local/cloud/command sibling modules (re-imported into origin) 2026-09-02 15:36:59 -07:00
Teknium 7cfc90bae8 refactor(terminal): split terminal_tool() into _plan_execution/_acquire_env/_run_foreground (dataclass plan) and unify the error envelope via _error_json 2026-09-02 15:36:30 -07:00
Teknium 50d63b2869 refactor(delegate): dedupe config knobs via _cfg(), split credential resolution by branch 2026-09-02 15:34:51 -07:00
Teknium 21c7037975 refactor(terminal): extract sudo/shell-rewrite cluster to terminal_tool_sudo and backend builders/checkers to terminal_tool_backends 2026-09-02 15:31:31 -07:00
Teknium 18700c60d5 Merge branch 'simp/tools' into simp/integration 2026-09-02 15:01:45 -07:00
Teknium aed2e156ad refactor(tools/files): finish file_operations wiring to extracted modules; restore WHY comments in file_tools; repoint test 2026-09-02 14:47:03 -07:00
Teknium 731a943ac4 refactor(tools/planning): compact kanban metadata merging and clarify tool/gateway helpers 2026-09-02 14:47:02 -07:00
Teknium ae67178be1 refactor(tools/infra): split cronjob god-methods into per-action helpers; extract tool_search catalog/validation; compact registry, lazy_deps, tool_backend_helpers, desktop_ui 2026-09-02 14:47:02 -07:00
Teknium 6723628de9 refactor(tools/messaging): split send_message into senders/targets tables; dedupe discord/bot_mode/relay helpers; extract feishu_lark shared plumbing; compact graph client/auth 2026-09-02 14:45:42 -07:00
Teknium 20258aad9e refactor(tools/mcp_oauth): extract mcp_oauth_provider; compact oauth manager/dashboard bridge and schema_sanitizer; repoint tests 2026-09-02 14:45:42 -07:00
Teknium 8b3dde5dcc refactor(tools/web): split web_tools into truncate/extract siblings; dispatch tables for backend probes and stdlib extractors; compact url_safety/website_policy/result cache 2026-09-02 14:45:42 -07:00
Teknium b6a3d3f440 refactor(tools): restore stdin=DEVNULL on subprocess probes dropped during compaction
voice_mode WSL playback probe, subagent_worktree._run_git, cua_backend
loginctl/xprop probes lost their explicit stdin= (and one line-wrapped past
the encoding= same-line rule); lazy_deps._run carries it via _SUBPROCESS_KW
so mark it for the guard. Fixes tests/tools/test_subprocess_stdin_guard.py
and tests/scripts/test_footgun_subprocess_encoding.py (green on base).
2026-09-02 14:45:42 -07:00
Teknium 8813eba345 refactor(tools): repoint tests to moved symbols; add voice_mode_transcript module 2026-09-02 14:45:15 -07:00
Teknium 9fe009f467 refactor(tools/media): extract vision_tools_image_prep; dedupe image/video generation providers; compact fal/xai helpers 2026-09-02 14:45:15 -07:00
Teknium 33ae442e94 refactor(tools/computer_use): split cua_backend into daemon/actions/... siblings; dispatch tables; compact doctor/tool 2026-09-02 14:45:15 -07:00
Teknium 3bbec90f23 refactor(tools/environments): split local/base into output/wait/session-env siblings; extract docker_egress + remote_common; dedupe remote backends; compact process_registry 2026-09-02 14:45:15 -07:00
Teknium d4ded67503 refactor(tools/terminal): split terminal_tool god method into helpers; dispatch tables; compact env probe/passthrough/daemon pool 2026-09-02 14:45:15 -07:00
Teknium 2ef1e8e4e0 refactor(tools/voice): extract tts delivery + wake_word engines; dedupe transcription/voice_mode helpers; compact tts providers 2026-09-02 14:45:15 -07:00
Teknium 49d15faae6 refactor(tools/skills_sync,memory): split skills_sync client wire/org and bundled/optional ops; extract memory_tool_store and session_search common/discover 2026-09-02 14:45:15 -07:00
Teknium 7c1ec19d4a refactor(tools/skills): split skills_hub into per-source modules; extract skill_manager guards/batch, skills_tool dedup/plugin/setup; compact skill_usage, skills_guard 2026-09-02 14:45:15 -07:00
Teknium 05526b028a refactor(tools/delegate): split delegate_tool into child_run/config/dispatch/progress/registry/results; compact delegation helpers 2026-09-02 14:45:15 -07:00
Teknium 9f6335bc44 refactor(tools/browser): extract eval-policy/lightpanda-fallback/real-profile/snapshot modules from browser_tool; split supervisor dialogs/frames; dedupe camofox/cli 2026-09-02 14:44:15 -07:00
Teknium c1f8af1e86 refactor(tools/mcp): split mcp_tool.py into transport/lifecycle/schema/handlers/... sibling modules; compact watchdog and schema cache 2026-09-02 14:44:15 -07:00
Teknium 606cb2de92 refactor(tools/code_exec): unify code_kernel local/remote helpers, split checkpoint_manager god methods, compact spill helpers 2026-09-02 14:44:15 -07:00
Teknium 5c919161d0 refactor(tools/approval): split approval.py into smart/human-wait/gateway-wait modules; dedupe guards 2026-09-02 14:44:14 -07:00
Teknium ec49ae7c0f refactor(tools): wire file_operations to extracted common/lint/search modules; restore literal security pins in lazy_deps 2026-09-02 14:43:45 -07:00
Teknium 3f5564d727 refactor(tools): discovery scan sees loop-registered tools; fix lint module import 2026-09-02 14:43:45 -07:00
Teknium d4cec15b47 refactor(tools): first-wave simplification of tools/ (file ops split, lazy_deps, code_exec, approval, browser, delegate, mcp, skills, terminal, voice, media)
Behavior-neutral structural pass over tools/*: god-file extractions into
sibling modules (file_operations_common/lint/search, file_tools_paths/
read_tracking/write, code_execution_env/rpc, tool_search_catalog/names/
validation, tts_command_provider, ...), duplicate helper unification,
if/elif -> dispatch tables, dead-code removal, docstring compaction.
Tool schemas (get_tool_definitions) verified byte-identical to base.
2026-09-02 14:43:45 -07:00
Teknium fcb5c64101 refactor(agent): model_tools — extract argument type coercion into tools/arg_coercion.py (re-exported; logger name preserved) 2026-09-02 13:29:39 -07:00
kshitijk4poor ff7233b815 fix(credential_files): apply the same exclusions to the symlink-safe mount copy
_safe_skills_path() is the sibling of iter_skills_files(): when a symlink in
skills/ forces a sanitized copy for mount-based backends (Docker/Singularity),
it rglob-copied the whole tree — .hub, .curator_backups, node_modules and all.
Prune EXCLUDED_SKILL_DIRS before descending, same rule as the sync generator,
so the mounted copy never carries (or walks) the bookkeeping trees either.
2026-09-03 01:33:18 +05:30
kshitijk4poor 1d06ef3a5d refactor(credential_files): prune excluded dirs before descending in the sync walk
Replaces the three hand-copied rglob loops + post-hoc parts check with one
os.walk generator that drops EXCLUDED_SKILL_DIRS from dirnames before
recursing. Same file set as the cherry-picked fix (the test binds it), but
the walk no longer stats every file under .hub/.curator_backups/node_modules
on each 5s FileSyncManager tick.

Bench (synthetic skills tree: 20 skills + 400 .hub files + 5x8MB curator
tarballs + 50 archived files): iter_skills_files() 35ms -> 2.4ms.
2026-09-03 01:33:18 +05:30
Carry00 edac49e473 fix(skills): stop syncing bookkeeping dirs to sandboxes
iter_skills_files() walked the skills tree with a bare rglob("*"), so the
.hub download cache, .archive, curator backups, and any node_modules/.git
under a skill package were uploaded to the sandbox on every sync. The
sandbox never reads them: skill content is resolved host-side.

EXCLUDED_SKILL_DIRS is already the canonical exclusion set, honoured by
discovery and backup. Apply it to the sync path too, across all three
roots iter_skills_files() walks (local, external, project-local), and add
.curator_backups to the set.

Measured on a local install: 900 files / 67.3 MB -> 771 files / 8.4 MB.

This is not just wasted bandwidth on the SSH backend, where the oversized
payload can exceed the 120s _ssh_bulk_upload deadline and surface as the
agent hanging on every tool call.

The filter intentionally does not reuse is_excluded_skill_path(), which
also prunes references/, templates/, assets/ and scripts/ -- those hold
support files and bundled scripts the sandbox does read and execute.
2026-09-03 01:33:18 +05:30
Teknium f6234d00c5 fix(security): close GitSpawn RCE class — malicious repo .git/config no longer executes on context gathering (GHSA-7x36-8jrh-v4pw)
Hermes gathers workspace context by running git against the session
directory automatically — the coding-workspace snapshot, gateway
project-tree build, /diff, @diff|@staged context refs, goal-gate
fingerprint, and -w startup worktree add — before any prompt, tool call,
approval, or trust gate. Those probes ran the system git without
stripping the repository's own config, so a repo delivered as files with
its .git directory intact (a shared zip, sync folder, or USB stick;
git clone never transfers .git/config) could set an execution-sink git
setting and get arbitrary host code execution as the user with nothing
on screen.

- core.fsmonitor / core.hooksPath / pager / editor / credential helper:
  neutralized by routing every automatic probe through
  noninteractive_git_env(), which pins those keys to inert values via
  GIT_CONFIG_* and ignores global/system config. bounded_git_probe (the
  reported sink, coding_context._git + tui_gateway.git_probe) now defaults
  to that env; worktree-add, working_diff, web_git, context_references,
  goals, and subagent_worktree route through it too.
- Attribute-scoped [diff "x"] command=/textconv= drivers: the attacker
  names the driver in .gitattributes, so GIT_CONFIG_KEY overrides can't
  enumerate them. Added harden_git_argv(), which inserts
  --no-ext-diff --no-textconv on diff-rendering subcommands (diff/show/
  log/blame) only — status et al reject the flags. Both flags required
  (verified empirically; each alone leaves the other live).

Builds on the noninteractive_git_env config-scrubbing from the
gemini-cli #28792 port. Real-git E2E regression suite arms a malicious
repo and asserts every automatic path neutralizes fsmonitor, hooks,
external-diff, and textconv; a baseline test proves the repo is armed.
2026-09-02 10:33:43 -07:00
Teknium 7840a0e2d9 feat: delegation batch tags read "set N" instead of a hex id slice
Interleaved subagent fan-outs were tagged with the first 4 hex chars of the
delegation id ([b2ac 3/9]), which is attributable but unreadable. Batches are
now numbered in order of appearance per process: [set 1 · 3/9], [set 2 · 1/7].
Desktop /agents already labels groups "Delegation N", so its duplicate hex
badge is dropped.
2026-09-02 10:12:54 -07:00
Victor Kyriazakos fd35e1ec5a fix(cron): delivery bookkeeping reads the failure lane it actually routed through
Review findings (Salt, NS-788):

B1: delivery_outcome classification, unresolved_origin, and incident
'alerted' marking all read the deliver lane while the notice itself was
routed through failure_deliver — a silenced failure recorded
delivery_outcome='delivered' and marked its incident alerted (corrupting
the 'failure seen' vs 'operator was pinged' distinction the incident
store documents), and a failure delivered via failure_deliver over an
unresolvable deliver=origin recorded 'not_configured'. New
_delivery_lane_value() helper feeds the SAME lane to routing and
bookkeeping at all five sites (both classifiers, both unresolved_origin
computations, both zero-target checks). Three regression tests assert
outcome + alerted-marking; verified to bite on the pre-fix classifier.

S1: failure_deliver now goes through _resolve_cron_context_deliver on
tool create/update, matching deliver — a job created from inside a cron
run can no longer store literal 'origin' in its failure lane.

S2/T1: corrected the false 'same helper' comment in create_job; the
str/list flatten mirrors the tool layer for direct callers.

Full cron suite + interrupt tests: 87 files, 1112 passed, 0 failed.
2026-09-02 20:16:14 +05:30
Victor Kyriazakos c9491e6a7d feat(cron): per-job failure_deliver — route or suppress failure notices (NS-788)
Coatue FR (Frank Long): jobs delivering into shared channels publish
engine failure notices ('⚠️ Cron X failed…') to those channels with no
opt-out. Adds an optional per-job failure_deliver field sharing
deliver's grammar: on failure, targets resolve from failure_deliver
when set (local = structural silence; state still recorded in
last_status/last_error/run history). Success delivery is unchanged;
absent field = today's behavior byte-for-byte.

Honored by every failure-category engine notice: the run_job failure
summary (+streak nudge), the escaped-failure retry path, drift-skip and
blocked-config alerts (composed into the same delivery), and the
gateway-shutdown interrupted-run notice (_notify_interrupted_cron_jobs).

Surfaces: cronjob tool create/update (same bot-chat validation as
deliver; '' clears on update), hermes cron create/edit
--failure-deliver, docs tip in automate-with-cron.

Existing fake_deliver test doubles gained **kwargs for the new
for_failure keyword — signature-compat only, no behavior change.
2026-09-02 20:16:14 +05:30
Teknium 8e4366d358 fix(tools): freeze tools[] across agent-cache eviction; make /reload-mcp the re-probe hatch
Policy: availability-gated tools (check_fn probes — Docker, HASS_TOKEN,
OAuth…) are frozen for the life of a session. tools[] only changes on
/new, /reload-mcp, or compaction. Two doors remained after #100638:

* Gateway agent-cache eviction (LRU/idle sweep/cross-process invalidation)
  rebuilds a fresh AIAgent for the SAME session and agent_init re-derives
  agent.tools from live probes with no predecessor to preserve. Persist
  the session's resolved tool-name order in a new `sessions.tool_names`
  JSON column (declarative reconciliation, SCHEMA_VERSION 28), written
  alongside the system prompt and re-pinned on every published refresh
  (so /reload-mcp and compaction naturally reset it; /new mints a new
  row). On restore-for-existing-session the fresh definitions are folded
  onto the saved order via the SAME `_merge_preserving_prefix` helper —
  a probe-flipped tool is carried forward from the registry schema, a
  deregistered one dropped, new tools appended at the tail.

* /reload-mcp (CLI, gateway, TUI RPC) now also calls
  `reprobe_tool_availability()` — drops the check_fn verdict cache and the
  get_tool_definitions memo — so a user can consciously pick up a
  credential/daemon that appeared mid-session. Docs updated.
2026-09-02 07:22:59 -07:00
joaomarcos 65b0f00002 fix(agent): stop the between-turns tool refresh from forking the cached prefix
The per-turn MCP refresh re-derives `agent.tools` from live availability and
publishes the result wholesale. Two kinds of bytes move as a result:

* a tool whose `check_fn` merely flapped (headless browser probe, expired
  credential, docker blip) disappears from the array, and
* a late-landing MCP tool splices into sorted position, which can be index 0.

Providers that render `tools` ahead of the messages re-prefill the entire
history behind any moved byte, so either case costs a full re-prefill of the
session — the measured 2% cache hit in #100336. The caller's own comment
claimed the refresh "only ever extends a fresh request prefix"; it did not.

`refresh_agent_mcp_tools(..., preserve_prefix=True)` makes that claim true.
The live order becomes authoritative: existing tools keep their slot (fresh
schemas still land), a tool that is still registered but momentarily
unavailable is carried forward, a tool that genuinely left the registry is
still dropped, and new tools are appended at the tail. Explicit `/reload-mcp`
and the compaction boundary keep the plain rebuild.

Refs #100336
2026-09-02 07:22:59 -07:00
Teknium ee0e234a2c fix(gateway): discover and reload MCP servers per profile under multiplex
A multiplexed gateway ran `discover_mcp_tools()` once, unscoped, at boot
and again on `/reload-mcp`, so only the launch profile's `mcp_servers`
ever connected; secondary profiles' servers never registered, and a
`/reload-mcp` from any profile tore down every profile's connections.

- `_discover_gateway_mcp_tools()`: under multiplex, run discovery once per
  served profile inside `_profile_runtime_scope`, carried into the
  executor via `copy_context()` (same shape as
  `_run_in_executor_with_context`). Single-profile path unchanged.
- `_execute_mcp_reload()`: enter the requesting profile's scope when the
  caller (e.g. button-confirm callback) did not; shut down / rediscover /
  report only that profile's servers; refresh only that profile's cached
  agents.
- `shutdown_mcp_servers(scope=)`: scoped teardown keyed by the new
  `_server_scope_keys` ownership map; leaves the shared MCP loop running
  while other profiles' servers are live. Unscoped call keeps the full
  historical behavior.
- MCP tools register into the owning profile's registry overlay
  (`registry.register(scope=...)`), and `registry.deregister()` gains a
  matching `scope=` kwarg. Plugin callers still cannot name another
  profile's scope; the plugin-vs-global guard is unchanged for them.

Fixes #95518

Co-authored-by: fangliquanflq <fangliquan@qq.com>
Co-authored-by: Kong <mgongzai@gmail.com>
Co-authored-by: roraag <232666910+roraag@users.noreply.github.com>
2026-09-02 07:00:13 -07:00
Teknium 4e7aa48716 fix(browser): reap idle multiplexed sessions under their owner profile scope
The inactivity janitor is one process-global thread started by whichever
profile first opens a browser, so under `gateway.multiplex_profiles` it runs
with no secret scope: `cleanup_browser` -> `is_camofox_mode` ->
`get_secret("CAMOFOX_URL")` raises UnscopedSecretError, the session entry is
never removed, and the same failure repeats every 30s while the Chromium
daemon leaks.

- `_update_session_activity` records the owning Hermes home per session;
  `_cleanup_inactive_browser_sessions` re-enters that owner's
  `set_hermes_home_override` + `build_profile_secret_scope` around each
  teardown (`_session_owner_scope`, mirroring `_profile_runtime_scope`).
  copy_context at thread spawn would pin the first profile's secrets onto
  every other profile's teardown; there is no os.environ fallthrough.
- 3 consecutive failures -> `_force_reap_browser_session`, which skips the
  failing `close` round-trips but still closes the cloud provider session
  and kills the local daemon via the shared `_release_session_resources`
  tail (extracted from `_cleanup_single_browser_session`, unchanged).
  An activity touch does not reset the failure budget.

Fixes #86402
Fixes #100738

Co-authored-by: fangliquanflq <fangliquan@qq.com>
2026-09-02 07:00:13 -07:00