computer_use kept its own approval decision: two module dicts
(_session_auto_approve / _always_allow) mirroring tools.approval's
session store and _persist_choice, a private verdict vocabulary
(approve_once/approve_session/always_approve) that hermes_cli mapped
back to once/session/always, and — the real problem — `if
_approval_callback is None: return None`. Only the interactive CLI ever
installed that callback, so every other host (gateway turns, cron,
api_server, tui_gateway, ACP) ran destructive desktop input with no
approval at all, ignoring cron_mode / unattended_mode / the permanent
allowlist, and "always" grants were invisible to `is_approved`,
`clear_session` and the messaging-platform approval buttons.
_request_approval now calls tools.approval._run_approval_gate with
pattern_key `cua:<action>:<background|foreground>` (the old scope shape,
so a background grant still never covers the visible foreground variant)
and fail_closed_when_no_human=True, the same posture as
request_tool_approval / the SSH-config write gate. The private dicts,
their release/atexit clearing, the verdict mapping in
hermes_cli/cli_modal_mixin.py and the extra callback install in cli.py
are deleted: the CLI's terminal_tool callback answers computer_use
prompts like any other tool. set_approval_callback stays as an optional
explicit-callback hook with the shared callback contract
(cb(command, description, **kw) -> once|session|always|deny|timeout);
no in-tree host uses it.
Behavior change:
- No approval callback and no gateway (cron, api_server/webhook,
headless -q, plain library use): destructive actions are now REFUSED
with a BLOCKED error and never reach the backend. Previously they
silently ran. cron honors approvals.cron_mode, unattended platforms
approvals.unattended_mode, -q approvals.single_query_mode.
- --yolo / gateway /yolo / approvals.mode: off still allow (unchanged).
- Gateway sessions (Telegram/Discord/Slack/...) now get a real pending
approval with once/session/always buttons instead of default-allow.
- session/always grants live in tools.approval's store; "always" is one
command_allowlist entry (`cua:click:background`) and is scoped to that
action+mode — the old blanket "always_approve unlocks everything for
the session" no longer exists.
- Denial wording is the shared gate's ("BLOCKED: User denied ...",
"BLOCKED: Action timed out ..."); the error JSON keeps `action`.
Tests: tests/tools/test_computer_use_approval_isolation.py
::test_no_callback_refuses_unless_yolo (blocked + no backend call, then
yolo executes) and ::test_always_grant_lands_in_the_shared_store
(is_approved sees the cua:<action>:<mode> key; second call served from
the store). Sabotage: restoring the `callback is None -> allow`
short-circuit fails the first; swapping the shared gate for a private
grant set fails the second plus the three delivery-ladder scope tests.
tests/tools/conftest.py gains `grant_computer_use_approvals` for
dispatch tests that only care about routing.
Sweep of `ruff check . --select F821 --target-version py311`: 2,234 hits. 2,201 are left
alone on purpose: tui_gateway (2,169; bind_module rebinds bodies onto server.py globals,
all names verified to resolve there), the Feishu adapter (27; globals().update() SDK
binding) and the godmode script (5; dead standalone script). The other 33 were all
genuine defects. No lint config change; no TYPE_CHECKING escape hatches — every
annotation names a real, imported type; ty on the touched files: 0 new diagnostics.
- gateway/slash_commands.py: HISTORY_UNREADABLE never imported after #102117
→ NameError on the /btw error branch (same one-liner as #102952).
- gateway/platforms/whatsapp_common.py: `-> Path` return annotation with no Path
import (the body uses `_Path`). Never raised at runtime thanks to
`from __future__ import annotations`, but `typing.get_type_hints()` and ty
both fail on it.
- gateway/run.py: ActivityProvenance imported at module level
(agent.session_activity has no gateway deps); stringly annotation and the
lazy in-function import are gone.
- tools/patch_parser.py: PatchResult imported at module level; real return
annotation. The "avoid circular import" lazy import guarded a cycle that
does not exist (file_operations_common never imports patch_parser).
- gateway/platforms/helpers.py: base.py imports helpers at module level, so
MessageEvent cannot be named here; TextBatchAggregator only reads .text and
.source, so it is typed by a BatchableEvent Protocol that MessageEvent
satisfies structurally.
- tools/mcp_tool_sampling.py: mcp_tool imports this module, so MCPServerTask
cannot be named here; ElicitationHandler only reads
owner._pending_call_context, typed by an ElicitationOwner Protocol.
- plugins/platforms/sms/adapter.py: aiohttp is an optional dep ([messaging] extra) →
module-level try/except ImportError binding `aiohttp = web = None`, the pattern the
homeassistant / webhook / whatsapp_cloud adapters already use. Retires three lazy
in-function imports and the `_aiohttp_available()` wrapper; `_handle_webhook` typed
`web.Request -> web.Response`.
- plugins/platforms/teams/summary_writer.py: plain module-level `import httpx` — httpx is a
hard core dependency (pyproject `httpx[socks]==0.28.1`), so the lazy import and the
"imported on every CLI start" docstring premise were both wrong (plugin discovery never
imports this module; it is reached only via the Teams adapter / meeting pipeline).
Tests:
- tests/hermes_cli/test_config.py: a test body orphaned by the wave-1 prune
(6b81590c55) sat inside the class as dead code with self/tmp_path unbound
— header restored, so the v11→12 custom_providers migration is covered.
- tests/tools/test_mcp_tool.py: @staticmethod recursing on `self` in the
win32 branch; call portalocker directly.
- tests/test_background_review_list_shapes.py: main() still ran 3 pruned tests.
- tests/agent/test_cursor_optimizations_parity.py: bench() used names only
imported inside a sibling test.
- GatewayRunner / FeishuAdapter / Dict / Optional: missing imports.
main guarded scroll's x/y independently (coordinate=[null,100] -> y=100),
while click treats a coordinate without x as no point. The shared _xy()
applied click semantics to scroll; split out _scroll_xy() with main's
per-axis guard and pin both behaviors in a test.
Real-profile browsing routes all in-page browser work through the Browser Use
CLI (browser_exec), which obsoletes the cua-driver typed-browser surface baked
into computer_use. Remove it so computer_use is a pure DESKTOP-control tool
(screenshots / mouse / keyboard / window management) and every call's schema
drops ~24 browser-only params + 9 actions.
- schema.py: 9 cua_browser_* actions and the typed-browser param block removed;
14 desktop actions + shared params kept; description drops the browser rung.
- tool.py: cua_browser entries out of _SAFE/_DESTRUCTIVE_ACTIONS; the whole
cua_browser dispatch block deleted; {"type","cua_browser_type"} → "type"
(desktop typing untouched); _config_preauthorized (browser-prepare-only, a
no-op for every desktop action) and the browser-page escalation hint removed.
- browser_route.py deleted (no importers outside the package); cua_backend.py
drops the import + typed_browser_* methods; backend.py drops the non-abstract
defaults.
- tests: browser-route/contract suites removed; browser assertions trimmed.
Desktop control unchanged. 233 computer_use tests pass; the 1 remaining failure
(test_gateway_session_key_yolo_maps_to_unrestricted_mode) is a pre-existing
cross-test state leak — fails identically on origin/main, passes in isolation.
An MCP call_tool deadline hit left the cua-driver session wedged for
all later computer-use calls. Mark the session suspect on a
concurrent.futures.TimeoutError and tear down + recreate it before the
next non-lifecycle call; healthy sessions are never restarted.
Fail-closed: the timed-out action may still have taken effect on the
remote screen, so it is never silently replayed — the error result
carries structuredContent.code=timeout_outcome_unknown with
next_step=fresh_state.
Informed by #74877 by BlackishGreen33.
Co-authored-by: BlackishGreen33 <s5460703@gmail.com>
mcp 2.0.0 implements MCP revision 2026-07-28 and makes three breaking
changes Hermes sits on top of: `mcp.server.fastmcp` is gone, every model
field is renamed to snake_case (camelCase survives only as a
serialization alias, which pydantic does not expose to attribute
access), and the SDK's own HTTP stack moved from `httpx` to `httpx2`.
Bump the pin across the dev/mcp/computer-use extras and port the tree:
- `mcp_serve.py` and `agent/transports/hermes_tools_mcp_server.py` move
from `FastMCP` to `mcp.server.MCPServer`, which has the same
decorator/add_tool surface. The hermes-tools server already
synthesised `__signature__` from Hermes' JSON Schema, which is exactly
what 2.0's `add_tool` reads.
- SDK model reads go through `mcp_field(obj, snake, camel)`, which reads
both spellings. A single-spelling read fails *silently* on the other
generation — empty tool schemas, dropped structured content, tool
results vanishing from sampling conversations — and `mcp` is an
optional extra users install at their own version.
- `sdk_httpx()` resolves the httpx flavour from the SDK's own transport
module, so objects handed to `streamable_http_client`, the `sse_client`
factory, and the OAuth metadata helpers come from the module the
installed SDK actually imports.
- HTTP support is gated on either streamable-HTTP entry point, not just
the deprecated alias 2.0 removed.
- OAuth: `OAuthClientProvider` lost its `timeout` argument (the
configured `oauth.timeout` now bounds the callback waiter's own poll
loop, where the browser round-trip was always awaited), and
`callback_handler` must return `AuthorizationCodeResult` rather than a
tuple. 2.0 also validates the RFC 9207 `iss` parameter, so the
callback handler and paste fallback capture it.
`mcp`/`mcp-types` 2.0.0 are inside the 14-day `exclude-newer` window, so
two narrow `exclude-newer-package` entries unblock `uv lock`, annotated
for removal on or after 2026-08-11. `httpx2` needs no exemption: 2.7.0 is
already outside the window and satisfies mcp's floor.
Refs #69931
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A same-day version-floor bump (0.20 runtime contract) left every install
with an older cua-driver hard-failing on all computer_use calls: the
start() gate fails closed, while the `hermes update` refresh defers to the
driver's own check-update verb — whose ~20h cache routinely answers "no
update available" right after we raise the floor. Hermes knew it required
0.20+ but never acted on that knowledge.
Two changes:
- tools_config.install_cua_driver(): a contract-failed installed driver is
repaired on the upgrade=True path too (previously only upgrade=False).
The contract failure itself is the confirmation, so the
require_confirmed_update gate and the check-update short-circuit are
bypassed for repairs — an indeterminate or stale-cached check can no
longer pin users on an unusable driver.
- cua_backend.CuaDriverBackend.start(): when the contract gate fails on an
installed binary, attempt one automatic repair per process via the
standard install path, then re-probe. HERMES_CUA_DRIVER_CMD overrides
are never repaired (explicit override is authoritative even when broken)
and a missing binary still just reports the install hint. A failing
installer can't loop: the second start() surfaces the original error.
Tests: contract-repair coverage in test_computer_use.py (auto-repair
success, failed repair surfaces the original error, once-per-process
guard, override never repaired, missing binary never repaired) and
test_install_cua_driver.py (incompatible driver repairs despite an
indeterminate check-update, check-update not consulted). All new tests
verified to fail against the unfixed source (sabotage run).
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.
Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
(DEFAULT_CONFIG smart-approval leaked in) — pinned approval
mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
silently probed live provider catalogs (~2s/test) — stubbed
cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
MagicMock agents — interrupt.side_effect now clears _running_agents
(22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
(24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
with rationale comment — the watchdog-dump assertion needs the loop
blocked well past deadline+grace under parallel load (flaked once in
the 40-worker verification run at a 0.2s margin).
Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
`_canon_key_combo` (the `_BLOCKED_KEY_COMBOS` gate in
`handle_computer_use`) split key strings on `+` only, but the cua-driver
backend's `_parse_key_combo` splits on both `+` and `-`. So a model could
issue `{"action":"key","keys":"ctrl-alt-delete"}` (or `alt-f4`,
`cmd-shift-q`): the gate saw a single unknown token and let it through
while the backend executed the real destructive shortcut.
Split the gate on both `+` and `-` so it canonicalizes combos the same
way the backend does. Non-destructive hyphen combos (`cmd-c`) and the
literal `-` zoom key (`cmd+-`) are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cua-driver 0.7.x can return list_windows/list_apps payloads under
structuredContent.windows, data.windows, data._legacy_windows, or
top-level windows/_legacy_windows (direct CLI responses). The wrapper
only read structuredContent.windows, so discovery came back empty
(capture 0x0, list_apps []) while raw cua-driver calls worked.
- add _windows_from_tool_result(): walks the known envelope shapes in
priority order, skipping empty higher-priority envelopes
- route _load_windows() (MCP + CLI re-fetch paths) through the helper,
covering capture() and focus_app()
- harden _ingest_windows(): skip non-dict members, normalize untrusted
app_name/title/z_index fields
- list_apps(): prefer structuredContent.apps, fall through populated
data/top-level envelopes, derive unique apps from window-shaped
payloads via _apps_from_windows(), keep the text-line fallback last
- tests for every envelope shape, precedence, malformed records, and
app derivation
Salvaged from #63037 (Reaper-Legion), which itself preserved the
original implementation from #57961 (kohoj); #73007 (umi008)
independently proposed the same normalization later.
Fixes#57905
Co-authored-by: Reaper <248977840+Reaper-Forge@users.noreply.github.com>
Co-authored-by: Ulises Millan Guerrero <ulises.millanguerrero@gmail.com>
_start_lifecycle_locked reassigns a fresh threading.Event() at line
786, so patching the pre-made instance's wait() is lost and the real
30s wait races pytest-timeout. Patch threading.Event itself with a
FakeEvent whose wait() returns False immediately (#69372).
Treat cua-driver's Linux `is_on_screen: null` as unknown instead of
off-screen, and skip GNOME Shell desktop/backdrop helper windows
(ding "Desktop Icons", @!x,y;BDHF) when selecting the default capture
target — they are targetable X11 windows but capture as empty.
Reconciled with the _NET_ACTIVE_WINDOW fallback from #58030: helper
windows are filtered out of the candidate pool first, then the tied
z-order active-window probe runs on the remaining real app windows.
Also falls back to the requested app name for _last_app when Linux
windows carry no app_name.
Salvaged from #54173 by @dnth.
cua_driver_update_check now short-circuits to None when no driver
resolves; CI has none installed, so pin resolve_cua_driver_cmd in the
update-check and env-sanitization tests.
- Add autouse fixture to TestMcpInvocationResolution to disable
--no-overlay flag so existing tests assert baseline args
- Make test_config_load_failure_fails_safe and test_missing_section_enables
platform-aware (Linux auto-detect returns True, macOS/Windows False)
* fix(computer-use): target Linux app windows reliably
Resolve app filters through the canonical cua-driver MCP app metadata and join running app PIDs back to windows. Preserve an exact selected window across capture_after, support direct capture by pid/window_id, and send the active window ID for coordinate pointer actions on Linux.
Co-authored-by: annguyenNous <annguyenNous@users.noreply.github.com>
Co-authored-by: grimmjoww578 <willies578@gmail.com>
Co-authored-by: ai-ag2026 <261867348+ai-ag2026@users.noreply.github.com>
* fix(computer-use): address review on PR #63725
Address three review comments from @f-trycua:
1. type_text, press_key, and hotkey now carry _active_window_id and
fail closed when it is missing. Previously they sent only the PID,
so CUA Driver fell back to the first window for that PID — input
could reach the wrong window in multi-window apps.
2. Coordinate scroll x/y are now capability-gated behind
input.scroll.coordinates. CUA Driver 0.7.1 Linux schema rejects
x/y on scroll; omitting them when the driver doesn't advertise
support avoids the schema rejection while still routing via
window_id.
3. Windows are sorted by z_index descending (higher = front, per CUA
Driver semantics) instead of ascending. Null z_index (Wayland) is
coerced to 0 in _ingest_windows so it doesn't crash the sort and
sorts to the back instead of being selected as the capture target.
---------
Co-authored-by: LeonSGP43 <cine.dreamer.one@gmail.com>
Co-authored-by: annguyenNous <annguyenNous@users.noreply.github.com>
Co-authored-by: grimmjoww578 <willies578@gmail.com>
Co-authored-by: ai-ag2026 <261867348+ai-ag2026@users.noreply.github.com>
The SOM/AX element list dropped labels for two extremely common cua-driver
render forms, leaving the model unable to target elements by name:
- [79] AXButton (Dark) -> parenthesised label
- [4] AXStaticText = "Wi-Fi" -> = "value" form
- [92] AXPopUpButton = "Automatic" -> = "value" form
The old regex only matched quoted "label" and id=Label, so System Settings
buttons/text/popups all surfaced with empty labels. That's why selecting the
macOS Appearance 'Dark' button by element index required guessing — the
labels weren't available to aim with.
Fix: extend _ELEMENT_LINE_RE to capture all four label forms (= "value",
"quoted", (parenthesised), id=Label), skipping a pure-digit (N) order number
in favour of the id= label. Verified live against System Settings: the
Appearance buttons now surface as Auto/Light/Dark.
Adds a regression test covering all label forms. Full suite: 84 passed.
The first fix handled the EAGAIN McpError path. But the persistent MCP
session (long-running gateway/desktop worker) has a second failure mode:
list_windows or get_window_state 'succeed' over MCP yet return a
degenerate/empty payload (no windows, or no screenshot + blank tree)
WITHOUT raising — typically when the bridge reconnected mid-call and
dropped the heavy response. That surfaced to the model as a silent 0x0
capture with no error and no fallback firing (0.00s empty return).
Fix: detect empty results in capture() and re-fetch over the CLI
transport before giving up:
- empty list_windows -> CLI re-fetch the window list
- empty get_window_state (som/ax) -> CLI re-fetch the AX tree + screenshot
- empty screenshot (vision) -> CLI re-fetch get_window_state for the PNG
Adds 2 regression tests. Full suite: 83 passed.
The cua-driver MCP stdio bridge intermittently (and on some machines
persistently) fails to forward heavier calls like get_window_state to
the daemon with POSIX EAGAIN — 'daemon transport error forwarding
get_window_state: Resource temporarily unavailable (os error 35)'.
The wrapper surfaced this as an empty 0x0 capture, so computer_use
returned blank screenshots even though the display, permissions, and
the daemon were all healthy (the direct 'cua-driver call' CLI path
worked fine throughout).
Fix: when the MCP path raises the transient/transport error, fall back
to the 'cua-driver call' subprocess transport, which talks to the
daemon over a different socket. The CLI fallback routes get_window_state
screenshots to a temp file via screenshot_out_file (tiny JSON response
instead of a multi-MB base64 blob that congests the socket), reads the
PNG back, retries with backoff, and remaps the JSON into the same
{data, images, structuredContent, isError} shape the MCP path produces
so capture()/_action() are transport-agnostic.
Adds _is_transient_daemon_error() classifier and 3 regression tests.
Verified live: captures that returned 0x0 now return full
1567x905 screenshots with the AX element tree.
The 'never reached ready' error (issue #57025) was undiagnosable — doctor
and MCP test pass while the wrapper times out, with no hint where startup
stalled. Track a phase marker through _lifecycle_coro (binary-check →
manifest-discovery → mcp-initialize → capability-discovery → ready) and
include it in the timeout RuntimeError plus a pointer to doctor and the
agent.log phase timings.
Complements the 15s→30s bump + success-path phase timing log from #58760.
capture(app='screen'|'desktop') now resolves to the OS shell/desktop
window (Windows Progman/WorkerW desktop or Shell_TrayWnd taskbar, macOS
Finder/Dock) so 'show me my screen' and 'click the taskbar' work.
Previously capture() only matched application windows, and the schema
advertised 'or the whole screen' without any code path delivering it.
cua-driver is window-oriented (no virtual-desktop or per-monitor MCP
tool), so a single image still cannot span multiple monitors — the
schema now states this and the no-desktop-window path returns a clear
message instead of silently grabbing the frontmost app.
cua-driver 0.6.x removed the standalone screenshot MCP tool, so
capture(mode='vision') hit 'Unknown tool: screenshot' and returned a
0x0 image with no PNG while som/ax (which use get_window_state) still
worked. Route vision through get_window_state(capture_mode='vision').
Salvaged from PR #50771; same fix submitted earlier as #39262 by
@Tranquil-Flow.
Make the computer_use toolset platform-agnostic by driving cua-driver on
macOS, Windows, and Linux. Consumes the 8 cua-driver decoupling surfaces
(capability discovery, structuredContent AX tree, opaque element_token,
click button enum, explicit mimeType, machine-readable manifest,
structured list_windows, structured health_report), each degrading
gracefully on older drivers.
Adds `hermes computer-use doctor` (drives cua-driver health_report with a
per-OS check matrix and an exit 0/1/2 ok/degraded/blocked contract), full
typed wrappers for the previously-uncovered cua-driver tools plus a generic
call_tool escape hatch, per-session agent-cursor lifecycle, platform-aware
system-prompt guidance (host-deterministic, cache-safe), and honors
HERMES_CUA_DRIVER_CMD end-to-end.
Replaces the macOS-only skills/apple/macos-computer-use skill with a
cross-platform skills/computer-use skill, and refreshes the EN + zh-Hans
docs.
Supersedes #44221 (Windows-enablement salvage of #30660).
Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
Review feedback from egilewski:
1. Remove trailing whitespace from test docstring and mock patches (lines 1430, 1469, 1476, 1482)
2. Expand test coverage: also verify ANTHROPIC_API_KEY is stripped (not just OPENAI_API_KEY)
Changes:
- Remove trailing whitespace from test file
- Add ANTHROPIC_API_KEY to test environment
- Add assertion verifying ANTHROPIC_API_KEY is stripped from cua-driver subprocess env
- Syntax verified: python3 -m py_compile tests/tools/test_computer_use.py ✓
- Use _sanitize_subprocess_env() to filter Hermes-managed credentials
from the cua-driver subprocess environment (issue #37878)
- Prevents credential exfiltration to the third-party cua-driver binary
- Aligns with existing pattern used by browser-tool and other tools
- Add regression test to verify environment sanitization
The cua-driver is a lower-trust MCP subprocess per SECURITY.md §2.3.
Its inherited environment is now scrubbed by default, removing provider
API keys, gateway tokens, and platform credentials that should not leak
to third-party binaries.
Fixes#37878
xAI's consent page renders the authorization code in-page instead of
redirecting to the loopback callback, so the listener just hangs and the
manual-paste flow demands a callback URL that never contains the token.
- auth.py: poll stdin non-blockingly while waiting for the xAI loopback
callback; accept a pasted bare Grok Build code and substitute the locally
generated state (PKCE code_verifier still binds the exchange). No need to
wait for timeout or re-run with --manual-paste.
- computer_use: parse PNG/JPEG dimensions from base64 and fall back to the
text/AX/SOM payload when the screenshot is below the provider minimum
(8x8), which xAI rejects with HTTP 400.
- model_setup_flows.py: xAI credential reuse prompt uses the standard radio
picker via a shared _prompt_auth_credentials_choice helper.
- main.py: thread a title through _prompt_provider_choice; re-home the helper
import (flows live in model_setup_flows.py post-decomposition).
Salvaged from #36781 onto current main (contributor's main.py edits re-homed
to model_setup_flows.py, where the flows were extracted since the PR opened).
Remove unused imports (F401) and duplicate/shadowed import
redefinitions (F811) across the codebase using ruff's safe
autofixes. No behavioral changes -- imports only.
- ~1400 safe autofixes applied across 644 files (net -1072 lines)
- __init__.py re-exports preserved (excluded from F401 removal so
public re-export surfaces stay intact)
- Re-exports that are imported or monkeypatched by tests but look
unused in their defining module are kept with explicit # noqa:
F401 (gateway/run.py load_dotenv; run_agent re-exports from
agent.message_sanitization, agent.context_compressor,
agent.retry_utils, agent.prompt_builder, agent.process_bootstrap,
agent.codex_responses_adapter)
- Unsafe F841 (unused-variable) fixes deliberately skipped -- those
can change behavior when the RHS has side effects
- ruff lints remain disabled in pyproject.toml (only PLW1514 is
selected); this is a one-time cleanup, not a config change
Verification:
- python -m compileall: clean
- pytest --collect-only: all 27161 tests collect (zero import errors)
- core entry points import clean (run_agent, model_tools, cli,
toolsets, hermes_state, batch_runner, gateway)
- static scan: every name any test imports directly from an edited
module still resolves
_maybe_follow_capture() issued a follow-up screenshot unconditionally
when capture_after=True, even when res.ok=False. The model then received
a normal-looking screenshot alongside an error message, and in practice
it often ignored ok=False and proceeded as if the action had succeeded.
Fix: return _text_response(res) early when res.ok is False so the model
receives only the error and can decide how to recover.
Tests added:
- test_capture_after_skipped_when_action_failed: patches click to return
ok=False and asserts no capture call is issued.
- test_capture_after_fires_when_action_succeeds: ensures the happy path
still triggers the follow-up capture.
_dispatch() routes action="set_value" to backend.set_value(), but:
- ComputerUseBackend did not declare set_value as @abstractmethod, so
subclasses could silently omit it without a TypeError at class load time.
- _NoopBackend (the test/CI stub) had no set_value method at all, causing
AttributeError in any test that exercises the set_value action path.
Fix:
- Add set_value as @abstractmethod to ComputerUseBackend in backend.py.
- Add a recording stub in _NoopBackend in tool.py.
- Add two TestDispatch cases: one verifying the call reaches the backend,
one verifying the missing-value guard returns a clean error.
Four findings from Copilot's review on PR #22891, all in the AX
elements-array cap added by 22fa1ed:
1. The truncation note ("response truncated to N of M elements") was
appended unconditionally — including in the som/vision multimodal
path, whose response carries a screenshot rather than an `elements`
array. The note described a payload field that wasn't present.
Moved the note into the AX-text branch where the array actually
appears.
2. `_format_elements(cap.elements)` ran on the full untrimmed list with
its own `max_lines=40` cap, so a caller passing `max_elements=10`
would see summary lines referencing `#11..#40` even though the JSON
`elements` array only held #1..#10. Format on `visible_elements`
instead so the summary indices always exist in the response.
3. `_coerce_max_elements` enforced a lower bound but no upper bound,
so `max_elements=10_000_000` silently disabled the safeguard and
reintroduced the original context-blow-up. Added a hard cap
(`_MAX_ALLOWED_MAX_ELEMENTS = 1000`) that clamps oversized values.
4. The schema string said "Default 100" but the property carried no
`default` field, and claimed `max_elements` had no effect on som/
vision while the image-missing fallback path can still return an
elements array. Added `"default": 100`, `"maximum": 1000`, and
clarified the fallback-path wording.
Each finding gets a regression test:
- test_capture_ax_clamps_oversized_max_elements_to_hard_cap
- test_capture_ax_summary_indices_match_returned_elements
- test_capture_multimodal_summary_omits_truncation_note
- test_schema_max_elements_documents_default_and_upper_bound
Verified with `pytest tests/tools/test_computer_use.py` (53 passed,
including the 5 new cases). Confirmed each new test fails on the
pre-fix code path before applying the production change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`computer_use(action='capture', mode='ax')` returned the full AX element
list verbatim in the JSON response. Dense Electron / Obsidian / JetBrains
UIs publish 500+ AX nodes (one reproduction in #22865 returned 597
elements against Obsidian), so a single capture could consume enough
context to trigger compression failures or render the session unusable.
The human-readable `_format_elements` summary is already capped at 40
lines, so the truncation gap was invisible to anyone reading the summary
output.
Add a `max_elements` argument to the tool schema, default 100, that
trims the AX `elements` array. When the cap fires, the response surfaces
`total_elements` and `truncated_elements` and appends a "raise
max_elements or pass app= to narrow" hint to the summary so the model
knows the JSON view is partial and can re-issue with a tighter scope.
Validation is centralized in `_coerce_max_elements`: missing /
non-integer / sub-1 inputs fall back to the default cap, so the
protection can never be silently disabled by a malformed tool-call
argument. The cap only affects AX-mode JSON; `mode='som'` and
`mode='vision'` keep returning a screenshot + image-aware summary
unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
teknium1