feat(computer-use): user-facing authorization for cua-driver browser attachment
Completes the typed cua_browser_* route (PR #74166 lineage) with the authorization surface that makes existing-profile attachment and repeatable bounded automation reachable by real users: - hermes computer-use browser-approve: CLI passthrough that mints cua-driver's five-minute single-use attachment token for one exact (pid, window_id). The user, never the model, is the token source. - approval_token passthrough on cua_browser_prepare (schema + dispatch + browser_route), forwarded only for existing_profile and only as a non-empty string. - computer_use.permission_mode: bounded + capability_manifest config: private per-session embedded daemon launched with --capability-manifest/--approve-capability-manifest; missing manifest fails loudly. 'unrestricted' is deliberately NOT a config value — it stays bound to the explicit per-session YOLO toggle. - Skill + system-prompt + docs guidance for the three authorization rungs and the isolated-profile-first default. E2E-verified against a temp HERMES_HOME: real config resolution to bounded, loud failure without a manifest, real argparse path driving a fake cua-driver binary, standard default preserved.
This commit is contained in:
@@ -638,9 +638,12 @@ def computer_use_guidance(platform_name: Optional[str] = None) -> str:
|
||||
"browser chrome, OS permission prompts, native dialogs, and unsupported "
|
||||
"targets. Browser setup is a separately approved action; attaching an "
|
||||
"existing profile is enforced by cua-driver's immutable permission "
|
||||
"mode: standard requires a certified protected host and fails closed "
|
||||
"when Hermes has none; explicit Hermes YOLO uses a private unrestricted "
|
||||
"daemon after the user's launch/session risk acceptance.\n\n"
|
||||
"mode: in standard mode ask the USER to run `hermes computer-use "
|
||||
"browser-approve` and pass the pasted single-use token as "
|
||||
"`approval_token` (you can never mint one); bounded mode authorizes "
|
||||
"via the user's reviewed capability manifest; explicit Hermes YOLO "
|
||||
"uses a private unrestricted daemon after the user's launch/session "
|
||||
"risk acceptance.\n\n"
|
||||
"## Background mode rules\n"
|
||||
"- Do NOT use `raise_window=true` on `focus_app` unless the user "
|
||||
"explicitly asked you to bring a window to front. Input routing to "
|
||||
|
||||
@@ -3274,6 +3274,22 @@ DEFAULT_CONFIG = {
|
||||
# True = always disable the overlay
|
||||
# False = always enable the overlay
|
||||
"no_overlay": None,
|
||||
# cua-driver permission mode for this Hermes install.
|
||||
# standard (default) — cua-driver's own approval boundary. Protected
|
||||
# operations (e.g. attaching to an existing signed-in browser
|
||||
# profile) fail closed unless the user mints a single-use token
|
||||
# via `hermes computer-use browser-approve`.
|
||||
# bounded — repeatable automation under a user-reviewed capability
|
||||
# manifest (set capability_manifest below). No runtime prompts;
|
||||
# anything outside the manifest fails closed inside cua-driver.
|
||||
# `unrestricted` is intentionally NOT accepted here: it stays bound to
|
||||
# the explicit per-session YOLO toggle so a config line can never
|
||||
# silently bypass approvals.
|
||||
"permission_mode": "standard",
|
||||
# Absolute or ~ path to the reviewed cua-driver capability manifest
|
||||
# YAML used when permission_mode is "bounded". See
|
||||
# https://cua.ai/docs/reference/cua-driver/permission-modes
|
||||
"capability_manifest": "",
|
||||
},
|
||||
|
||||
# =========================================================================
|
||||
|
||||
@@ -12377,6 +12377,38 @@ def main():
|
||||
"grant",
|
||||
help="Request the grants (opens the dialog attributed to CuaDriver)",
|
||||
)
|
||||
computer_use_browser_approve = computer_use_sub.add_parser(
|
||||
"browser-approve",
|
||||
help="Mint a single-use token authorizing browser attachment for one exact window",
|
||||
description=(
|
||||
"Runs `cua-driver browser-approve` to mint a five-minute,\n"
|
||||
"single-use token that authorizes ONE browser preparation for the\n"
|
||||
"exact process (and window) you name. Give the printed token to\n"
|
||||
"the agent; it passes it as approval_token on the\n"
|
||||
"cua_browser_prepare action.\n\n"
|
||||
"This is the explicit human boundary for attaching to a browser —\n"
|
||||
"especially an existing signed-in profile, where the DevTools\n"
|
||||
"protocol can see that profile's live pages, cookies, and storage.\n"
|
||||
"Ordinary tool approval never substitutes for this grant, so a\n"
|
||||
"model can never mint or guess the token itself.\n\n"
|
||||
"Find the pid/window_id via the agent (list_windows) or ask it to\n"
|
||||
"read them from a native capture."
|
||||
),
|
||||
)
|
||||
computer_use_browser_approve.add_argument(
|
||||
"--pid", type=int, required=True,
|
||||
help="Exact browser process id to authorize",
|
||||
)
|
||||
computer_use_browser_approve.add_argument(
|
||||
"--window-id", type=int, default=None,
|
||||
help="Exact native window id (required for existing-profile attachment)",
|
||||
)
|
||||
computer_use_browser_approve.add_argument(
|
||||
"--profile-mode",
|
||||
choices=["isolated_new", "isolated_named", "existing_profile"],
|
||||
default="isolated_new",
|
||||
help="Preparation the token authorizes (default: isolated_new)",
|
||||
)
|
||||
|
||||
def cmd_computer_use(args):
|
||||
action = getattr(args, "computer_use_action", None)
|
||||
@@ -12433,6 +12465,36 @@ def main():
|
||||
json_output=bool(getattr(args, "json", False)),
|
||||
)
|
||||
sys.exit(code)
|
||||
if action == "browser-approve":
|
||||
import subprocess
|
||||
from tools.computer_use.cua_backend import (
|
||||
cua_driver_child_env,
|
||||
cua_driver_install_hint,
|
||||
resolve_cua_driver_cmd,
|
||||
)
|
||||
binary = resolve_cua_driver_cmd()
|
||||
if not binary:
|
||||
print(cua_driver_install_hint())
|
||||
sys.exit(2)
|
||||
cmd = [binary, "browser-approve", "--pid", str(args.pid)]
|
||||
window_id = getattr(args, "window_id", None)
|
||||
if window_id is not None:
|
||||
cmd += ["--window-id", str(window_id)]
|
||||
cmd += ["--profile-mode", getattr(args, "profile_mode", "isolated_new")]
|
||||
try:
|
||||
# Interactive passthrough: cua-driver requires a TTY to mint
|
||||
# the grant, prints the token itself, and owns the expiry.
|
||||
proc = subprocess.run(cmd, env=cua_driver_child_env())
|
||||
except OSError as exc:
|
||||
print(f"cua-driver browser-approve failed to launch: {exc}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
if proc.returncode == 0:
|
||||
print(
|
||||
"\nGive the token above to the agent — it passes it as "
|
||||
"approval_token on cua_browser_prepare. Single use, "
|
||||
"expires in ~5 minutes."
|
||||
)
|
||||
sys.exit(proc.returncode)
|
||||
if action == "permissions":
|
||||
perms_action = getattr(args, "computer_use_perms_action", None)
|
||||
if perms_action == "grant":
|
||||
|
||||
@@ -180,11 +180,29 @@ browser tools. The contract is capability-based:
|
||||
`cua_browser_prepare` is a separate approved setup action. Driver-owned
|
||||
`isolated_new`/`isolated_named` profiles require explicit `allow_launch=true`.
|
||||
An `existing_profile` is decided by cua-driver's immutable permission mode.
|
||||
Normal Hermes sessions use `standard`, which requires a certified protected
|
||||
host and fails closed when Hermes has none. Explicit Hermes YOLO (`--yolo`,
|
||||
`/yolo`, or `approvals.mode: off`) launches a private embedded cua-driver in
|
||||
`unrestricted` after that risk acceptance, so there are no runtime Cua
|
||||
approval prompts. Never invent, store, log, or reuse a grant token.
|
||||
Prefer `isolated_new` unless the task genuinely needs the user's signed-in
|
||||
session — attaching to an existing profile exposes its live pages, cookies,
|
||||
and storage over the browser protocol.
|
||||
|
||||
Authorization paths for `existing_profile`, in preference order:
|
||||
|
||||
1. **User-minted token (standard mode).** Ask the user to run
|
||||
`hermes computer-use browser-approve --pid <pid> --window-id <window_id>
|
||||
--profile-mode existing_profile` in their terminal and paste back the
|
||||
token. Pass it as `approval_token` on `cua_browser_prepare`. The token is
|
||||
single-use and expires in ~5 minutes, so mint it right before the call.
|
||||
You cannot create this token yourself; only the user can.
|
||||
2. **Bounded manifest.** When `computer_use.permission_mode: bounded` is
|
||||
configured with a reviewed `capability_manifest`, prepares inside the
|
||||
manifest's scope succeed without prompts and everything else fails closed.
|
||||
No token needed; do not ask for one.
|
||||
3. **Explicit Hermes YOLO** (`--yolo`, `/yolo`, or `approvals.mode: off`)
|
||||
launches a private embedded cua-driver in `unrestricted` after that risk
|
||||
acceptance, so there are no runtime Cua approval prompts.
|
||||
|
||||
Without any of these, `existing_profile` fails closed — report the refusal and
|
||||
offer path 1 to the user; do not retry, downgrade trust, or work around it.
|
||||
Never invent, store, log, or reuse a grant token.
|
||||
|
||||
Use the native capture/AX/pixel/foreground ladder for browser chrome, browser
|
||||
permission UI, OS prompts, native dialogs, extension surfaces, unsupported
|
||||
|
||||
@@ -0,0 +1,364 @@
|
||||
"""Authorization plumbing for the cua-driver typed browser route.
|
||||
|
||||
Covers the three rungs that let ``existing_profile`` attachment (and bounded
|
||||
automation generally) actually work from Hermes:
|
||||
|
||||
* ``approval_token`` passthrough — the user-minted single-use token from
|
||||
``hermes computer-use browser-approve`` reaches ``browser_prepare`` and is
|
||||
never fabricated by the wrapper.
|
||||
* ``bounded`` permission mode — a private embedded daemon launched with a
|
||||
user-reviewed capability manifest (``--capability-manifest`` +
|
||||
``--approve-capability-manifest``), failing loudly when the manifest is
|
||||
missing.
|
||||
* mode resolution — config supplies standard/bounded only; explicit session
|
||||
YOLO still (and exclusively) selects unrestricted.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict
|
||||
|
||||
import pytest
|
||||
|
||||
from tools.computer_use import cua_backend as cb
|
||||
from tools.computer_use.browser_route import CuaTypedBrowserRoute
|
||||
from tools.computer_use.cua_backend import _EmbeddedCuaDaemon
|
||||
|
||||
|
||||
def _driver_result(payload: Dict[str, Any]) -> Dict[str, Any]:
|
||||
return {"structuredContent": dict(payload)}
|
||||
|
||||
|
||||
class _PrepareDriver:
|
||||
def __init__(self) -> None:
|
||||
self.calls: list[tuple[str, Dict[str, Any]]] = []
|
||||
|
||||
def has_tool(self, name: str) -> bool:
|
||||
return True
|
||||
|
||||
def call(self, name: str, args: Dict[str, Any]) -> Dict[str, Any]:
|
||||
self.calls.append((name, dict(args)))
|
||||
return _driver_result({"status": "ok"})
|
||||
|
||||
|
||||
def _route(driver: _PrepareDriver) -> CuaTypedBrowserRoute:
|
||||
return CuaTypedBrowserRoute(
|
||||
session_id="hermes-a",
|
||||
call_tool=driver.call,
|
||||
has_tool=driver.has_tool,
|
||||
)
|
||||
|
||||
|
||||
# ── approval_token passthrough ──────────────────────────────────────────
|
||||
|
||||
|
||||
def test_existing_profile_prepare_forwards_user_minted_approval_token():
|
||||
driver = _PrepareDriver()
|
||||
result = _route(driver).prepare(
|
||||
pid=101,
|
||||
window_id=202,
|
||||
profile_mode="existing_profile",
|
||||
approval_token="tok-from-user-terminal",
|
||||
)
|
||||
|
||||
assert result["status"] == "ok"
|
||||
assert driver.calls == [
|
||||
(
|
||||
"browser_prepare",
|
||||
{
|
||||
"pid": 101,
|
||||
"window_id": 202,
|
||||
"strategy": {"kind": "existing_profile"},
|
||||
"approval_token": "tok-from-user-terminal",
|
||||
"session": "hermes-a",
|
||||
},
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def test_existing_profile_prepare_without_token_sends_none():
|
||||
"""No token → the field is absent; the driver's own gate decides."""
|
||||
driver = _PrepareDriver()
|
||||
_route(driver).prepare(pid=101, window_id=202, profile_mode="existing_profile")
|
||||
|
||||
(_, args), = driver.calls
|
||||
assert "approval_token" not in args
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bogus", ["", None, 7, True])
|
||||
def test_non_string_or_empty_token_is_never_forwarded(bogus):
|
||||
driver = _PrepareDriver()
|
||||
_route(driver).prepare(
|
||||
pid=101,
|
||||
window_id=202,
|
||||
profile_mode="existing_profile",
|
||||
approval_token=bogus,
|
||||
)
|
||||
|
||||
(_, args), = driver.calls
|
||||
assert "approval_token" not in args
|
||||
|
||||
|
||||
def test_isolated_prepare_ignores_approval_token():
|
||||
"""The token authorizes existing-profile attachment only."""
|
||||
driver = _PrepareDriver()
|
||||
_route(driver).prepare(
|
||||
pid=101,
|
||||
profile_mode="isolated_new",
|
||||
allow_launch=True,
|
||||
approval_token="tok",
|
||||
)
|
||||
|
||||
(_, args), = driver.calls
|
||||
assert "approval_token" not in args
|
||||
|
||||
|
||||
def test_dispatch_forwards_approval_token_to_backend():
|
||||
from unittest.mock import Mock
|
||||
|
||||
from tools.computer_use.tool import _dispatch
|
||||
|
||||
backend = Mock()
|
||||
backend.typed_browser_prepare.return_value = {"status": "ok"}
|
||||
|
||||
_dispatch(
|
||||
backend,
|
||||
"cua_browser_prepare",
|
||||
{
|
||||
"pid": 101,
|
||||
"window_id": 202,
|
||||
"profile_mode": "existing_profile",
|
||||
"approval_token": "tok-abc",
|
||||
},
|
||||
)
|
||||
|
||||
kwargs = backend.typed_browser_prepare.call_args.kwargs
|
||||
assert kwargs["approval_token"] == "tok-abc"
|
||||
assert kwargs["profile_mode"] == "existing_profile"
|
||||
|
||||
|
||||
def test_schema_documents_approval_token_as_user_minted():
|
||||
from tools.computer_use.schema import COMPUTER_USE_SCHEMA
|
||||
|
||||
prop = COMPUTER_USE_SCHEMA["parameters"]["properties"]["approval_token"]
|
||||
desc = prop["description"]
|
||||
assert "browser-approve" in desc
|
||||
assert "never invent" in desc.lower()
|
||||
|
||||
|
||||
# ── bounded embedded daemon ─────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_bounded_daemon_requires_a_manifest():
|
||||
with pytest.raises(ValueError, match="capability_manifest"):
|
||||
_EmbeddedCuaDaemon("cua-driver", "bounded")
|
||||
|
||||
|
||||
def test_bounded_daemon_requires_manifest_file_to_exist(tmp_path):
|
||||
with pytest.raises(ValueError, match="not found"):
|
||||
_EmbeddedCuaDaemon(
|
||||
"cua-driver", "bounded",
|
||||
capability_manifest=str(tmp_path / "missing.yaml"),
|
||||
)
|
||||
|
||||
|
||||
def test_bounded_daemon_env_does_not_bypass_approvals(tmp_path):
|
||||
manifest = tmp_path / "manifest.yaml"
|
||||
manifest.write_text("version: 3\n", encoding="utf-8")
|
||||
daemon = _EmbeddedCuaDaemon(
|
||||
"cua-driver", "bounded", capability_manifest=str(manifest)
|
||||
)
|
||||
|
||||
env = daemon.child_env()
|
||||
assert env["CUA_DRIVER_PERMISSION_MODE"] == "bounded"
|
||||
assert "CUA_DRIVER_DANGEROUSLY_BYPASS_APPROVALS" not in env
|
||||
|
||||
|
||||
def test_unrestricted_daemon_env_keeps_explicit_bypass():
|
||||
daemon = _EmbeddedCuaDaemon("cua-driver", "unrestricted")
|
||||
env = daemon.child_env()
|
||||
assert env["CUA_DRIVER_PERMISSION_MODE"] == "unrestricted"
|
||||
assert env["CUA_DRIVER_DANGEROUSLY_BYPASS_APPROVALS"] == "1"
|
||||
|
||||
|
||||
def test_bounded_daemon_serves_with_approved_manifest(tmp_path, monkeypatch):
|
||||
"""The spawn command carries the manifest + launch-time approval flags."""
|
||||
manifest = tmp_path / "manifest.yaml"
|
||||
manifest.write_text("version: 3\n", encoding="utf-8")
|
||||
daemon = _EmbeddedCuaDaemon(
|
||||
"cua-driver", "bounded", capability_manifest=str(manifest)
|
||||
)
|
||||
|
||||
captured: Dict[str, Any] = {}
|
||||
|
||||
class _FakeProc:
|
||||
stderr = None
|
||||
|
||||
def poll(self):
|
||||
return None
|
||||
|
||||
def _fake_popen(command, **kwargs):
|
||||
captured["command"] = list(command)
|
||||
return _FakeProc()
|
||||
|
||||
def _fake_run(command, **kwargs):
|
||||
class _Probe:
|
||||
returncode = 0
|
||||
return _Probe()
|
||||
|
||||
monkeypatch.setattr(cb.subprocess, "Popen", _fake_popen)
|
||||
monkeypatch.setattr(cb.subprocess, "run", _fake_run)
|
||||
monkeypatch.setattr(
|
||||
cb, "_resolve_mcp_invocation", lambda cmd: (cmd, ["mcp"])
|
||||
)
|
||||
|
||||
daemon.start()
|
||||
|
||||
command = captured["command"]
|
||||
assert "--permission-mode" in command
|
||||
assert command[command.index("--permission-mode") + 1] == "bounded"
|
||||
assert "--capability-manifest" in command
|
||||
assert (
|
||||
command[command.index("--capability-manifest") + 1]
|
||||
== str(manifest)
|
||||
)
|
||||
assert "--approve-capability-manifest" in command
|
||||
assert "--dangerously-bypass-approvals" not in command
|
||||
|
||||
|
||||
def test_unrestricted_daemon_serve_command_unchanged(monkeypatch):
|
||||
daemon = _EmbeddedCuaDaemon("cua-driver", "unrestricted")
|
||||
|
||||
captured: Dict[str, Any] = {}
|
||||
|
||||
class _FakeProc:
|
||||
stderr = None
|
||||
|
||||
def poll(self):
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(
|
||||
cb.subprocess, "Popen",
|
||||
lambda command, **kw: captured.update(command=list(command)) or _FakeProc(),
|
||||
)
|
||||
|
||||
def _fake_run(command, **kwargs):
|
||||
class _Probe:
|
||||
returncode = 0
|
||||
return _Probe()
|
||||
|
||||
monkeypatch.setattr(cb.subprocess, "run", _fake_run)
|
||||
monkeypatch.setattr(
|
||||
cb, "_resolve_mcp_invocation", lambda cmd: (cmd, ["mcp"])
|
||||
)
|
||||
|
||||
daemon.start()
|
||||
|
||||
command = captured["command"]
|
||||
assert "--dangerously-bypass-approvals" in command
|
||||
assert "--capability-manifest" not in command
|
||||
|
||||
|
||||
# ── permission-mode resolution ──────────────────────────────────────────
|
||||
|
||||
|
||||
def test_configured_mode_defaults_to_standard(monkeypatch):
|
||||
monkeypatch.setattr(cb, "_computer_use_cfg", dict)
|
||||
assert cb._cua_configured_permission_mode() == "standard"
|
||||
|
||||
|
||||
def test_configured_mode_honors_bounded(monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
cb, "_computer_use_cfg", lambda: {"permission_mode": "Bounded"}
|
||||
)
|
||||
assert cb._cua_configured_permission_mode() == "bounded"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["unrestricted", "yolo", "off", 3, None])
|
||||
def test_configured_mode_never_yields_unrestricted(monkeypatch, value):
|
||||
"""A config line must never silently bypass approvals."""
|
||||
monkeypatch.setattr(
|
||||
cb, "_computer_use_cfg", lambda: {"permission_mode": value}
|
||||
)
|
||||
assert cb._cua_configured_permission_mode() == "standard"
|
||||
|
||||
|
||||
def test_capability_manifest_reads_config(monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
cb, "_computer_use_cfg",
|
||||
lambda: {"capability_manifest": " ~/manifests/cua.yaml "},
|
||||
)
|
||||
assert cb._cua_capability_manifest() == "~/manifests/cua.yaml"
|
||||
monkeypatch.setattr(cb, "_computer_use_cfg", dict)
|
||||
assert cb._cua_capability_manifest() is None
|
||||
|
||||
|
||||
def test_session_yolo_overrides_configured_bounded(monkeypatch):
|
||||
import tools.computer_use.tool as cu_tool
|
||||
|
||||
monkeypatch.setattr(
|
||||
cb, "_computer_use_cfg", lambda: {"permission_mode": "bounded"}
|
||||
)
|
||||
import tools.approval as approval
|
||||
|
||||
monkeypatch.setattr(
|
||||
approval, "is_approval_bypass_active_for_session", lambda sid: True
|
||||
)
|
||||
assert cu_tool._cua_permission_mode("sess-1") == "unrestricted"
|
||||
|
||||
|
||||
def test_no_yolo_uses_configured_bounded(monkeypatch):
|
||||
import tools.computer_use.tool as cu_tool
|
||||
|
||||
monkeypatch.setattr(
|
||||
cb, "_computer_use_cfg", lambda: {"permission_mode": "bounded"}
|
||||
)
|
||||
import tools.approval as approval
|
||||
|
||||
monkeypatch.setattr(
|
||||
approval, "is_approval_bypass_active_for_session", lambda sid: False
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
approval, "get_current_session_key", lambda default="": ""
|
||||
)
|
||||
assert cu_tool._cua_permission_mode("sess-1") == "bounded"
|
||||
|
||||
|
||||
def test_no_yolo_no_config_stays_standard(monkeypatch):
|
||||
import tools.computer_use.tool as cu_tool
|
||||
|
||||
monkeypatch.setattr(cb, "_computer_use_cfg", dict)
|
||||
import tools.approval as approval
|
||||
|
||||
monkeypatch.setattr(
|
||||
approval, "is_approval_bypass_active_for_session", lambda sid: False
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
approval, "get_current_session_key", lambda default="": ""
|
||||
)
|
||||
assert cu_tool._cua_permission_mode("sess-1") == "standard"
|
||||
|
||||
|
||||
def test_backend_accepts_bounded_with_manifest(tmp_path, monkeypatch):
|
||||
manifest = tmp_path / "manifest.yaml"
|
||||
manifest.write_text("version: 3\n", encoding="utf-8")
|
||||
monkeypatch.setattr(
|
||||
cb, "_cua_capability_manifest", lambda: str(manifest)
|
||||
)
|
||||
monkeypatch.setattr(cb, "resolve_cua_driver_cmd", lambda override=None: "cua-driver")
|
||||
|
||||
backend = cb.CuaDriverBackend(permission_mode="bounded")
|
||||
assert backend.permission_mode == "bounded"
|
||||
assert backend._embedded_daemon is not None
|
||||
assert backend._embedded_daemon.capability_manifest == str(manifest)
|
||||
|
||||
|
||||
def test_backend_bounded_without_manifest_fails_loudly(monkeypatch):
|
||||
monkeypatch.setattr(cb, "_cua_capability_manifest", lambda: None)
|
||||
monkeypatch.setattr(cb, "resolve_cua_driver_cmd", lambda override=None: "cua-driver")
|
||||
|
||||
with pytest.raises(ValueError, match="capability_manifest"):
|
||||
cb.CuaDriverBackend(permission_mode="bounded")
|
||||
|
||||
|
||||
def test_backend_rejects_unknown_mode():
|
||||
with pytest.raises(ValueError, match="unsupported"):
|
||||
cb.CuaDriverBackend(permission_mode="wide-open")
|
||||
@@ -761,6 +761,7 @@ def test_namespaced_state_and_prepare_actions_use_typed_backend_wrappers():
|
||||
profile_mode="isolated_new",
|
||||
profile_name=None,
|
||||
allow_launch=True,
|
||||
approval_token=None,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -343,6 +343,7 @@ class CuaTypedBrowserRoute:
|
||||
profile_mode: str,
|
||||
profile_name: Optional[str] = None,
|
||||
allow_launch: bool = False,
|
||||
approval_token: Optional[str] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Run explicit setup through the driver's authoritative mode gate."""
|
||||
missing = self._require_tool("browser_prepare")
|
||||
@@ -361,17 +362,22 @@ class CuaTypedBrowserRoute:
|
||||
"Existing-profile attachment requires an exact positive pid and window_id pair.",
|
||||
)
|
||||
# The driver owns the immutable standard/bounded/unrestricted
|
||||
# decision. Standard fails closed without a certified host;
|
||||
# explicit Hermes YOLO owns a private unrestricted daemon.
|
||||
# decision. Standard fails closed without a certified host,
|
||||
# a user-minted single-use approval token, or an approved
|
||||
# bounded manifest; explicit Hermes YOLO owns a private
|
||||
# unrestricted daemon.
|
||||
self.state.clear()
|
||||
return self._call(
|
||||
"browser_prepare",
|
||||
{
|
||||
"pid": exact_pid,
|
||||
"window_id": exact_window,
|
||||
"strategy": {"kind": "existing_profile"},
|
||||
},
|
||||
)
|
||||
args: Dict[str, Any] = {
|
||||
"pid": exact_pid,
|
||||
"window_id": exact_window,
|
||||
"strategy": {"kind": "existing_profile"},
|
||||
}
|
||||
if isinstance(approval_token, str) and approval_token:
|
||||
# Minted out-of-band by `hermes computer-use browser-approve`
|
||||
# (cua-driver browser-approve): five-minute, single-use. The
|
||||
# user, never the model, is the source of this value.
|
||||
args["approval_token"] = approval_token
|
||||
return self._call("browser_prepare", args)
|
||||
if profile_mode not in {"isolated_new", "isolated_named"}:
|
||||
return _refusal(
|
||||
"browser_profile_mode_invalid",
|
||||
|
||||
@@ -240,6 +240,32 @@ def _cua_telemetry_disabled() -> bool:
|
||||
return not bool(_computer_use_cfg().get("cua_telemetry", False))
|
||||
|
||||
|
||||
def _cua_configured_permission_mode() -> str:
|
||||
"""The user-configured cua-driver permission mode.
|
||||
|
||||
Reads ``computer_use.permission_mode`` (default ``standard``). Only
|
||||
``standard`` and ``bounded`` are honored here — ``unrestricted`` is
|
||||
deliberately NOT a config value: it stays tied to the explicit
|
||||
per-session Hermes YOLO toggle so a stale config line can never
|
||||
silently bypass approvals. Unknown values fall closed to ``standard``.
|
||||
"""
|
||||
raw = str(_computer_use_cfg().get("permission_mode", "standard") or "").strip().lower()
|
||||
return raw if raw in {"standard", "bounded"} else "standard"
|
||||
|
||||
|
||||
def _cua_capability_manifest() -> Optional[str]:
|
||||
"""Path of the reviewed capability manifest for bounded mode, or None.
|
||||
|
||||
Reads ``computer_use.capability_manifest``. Existence is validated by
|
||||
``_EmbeddedCuaDaemon`` so a missing file fails loudly at session start
|
||||
instead of silently degrading the authorization story.
|
||||
"""
|
||||
raw = _computer_use_cfg().get("capability_manifest")
|
||||
if not isinstance(raw, str) or not raw.strip():
|
||||
return None
|
||||
return raw.strip()
|
||||
|
||||
|
||||
def _computer_use_max_image_dimension() -> Optional[int]:
|
||||
"""Longest-edge cap for cua-driver screenshots, or None to leave unset.
|
||||
|
||||
@@ -381,19 +407,48 @@ def _wsl_windows_path_to_posix(path: str) -> str:
|
||||
|
||||
|
||||
class _EmbeddedCuaDaemon:
|
||||
"""Private host-owned daemon used for an explicit unrestricted session.
|
||||
"""Private host-owned daemon for a non-standard permission mode.
|
||||
|
||||
Cua Driver permission mode is immutable after daemon startup. Reusing the
|
||||
machine-wide daemon would therefore let one Hermes session's YOLO choice
|
||||
affect another session. A private embedded daemon gives the requesting
|
||||
session its own socket, process, and launch-time risk acknowledgement.
|
||||
session its own socket, process, and launch-time authorization:
|
||||
|
||||
* ``unrestricted`` — explicit Hermes YOLO; launch-time risk
|
||||
acknowledgement via ``--dangerously-bypass-approvals``.
|
||||
* ``bounded`` — a user-reviewed capability manifest
|
||||
(``computer_use.capability_manifest`` in config.yaml) approved at
|
||||
launch via ``--approve-capability-manifest``. The manifest, not a
|
||||
runtime prompt, is the authorization boundary; calls outside it fail
|
||||
closed inside cua-driver.
|
||||
"""
|
||||
|
||||
_START_TIMEOUT_SECONDS = 15.0
|
||||
|
||||
def __init__(self, driver_cmd: str, permission_mode: str) -> None:
|
||||
if permission_mode != "unrestricted":
|
||||
raise ValueError("embedded permission override supports unrestricted only")
|
||||
def __init__(
|
||||
self,
|
||||
driver_cmd: str,
|
||||
permission_mode: str,
|
||||
capability_manifest: Optional[str] = None,
|
||||
) -> None:
|
||||
if permission_mode not in {"unrestricted", "bounded"}:
|
||||
raise ValueError(
|
||||
"embedded permission override supports unrestricted or bounded only"
|
||||
)
|
||||
if permission_mode == "bounded":
|
||||
manifest = str(capability_manifest or "").strip()
|
||||
if not manifest:
|
||||
raise ValueError(
|
||||
"bounded permission mode requires computer_use.capability_manifest"
|
||||
)
|
||||
manifest = os.path.abspath(os.path.expanduser(manifest))
|
||||
if not os.path.isfile(manifest):
|
||||
raise ValueError(
|
||||
f"capability manifest not found: {manifest}"
|
||||
)
|
||||
self.capability_manifest: Optional[str] = manifest
|
||||
else:
|
||||
self.capability_manifest = None
|
||||
self.permission_mode = permission_mode
|
||||
self._driver_cmd = driver_cmd
|
||||
self._command = driver_cmd
|
||||
@@ -411,8 +466,9 @@ class _EmbeddedCuaDaemon:
|
||||
|
||||
def child_env(self) -> Dict[str, str]:
|
||||
env = cua_driver_child_env()
|
||||
env["CUA_DRIVER_PERMISSION_MODE"] = "unrestricted"
|
||||
env["CUA_DRIVER_DANGEROUSLY_BYPASS_APPROVALS"] = "1"
|
||||
env["CUA_DRIVER_PERMISSION_MODE"] = self.permission_mode
|
||||
if self.permission_mode == "unrestricted":
|
||||
env["CUA_DRIVER_DANGEROUSLY_BYPASS_APPROVALS"] = "1"
|
||||
return env
|
||||
|
||||
def _drain_stderr(self, process: Any) -> None:
|
||||
@@ -447,9 +503,18 @@ class _EmbeddedCuaDaemon:
|
||||
self.socket_path,
|
||||
"--no-permissions-gate",
|
||||
"--permission-mode",
|
||||
"unrestricted",
|
||||
"--dangerously-bypass-approvals",
|
||||
self.permission_mode,
|
||||
]
|
||||
if self.permission_mode == "unrestricted":
|
||||
command.append("--dangerously-bypass-approvals")
|
||||
else: # bounded — manifest validated in __init__
|
||||
command.extend(
|
||||
[
|
||||
"--capability-manifest",
|
||||
str(self.capability_manifest),
|
||||
"--approve-capability-manifest",
|
||||
]
|
||||
)
|
||||
self._process = subprocess.Popen(
|
||||
command,
|
||||
stdin=subprocess.DEVNULL,
|
||||
@@ -1917,14 +1982,23 @@ class CuaDriverBackend(ComputerUseBackend):
|
||||
"""Default computer-use backend. Cross-platform via cua-driver MCP."""
|
||||
|
||||
def __init__(self, permission_mode: str = "standard") -> None:
|
||||
if permission_mode not in {"standard", "unrestricted"}:
|
||||
if permission_mode not in {"standard", "bounded", "unrestricted"}:
|
||||
raise ValueError(f"unsupported cua-driver permission mode: {permission_mode}")
|
||||
self.permission_mode = permission_mode
|
||||
self._embedded_daemon = (
|
||||
_EmbeddedCuaDaemon(resolve_cua_driver_cmd() or "", permission_mode)
|
||||
if permission_mode == "unrestricted"
|
||||
else None
|
||||
)
|
||||
if permission_mode == "unrestricted":
|
||||
self._embedded_daemon: Optional[_EmbeddedCuaDaemon] = _EmbeddedCuaDaemon(
|
||||
resolve_cua_driver_cmd() or "", permission_mode
|
||||
)
|
||||
elif permission_mode == "bounded":
|
||||
# Manifest path comes from config.yaml; _EmbeddedCuaDaemon
|
||||
# validates existence and raises a clear error otherwise.
|
||||
self._embedded_daemon = _EmbeddedCuaDaemon(
|
||||
resolve_cua_driver_cmd() or "",
|
||||
permission_mode,
|
||||
capability_manifest=_cua_capability_manifest(),
|
||||
)
|
||||
else:
|
||||
self._embedded_daemon = None
|
||||
self._bridge = _AsyncBridge()
|
||||
self._session = _CuaDriverSession(self._bridge, self._embedded_daemon)
|
||||
# Sticky context — updated by capture(), used by action tools.
|
||||
|
||||
@@ -291,12 +291,25 @@ COMPUTER_USE_SCHEMA: Dict[str, Any] = {
|
||||
"enum": ["isolated_new", "isolated_named", "existing_profile"],
|
||||
"description": (
|
||||
"Browser preparation mode. existing_profile is decided by "
|
||||
"cua-driver's immutable permission mode: standard requires a "
|
||||
"certified protected host; explicit Hermes YOLO uses a private "
|
||||
"cua-driver's immutable permission mode: in standard mode "
|
||||
"ask the user to mint a single-use token with `hermes "
|
||||
"computer-use browser-approve` and pass it as "
|
||||
"approval_token; bounded mode authorizes via the reviewed "
|
||||
"capability manifest; explicit Hermes YOLO uses a private "
|
||||
"unrestricted daemon."
|
||||
),
|
||||
},
|
||||
"profile_name": {"type": "string", "description": "Name for isolated_named setup."},
|
||||
"approval_token": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"Single-use existing-profile attachment token the USER "
|
||||
"minted with `hermes computer-use browser-approve`. Ask "
|
||||
"the user to run that command and give you the token; "
|
||||
"never invent one. Only meaningful with "
|
||||
"profile_mode='existing_profile'."
|
||||
),
|
||||
},
|
||||
"allow_launch": {
|
||||
"type": "boolean",
|
||||
"description": "Explicitly allow launch of a driver-owned isolated browser.",
|
||||
|
||||
@@ -195,7 +195,15 @@ def _cua_permission_mode(session_id: str) -> str:
|
||||
except Exception:
|
||||
# Approval state must fail closed if it cannot be resolved.
|
||||
pass
|
||||
return "standard"
|
||||
try:
|
||||
# Without YOLO, honor the configured mode (standard | bounded).
|
||||
# bounded requires computer_use.capability_manifest; the backend
|
||||
# fails loudly at session start when the manifest is missing.
|
||||
from tools.computer_use.cua_backend import _cua_configured_permission_mode
|
||||
|
||||
return _cua_configured_permission_mode()
|
||||
except Exception:
|
||||
return "standard"
|
||||
|
||||
|
||||
def _get_backend(session_id: str = "") -> ComputerUseBackend:
|
||||
@@ -649,6 +657,7 @@ def _dispatch(backend: ComputerUseBackend, action: str, args: Dict[str, Any]) ->
|
||||
profile_mode=args.get("profile_mode", "isolated_new"),
|
||||
profile_name=args.get("profile_name"),
|
||||
allow_launch=bool(args.get("allow_launch")),
|
||||
approval_token=args.get("approval_token"),
|
||||
))
|
||||
|
||||
browser_tools = {
|
||||
|
||||
@@ -80,23 +80,67 @@ or add `computer_use` to your enabled toolsets in `~/.hermes/config.yaml`.
|
||||
|
||||
## Permission modes and logged-in browser profiles
|
||||
|
||||
Hermes maps its existing approval UX onto cua-driver 0.10's immutable daemon
|
||||
Hermes maps its existing approval UX onto cua-driver's immutable daemon
|
||||
modes. There is no second permission toggle to keep in sync:
|
||||
|
||||
| Hermes session | cua-driver mode | Human intervention | `existing_profile` |
|
||||
|---|---|---|---|
|
||||
| Manual or smart approvals (default) | `standard` | Normal Hermes approvals; Cua stops at its protected boundary | Refuses unless a certified protected host is available; Hermes does not claim one today |
|
||||
| Manual or smart approvals (default) | `standard` | Normal Hermes approvals; Cua stops at its protected boundary | Refuses unless you mint a single-use token with `hermes computer-use browser-approve` |
|
||||
| `computer_use.permission_mode: bounded` + reviewed manifest | private `bounded` daemon | You review and approve the capability manifest once, at launch | Allowed only within the manifest's declared profiles/origins/tools; everything else fails closed |
|
||||
| `--yolo`, `/yolo`, or `approvals.mode: off` | private `unrestricted` daemon | One explicit Hermes risk acceptance; no runtime Cua prompts | Allowed within Cua's built-in, managed, and user policy ceilings |
|
||||
|
||||
The unrestricted daemon is private to that Hermes session. Turning `/yolo`
|
||||
off, resetting/closing the session, cancellation cleanup, or process exit ends
|
||||
the Cua session and stops that daemon. It never changes the machine-wide
|
||||
daemon's mode or grants another Hermes conversation the same authority.
|
||||
### Attaching to your signed-in browser (`browser-approve`)
|
||||
|
||||
The agent can drive a Chrome/Edge window you already have open — including a
|
||||
signed-in profile — **without restarting the browser, copying the profile, or
|
||||
touching your tabs**. Because DevTools access exposes that profile's live
|
||||
pages, cookies, and storage, cua-driver requires an explicit human grant that
|
||||
ordinary tool approval cannot substitute for. Hermes surfaces it as:
|
||||
|
||||
```bash
|
||||
# The agent will tell you the exact pid/window_id it discovered:
|
||||
hermes computer-use browser-approve --pid 844 --window-id 10725 --profile-mode existing_profile
|
||||
```
|
||||
|
||||
This mints a five-minute, single-use token bound to that exact process and
|
||||
window. Paste the token to the agent; it passes it as `approval_token` on the
|
||||
`cua_browser_prepare` action and cua-driver verifies the binding before
|
||||
attaching. The grant lives only in driver memory and dies with the session.
|
||||
Prefer letting the agent use an isolated profile (`isolated_new`, no token
|
||||
needed beyond the normal tool approval) unless the task genuinely needs your
|
||||
authenticated session.
|
||||
|
||||
### Bounded mode for repeatable automation
|
||||
|
||||
For recurring browser automation (cron jobs, scheduled research against an
|
||||
authenticated app), per-run tokens are impractical. `bounded` mode replaces
|
||||
prompts with a capability manifest you review once:
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
computer_use:
|
||||
permission_mode: bounded
|
||||
capability_manifest: ~/.hermes/cua-manifest.yaml
|
||||
```
|
||||
|
||||
The manifest names the apps, browser profile kinds, allowed origins, and
|
||||
typed tools the session may use (see the
|
||||
[cua-driver permission modes reference](https://cua.ai/docs/reference/cua-driver/permission-modes)
|
||||
for the format). Hermes launches a private per-session daemon with
|
||||
`--capability-manifest ... --approve-capability-manifest`; anything outside
|
||||
the manifest fails closed inside cua-driver. A missing or unreadable manifest
|
||||
fails loudly at session start rather than silently downgrading. Session YOLO
|
||||
still overrides bounded for that one session.
|
||||
|
||||
The bounded and unrestricted daemons are private to that Hermes session.
|
||||
Turning `/yolo` off, resetting/closing the session, cancellation cleanup, or
|
||||
process exit ends the Cua session and stops that daemon. It never changes the
|
||||
machine-wide daemon's mode or grants another Hermes conversation the same
|
||||
authority.
|
||||
|
||||
`smart` approval remains `standard`: an LLM classification is not protected
|
||||
human consent. Cua's `bounded` manifest mode is also not inferred from smart
|
||||
approval or a normal tool confirmation; it needs a separately trusted host
|
||||
that reviews and launches the exact manifest.
|
||||
human consent, and it cannot mint a `browser-approve` token or stand in for a
|
||||
reviewed manifest.
|
||||
|
||||
<div class="alert alert--warning">
|
||||
|
||||
@@ -320,6 +364,15 @@ real headless Chromium and is the right answer for web-only tasks.
|
||||
|
||||
## Configuration
|
||||
|
||||
Permission mode and manifest (see
|
||||
[Permission modes](#permission-modes-and-logged-in-browser-profiles) above):
|
||||
|
||||
```yaml
|
||||
computer_use:
|
||||
permission_mode: standard # standard (default) | bounded
|
||||
capability_manifest: "" # manifest path, required for bounded
|
||||
```
|
||||
|
||||
Override the driver binary path (tests / CI / local builds):
|
||||
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user