feat(computer-use): user-facing authorization for cua-driver browser attachment

Completes the typed cua_browser_* route (PR #74166 lineage) with the
authorization surface that makes existing-profile attachment and
repeatable bounded automation reachable by real users:

- hermes computer-use browser-approve: CLI passthrough that mints
  cua-driver's five-minute single-use attachment token for one exact
  (pid, window_id). The user, never the model, is the token source.
- approval_token passthrough on cua_browser_prepare (schema + dispatch +
  browser_route), forwarded only for existing_profile and only as a
  non-empty string.
- computer_use.permission_mode: bounded + capability_manifest config:
  private per-session embedded daemon launched with
  --capability-manifest/--approve-capability-manifest; missing manifest
  fails loudly. 'unrestricted' is deliberately NOT a config value —
  it stays bound to the explicit per-session YOLO toggle.
- Skill + system-prompt + docs guidance for the three authorization
  rungs and the isolated-profile-first default.

E2E-verified against a temp HERMES_HOME: real config resolution to
bounded, loud failure without a manifest, real argparse path driving a
fake cua-driver binary, standard default preserved.
This commit is contained in:
Teknium
2026-08-14 13:50:20 -07:00
parent fe0a56ed16
commit 48dd9c87cf
11 changed files with 664 additions and 45 deletions
+6 -3
View File
@@ -638,9 +638,12 @@ def computer_use_guidance(platform_name: Optional[str] = None) -> str:
"browser chrome, OS permission prompts, native dialogs, and unsupported "
"targets. Browser setup is a separately approved action; attaching an "
"existing profile is enforced by cua-driver's immutable permission "
"mode: standard requires a certified protected host and fails closed "
"when Hermes has none; explicit Hermes YOLO uses a private unrestricted "
"daemon after the user's launch/session risk acceptance.\n\n"
"mode: in standard mode ask the USER to run `hermes computer-use "
"browser-approve` and pass the pasted single-use token as "
"`approval_token` (you can never mint one); bounded mode authorizes "
"via the user's reviewed capability manifest; explicit Hermes YOLO "
"uses a private unrestricted daemon after the user's launch/session "
"risk acceptance.\n\n"
"## Background mode rules\n"
"- Do NOT use `raise_window=true` on `focus_app` unless the user "
"explicitly asked you to bring a window to front. Input routing to "
+16
View File
@@ -3274,6 +3274,22 @@ DEFAULT_CONFIG = {
# True = always disable the overlay
# False = always enable the overlay
"no_overlay": None,
# cua-driver permission mode for this Hermes install.
# standard (default) — cua-driver's own approval boundary. Protected
# operations (e.g. attaching to an existing signed-in browser
# profile) fail closed unless the user mints a single-use token
# via `hermes computer-use browser-approve`.
# bounded — repeatable automation under a user-reviewed capability
# manifest (set capability_manifest below). No runtime prompts;
# anything outside the manifest fails closed inside cua-driver.
# `unrestricted` is intentionally NOT accepted here: it stays bound to
# the explicit per-session YOLO toggle so a config line can never
# silently bypass approvals.
"permission_mode": "standard",
# Absolute or ~ path to the reviewed cua-driver capability manifest
# YAML used when permission_mode is "bounded". See
# https://cua.ai/docs/reference/cua-driver/permission-modes
"capability_manifest": "",
},
# =========================================================================
+62
View File
@@ -12377,6 +12377,38 @@ def main():
"grant",
help="Request the grants (opens the dialog attributed to CuaDriver)",
)
computer_use_browser_approve = computer_use_sub.add_parser(
"browser-approve",
help="Mint a single-use token authorizing browser attachment for one exact window",
description=(
"Runs `cua-driver browser-approve` to mint a five-minute,\n"
"single-use token that authorizes ONE browser preparation for the\n"
"exact process (and window) you name. Give the printed token to\n"
"the agent; it passes it as approval_token on the\n"
"cua_browser_prepare action.\n\n"
"This is the explicit human boundary for attaching to a browser —\n"
"especially an existing signed-in profile, where the DevTools\n"
"protocol can see that profile's live pages, cookies, and storage.\n"
"Ordinary tool approval never substitutes for this grant, so a\n"
"model can never mint or guess the token itself.\n\n"
"Find the pid/window_id via the agent (list_windows) or ask it to\n"
"read them from a native capture."
),
)
computer_use_browser_approve.add_argument(
"--pid", type=int, required=True,
help="Exact browser process id to authorize",
)
computer_use_browser_approve.add_argument(
"--window-id", type=int, default=None,
help="Exact native window id (required for existing-profile attachment)",
)
computer_use_browser_approve.add_argument(
"--profile-mode",
choices=["isolated_new", "isolated_named", "existing_profile"],
default="isolated_new",
help="Preparation the token authorizes (default: isolated_new)",
)
def cmd_computer_use(args):
action = getattr(args, "computer_use_action", None)
@@ -12433,6 +12465,36 @@ def main():
json_output=bool(getattr(args, "json", False)),
)
sys.exit(code)
if action == "browser-approve":
import subprocess
from tools.computer_use.cua_backend import (
cua_driver_child_env,
cua_driver_install_hint,
resolve_cua_driver_cmd,
)
binary = resolve_cua_driver_cmd()
if not binary:
print(cua_driver_install_hint())
sys.exit(2)
cmd = [binary, "browser-approve", "--pid", str(args.pid)]
window_id = getattr(args, "window_id", None)
if window_id is not None:
cmd += ["--window-id", str(window_id)]
cmd += ["--profile-mode", getattr(args, "profile_mode", "isolated_new")]
try:
# Interactive passthrough: cua-driver requires a TTY to mint
# the grant, prints the token itself, and owns the expiry.
proc = subprocess.run(cmd, env=cua_driver_child_env())
except OSError as exc:
print(f"cua-driver browser-approve failed to launch: {exc}", file=sys.stderr)
sys.exit(2)
if proc.returncode == 0:
print(
"\nGive the token above to the agent — it passes it as "
"approval_token on cua_browser_prepare. Single use, "
"expires in ~5 minutes."
)
sys.exit(proc.returncode)
if action == "permissions":
perms_action = getattr(args, "computer_use_perms_action", None)
if perms_action == "grant":
@@ -180,11 +180,29 @@ browser tools. The contract is capability-based:
`cua_browser_prepare` is a separate approved setup action. Driver-owned
`isolated_new`/`isolated_named` profiles require explicit `allow_launch=true`.
An `existing_profile` is decided by cua-driver's immutable permission mode.
Normal Hermes sessions use `standard`, which requires a certified protected
host and fails closed when Hermes has none. Explicit Hermes YOLO (`--yolo`,
`/yolo`, or `approvals.mode: off`) launches a private embedded cua-driver in
`unrestricted` after that risk acceptance, so there are no runtime Cua
approval prompts. Never invent, store, log, or reuse a grant token.
Prefer `isolated_new` unless the task genuinely needs the user's signed-in
session — attaching to an existing profile exposes its live pages, cookies,
and storage over the browser protocol.
Authorization paths for `existing_profile`, in preference order:
1. **User-minted token (standard mode).** Ask the user to run
`hermes computer-use browser-approve --pid <pid> --window-id <window_id>
--profile-mode existing_profile` in their terminal and paste back the
token. Pass it as `approval_token` on `cua_browser_prepare`. The token is
single-use and expires in ~5 minutes, so mint it right before the call.
You cannot create this token yourself; only the user can.
2. **Bounded manifest.** When `computer_use.permission_mode: bounded` is
configured with a reviewed `capability_manifest`, prepares inside the
manifest's scope succeed without prompts and everything else fails closed.
No token needed; do not ask for one.
3. **Explicit Hermes YOLO** (`--yolo`, `/yolo`, or `approvals.mode: off`)
launches a private embedded cua-driver in `unrestricted` after that risk
acceptance, so there are no runtime Cua approval prompts.
Without any of these, `existing_profile` fails closed — report the refusal and
offer path 1 to the user; do not retry, downgrade trust, or work around it.
Never invent, store, log, or reuse a grant token.
Use the native capture/AX/pixel/foreground ladder for browser chrome, browser
permission UI, OS prompts, native dialogs, extension surfaces, unsupported
@@ -0,0 +1,364 @@
"""Authorization plumbing for the cua-driver typed browser route.
Covers the three rungs that let ``existing_profile`` attachment (and bounded
automation generally) actually work from Hermes:
* ``approval_token`` passthrough — the user-minted single-use token from
``hermes computer-use browser-approve`` reaches ``browser_prepare`` and is
never fabricated by the wrapper.
* ``bounded`` permission mode — a private embedded daemon launched with a
user-reviewed capability manifest (``--capability-manifest`` +
``--approve-capability-manifest``), failing loudly when the manifest is
missing.
* mode resolution — config supplies standard/bounded only; explicit session
YOLO still (and exclusively) selects unrestricted.
"""
from typing import Any, Dict
import pytest
from tools.computer_use import cua_backend as cb
from tools.computer_use.browser_route import CuaTypedBrowserRoute
from tools.computer_use.cua_backend import _EmbeddedCuaDaemon
def _driver_result(payload: Dict[str, Any]) -> Dict[str, Any]:
return {"structuredContent": dict(payload)}
class _PrepareDriver:
def __init__(self) -> None:
self.calls: list[tuple[str, Dict[str, Any]]] = []
def has_tool(self, name: str) -> bool:
return True
def call(self, name: str, args: Dict[str, Any]) -> Dict[str, Any]:
self.calls.append((name, dict(args)))
return _driver_result({"status": "ok"})
def _route(driver: _PrepareDriver) -> CuaTypedBrowserRoute:
return CuaTypedBrowserRoute(
session_id="hermes-a",
call_tool=driver.call,
has_tool=driver.has_tool,
)
# ── approval_token passthrough ──────────────────────────────────────────
def test_existing_profile_prepare_forwards_user_minted_approval_token():
driver = _PrepareDriver()
result = _route(driver).prepare(
pid=101,
window_id=202,
profile_mode="existing_profile",
approval_token="tok-from-user-terminal",
)
assert result["status"] == "ok"
assert driver.calls == [
(
"browser_prepare",
{
"pid": 101,
"window_id": 202,
"strategy": {"kind": "existing_profile"},
"approval_token": "tok-from-user-terminal",
"session": "hermes-a",
},
)
]
def test_existing_profile_prepare_without_token_sends_none():
"""No token → the field is absent; the driver's own gate decides."""
driver = _PrepareDriver()
_route(driver).prepare(pid=101, window_id=202, profile_mode="existing_profile")
(_, args), = driver.calls
assert "approval_token" not in args
@pytest.mark.parametrize("bogus", ["", None, 7, True])
def test_non_string_or_empty_token_is_never_forwarded(bogus):
driver = _PrepareDriver()
_route(driver).prepare(
pid=101,
window_id=202,
profile_mode="existing_profile",
approval_token=bogus,
)
(_, args), = driver.calls
assert "approval_token" not in args
def test_isolated_prepare_ignores_approval_token():
"""The token authorizes existing-profile attachment only."""
driver = _PrepareDriver()
_route(driver).prepare(
pid=101,
profile_mode="isolated_new",
allow_launch=True,
approval_token="tok",
)
(_, args), = driver.calls
assert "approval_token" not in args
def test_dispatch_forwards_approval_token_to_backend():
from unittest.mock import Mock
from tools.computer_use.tool import _dispatch
backend = Mock()
backend.typed_browser_prepare.return_value = {"status": "ok"}
_dispatch(
backend,
"cua_browser_prepare",
{
"pid": 101,
"window_id": 202,
"profile_mode": "existing_profile",
"approval_token": "tok-abc",
},
)
kwargs = backend.typed_browser_prepare.call_args.kwargs
assert kwargs["approval_token"] == "tok-abc"
assert kwargs["profile_mode"] == "existing_profile"
def test_schema_documents_approval_token_as_user_minted():
from tools.computer_use.schema import COMPUTER_USE_SCHEMA
prop = COMPUTER_USE_SCHEMA["parameters"]["properties"]["approval_token"]
desc = prop["description"]
assert "browser-approve" in desc
assert "never invent" in desc.lower()
# ── bounded embedded daemon ─────────────────────────────────────────────
def test_bounded_daemon_requires_a_manifest():
with pytest.raises(ValueError, match="capability_manifest"):
_EmbeddedCuaDaemon("cua-driver", "bounded")
def test_bounded_daemon_requires_manifest_file_to_exist(tmp_path):
with pytest.raises(ValueError, match="not found"):
_EmbeddedCuaDaemon(
"cua-driver", "bounded",
capability_manifest=str(tmp_path / "missing.yaml"),
)
def test_bounded_daemon_env_does_not_bypass_approvals(tmp_path):
manifest = tmp_path / "manifest.yaml"
manifest.write_text("version: 3\n", encoding="utf-8")
daemon = _EmbeddedCuaDaemon(
"cua-driver", "bounded", capability_manifest=str(manifest)
)
env = daemon.child_env()
assert env["CUA_DRIVER_PERMISSION_MODE"] == "bounded"
assert "CUA_DRIVER_DANGEROUSLY_BYPASS_APPROVALS" not in env
def test_unrestricted_daemon_env_keeps_explicit_bypass():
daemon = _EmbeddedCuaDaemon("cua-driver", "unrestricted")
env = daemon.child_env()
assert env["CUA_DRIVER_PERMISSION_MODE"] == "unrestricted"
assert env["CUA_DRIVER_DANGEROUSLY_BYPASS_APPROVALS"] == "1"
def test_bounded_daemon_serves_with_approved_manifest(tmp_path, monkeypatch):
"""The spawn command carries the manifest + launch-time approval flags."""
manifest = tmp_path / "manifest.yaml"
manifest.write_text("version: 3\n", encoding="utf-8")
daemon = _EmbeddedCuaDaemon(
"cua-driver", "bounded", capability_manifest=str(manifest)
)
captured: Dict[str, Any] = {}
class _FakeProc:
stderr = None
def poll(self):
return None
def _fake_popen(command, **kwargs):
captured["command"] = list(command)
return _FakeProc()
def _fake_run(command, **kwargs):
class _Probe:
returncode = 0
return _Probe()
monkeypatch.setattr(cb.subprocess, "Popen", _fake_popen)
monkeypatch.setattr(cb.subprocess, "run", _fake_run)
monkeypatch.setattr(
cb, "_resolve_mcp_invocation", lambda cmd: (cmd, ["mcp"])
)
daemon.start()
command = captured["command"]
assert "--permission-mode" in command
assert command[command.index("--permission-mode") + 1] == "bounded"
assert "--capability-manifest" in command
assert (
command[command.index("--capability-manifest") + 1]
== str(manifest)
)
assert "--approve-capability-manifest" in command
assert "--dangerously-bypass-approvals" not in command
def test_unrestricted_daemon_serve_command_unchanged(monkeypatch):
daemon = _EmbeddedCuaDaemon("cua-driver", "unrestricted")
captured: Dict[str, Any] = {}
class _FakeProc:
stderr = None
def poll(self):
return None
monkeypatch.setattr(
cb.subprocess, "Popen",
lambda command, **kw: captured.update(command=list(command)) or _FakeProc(),
)
def _fake_run(command, **kwargs):
class _Probe:
returncode = 0
return _Probe()
monkeypatch.setattr(cb.subprocess, "run", _fake_run)
monkeypatch.setattr(
cb, "_resolve_mcp_invocation", lambda cmd: (cmd, ["mcp"])
)
daemon.start()
command = captured["command"]
assert "--dangerously-bypass-approvals" in command
assert "--capability-manifest" not in command
# ── permission-mode resolution ──────────────────────────────────────────
def test_configured_mode_defaults_to_standard(monkeypatch):
monkeypatch.setattr(cb, "_computer_use_cfg", dict)
assert cb._cua_configured_permission_mode() == "standard"
def test_configured_mode_honors_bounded(monkeypatch):
monkeypatch.setattr(
cb, "_computer_use_cfg", lambda: {"permission_mode": "Bounded"}
)
assert cb._cua_configured_permission_mode() == "bounded"
@pytest.mark.parametrize("value", ["unrestricted", "yolo", "off", 3, None])
def test_configured_mode_never_yields_unrestricted(monkeypatch, value):
"""A config line must never silently bypass approvals."""
monkeypatch.setattr(
cb, "_computer_use_cfg", lambda: {"permission_mode": value}
)
assert cb._cua_configured_permission_mode() == "standard"
def test_capability_manifest_reads_config(monkeypatch):
monkeypatch.setattr(
cb, "_computer_use_cfg",
lambda: {"capability_manifest": " ~/manifests/cua.yaml "},
)
assert cb._cua_capability_manifest() == "~/manifests/cua.yaml"
monkeypatch.setattr(cb, "_computer_use_cfg", dict)
assert cb._cua_capability_manifest() is None
def test_session_yolo_overrides_configured_bounded(monkeypatch):
import tools.computer_use.tool as cu_tool
monkeypatch.setattr(
cb, "_computer_use_cfg", lambda: {"permission_mode": "bounded"}
)
import tools.approval as approval
monkeypatch.setattr(
approval, "is_approval_bypass_active_for_session", lambda sid: True
)
assert cu_tool._cua_permission_mode("sess-1") == "unrestricted"
def test_no_yolo_uses_configured_bounded(monkeypatch):
import tools.computer_use.tool as cu_tool
monkeypatch.setattr(
cb, "_computer_use_cfg", lambda: {"permission_mode": "bounded"}
)
import tools.approval as approval
monkeypatch.setattr(
approval, "is_approval_bypass_active_for_session", lambda sid: False
)
monkeypatch.setattr(
approval, "get_current_session_key", lambda default="": ""
)
assert cu_tool._cua_permission_mode("sess-1") == "bounded"
def test_no_yolo_no_config_stays_standard(monkeypatch):
import tools.computer_use.tool as cu_tool
monkeypatch.setattr(cb, "_computer_use_cfg", dict)
import tools.approval as approval
monkeypatch.setattr(
approval, "is_approval_bypass_active_for_session", lambda sid: False
)
monkeypatch.setattr(
approval, "get_current_session_key", lambda default="": ""
)
assert cu_tool._cua_permission_mode("sess-1") == "standard"
def test_backend_accepts_bounded_with_manifest(tmp_path, monkeypatch):
manifest = tmp_path / "manifest.yaml"
manifest.write_text("version: 3\n", encoding="utf-8")
monkeypatch.setattr(
cb, "_cua_capability_manifest", lambda: str(manifest)
)
monkeypatch.setattr(cb, "resolve_cua_driver_cmd", lambda override=None: "cua-driver")
backend = cb.CuaDriverBackend(permission_mode="bounded")
assert backend.permission_mode == "bounded"
assert backend._embedded_daemon is not None
assert backend._embedded_daemon.capability_manifest == str(manifest)
def test_backend_bounded_without_manifest_fails_loudly(monkeypatch):
monkeypatch.setattr(cb, "_cua_capability_manifest", lambda: None)
monkeypatch.setattr(cb, "resolve_cua_driver_cmd", lambda override=None: "cua-driver")
with pytest.raises(ValueError, match="capability_manifest"):
cb.CuaDriverBackend(permission_mode="bounded")
def test_backend_rejects_unknown_mode():
with pytest.raises(ValueError, match="unsupported"):
cb.CuaDriverBackend(permission_mode="wide-open")
+1
View File
@@ -761,6 +761,7 @@ def test_namespaced_state_and_prepare_actions_use_typed_backend_wrappers():
profile_mode="isolated_new",
profile_name=None,
allow_launch=True,
approval_token=None,
)
+16 -10
View File
@@ -343,6 +343,7 @@ class CuaTypedBrowserRoute:
profile_mode: str,
profile_name: Optional[str] = None,
allow_launch: bool = False,
approval_token: Optional[str] = None,
) -> Dict[str, Any]:
"""Run explicit setup through the driver's authoritative mode gate."""
missing = self._require_tool("browser_prepare")
@@ -361,17 +362,22 @@ class CuaTypedBrowserRoute:
"Existing-profile attachment requires an exact positive pid and window_id pair.",
)
# The driver owns the immutable standard/bounded/unrestricted
# decision. Standard fails closed without a certified host;
# explicit Hermes YOLO owns a private unrestricted daemon.
# decision. Standard fails closed without a certified host,
# a user-minted single-use approval token, or an approved
# bounded manifest; explicit Hermes YOLO owns a private
# unrestricted daemon.
self.state.clear()
return self._call(
"browser_prepare",
{
"pid": exact_pid,
"window_id": exact_window,
"strategy": {"kind": "existing_profile"},
},
)
args: Dict[str, Any] = {
"pid": exact_pid,
"window_id": exact_window,
"strategy": {"kind": "existing_profile"},
}
if isinstance(approval_token, str) and approval_token:
# Minted out-of-band by `hermes computer-use browser-approve`
# (cua-driver browser-approve): five-minute, single-use. The
# user, never the model, is the source of this value.
args["approval_token"] = approval_token
return self._call("browser_prepare", args)
if profile_mode not in {"isolated_new", "isolated_named"}:
return _refusal(
"browser_profile_mode_invalid",
+89 -15
View File
@@ -240,6 +240,32 @@ def _cua_telemetry_disabled() -> bool:
return not bool(_computer_use_cfg().get("cua_telemetry", False))
def _cua_configured_permission_mode() -> str:
"""The user-configured cua-driver permission mode.
Reads ``computer_use.permission_mode`` (default ``standard``). Only
``standard`` and ``bounded`` are honored here — ``unrestricted`` is
deliberately NOT a config value: it stays tied to the explicit
per-session Hermes YOLO toggle so a stale config line can never
silently bypass approvals. Unknown values fall closed to ``standard``.
"""
raw = str(_computer_use_cfg().get("permission_mode", "standard") or "").strip().lower()
return raw if raw in {"standard", "bounded"} else "standard"
def _cua_capability_manifest() -> Optional[str]:
"""Path of the reviewed capability manifest for bounded mode, or None.
Reads ``computer_use.capability_manifest``. Existence is validated by
``_EmbeddedCuaDaemon`` so a missing file fails loudly at session start
instead of silently degrading the authorization story.
"""
raw = _computer_use_cfg().get("capability_manifest")
if not isinstance(raw, str) or not raw.strip():
return None
return raw.strip()
def _computer_use_max_image_dimension() -> Optional[int]:
"""Longest-edge cap for cua-driver screenshots, or None to leave unset.
@@ -381,19 +407,48 @@ def _wsl_windows_path_to_posix(path: str) -> str:
class _EmbeddedCuaDaemon:
"""Private host-owned daemon used for an explicit unrestricted session.
"""Private host-owned daemon for a non-standard permission mode.
Cua Driver permission mode is immutable after daemon startup. Reusing the
machine-wide daemon would therefore let one Hermes session's YOLO choice
affect another session. A private embedded daemon gives the requesting
session its own socket, process, and launch-time risk acknowledgement.
session its own socket, process, and launch-time authorization:
* ``unrestricted`` — explicit Hermes YOLO; launch-time risk
acknowledgement via ``--dangerously-bypass-approvals``.
* ``bounded`` — a user-reviewed capability manifest
(``computer_use.capability_manifest`` in config.yaml) approved at
launch via ``--approve-capability-manifest``. The manifest, not a
runtime prompt, is the authorization boundary; calls outside it fail
closed inside cua-driver.
"""
_START_TIMEOUT_SECONDS = 15.0
def __init__(self, driver_cmd: str, permission_mode: str) -> None:
if permission_mode != "unrestricted":
raise ValueError("embedded permission override supports unrestricted only")
def __init__(
self,
driver_cmd: str,
permission_mode: str,
capability_manifest: Optional[str] = None,
) -> None:
if permission_mode not in {"unrestricted", "bounded"}:
raise ValueError(
"embedded permission override supports unrestricted or bounded only"
)
if permission_mode == "bounded":
manifest = str(capability_manifest or "").strip()
if not manifest:
raise ValueError(
"bounded permission mode requires computer_use.capability_manifest"
)
manifest = os.path.abspath(os.path.expanduser(manifest))
if not os.path.isfile(manifest):
raise ValueError(
f"capability manifest not found: {manifest}"
)
self.capability_manifest: Optional[str] = manifest
else:
self.capability_manifest = None
self.permission_mode = permission_mode
self._driver_cmd = driver_cmd
self._command = driver_cmd
@@ -411,8 +466,9 @@ class _EmbeddedCuaDaemon:
def child_env(self) -> Dict[str, str]:
env = cua_driver_child_env()
env["CUA_DRIVER_PERMISSION_MODE"] = "unrestricted"
env["CUA_DRIVER_DANGEROUSLY_BYPASS_APPROVALS"] = "1"
env["CUA_DRIVER_PERMISSION_MODE"] = self.permission_mode
if self.permission_mode == "unrestricted":
env["CUA_DRIVER_DANGEROUSLY_BYPASS_APPROVALS"] = "1"
return env
def _drain_stderr(self, process: Any) -> None:
@@ -447,9 +503,18 @@ class _EmbeddedCuaDaemon:
self.socket_path,
"--no-permissions-gate",
"--permission-mode",
"unrestricted",
"--dangerously-bypass-approvals",
self.permission_mode,
]
if self.permission_mode == "unrestricted":
command.append("--dangerously-bypass-approvals")
else: # bounded — manifest validated in __init__
command.extend(
[
"--capability-manifest",
str(self.capability_manifest),
"--approve-capability-manifest",
]
)
self._process = subprocess.Popen(
command,
stdin=subprocess.DEVNULL,
@@ -1917,14 +1982,23 @@ class CuaDriverBackend(ComputerUseBackend):
"""Default computer-use backend. Cross-platform via cua-driver MCP."""
def __init__(self, permission_mode: str = "standard") -> None:
if permission_mode not in {"standard", "unrestricted"}:
if permission_mode not in {"standard", "bounded", "unrestricted"}:
raise ValueError(f"unsupported cua-driver permission mode: {permission_mode}")
self.permission_mode = permission_mode
self._embedded_daemon = (
_EmbeddedCuaDaemon(resolve_cua_driver_cmd() or "", permission_mode)
if permission_mode == "unrestricted"
else None
)
if permission_mode == "unrestricted":
self._embedded_daemon: Optional[_EmbeddedCuaDaemon] = _EmbeddedCuaDaemon(
resolve_cua_driver_cmd() or "", permission_mode
)
elif permission_mode == "bounded":
# Manifest path comes from config.yaml; _EmbeddedCuaDaemon
# validates existence and raises a clear error otherwise.
self._embedded_daemon = _EmbeddedCuaDaemon(
resolve_cua_driver_cmd() or "",
permission_mode,
capability_manifest=_cua_capability_manifest(),
)
else:
self._embedded_daemon = None
self._bridge = _AsyncBridge()
self._session = _CuaDriverSession(self._bridge, self._embedded_daemon)
# Sticky context — updated by capture(), used by action tools.
+15 -2
View File
@@ -291,12 +291,25 @@ COMPUTER_USE_SCHEMA: Dict[str, Any] = {
"enum": ["isolated_new", "isolated_named", "existing_profile"],
"description": (
"Browser preparation mode. existing_profile is decided by "
"cua-driver's immutable permission mode: standard requires a "
"certified protected host; explicit Hermes YOLO uses a private "
"cua-driver's immutable permission mode: in standard mode "
"ask the user to mint a single-use token with `hermes "
"computer-use browser-approve` and pass it as "
"approval_token; bounded mode authorizes via the reviewed "
"capability manifest; explicit Hermes YOLO uses a private "
"unrestricted daemon."
),
},
"profile_name": {"type": "string", "description": "Name for isolated_named setup."},
"approval_token": {
"type": "string",
"description": (
"Single-use existing-profile attachment token the USER "
"minted with `hermes computer-use browser-approve`. Ask "
"the user to run that command and give you the token; "
"never invent one. Only meaningful with "
"profile_mode='existing_profile'."
),
},
"allow_launch": {
"type": "boolean",
"description": "Explicitly allow launch of a driver-owned isolated browser.",
+10 -1
View File
@@ -195,7 +195,15 @@ def _cua_permission_mode(session_id: str) -> str:
except Exception:
# Approval state must fail closed if it cannot be resolved.
pass
return "standard"
try:
# Without YOLO, honor the configured mode (standard | bounded).
# bounded requires computer_use.capability_manifest; the backend
# fails loudly at session start when the manifest is missing.
from tools.computer_use.cua_backend import _cua_configured_permission_mode
return _cua_configured_permission_mode()
except Exception:
return "standard"
def _get_backend(session_id: str = "") -> ComputerUseBackend:
@@ -649,6 +657,7 @@ def _dispatch(backend: ComputerUseBackend, action: str, args: Dict[str, Any]) ->
profile_mode=args.get("profile_mode", "isolated_new"),
profile_name=args.get("profile_name"),
allow_launch=bool(args.get("allow_launch")),
approval_token=args.get("approval_token"),
))
browser_tools = {
@@ -80,23 +80,67 @@ or add `computer_use` to your enabled toolsets in `~/.hermes/config.yaml`.
## Permission modes and logged-in browser profiles
Hermes maps its existing approval UX onto cua-driver 0.10's immutable daemon
Hermes maps its existing approval UX onto cua-driver's immutable daemon
modes. There is no second permission toggle to keep in sync:
| Hermes session | cua-driver mode | Human intervention | `existing_profile` |
|---|---|---|---|
| Manual or smart approvals (default) | `standard` | Normal Hermes approvals; Cua stops at its protected boundary | Refuses unless a certified protected host is available; Hermes does not claim one today |
| Manual or smart approvals (default) | `standard` | Normal Hermes approvals; Cua stops at its protected boundary | Refuses unless you mint a single-use token with `hermes computer-use browser-approve` |
| `computer_use.permission_mode: bounded` + reviewed manifest | private `bounded` daemon | You review and approve the capability manifest once, at launch | Allowed only within the manifest's declared profiles/origins/tools; everything else fails closed |
| `--yolo`, `/yolo`, or `approvals.mode: off` | private `unrestricted` daemon | One explicit Hermes risk acceptance; no runtime Cua prompts | Allowed within Cua's built-in, managed, and user policy ceilings |
The unrestricted daemon is private to that Hermes session. Turning `/yolo`
off, resetting/closing the session, cancellation cleanup, or process exit ends
the Cua session and stops that daemon. It never changes the machine-wide
daemon's mode or grants another Hermes conversation the same authority.
### Attaching to your signed-in browser (`browser-approve`)
The agent can drive a Chrome/Edge window you already have open — including a
signed-in profile — **without restarting the browser, copying the profile, or
touching your tabs**. Because DevTools access exposes that profile's live
pages, cookies, and storage, cua-driver requires an explicit human grant that
ordinary tool approval cannot substitute for. Hermes surfaces it as:
```bash
# The agent will tell you the exact pid/window_id it discovered:
hermes computer-use browser-approve --pid 844 --window-id 10725 --profile-mode existing_profile
```
This mints a five-minute, single-use token bound to that exact process and
window. Paste the token to the agent; it passes it as `approval_token` on the
`cua_browser_prepare` action and cua-driver verifies the binding before
attaching. The grant lives only in driver memory and dies with the session.
Prefer letting the agent use an isolated profile (`isolated_new`, no token
needed beyond the normal tool approval) unless the task genuinely needs your
authenticated session.
### Bounded mode for repeatable automation
For recurring browser automation (cron jobs, scheduled research against an
authenticated app), per-run tokens are impractical. `bounded` mode replaces
prompts with a capability manifest you review once:
```yaml
# config.yaml
computer_use:
permission_mode: bounded
capability_manifest: ~/.hermes/cua-manifest.yaml
```
The manifest names the apps, browser profile kinds, allowed origins, and
typed tools the session may use (see the
[cua-driver permission modes reference](https://cua.ai/docs/reference/cua-driver/permission-modes)
for the format). Hermes launches a private per-session daemon with
`--capability-manifest ... --approve-capability-manifest`; anything outside
the manifest fails closed inside cua-driver. A missing or unreadable manifest
fails loudly at session start rather than silently downgrading. Session YOLO
still overrides bounded for that one session.
The bounded and unrestricted daemons are private to that Hermes session.
Turning `/yolo` off, resetting/closing the session, cancellation cleanup, or
process exit ends the Cua session and stops that daemon. It never changes the
machine-wide daemon's mode or grants another Hermes conversation the same
authority.
`smart` approval remains `standard`: an LLM classification is not protected
human consent. Cua's `bounded` manifest mode is also not inferred from smart
approval or a normal tool confirmation; it needs a separately trusted host
that reviews and launches the exact manifest.
human consent, and it cannot mint a `browser-approve` token or stand in for a
reviewed manifest.
<div class="alert alert--warning">
@@ -320,6 +364,15 @@ real headless Chromium and is the right answer for web-only tasks.
## Configuration
Permission mode and manifest (see
[Permission modes](#permission-modes-and-logged-in-browser-profiles) above):
```yaml
computer_use:
permission_mode: standard # standard (default) | bounded
capability_manifest: "" # manifest path, required for bounded
```
Override the driver binary path (tests / CI / local builds):
```