fix(vault): make browser_vault_fill work on the default Browser Use backend
On the default backend (browser.backend unset → browser_exec) the vault tools were advertised but could never fill: the CDP supervisor that carries the secret-bearing eval is started only by the built-in browser_* session path, so _eval_js_secret failed closed with supervisor_required and the origin pre-check fell back to an agent-browser CLI eval against a browser browser_exec never touched. - browser_exec now attaches SUPERVISOR_REGISTRY to the CDP endpoint it just routed the harness to (BU_CDP_WS/BU_CDP_URL), so the fill talks to the SAME browser over the same secret-capable WebSocket. BU direct-cloud (BU_AUTOSPAWN) exposes no endpoint and keeps the supervisor_required refusal. - CDPSupervisor.focus_page(origin, accept=<js>) (used by browser_vault_fill, next commits) re-attaches the page session to the open tab on the item's origin whose DOM holds the form being filled (browser_exec opens its own tabs; the supervisor's initial attach picks the first page target, which is chrome://new-tab-page). browser_vault_fill uses it before the origin pre-check with a per-kind probe (password input / card fields / address fields). Live: evals/vault_fill_live_e2e.py drives the real browser_exec tool against Hermes' packaged Chromium with the login page in the third tab; A/B with the attach line disabled fails at "did not attach a supervisor", enabled fills the password into the /login tab and card fields into the /checkout tab with every model-facing read scrubbed. Also: browser_vault_list/fill described the workflow as "type the identifier with fill_input", a helper that exists only inside browser_exec code (toolset browser-use) and is a ghost on the built-in stack. model_tools._rewrite_browser_vault substitutes the concrete name from the session's actual tool set (`fill_input` inside browser_exec, or browser_type), the same dynamic cross-reference pattern browser_navigate uses for web_search.
This commit is contained in:
@@ -257,6 +257,53 @@ class CDPSupervisor(DialogSupervisionMixin, FrameTrackingMixin):
|
||||
value = result_obj.get("description") or result_obj.get("unserializableValue")
|
||||
return {"ok": True, "result": value, "result_type": result_type}
|
||||
|
||||
def focus_page(self, origin: str, *, accept: Optional[str] = None, timeout: float = 10.0) -> Dict[str, Any]:
|
||||
"""Re-attach the supervisor's page session to an open page target on ``origin``
|
||||
(``scheme://host[:port]``). The initial attach picks the FIRST page target, but tools
|
||||
that open their own tabs (browser_exec) put the login form somewhere else. With
|
||||
``accept`` (a JS expression) the first same-origin tab where it evaluates truthy wins,
|
||||
so a login and a checkout tab on one site resolve to the right one. Returns
|
||||
``{"ok": True, "url"}`` or ``{"ok": False, "error"}``; on failure the previous session stays."""
|
||||
loop = self._loop
|
||||
if loop is None or not loop.is_running():
|
||||
return _fail("supervisor loop is not running")
|
||||
|
||||
async def _attach(target_id: str) -> str:
|
||||
attach = await self._cdp("Target.attachToTarget", {"targetId": target_id, "flatten": True}, timeout=timeout)
|
||||
sid = attach["result"]["sessionId"]
|
||||
await self._enable_page_domains(sid, timeout=timeout)
|
||||
await self._install_dialog_bridge(sid)
|
||||
return sid
|
||||
|
||||
async def _focus() -> Dict[str, Any]:
|
||||
from agent.vault_store import normalize_origin
|
||||
targets = (await self._cdp("Target.getTargets", timeout=timeout)).get("result", {}).get("targetInfos", [])
|
||||
candidates = []
|
||||
for t in targets:
|
||||
url = str(t.get("url") or "")
|
||||
try:
|
||||
if t.get("type") == "page" and normalize_origin(url) == origin:
|
||||
candidates.append((t["targetId"], url))
|
||||
except Exception:
|
||||
continue
|
||||
for target_id, url in candidates:
|
||||
sid = await _attach(target_id)
|
||||
if accept:
|
||||
probe = await self._cdp("Runtime.evaluate", {"expression": accept, "returnByValue": True},
|
||||
session_id=sid, timeout=timeout)
|
||||
if not probe.get("result", {}).get("result", {}).get("value"):
|
||||
await self._cdp("Target.detachFromTarget", {"sessionId": sid}, timeout=timeout)
|
||||
continue
|
||||
with self._state_lock:
|
||||
self._page_session_id = sid
|
||||
return {"ok": True, "url": url}
|
||||
return _fail(f"no open page on {origin}" + (" with the expected form" if accept and candidates else ""))
|
||||
|
||||
try:
|
||||
return _schedule(_focus(), loop, timeout=timeout + 1)
|
||||
except Exception as exc:
|
||||
return _err(exc)
|
||||
|
||||
# ── Supervisor loop internals ────────────────────────────────────────────
|
||||
|
||||
def _thread_main(self) -> None:
|
||||
|
||||
Reference in New Issue
Block a user