Teknium
9d6d772837
feat(computer_use): follow cua-driver's verify → escalate ladder ( #67123 )
...
Hermes' computer_use wrapper dropped cua-driver's structured action verdicts,
exposed no delivery_mode, and injected background-only guidance — so the agent
reported unverified no-ops as success and concluded cua-driver 'cannot drive'
Electron/Chromium surfaces (observed live on tldraw offline). Fixes #67052 .
Phase A — preserve the result contract:
- ActionResult carries verified/effect/escalation/path/degraded/code/delivery_mode
- CuaDriverBackend._action() reads structuredContent (was data-only); a helper
normalizes it, additive and None-safe on old drivers
- _text_response surfaces the fields additively (ok stays transport-only)
Phase B — bounded, model-reachable foreground:
- delivery_mode (background|foreground) + bring_to_front on the schema, dispatcher,
ABC, and all input methods
- foreground is capability-gated (input.delivery_mode); old drivers get a
structured foreground_unsupported refusal, never a silent background downgrade
- no automatic/hidden foreground retry — the model selects it from the signal
Phase C — guidance + isolation:
- system prompt (prompt_builder) and bundled skills/computer-use/SKILL.md go from
background-ONLY to background-FIRST, teaching the AX→PX→foreground ladder driven
by returned effect/escalation, not predicted from the app being Electron
- foreground approval scoped by (action, delivery_mode): a background approval
never silently authorizes foreground
- approval state keyed per session_id so concurrent gateway runs don't leak unlocks
Tests: tests/tools/test_computer_use_delivery_ladder.py (15) cover confirmed/
unverifiable/suspected_noop/degraded/old-driver verdicts, delivery_mode gating +
foreground_unsupported, and session-scoped foreground approval. Existing 265
computer_use tests still green.
Live E2E (real cua-driver 0.8.3 + tldraw offline on Linux/X11): a background click
returned effect='unverifiable'/path='ax' (no fabricated success), and a foreground
request returned code='foreground_unsupported' — correct on a driver that predates
the input.delivery_mode capability.
2026-07-18 13:59:35 -07:00
..
2026-05-22 03:33:01 -07:00
2026-05-28 22:26:25 -07:00
2026-07-17 04:53:38 -07:00
2026-07-05 14:48:40 -07:00
2026-05-28 22:26:25 -07:00
2026-07-12 22:43:57 -07:00
2026-07-13 04:31:55 -07:00
2026-07-17 04:53:04 -07:00
2026-07-06 04:58:42 -07:00
2026-07-14 23:36:37 -07:00
2026-07-16 01:28:24 -07:00
2026-06-11 10:49:47 -07:00
2026-06-29 01:26:24 -07:00
2026-06-29 01:26:24 -07:00
2026-06-29 01:26:24 -07:00
2026-07-03 03:27:47 -07:00
2026-05-28 22:26:25 -07:00
2026-06-29 01:26:24 -07:00
2026-06-25 22:02:22 -07:00
2026-07-01 05:09:35 -07:00
2026-07-03 03:27:47 -07:00
2026-06-28 12:25:15 -05:00
2026-06-28 13:49:08 -07:00
2026-05-28 22:26:25 -07:00
2026-05-09 13:35:39 -07:00
2026-06-29 02:24:57 -07:00
2026-07-01 05:23:17 -07:00
2026-07-16 23:41:26 -07:00
2026-07-16 23:41:26 -07:00
2026-07-02 13:10:30 +05:30
2026-05-30 07:31:25 -07:00
2026-06-28 14:25:10 -07:00
2026-07-16 23:41:26 -07:00
2026-07-18 10:07:46 -07:00
2026-06-28 13:49:08 -07:00
2026-07-01 05:04:41 -07:00
2026-06-28 12:14:21 -05:00
2026-06-21 15:23:47 -07:00
2026-07-01 20:01:55 +03:00
2026-07-01 05:04:41 -07:00
2026-06-28 02:42:01 -07:00
2026-06-21 07:26:18 -07:00
2026-05-28 22:26:25 -07:00
2026-06-25 22:02:22 -07:00
2026-06-21 17:46:38 +05:30
2026-05-28 22:26:25 -07:00
2026-07-17 20:55:24 +00:00
2026-06-19 06:31:08 -07:00
2026-05-28 22:26:25 -07:00
2026-07-16 04:24:07 -07:00
2026-05-29 03:44:49 -07:00
2026-07-16 04:25:19 -07:00
2026-06-18 12:48:36 +10:00
2026-06-22 06:42:30 -07:00
2026-07-18 13:59:35 -07:00
2026-07-04 02:37:33 -07:00
2026-06-07 02:09:20 -07:00
2026-07-16 00:34:20 +05:30
2026-07-14 02:59:39 +05:30
2026-07-16 00:18:38 -07:00
2026-06-27 04:57:15 +05:30
2026-06-30 17:29:03 -07:00
2026-07-01 00:13:36 -07:00
2026-06-26 01:11:11 -07:00
2026-06-21 13:28:04 +05:30
2026-07-01 14:23:01 +05:30
2026-05-28 22:26:25 -07:00
2026-07-02 04:20:43 -07:00
2026-06-28 14:33:03 -07:00
2026-05-28 22:26:25 -07:00
2026-06-16 14:30:11 -04:00
2026-06-30 03:07:40 -07:00
2026-06-30 03:16:22 -07:00
2026-07-18 01:30:17 -07:00
2026-07-09 15:38:32 +05:30
2026-07-07 02:40:04 -07:00
2026-06-29 11:01:08 +10:00
2026-06-24 15:23:23 +10:00
2026-07-17 20:55:24 +00:00
2026-07-05 14:41:51 -07:00
2026-05-29 11:49:54 +10:00
2026-07-16 08:52:47 +10:00
2026-06-26 19:15:18 -07:00
2026-06-18 09:09:21 +10:00
2026-06-10 19:23:06 +10:00
2026-07-01 14:37:22 +05:30
2026-05-28 22:26:09 -07:00
2026-07-13 04:31:55 -07:00
2026-07-17 20:55:24 +00:00
2026-05-30 07:01:22 -07:00
2026-07-13 01:59:58 -04:00
2026-05-28 22:26:25 -07:00
2026-07-05 14:42:05 -07:00
2026-07-16 00:18:38 -07:00
2026-05-28 22:26:25 -07:00
2026-07-06 01:55:58 -07:00
2026-07-01 01:00:31 -07:00
2026-06-15 14:00:42 +05:30
2026-07-16 00:18:38 -07:00
2026-05-28 22:26:25 -07:00
2026-07-16 00:18:38 -07:00
2026-07-16 00:18:38 -07:00
2026-07-14 17:09:40 +05:30
2026-07-16 22:46:46 -04:00
2026-07-01 17:48:32 +05:30
2026-06-01 16:55:04 -07:00
2026-06-30 17:32:28 -07:00
2026-07-05 02:24:00 -07:00
2026-05-28 22:26:25 -07:00
2026-07-01 14:37:22 +05:30
2026-05-28 22:26:25 -07:00
2026-05-21 16:40:04 +05:30
2026-07-05 13:51:26 -07:00
2026-06-18 22:13:07 -07:00
2026-05-15 01:33:13 -07:00
2026-06-19 16:59:54 -07:00
2026-07-14 02:59:39 +05:30
2026-06-25 12:38:33 -07:00
2026-07-04 15:30:50 -07:00
2026-07-17 20:55:24 +00:00
2026-06-21 12:02:30 -07:00
2026-07-17 15:47:39 -07:00
2026-06-25 09:20:13 +10:00
2026-07-16 12:48:48 -07:00
2026-05-28 22:26:25 -07:00
2026-06-25 13:52:06 -07:00
2026-07-17 20:55:24 +00:00
2026-07-17 06:54:51 -07:00
2026-07-02 02:13:30 -07:00
2026-07-16 01:28:24 -07:00
2026-07-16 01:28:13 -07:00
2026-07-01 15:42:19 +05:30
2026-07-16 01:28:24 -07:00
2026-07-17 20:55:24 +00:00
2026-06-11 19:50:08 -05:00
2026-06-07 03:57:21 -07:00
2026-07-03 05:30:16 -05:00
2026-05-28 22:26:25 -07:00
2026-05-30 07:58:08 -07:00
2026-07-17 20:55:24 +00:00
2026-06-21 16:02:56 +05:30
2026-07-17 20:55:24 +00:00
2026-05-28 22:26:25 -07:00
2026-07-17 04:50:47 -07:00
2026-07-05 13:40:21 -07:00
2026-06-19 11:46:25 -07:00
2026-05-28 22:26:25 -07:00
2026-05-28 22:26:25 -07:00
2026-05-16 13:06:56 -07:00
2026-07-17 04:57:12 -07:00
2026-06-11 04:37:01 -07:00
2026-06-15 04:22:07 -07:00
2026-07-17 04:50:47 -07:00
2026-07-17 04:50:47 -07:00
2026-07-17 20:55:24 +00:00
2026-07-05 21:36:19 -07:00
2026-06-14 02:09:45 -07:00
2026-07-17 20:55:24 +00:00
2026-07-05 21:48:36 -07:00
2026-07-14 13:59:38 -07:00
2026-07-05 02:06:39 -07:00
2026-05-07 07:08:04 -07:00
2026-07-17 20:55:24 +00:00
2026-07-07 15:16:00 -07:00
2026-07-18 04:15:13 -07:00
2026-05-15 14:00:01 -07:00
2026-07-07 15:16:00 -07:00
2026-07-05 21:48:36 -07:00
2026-07-17 20:55:24 +00:00
2026-05-28 22:26:25 -07:00
2026-07-09 15:38:32 +05:30
2026-06-18 10:19:33 -07:00
2026-07-07 02:10:43 -07:00
2026-05-08 09:27:26 -07:00
2026-05-08 09:27:26 -07:00
2026-05-28 22:26:25 -07:00
2026-07-09 20:03:11 -07:00
2026-06-21 11:11:23 -07:00
2026-06-06 18:30:39 -07:00
2026-05-25 15:18:45 -07:00
2026-07-13 02:42:18 -07:00
2026-05-25 06:13:36 -07:00
2026-07-17 20:55:24 +00:00
2026-06-13 14:42:51 -07:00
2026-06-17 13:45:25 -07:00
2026-06-19 11:57:43 -07:00
2026-07-17 20:55:24 +00:00
2026-07-07 15:14:30 -07:00
2026-07-16 00:18:38 -07:00
2026-07-14 22:14:10 -07:00
2026-07-07 02:52:17 -07:00
2026-06-13 14:35:21 -07:00
2026-06-20 23:23:47 -07:00
2026-05-28 22:26:25 -07:00
2026-06-20 10:26:45 -07:00
2026-06-12 01:07:38 -07:00
2026-06-15 05:51:47 -07:00
2026-05-18 22:19:50 -07:00
2026-07-09 15:38:32 +05:30
2026-07-16 00:18:38 -07:00
2026-06-27 04:41:22 -07:00
2026-06-07 23:04:54 -07:00
2026-07-01 01:39:10 -07:00
2026-06-20 10:26:45 -07:00
2026-05-17 02:29:41 -07:00
2026-07-12 02:59:27 -07:00
2026-05-28 22:26:25 -07:00
2026-05-28 22:26:25 -07:00
2026-07-18 20:17:22 +05:30
2026-05-28 22:26:25 -07:00
2026-05-28 22:26:25 -07:00
2026-06-25 22:03:02 -07:00
2026-06-06 18:29:52 -07:00
2026-05-28 22:26:25 -07:00
2026-07-05 13:42:46 -07:00
2026-05-19 14:17:38 -07:00
2026-06-11 18:03:11 -04:00
2026-07-05 22:21:05 -07:00
2026-06-18 12:26:20 +05:30
2026-06-27 21:07:53 -07:00
2026-07-09 15:38:14 +05:30
2026-07-07 05:14:00 -07:00
2026-07-17 04:52:02 -07:00
2026-06-21 16:39:48 -07:00
2026-06-20 23:23:47 -07:00
2026-06-04 18:06:21 -07:00
2026-06-26 12:09:52 +10:00
2026-06-26 12:09:52 +10:00
2026-06-08 22:46:57 -07:00
2026-05-28 22:26:25 -07:00
2026-05-28 22:26:25 -07:00
2026-07-09 15:38:32 +05:30
2026-05-28 22:26:25 -07:00
2026-06-22 15:41:23 +10:00
2026-07-16 08:55:10 -07:00
2026-05-28 22:26:25 -07:00
2026-07-14 22:00:28 -07:00
2026-05-28 22:26:25 -07:00
2026-07-16 00:18:38 -07:00
2026-06-27 19:29:00 -07:00
2026-07-04 14:17:11 +05:30
2026-07-01 01:05:28 -07:00
2026-06-26 15:26:08 +05:30
2026-06-25 23:42:42 +05:30
2026-06-07 18:06:27 -07:00
2026-05-29 13:35:39 +10:00
2026-05-31 00:50:19 -07:00
2026-06-30 16:39:41 -07:00
2026-06-03 06:36:46 -07:00
2026-05-28 22:26:25 -07:00
2026-07-14 02:59:39 +05:30
2026-05-31 17:46:56 -05:00
2026-05-25 01:41:19 -07:00
2026-05-31 17:46:56 -05:00
2026-05-28 22:26:25 -07:00
2026-07-14 02:59:39 +05:30
2026-07-14 02:59:39 +05:30
2026-07-14 02:59:39 +05:30
2026-07-13 22:26:19 -07:00
2026-05-13 17:32:22 -07:00
2026-06-10 02:57:39 -07:00
2026-07-14 02:59:39 +05:30
2026-05-18 22:29:45 -07:00
2026-05-25 05:15:55 -07:00
2026-07-14 02:59:39 +05:30
2026-05-24 18:04:54 -07:00
2026-07-03 05:30:16 -05:00
2026-06-19 07:26:56 -07:00
2026-07-07 02:39:36 -07:00
2026-07-05 00:47:54 -07:00
2026-06-29 21:11:58 -07:00
2026-07-14 02:59:39 +05:30
2026-06-29 21:11:58 -07:00
2026-07-04 15:30:50 -07:00
2026-07-05 00:47:54 -07:00
2026-07-17 20:55:24 +00:00
2026-07-17 20:55:24 +00:00
2026-05-28 22:26:25 -07:00
2026-06-30 00:19:49 -07:00
2026-06-04 18:04:47 -07:00
2026-06-25 01:45:06 +05:30
2026-06-14 16:47:57 -07:00
2026-07-17 11:37:25 +05:30
2026-07-06 04:38:17 -07:00
2026-07-09 20:28:44 -07:00
2026-07-10 19:14:06 +05:30
2026-06-29 10:00:49 -07:00
2026-06-29 10:00:49 -07:00
2026-07-01 00:49:38 -07:00
2026-07-09 15:38:32 +05:30
2026-06-27 03:08:41 +05:30
2026-06-27 15:59:00 -07:00
2026-07-08 06:17:56 -07:00
2026-06-14 02:07:32 -07:00
2026-07-07 22:14:33 -07:00
2026-05-21 02:38:45 +05:30
2026-06-29 21:11:58 -07:00
2026-05-25 03:35:33 -07:00
2026-06-21 11:35:09 -07:00