An unset checkbox's `.value` is "on" per the HTML spec, and the inventory read
the value before the state — so a ticked box and an empty one both came back as
`value: "on"`, and the agent had no way to tell them apart. Anything built on
reading a form back ("is the newsletter box already checked?") was answering
from a coin flip.
The state check now runs first, gated on the input's type rather than on
`checked` being defined — it is defined, as false, on every input including
text fields, which is what made the original ordering look reasonable. ARIA
checkboxes and switches built out of divs read `aria-checked`, which the old
branch would have missed anyway since a div has no `.checked`.
Found by putting the extracted helper under a direct test. It was unreachable
before: the only way to observe it was to run a whole action against a whole
document.
act-in-page.ts had grown to 1,054 lines: one function holding twenty-eight
closures, of which roughly four hundred lines were pure string and geometry
work that never touched the holder, the action, or any page state. None of it
could be tested. `slug`, `affinity`, `coin` and the rest were reachable only by
running a whole action against a whole document and inferring what they must
have done.
Four modules now, at the seams the dependency graph actually has:
types.ts the six shared interfaces, re-exported from act-in-page.ts so
no import site anywhere changes
naming.ts what an element is called — labels, slugs, stems, anchors
visibility.ts whether it is really there and whether it is on screen
identity.ts the re-bind ladder and handle minting
Each is a factory rather than a bag of exports, and that shape is load-bearing
rather than taste. These sources are stringified into the guest page, where
module scope does not exist; separate exports would be separate names for the
bundler to mangle independently of the call sites inside the stringified core.
A factory that stringifies whole has no cross-module reference to break. The
core takes the kits as a parameter for the same reason — it names nothing it
did not receive.
That failure mode deserves spelling out, because it is why this is not a plain
import. The renderer minifies. An imported binding referenced inside the core
would be renamed to something the injected bundle never declares, and it would
break in packaged builds ONLY — green in dev, green under vitest, broken for
users. `actEngineSource()` is now the single supported way to obtain the
source, so a partial injection is not something a caller can express.
The self-containment test moves with it, and gets stronger: it evaluates the
assembled bundle rather than one function, which is what the guest actually
receives. It caught the contract change on the first run.
Bodies moved unchanged; the only edits are closure captures becoming
parameters, and `coin` taking the counter it used to read off the holder. The
engine is 634 lines, all of it orchestration. The 53 existing engine tests pass
untouched, which is the evidence the move preserved behaviour, and 27 new tests
exercise the extracted helpers directly for the first time.
Every drive_preview action answered with the entire inventory — around 120
elements of ref, role, label, and an up-to-eight-rung `:nth-child` selector
chain. On a real app shell that was ~24.5k characters, re-sent after every
click, so a ten-step task paid for ten copies of a page that had barely moved.
Handles are now durable and legible. An element is named after what it is and
what it says — `btn-sign-in`, `inp-email`, `srch-search-projects` — minted once
per page and never reused, with duplicates disambiguated as `btn-edit`,
`btn-edit-1`. Each one remembers a stable attribute, its role, its accessible
name, and the nearest landmark it sits in, so when a framework destroys the
node and builds a new one the handle moves across and the agent is told
`rebound` rather than being handed a removal it has to react to and an addition
it has to re-read. The re-bind ladder is anchortree's (Apache-2.0), minus its
geometry rung, which can never clear the threshold on its own.
Because the handles hold, the first look at a page returns the inventory and
every look after it returns only what moved. `changed` carries the ref and
whichever of label/value/disabled actually shifted — role and selector are
absent by construction, since a change in either would mean the re-bind ladder
was looking at a different element. A delta gives way to a full re-read when
half the page is new, where there is nothing left to reuse.
The selector column is gone with it. It was 74% of the inventory on an
85-element page, nothing downstream ever read it, and a positional chain is
wrong the moment a sibling appears. An `#id` or `[data-testid]` survives when
the page offers one; everything else is addressed by handle.
Legibility is what makes the delta work rather than a nicety. `+ btn-sign-in`
on turn nine reads on its own, where `+ @e42` sends the model back to an
inventory twenty thousand tokens ago.
Measured on an 85-element app shell: 18,693 -> 4,930 characters for a baseline,
and a steady turn that moved two things costs ~200.
Driving someone's browser invisibly is unnerving, and this browser is the one
they are signed into. The pane now draws the field the agent can reach, a box
round what it is touching, a cursor that goes there, and a wipe over text it
just read — one cursor primitive and one mark primitive, in a closed shadow
root so the agent's own inventory cannot see them. Marks carry the same handle
the agent addresses them by, so the word on screen and the word in the
transcript are the same string.
The point is supervision rather than decoration: a person glancing at the pane
can tell what is about to happen to their live session, and stop it.
It also has to cover the waiting. The agent flashes through a click in under a
second and then sits idle for the twenty to a hundred seconds the model spends
deciding what to do next, which is most of the wall clock of any task — so the
surface used to look broken during the part where it was working hardest. A
think stage runs off the $busy edge, sparsely flashing elements from the field
the last action left behind, and rest stops it. It guards itself: started
before there is an overlay or a field, it idles until there is one, so it can
be raised on the turn boundary without knowing whether anything has been
inventoried yet.
read_preview had the same hole from the other side. Reading is the cheapest
thing the agent does — hundredths of a second between two model round trips —
so paging through a document left the pane dark for twenty seconds immediately
after the one moment that showed anything. It draws a top-to-bottom wipe over
the text it took, and that is the one stage allowed to be a wipe: reading is
the only thing the agent does to a page in an order a person could follow.
Both go through preview-nudge, which says a single stage to an overlay the page
already has rather than re-shipping the engine to narrate. On a page the agent
never acted on it is a no-op, which is the honest answer — chrome there would
be a lie about what it did.
Everything respects prefers-reduced-motion.
The in-app browser was a one-way mirror. open_preview put a page in the pane
and read_preview read its text back, but nothing could touch it. A click meant
falling back to the browser_* tools, which drive a separate Chromium the user
cannot see — so "log into this and pull my invoices" happened in a different
browser from the one on screen, with none of the sessions the user is already
signed into.
Four pieces, and they only make sense together:
· an in-page engine that inventories what is interactable and performs the
verb, injected as source because it has to run inside the guest page;
· the preview.act.request bridge from the gateway into the pane;
· drive_preview, for acting: elements, click, type, scroll, press, and the
pane's own back/forward/reload;
· annotate_preview, for marking without acting.
Those last two started as one tool doing two unrelated jobs. Leaving a mark is
not an action — it outlives the turn that drew it — so it gets its own verb,
and the interaction verb gets a name that says what it does.
Gating is the existing surface rule: desktop_ui folds in on session
source: 'desktop', and the bridge refuses to act for a background session, so a
turn running behind the user's back cannot reach into the page they are working
in.
Two details worth a reviewer's attention. Typing assigns through the
prototype's value setter, because React shadows value with its own accessor and
ignores an input event whose value it believes it already wrote — a plain
el.value = … types into a field that snaps back on the next render. And
clicking replays the pointer/mouse pair before activation, because frameworks
bind to mousedown as often as to click.
hermes-seaeye[bot]