A dmg user can also update from the terminal (hermes update) or by
re-running the install one-liner; the dmg arm only routed the two
app-button methods, leaving three declared cells as permanent TODOs.
Port the POSIX driver's method blocks into the macos driver's update
phase (hermes-update with the --yes probe, installer re-run with per-ref
flag probing, the +desktop built-app assert) and open the workflow gate.
The dmg bootstrap driver also learns to recover instead of waiting out
its bound when a stage fails: the bootstrap parks on an error screen
with a Retry button (seen live: HTTP 429 downloading install.sh under
full-matrix runner load), so the driver watches the bootstrap log for
real failure shapes, requires two consecutive error probes before
retargeting the click at Retry's measured position, unlatches when the
log goes healthy, and gives up with the true cause after three retries.
Driver rule learned three times in this suite (lsof +D, git show and
find piped to grep -q): under set -euo pipefail, never feed grep -q
from a pipe... grep exits at first match, the producer takes SIGPIPE,
and a TRUE condition reads as failure. Capture to a variable or test
paths directly.
Verified end to end: run 33506177130, macos slice 13/13 green.
Hermes-Setup is a Tauri app that boots to a setup-choice screen and waits
for a click on Install Hermes before any install work starts; run bare it
blocked until the 120-minute job cap (both dmg legs, every run). Launch it
in the background with the driver's env, read the window geometry via
System Events (position and size need no assistive grant), post a real
CGEvent click with cliclick at the button's measured position (65% of
window height; System Events' own click needs assistive access the runners
deny), then wait for the full install to land: checkout, venv console
script, AND the built Hermes.app, since the cleanup trap would otherwise
kill the installer before its desktop-build stage. Bounded at 45 minutes
with desktop screenshots on every phase and failure. Adds a macos-desktop
dispatch route so this arm iterates without the full matrix.
Verified end to end: run 33407401698, both dmg legs green (first ever),
macos slice 10/10.
Fires only on the failure path, at most every fifth iteration: what wins elementsFromPoint at the button's center, cluster and bar rects with computed z-index, the titlebar CSS vars, and window size with dpr. Makes a CI-only click interception attributable from the log alone.
On CI runners the rebuilt app cannot self-relaunch (chrome-sandbox ownership), so it parks on a reopen-to-finish overlay and never exits; a bare app.close() waits on it forever. Close with a bounded ladder instead (graceful, SIGTERM, SIGKILL), sweep the app's descendant processes so the spawned backend cannot keep writing into the install dir, then relaunch from the same captured spec and require the updated app to present a live UI. The driver gets explicit exits plus an unref'd self-deadline so it cannot outlive its own test, and the posix driver quiesces the install dir (processes whose cwd is inside it) before the head desktop smoke because the in-app update's npm is detached from the Electron process tree.
The boot-time auto-check can fail transiently and latch the error UI even though a fresh check succeeds. Click Check now whenever it is clickable, re-test Update now, 3 minute ceiling. On final failure, pull the update status over the same IPC the About panel uses so the log carries the real git error instead of the UI's generic one.
The app ships a 90% UI-zoom default, so a fresh install renders at devicePixelRatio 0.9 and Playwright's input coordinates land ~10% off target on the CI runners: clicks aimed at the titlebar settings gear hit the bar beside it, which read as a phantom overlay interception and failed every app-update leg. Set zoom to 100% through webContents after boot, retrying until dpr reads 1 because the boot path re-applies the default asynchronously.
The overlay mounts in phases (boot-progress card, then the provider picker) and the settings gear reports visible while sitting under it, so a one-shot dismiss probe and a visibility gate both race it. Alternate short-timeout dismiss clicks with settings clicks until a settings click lands; a landed click is proof the overlay is gone. Also pick the window that renders the app UI instead of trusting firstWindow(), which can return a helper webContents.
The get-url shim self-check referenced REPO_DIR, which is never defined
(the script defines REPO_ROOT). Under set -u the stage phase dies
immediately, so every macos desktop-installer@latest leg fails before
install. Introduced in 368164d7 with the shim itself.
Verified: bash -n clean; shim + self-check block replayed standalone
with the fix and passes.
ts_prefix captured its own start per pipe, so each log's [+MM:SS]
was relative to that log's creation - the install log started at
+0:00, the app-update log at +0:00 too, and no single offset could
align all files with the recording. The drivers now stamp TS_BASE=
$SECONDS once at start and ts_prefix stamps every line relative to
it (falling back to its own start when unset): all logs in a leg
share the driver's clock, and playback.html's one offset slider
(recording start vs driver start) aligns every file at once. The ps1
twin was already driver-relative (TsPrefixStart is captured at
dot-source time, near the driver's top).
Verified: two pipes in one shell - the second starts at [+00:03],
continuing the driver clock instead of resetting to [+00:00].
The merged view interleaves every *.log in time order across the
whole leg, filename-prefixed per line (green .file span), and syncs
like any other tab: the video highlights the right file's line at the
right moment, clicking a line seeks the video. Untimed lines inherit
their file's previous timestamp so intra-file order survives the
sort; the merged tab is first and the default.
Verified in a real browser with an interleaved synthetic zip (two
logs, overlapping timelines): time-order merge, prefixes, follow-sync
at t=22 highlighting the app-update.log line, click-to-seek to t=30.
Windows transcripts were ZERO bytes: ts-prefix.ps1 formatted with
{0:D2}, but Floor() returns a double and the D specifier is
integer-only - it threw per line, and under the driver's relaxed EAP
every line errored into the void. {0:00} fixes it (custom numeric
format works on doubles). Reproduced the exact pipeline locally
(empty file + Format specifier invalid), verified the fix produces
prefixed merged stdout+stderr with exit code intact. That is also
why the log timeline never auto-synced: there was nothing in the
files to sync.
The GitHub artifact URL 307s to /suites/... server-side and strips
the ?zip= query param. The player now reads the zip URL from a
#zip= HASH param (client-side, survives the redirect) with ?zip=
as fallback; the hash path was verified in a real browser against a
real leg zip (auto-fetch + boot).
Per ethie's design, one player artifact for the whole run: new
leg-player job uploads playback.html (archive:false) before the
matrix legs, the report job needs it, and each ran cell gets TWO
links - 📼 to the player with #zip=<that leg's logs zip> and ⬇️ to
the raw zip. Per-leg player uploads removed from all three run
workflows.
Each leg uploads playback.html as a single-file artifact (archive:
false) before the driver runs, so it exists even on failure. The
results chart now links every leg that RAN (pass or fail, not skip)
to its player with ?zip= pointing at that leg's logs artifact.
Leg<->artifact mapping: the generator mints a leg_id per matrix entry
(sanitized matrix name, exported legId()), every run workflow names
its artifacts install-e2e-{player,logs}-<leg-id>, and the report job
feeds the run's artifact name->id list to the results renderer, which
rebuilds the leg id from the parsed job name. GitHub does not link
jobs to artifacts, so the deterministic name is the join key.
Empirical finding: GitHub artifact downloads are auth-gated (the
download URL 307s to /suites/... which is 404 anonymous), so a
locally-opened player page cannot fetch the zip cross-origin. The
player now degrades gracefully: ?zip= fetch failure renders a real
download link for the zip (a normal click carries the user's session)
plus a drag-and-drop / file-picker path, and no-param opens as a pure
drop target. Verified in a real browser against a real artifact URL.
Verified: generator emits leg_id, results renderer emits
✅/❌ [📼](...?zip=...) only on ran cells, npm run check PASS,
install tests 36/36, strict tsc PASS, actionlint x4 PASS.
A static single-file player (tests/install/e2e-assets/playback.html):
?zip=<artifact zip url> unzips in-browser (JSZip), plays the screen
recording with a timer pinned top-left, and renders every *.log with
video<->log sync: the video follows the driver's transcript, clicking
a log line seeks the video. A sync-offset slider aligns the recording
start (ffmpeg comes up first) with the driver's relative clock.
Sync axis: drivers now prefix every transcript line with [+MM:SS]
relative to driver start (ts-prefix.sh / ts-prefix.ps1, pipe-safe
under pipefail / relaxed EAP). Browsers cannot play Matroska, so each
leg remuxes recording.mkv -> recording.mp4 (-c copy, no re-encode)
before the artifact upload, on all three OSes.
Verified end-to-end in a real browser against a generated artifact
zip: zip load, mp4 playback, timer, tab switching, follow-sync at
t=6/t=12, click-to-seek, autoplay policy (expected NotAllowedError on
synthetic play; real clicks fine).
Also fixes the shim fail message's dead variable ( ->
observed_git_url) in both posix drivers.
The app-update legs died on the onboarding overlay - a fullscreen div
that intercepts every click (the Settings click timed out under it).
The old plan seeded a fake provider key, which lies: the overlay
vanishes but the app is broken. Instead the driver now runs the
desktop E2E suite's own mock inference server
(tests-js/scripts/mock-server.ts, zero deps, bare-node type
stripping >=22.18) and configures it into HERMES_HOME byte-for-byte
like the dev:mock flow: config.yaml provider + MOCK_API_KEY env. The
app boots genuinely configured - no overlay, real chat surface.
e2e-assets/mock-provider.{sh,mjs} own start/stop (pid + url files;
the wrapper lives until SIGTERM - gating on stdin-close made the
server die instantly, a background process's stdin is already EOF)
and the config write. Wired into the posix script driver's
hermes-desktop-app-update arm and the macos driver's update phase
(both app-update methods). The Playwright flow keeps its defense-in-
depth: the real escape hatch ('I'll choose a provider later') and the
verified 'Open settings' selector.
Probed locally: models + streamed/non-streamed completions answer,
server stops cleanly on kill.
The stage shim (git.bat, lying to the product about origin's URL) sits
on PATH, so Invoke-Git's bare 'git' routed the driver's own plumbing
through cmd - whose parser eats unquoted carets. PowerShell only quotes
args containing whitespace, so rev-parse 'v2026.8.3^{commit}' reached
the bat as v2026.8.3{commit}: bad revision.
The shim must stay a .bat: its audience is the product's python callers
(fork detection's remote get-url), which resolve via PATHEXT and never
see a .ps1. So the split is by audience - Invoke-Git pins the resolved
git.exe for every driver call; the shim serves the product, whose
shimmed flows use no caret revs (documented as the accepted hole, with
a loud bad-revision failure if that ever changes). The shim self-check
now probes through PATH, since Invoke-Git deliberately bypasses it.
macos gains the desktop-installer@latest install method: the website's
Hermes-Setup.dmg (verified live), mounted with hdiutil and its app
binary run DIRECTLY - an open-launched app inherits none of the git
redirect env, so direct exec is what keeps the isolation honest while
staying the same binary and first-launch flow.
install-e2e-macos-run.yml takes the windows shape: one workflow, one
inner job per driver arm, native skips. Arm 1 delegates script installs
to the shared OS-agnostic run workflow; arm 2 stages, installs from the
dmg, and drives both app-update methods through launch-from-spec.mjs -
open-app-update launches the installed .app (the double-click surface,
env via Playwright), hermes-desktop-app-update captures the product's
own hermes desktop spawn. Both end on sha asserts, never version
strings.
windows-desktop-gui-e2e.ps1 and windows-installer-script-e2e.ps1 fold
into tests/install/windows-e2e.ps1 with orthogonal -InstallMethod and
-Route axes: the install phase dispatches on one, the update phase on
the other, and shared workroot state carries how OLD landed - so any
implemented update method can follow any implemented install method.
Implementing a new pair is now a driver function plus a gate edit,
never a new job.
The run workflow collapses to ONE inner job whose if: is the
implemented-pairs table. Newly cheap pairs go live with the merge:
desktop-installer@latest -> hermes-update / installer-script /
installer-script+desktop / hermes-desktop-app-update
installer-script(+desktop) -> hermes-desktop-app-update
installer-script+desktop -> open-app-update (the -IncludeDesktop
install registers real Start Menu / Desktop shortcuts)
Only desktop-installer@latest as an UPDATE method stays a declared
TODO. scripts/windows_e2e_harness.ps1 executes the parse/parameter/
dispatch checks under pwsh before any Windows runner spins up.
git show | grep -qF exits at grep's first match; install.sh is ~140KB
with the flag strings in the first few KB, so git show takes SIGPIPE
on its next write and the pipeline reports 141 under set -o pipefail.
The probe answered NO for flags the ref HAS - timing-dependent, green
without pipefail (every local check), red on the runner.
It hid while a probe miss just meant omitting --skip-browser; the
first probe where NO is a hard failure (--include-desktop) exposed it
on its first CI leg. Buffer git show into a variable and grep the
string: git always completes, grep judges bytes.
Playwright must own the spawn (it needs the inspection pipe), but
hermes desktop is not just build+launch - stamp checks, integrity
gates, sandbox fixups, and a constructed child environment. So the
driver intercepts the product's own launch: a sitecustomize.py on
PYTHONPATH (opt-in via HERMES_E2E_CAPTURE_LAUNCH) wraps subprocess.run,
captures argv/cwd/env at the spawn site, and fakes success instead of
spawning; launch-from-spec.mjs then _electron.launch-es exactly that
spec and clicks Settings -> About -> Update now. Completion is product
state, not a Playwright event: the handoff result file or the checkout
reaching the expected sha (source installs write no result file).
Ships with the driver, so it works unchanged on every sampled OLD ref
- no product flag, no pre-flag fallback split. Both launch shapes are
matched (npm exec electron / packaged exe under apps/desktop/release);
npm BUILD calls pass through untouched. Exit 0 without a capture fails
the leg: a version that never reached its launch must not pass.
Probe-the-probe: scripts/launch_capture_probe.sh runs control rows
(no opt-in, non-launch argv) and both treatment shapes - all green
locally. Gate flips on the shared run workflow for linux/macos;
windows adopts the same path with the driver restructuring.
The composite action .github/actions/e2e-screen-record owns setup and
lifecycle on all three OSes: ffmpeg via apt/brew-verify/winget+cache,
capture via x11grab/gdigrab/avfoundation, mkv at 15fps stopped by 'q'
on live stdin with kill fallback. Linux runners have no display, so
start brings up a dedicated Xvfb :99 and exports DISPLAY - one display
serves both the recorder and any app a later step launches.
Recording moves out of the GUI driver into workflow infrastructure -
that is what makes it uniform - and a missing ffmpeg or a zero-frame
file now FAILS the leg instead of skipping silently: the graceful-skip
path is how the windows leg shipped no recording.mkv while green.
Lifecycle proven locally: start against lavfi testsrc, q-stop, ffprobe
duration check (record-start.sh/record-stop.sh under nix ffmpeg).
The one-liner with its desktop stage opted in (--include-desktop /
-IncludeDesktop) is a real install kind, distinct on both sides:
on windows the stage builds Hermes.exe AND registers Start Menu /
Desktop shortcuts - a second path to a hand-launchable app - while
on linux/macos it builds into the checkout and registers no OS
entry point.
Declared on every OS and driven by both script drivers: the drivers
pass the flag through (hard failure if the ref predates it - the
tag-has-desktop gate already skips pre-desktop tags upstream) and
assert the built app exists under apps/desktop/release afterwards.
The run-workflow gates run +desktop pairs only on desktop-bearing
tags; app-update pairs from +desktop installs stay declared TODOs.
The desktop app has two launch paths, so app-update becomes two
methods. open-app-update starts the app from the OS entry point the
desktop installer created (the installed exe / the .app), so it exists
only where a desktop installer does. hermes-desktop-app-update starts
the app via hermes desktop, which every install method provides on
every OS that ships the desktop app - on linux it is the only app
surface, since no desktop installer or packaged artifact exists there.
Both variants are desktop-surface methods on every OS, so the
tag_has_desktop annotation moves from windows-only to every matrix
entry, install-e2e-run.yml grows the input, and the plan chart marks
pre-desktop cells on all OSes.
The windows GUI arm's implemented pair renames to open-app-update;
every other new combination is a declared TODO that natively skips.
Each script-driver leg now proves the installed CLI can build the
desktop app, after the install phase and again after the update.
--build-only runs the full desktop pipeline and stops before the
launch - the same call hermes update makes. Old releases that predate
the flag skip the phase after a --help probe of the installed binary.
Actually launching the app is a TODO: it needs the spawn-interception
launcher and, on linux runners, a virtual display.
Also adds tests/install/README.md describing how the test family
works: the four layers, the git-redirect isolation, the phases, the
probe-do-not-assume rule for old versions, skips, triggers, artifacts.
All three drivers (installer-script-e2e.sh, windows-installer-script
-e2e.ps1, windows-desktop-gui-e2e.ps1) now emit the complete
install/update transcript into the job log wrapped in ::group::/
::endgroup:: - collapsed by default, one click to expand, win or
lose. Replaces the tail-50-only-on-failure pattern: a green
install's transcript is how you diagnose the leg that fails next,
and the artifact download was the only way to see it before. The
GUI driver's bootstrap-installer.log / desktop-update-handoff.log
tails become full folded dumps too.
First real run of windows-installer-script-e2e.ps1 died on
'Cloning into ...': git clone writes progress to stderr, and under
EAP=Stop the outer PowerShell wraps a child's stderr into a
terminating NativeCommandError. Drop to EAP=Continue around every
native invocation that redirects with *> (install.ps1 run, hermes
--version, hermes update) and judge by exit code alone - the same
prevEap pattern the GUI driver already uses.
Run 31553293275: the desktop leg from v2026.6.19 failed at
'@playwright/test resolvable from installed apps/desktop' - that
release predates the dependency, and newer releases move it around
via workspace hoisting, so resolving it from the installed tree made
the leg's tooling a function of the version under test.
Install a pinned @playwright/test (param, default 1.58.2 - the repo
lockfile's current version) into a scratch driver dir with the
managed node/npm every run and launch the driver from there. The
driver talks to the app over Playwright's inspection pipe, so its
Playwright is independent of the app; every OLD ref now runs the
exact same driver stack.
The install.ps1 sibling of installer-script-e2e.sh: stage serve.git
(main parked at OLD, GIT_CONFIG_GLOBAL insteadOf redirect - NOT env
config, which install.ps1 clobbers), run the install.ps1 shipped AT
the OLD ref headless (-SkipSetup -HermesHome/-InstallDir explicit
because the oldest tags predate the HERMES_HOME env override;
-NonInteractive probed from the ref's own script text), assert the
checkout + venv hermes.exe, advance served main, update via
hermes-update (--yes probed) or HEAD's install.ps1, assert HEAD.
install-e2e-windows-run.yml grows a second job for the arm: the
installer-script x {hermes-update, installer-script} pairs flip from
grey to live, app-update from a script install stays a declared TODO.
PS 5.1-safe pure ASCII.
Stage logic verified behaviorally under pwsh (redirect resolves the
canonical URL to serve.git at OLD, per-ref install.ps1 extraction
parses, advance lands HEAD); the install legs themselves need a real
Windows runner - dispatched next.
The fake Internet (bubblewrap + slirp4netns + MITM proxy +
upload-pack shim, 883 lines across dev-sandbox.sh, stage2-run.sh,
proxy.py, ssh-shim.sh, openssl.cnf, install-update-e2e.sh) existed to
isolate install.sh's network. The GIT_CONFIG_GLOBAL insteadOf redirect
the windows driver introduced does the same job with a gitconfig file
and works on any OS, so:
* install-e2e-run.yml now runs tests/install/installer-script-e2e.sh
directly on the bare runner - no sandbox deps, no userns sysctls -
and takes a runner input;
* the macos matrix calls the SAME workflow on macos-latest, deleting
install-e2e-macos-run.yml: installer-script -> installer-script /
hermes-update flip from grey to live, app-update pairs stay TODO
inside the shared gate;
* install.sh is no longer curl'd through a fake CA - each leg runs
the copy from the ref a user of that version actually executed;
* scripts/dev-sandbox.sh becomes the minimal isolation sandbox from
ab6b9492f (separate HERMES_HOME / Electron userData / app name,
same CLI surface: --persistent, --from, --delete), keeping its
.hermes-sandbox dir name so gitignore and docs hold;
* nix/sandbox.nix drops the bwrap/proxy closure and keeps only the
Electron runtime LD_LIBRARY_PATH the desktop app needs.
Verified: nix build .#sandbox + smoke run (isolated HERMES_HOME
created, ephemeral cleanup), shellcheck/bash -n on both scripts,
actionlint on all three workflows, and the new driver ran the full
v0.20.2 -> HEAD hermes-update pass locally before this commit.
The POSIX sibling of windows-desktop-gui-e2e.ps1, sharing its staging
trick: bare-clone the checkout to serve.git, park main at OLD, point
every git process at it with url.<file://serve.git>.insteadOf in a
driver-owned GIT_CONFIG_GLOBAL. The installer and updater run
byte-for-byte against their real URLs; no bwrap, no MITM proxy, no
TLS interception - a disposable CI runner IS the sandbox, so the
same driver can run on macos-latest unchanged.
install.sh is not curl'd: the install leg runs the copy shipped AT
the OLD ref (what a user who installed then actually executed), the
installer-script update leg runs HEAD's copy (what the website
serves at update time). Flags are probed per-ref (--skip-browser is
newer than sampled tags); HOME is isolated because old installers
hardcode ~/.hermes; .skip_upstream_prompt suppresses the updater's
fork prompt on the file:// origin; the dirty-tree guard checks
tracked files only (-uno) since untracked files cannot leak into a
bare clone.
Verified locally end-to-end: v0.20.2 installed via its own
install.sh (uv, managed Python, Node, venv; hermes --version OK),
served main advanced, hermes update landed the checkout on HEAD
with a working hermes. bash -n + shellcheck clean.
Per review the unions were overcomplicated. Install methods are now
just: installer-script (the platform one-liner - curl | bash on
linux/macos, irm | iex on windows), desktop-installer, and
packaged-app (declared, unused). Update methods are every install
method (re-run it over the existing install) plus hermes-update and
app-update. desktop-installer-rerun, desktop-app, curl-bash, and
irm-iex are gone as ids; the windows driver's ValidateSet, switch
arms, and both run workflows' gates renamed to match. tsc --checkJs
clean; generator output re-verified (4 linux / 16 windows / 6 macos
legs for 2 tags).
Revert the hardcoded 16-job experiment: the combination spec belongs
in scripts/sandbox/generate-e2e-matrix.mjs (restored), not copy-pasted
YAML blocks. What survives from the experiment:
* leg names carry everything - 'os: install -> update (tag -> HEAD)' -
generated per entry, since slash-joined names are all the graph
renders;
* pick-releases annotates each tag ({ref, desktop}) and the generator
threads tag_has_desktop onto windows entries, so the windows run
workflow still gates pre-desktop tags without a probe job;
* the per-OS run workflows are untouched: single job, static 'e2e'
name, native skip gates own all capability knowledge.
install-e2e.yml is one generate job + three per-OS matrix fanouts.
Generator shape (4/16/12 legs for 2 tags), annotation threading, and
all six error paths verified; all four workflows pass actionlint;
driver parses clean pure-ASCII.
GitHub only draws matrix boxes for the PRIMARY workflow's matrices;
everything inside a called workflow flattens into slash-joined names.
The generator + per-tag sub-workflow therefore bought no structure in
the graph and hid the support matrix in a script.
Invert it: install-e2e.yml now declares one job per {os,
install-method -> update-method} combination (2 linux + 8 windows +
6 macos - same 16 the generator produced, verified by inventory
before/after), each a matrix over the picked release tags. The graph
now renders one titled box per combination whose legs read
'... from vX' - the tag axis inside the combo axis. The per-OS run
workflows are unchanged: they own capability knowledge and natively
skip unimplemented method pairs and pre-desktop tags.
install-e2e-tag.yml and generate-e2e-matrix.mjs are deleted; adding a
method is now adding one job block here, implementing one is flipping
the run workflow's gate.
Two structural changes to the combination fanout:
1. Tags become the OUTER axis, as a sub-graph per starting version:
install-e2e.yml fans a plain matrix over the picked tags into a new
per-tag reusable workflow (install-e2e-tag.yml), which runs the
combination generator for that one tag and fans out one job per
{os, install-method, update-method}. The Actions graph now reads
'from vX -> windows: install -> update' per leg. Nothing is
hardcoded in the workflows: the tag workflow calls the generator
itself.
2. Native skips move to the point that owns the capability knowledge:
macOS combos (no driving workflow exists) grey out in the tag
workflow via install-e2e-skip.yml, untouched by the tag axis; ALL
windows combos dispatch to install-e2e-windows-run.yml, which takes
install-method/update-method inputs and natively skips the pairs
its driver cannot run yet - so implementing a windows method is a
change in the run workflow + driver only. The driver's -Route ids
now match the generator's method ids verbatim.
Generator output shape, route filters, and all error paths re-verified
locally; all four workflows pass actionlint; driver re-parses clean
pure-ASCII.
Bring back the continuous recording the retired AHK-only driver had,
alongside the 3s frame captures: gdigrab 15fps to proof/<phase>/
recording.mkv, started before the installer/app launches and q-stopped
in the finally block win or lose. mkv stays playable when the process
dies unfinalized; skip gracefully when ffmpeg is not on PATH (it ships
on windows-latest). Lifecycle (start, null-skip, graceful q stop, clean
exit, playable output) verified locally with a lavfi source.
Run 31520559900: stage passed the whole multiline tag list to
rev-parse ('Filename too long'). In
(Invoke-Git @(...) -split pattern | ...) PowerShell parses -split as
ANOTHER ARGUMENT to Invoke-Git, not as an operator on its result - the
function got '-split' and the regex appended to its array and rev-parse
received every tag at once. Split via an intermediate variable instead.
Verified locally: pwsh picks [v0.20.2] and resolves it to a single
commit.
Run 31520267702 died in 3s: 'Missing an argument for parameter
InstallRef'. powershell.exe -File drops a "" argument from the command
line entirely, so the parameter binder saw -InstallRef followed by
-SetupExeUrl. Default both the workflow input and the script parameter
to 'auto' (= newest release tag) instead of empty.
Run 31519103491 failed the 'update genuinely available' assert with
the installer landing on HEAD itself. The staging assumed the website
exe installs a baked release pin, but the bootstrap log shows
Pin { commit: None, branch: main } - the published installer installs
whatever main serves, and serve.git's main was parked at HEAD.
Stage the way the linux axis does: park served main at OLD
(-InstallRef, default newest release tag; threaded through the
reusable workflow as install-ref) for the install phase, assert the
install lands exactly there, then advance main to HEAD in the update
phase - an update becomes available the same way it does for a real
user. allowAnySHA1InWant stays as belt-and-braces for installer builds
that DO bake a pin.
Restructure tek's two-job desktop-windows-e2e.yml into the shape the
linux axis already has: install-e2e.yml keeps its update/installer
routes untouched and windows-desktop returns as a route in the same
family, calling a reusable install-e2e-windows-run.yml.
Behind that route is now ONLY the real user flow - the headless
contract job (install.ps1 at HEAD~1, desktop-update.ps1 -NoUi,
BASE/CURRENT/NEXT ref dance) is gone, along with its driver. Every leg
goes through a surface a user touches: website Hermes-Setup.exe headed
with AutoHotkey clicking Install -> Launch, then the installed
Hermes.exe under Playwright's Electron driver clicking Settings ->
About -> 'Update now', through the detached hand-off to a relaunched
window asserted on HEAD.
The driver drops the synthetic-NEXT staging with the contract job:
serve.git just serves HEAD as main and OLD is the release pin baked
into the website exe - the literal starting point of every real GUI
user, same philosophy as the linux axis's release-tag matrix. The
-Route parameter (desktop today) declares the future update mechanisms
as arms: 'update' (hermes update from the installed venv) and
'installer' (re-run the bootstrap exe) raise until implemented, so the
workflow surface is stable when they land.
The cherry-picked desktop-windows-e2e.yml covers everything the
install-e2e-windows-run.yml axis did and more: the contract job drives
the same desktop-update.ps1 hand-off (plus a CURRENT->NEXT forward
leg), and the GUI job replaces AHK-only driving with the full real
user flow - website Hermes-Setup.exe, clicked Install/Launch, then
Playwright clicking Settings -> About -> 'Update now' in the packaged
app, through the detached hand-off to a relaunched window.
Remove the superseded workflow, its driver, and the AHK/button assets
under tests/install/windows/ (the GUI job's e2e-assets carry the
re-captured templates), and drop the windows-desktop route from
install-e2e.yml's dispatch options.
The real-user-flow job passed end to end (run 31492613931, 10m57s):
website Hermes-Setup.exe installed headed (AHK Install+Launch, real app
window), then TWO GUI updates driven by real Settings -> About ->
'Update now' clicks, each carried through the detached hand-off to a
relaunched desktop on the target commit. Every assertion green on both
legs (marker cleanup, checkout on target sha, working hermes, relaunch).
Two finishing touches:
* Foreground the relaunched Hermes window before the 99-relaunched proof
screenshot — the full-desktop grab is z-order dependent and one run
caught VS Code on top. The relaunch ASSERT already passed on the
process signal; this is purely to make the proof image show Hermes.
* Remove the temporary branch push trigger used for pre-merge validation;
back to main + nightly + release tags + manual dispatch only.
Attempt 10 ran 1h40m and the diagnosis is precise: the GUI update hung on
a bare input() in hermes update's _sync_with_upstream_if_needed. Our
serve.git origin is a file:// URL, so _is_fork() is true and the updater
asks 'Add official repo as upstream? [Y/n]' via raw input(). When the
Desktop spawns the hand-off through 'cmd start /min' that child has a real
but EMPTY console, so input() blocks forever (no EOF, no keystroke). The
contract job spawns the hand-off with inherited non-interactive stdin, so
input() hits EOF and defaults immediately -- which is why it never hung.
The proof chain confirmed everything else worked: backend exited, venv
unlocked, git pull found the commit and applied it; the process then just
sat in input(). update.log was never created because the hang is BEFORE
the desktop-build step.
Fix: create HERMES_HOME/.skip_upstream_prompt after install -- the
product's own 'don't ask about upstream' marker (_should_skip_upstream_
prompt). Real GUI users install from the official github origin where
_is_fork() is false and this prompt never fires, so this only neutralizes
a staging artifact of the file:// serve repo, not real behavior.
(Noted for a separate product follow-up: hermes update --gateway should
route this input() through _gateway_prompt like its other prompts, so a
fork-origin GUI update can't hang even without the marker.)
Attempt 9 drove the full real update through the hand-off: marker
detected, desktop exited, hermes update fetched from serve.git, found the
commit, pulled, restored -- all correct. It then timed out because the
updater legitimately runs LONG here: the website release we install
(v0.20.0) is weeks of main behind CURRENT, so the update pulls a large
diff AND does a full Electron desktop rebuild (vite + electron-builder)
plus uv sync. The contract job's BASE->CURRENT is a 1-commit tests-only
diff that skips the rebuild, which is why it finishes in ~1 min; the GUI
job's release->CURRENT does not.
* wait window 40 -> 90 min per leg; job timeout 180 -> 240 min
* tail logs/update.log during the wait so the desktop-rebuild phase is
visible in CI output instead of tens of minutes of silence (the rebuild
streams there, not to the handoff log)
The CURRENT->NEXT leg stays fast (NEXT is a same-tree child of CURRENT,
no rebuild), so total stays well within 240 min.
Attempt 8 drove the ENTIRE GUI update click-path successfully: onboarding
dismissed, Settings opened, About opened, Update now clicked, updating
overlay shown. The hand-off log proves the real update then ran: desktop
(pid 8880) exited, venv unlocked, 'hermes update --yes --gateway --force
--branch main' fetched from serve.git, found 1 new commit, pulled, and
restored. Everything worked.
The only failure was the driver waiting on Playwright's app 'close'
event, which doesn't fire reliably when the Electron app self-quits for
the hand-off. Switch to the authoritative signal: poll for the
HERMES_HOME/.hermes-update-in-progress marker (or the result JSON, or a
genuine window-gone), which the hand-off writes ~4s after the click. The
PowerShell driver still owns asserting the OUTCOME (target sha, marker
cleanup, working hermes, relaunched app) after the driver returns.
Attempt 7 got the whole way into the GUI update leg: the installed
Electron app launched under Playwright, booted, composer attached, first
screenshot captured. It then couldn't find the settings gear -- the
ERROR screenshot showed why: a fresh install with no CONFIGURED provider
(the seeded .env key isn't read as model.provider) shows the onboarding
card ('Let''s get you setup with Hermes Agent'), which covers the shell
and its settings gear.
The update path needs no provider, so the driver now clicks 'I'll choose
a provider later' (with skip fallbacks) to dismiss onboarding and reach
the shell before looking for the gear. Harmless no-op when onboarding
isn't shown. Gear (aria-label 'Open settings') and About nav ('About')
selectors already match the real components.
yoniebans