A dmg user can also update from the terminal (hermes update) or by
re-running the install one-liner; the dmg arm only routed the two
app-button methods, leaving three declared cells as permanent TODOs.
Port the POSIX driver's method blocks into the macos driver's update
phase (hermes-update with the --yes probe, installer re-run with per-ref
flag probing, the +desktop built-app assert) and open the workflow gate.
The dmg bootstrap driver also learns to recover instead of waiting out
its bound when a stage fails: the bootstrap parks on an error screen
with a Retry button (seen live: HTTP 429 downloading install.sh under
full-matrix runner load), so the driver watches the bootstrap log for
real failure shapes, requires two consecutive error probes before
retargeting the click at Retry's measured position, unlatches when the
log goes healthy, and gives up with the true cause after three retries.
Driver rule learned three times in this suite (lsof +D, git show and
find piped to grep -q): under set -euo pipefail, never feed grep -q
from a pipe... grep exits at first match, the producer takes SIGPIPE,
and a TRUE condition reads as failure. Capture to a variable or test
paths directly.
Verified end to end: run 33506177130, macos slice 13/13 green.
Hermes-Setup is a Tauri app that boots to a setup-choice screen and waits
for a click on Install Hermes before any install work starts; run bare it
blocked until the 120-minute job cap (both dmg legs, every run). Launch it
in the background with the driver's env, read the window geometry via
System Events (position and size need no assistive grant), post a real
CGEvent click with cliclick at the button's measured position (65% of
window height; System Events' own click needs assistive access the runners
deny), then wait for the full install to land: checkout, venv console
script, AND the built Hermes.app, since the cleanup trap would otherwise
kill the installer before its desktop-build stage. Bounded at 45 minutes
with desktop screenshots on every phase and failure. Adds a macos-desktop
dispatch route so this arm iterates without the full matrix.
Verified end to end: run 33407401698, both dmg legs green (first ever),
macos slice 10/10.
Windows transcripts were ZERO bytes: ts-prefix.ps1 formatted with
{0:D2}, but Floor() returns a double and the D specifier is
integer-only - it threw per line, and under the driver's relaxed EAP
every line errored into the void. {0:00} fixes it (custom numeric
format works on doubles). Reproduced the exact pipeline locally
(empty file + Format specifier invalid), verified the fix produces
prefixed merged stdout+stderr with exit code intact. That is also
why the log timeline never auto-synced: there was nothing in the
files to sync.
The GitHub artifact URL 307s to /suites/... server-side and strips
the ?zip= query param. The player now reads the zip URL from a
#zip= HASH param (client-side, survives the redirect) with ?zip=
as fallback; the hash path was verified in a real browser against a
real leg zip (auto-fetch + boot).
Per ethie's design, one player artifact for the whole run: new
leg-player job uploads playback.html (archive:false) before the
matrix legs, the report job needs it, and each ran cell gets TWO
links - 📼 to the player with #zip=<that leg's logs zip> and ⬇️ to
the raw zip. Per-leg player uploads removed from all three run
workflows.
Each leg uploads playback.html as a single-file artifact (archive:
false) before the driver runs, so it exists even on failure. The
results chart now links every leg that RAN (pass or fail, not skip)
to its player with ?zip= pointing at that leg's logs artifact.
Leg<->artifact mapping: the generator mints a leg_id per matrix entry
(sanitized matrix name, exported legId()), every run workflow names
its artifacts install-e2e-{player,logs}-<leg-id>, and the report job
feeds the run's artifact name->id list to the results renderer, which
rebuilds the leg id from the parsed job name. GitHub does not link
jobs to artifacts, so the deterministic name is the join key.
Empirical finding: GitHub artifact downloads are auth-gated (the
download URL 307s to /suites/... which is 404 anonymous), so a
locally-opened player page cannot fetch the zip cross-origin. The
player now degrades gracefully: ?zip= fetch failure renders a real
download link for the zip (a normal click carries the user's session)
plus a drag-and-drop / file-picker path, and no-param opens as a pure
drop target. Verified in a real browser against a real artifact URL.
Verified: generator emits leg_id, results renderer emits
✅/❌ [📼](...?zip=...) only on ran cells, npm run check PASS,
install tests 36/36, strict tsc PASS, actionlint x4 PASS.
A static single-file player (tests/install/e2e-assets/playback.html):
?zip=<artifact zip url> unzips in-browser (JSZip), plays the screen
recording with a timer pinned top-left, and renders every *.log with
video<->log sync: the video follows the driver's transcript, clicking
a log line seeks the video. A sync-offset slider aligns the recording
start (ffmpeg comes up first) with the driver's relative clock.
Sync axis: drivers now prefix every transcript line with [+MM:SS]
relative to driver start (ts-prefix.sh / ts-prefix.ps1, pipe-safe
under pipefail / relaxed EAP). Browsers cannot play Matroska, so each
leg remuxes recording.mkv -> recording.mp4 (-c copy, no re-encode)
before the artifact upload, on all three OSes.
Verified end-to-end in a real browser against a generated artifact
zip: zip load, mp4 playback, timer, tab switching, follow-sync at
t=6/t=12, click-to-seek, autoplay policy (expected NotAllowedError on
synthetic play; real clicks fine).
Also fixes the shim fail message's dead variable ( ->
observed_git_url) in both posix drivers.
Skipped cells split into their reason: pre-desktop (a desktop-surface
method against a tag that predates apps/desktop) vs TODO (declared,
no driver arm yet). The report job passes pick-releases' annotated
tags into --format results; the shared methodNeedsDesktop() is the
same predicate the plan chart uses, so plan and results agree about
what pre-desktop means. Without --tags the renderer keeps the flat
skip label (backward compatible).
Verified against run 31579084845 real job list: 82 legs, 44 skips
labeled correctly.
macos gains the desktop-installer@latest install method: the website's
Hermes-Setup.dmg (verified live), mounted with hdiutil and its app
binary run DIRECTLY - an open-launched app inherits none of the git
redirect env, so direct exec is what keeps the isolation honest while
staying the same binary and first-launch flow.
install-e2e-macos-run.yml takes the windows shape: one workflow, one
inner job per driver arm, native skips. Arm 1 delegates script installs
to the shared OS-agnostic run workflow; arm 2 stages, installs from the
dmg, and drives both app-update methods through launch-from-spec.mjs -
open-app-update launches the installed .app (the double-click surface,
env via Playwright), hermes-desktop-app-update captures the product's
own hermes desktop spawn. Both end on sha asserts, never version
strings.
windows-desktop-gui-e2e.ps1 and windows-installer-script-e2e.ps1 fold
into tests/install/windows-e2e.ps1 with orthogonal -InstallMethod and
-Route axes: the install phase dispatches on one, the update phase on
the other, and shared workroot state carries how OLD landed - so any
implemented update method can follow any implemented install method.
Implementing a new pair is now a driver function plus a gate edit,
never a new job.
The run workflow collapses to ONE inner job whose if: is the
implemented-pairs table. Newly cheap pairs go live with the merge:
desktop-installer@latest -> hermes-update / installer-script /
installer-script+desktop / hermes-desktop-app-update
installer-script(+desktop) -> hermes-desktop-app-update
installer-script+desktop -> open-app-update (the -IncludeDesktop
install registers real Start Menu / Desktop shortcuts)
Only desktop-installer@latest as an UPDATE method stays a declared
TODO. scripts/windows_e2e_harness.ps1 executes the parse/parameter/
dispatch checks under pwsh before any Windows runner spins up.
Playwright must own the spawn (it needs the inspection pipe), but
hermes desktop is not just build+launch - stamp checks, integrity
gates, sandbox fixups, and a constructed child environment. So the
driver intercepts the product's own launch: a sitecustomize.py on
PYTHONPATH (opt-in via HERMES_E2E_CAPTURE_LAUNCH) wraps subprocess.run,
captures argv/cwd/env at the spawn site, and fakes success instead of
spawning; launch-from-spec.mjs then _electron.launch-es exactly that
spec and clicks Settings -> About -> Update now. Completion is product
state, not a Playwright event: the handoff result file or the checkout
reaching the expected sha (source installs write no result file).
Ships with the driver, so it works unchanged on every sampled OLD ref
- no product flag, no pre-flag fallback split. Both launch shapes are
matched (npm exec electron / packaged exe under apps/desktop/release);
npm BUILD calls pass through untouched. Exit 0 without a capture fails
the leg: a version that never reached its launch must not pass.
Probe-the-probe: scripts/launch_capture_probe.sh runs control rows
(no opt-in, non-launch argv) and both treatment shapes - all green
locally. Gate flips on the shared run workflow for linux/macos;
windows adopts the same path with the driver restructuring.
The composite action .github/actions/e2e-screen-record owns setup and
lifecycle on all three OSes: ffmpeg via apt/brew-verify/winget+cache,
capture via x11grab/gdigrab/avfoundation, mkv at 15fps stopped by 'q'
on live stdin with kill fallback. Linux runners have no display, so
start brings up a dedicated Xvfb :99 and exports DISPLAY - one display
serves both the recorder and any app a later step launches.
Recording moves out of the GUI driver into workflow infrastructure -
that is what makes it uniform - and a missing ffmpeg or a zero-frame
file now FAILS the leg instead of skipping silently: the graceful-skip
path is how the windows leg shipped no recording.mkv while green.
Lifecycle proven locally: start against lavfi testsrc, q-stop, ffprobe
duration check (record-start.sh/record-stop.sh under nix ffmpeg).
The one-liner with its desktop stage opted in (--include-desktop /
-IncludeDesktop) is a real install kind, distinct on both sides:
on windows the stage builds Hermes.exe AND registers Start Menu /
Desktop shortcuts - a second path to a hand-launchable app - while
on linux/macos it builds into the checkout and registers no OS
entry point.
Declared on every OS and driven by both script drivers: the drivers
pass the flag through (hard failure if the ref predates it - the
tag-has-desktop gate already skips pre-desktop tags upstream) and
assert the built app exists under apps/desktop/release afterwards.
The run-workflow gates run +desktop pairs only on desktop-bearing
tags; app-update pairs from +desktop installs stay declared TODOs.
The desktop app has two launch paths, so app-update becomes two
methods. open-app-update starts the app from the OS entry point the
desktop installer created (the installed exe / the .app), so it exists
only where a desktop installer does. hermes-desktop-app-update starts
the app via hermes desktop, which every install method provides on
every OS that ships the desktop app - on linux it is the only app
surface, since no desktop installer or packaged artifact exists there.
Both variants are desktop-surface methods on every OS, so the
tag_has_desktop annotation moves from windows-only to every matrix
entry, install-e2e-run.yml grows the input, and the plan chart marks
pre-desktop cells on all OSes.
The windows GUI arm's implemented pair renames to open-app-update;
every other new combination is a declared TODO that natively skips.
generate-e2e-matrix.mjs grows --format results: reads the run's own
job list as NDJSON {name, conclusion} on stdin (per-leg conclusions
are NOT reachable through needs - a matrix job collapses to one
aggregate result) and re-renders the plan chart with each cell's
outcome. Legs are recognized by the exact name shape buildMatrices
mints, so unrelated jobs fall out; duplicate leg names (one windows
job per driver arm, only one runs) merge by significance - real
outcomes beat skips, failures beat successes. A final report job
(if: always, needs all three OS jobs) appends the chart to its step
summary via gh api with the default token.
Verified against two real runs: 31536931863 renders 11 passed / 0
failed / 54 skipped all-green; 31557865241 (the pre-EAP-fix run)
renders its 4 real failures + cancellations over the sibling arm's
skips.
generate-e2e-matrix.mjs grows --format markdown: one row per
{os, install -> update} combination, one column per starting tag,
appended to GITHUB_STEP_SUMMARY by the expand job. Cells mark
dispatched legs; run-vs-grey stays the run workflows' call, so the
only special cell is pre-desktop (the one annotation the plan owns).
JSON mode unchanged.
The install.ps1 sibling of installer-script-e2e.sh: stage serve.git
(main parked at OLD, GIT_CONFIG_GLOBAL insteadOf redirect - NOT env
config, which install.ps1 clobbers), run the install.ps1 shipped AT
the OLD ref headless (-SkipSetup -HermesHome/-InstallDir explicit
because the oldest tags predate the HERMES_HOME env override;
-NonInteractive probed from the ref's own script text), assert the
checkout + venv hermes.exe, advance served main, update via
hermes-update (--yes probed) or HEAD's install.ps1, assert HEAD.
install-e2e-windows-run.yml grows a second job for the arm: the
installer-script x {hermes-update, installer-script} pairs flip from
grey to live, app-update from a script install stays a declared TODO.
PS 5.1-safe pure ASCII.
Stage logic verified behaviorally under pwsh (redirect resolves the
canonical URL to serve.git at OLD, per-ref install.ps1 extraction
parses, advance lands HEAD); the install legs themselves need a real
Windows runner - dispatched next.
The fake Internet (bubblewrap + slirp4netns + MITM proxy +
upload-pack shim, 883 lines across dev-sandbox.sh, stage2-run.sh,
proxy.py, ssh-shim.sh, openssl.cnf, install-update-e2e.sh) existed to
isolate install.sh's network. The GIT_CONFIG_GLOBAL insteadOf redirect
the windows driver introduced does the same job with a gitconfig file
and works on any OS, so:
* install-e2e-run.yml now runs tests/install/installer-script-e2e.sh
directly on the bare runner - no sandbox deps, no userns sysctls -
and takes a runner input;
* the macos matrix calls the SAME workflow on macos-latest, deleting
install-e2e-macos-run.yml: installer-script -> installer-script /
hermes-update flip from grey to live, app-update pairs stay TODO
inside the shared gate;
* install.sh is no longer curl'd through a fake CA - each leg runs
the copy from the ref a user of that version actually executed;
* scripts/dev-sandbox.sh becomes the minimal isolation sandbox from
ab6b9492f (separate HERMES_HOME / Electron userData / app name,
same CLI surface: --persistent, --from, --delete), keeping its
.hermes-sandbox dir name so gitignore and docs hold;
* nix/sandbox.nix drops the bwrap/proxy closure and keeps only the
Electron runtime LD_LIBRARY_PATH the desktop app needs.
Verified: nix build .#sandbox + smoke run (isolated HERMES_HOME
created, ephemeral cleanup), shellcheck/bash -n on both scripts,
actionlint on all three workflows, and the new driver ran the full
v0.20.2 -> HEAD hermes-update pass locally before this commit.
Per review the unions were overcomplicated. Install methods are now
just: installer-script (the platform one-liner - curl | bash on
linux/macos, irm | iex on windows), desktop-installer, and
packaged-app (declared, unused). Update methods are every install
method (re-run it over the existing install) plus hermes-update and
app-update. desktop-installer-rerun, desktop-app, curl-bash, and
irm-iex are gone as ids; the windows driver's ValidateSet, switch
arms, and both run workflows' gates renamed to match. tsc --checkJs
clean; generator output re-verified (4 linux / 16 windows / 6 macos
legs for 2 tags).
Per review: the method/version vocabulary is now closed TYPE unions
(@ts-check + jsdoc typedefs - InstallerVersion, InstallMethod,
UpdateMethod - checked with tsc --checkJs, which rejects a SPEC entry
outside the unions; verified by corrupting a copy: 6 errors) instead
of runtime KNOWN_METHODS/ALLOWED_VERSIONS sets. validateEntry and
routeWants are deleted with all the paranoia: the generator always
emits every OS matrix and the dispatch route filter moved to plain
job-level ifs in install-e2e.yml, where the OS jobs already live.
secondUpdate is typed never[] so declaring one is a type error until
a leg implements it. Anything types cannot catch is self-evident on
the next CI run.
Revert the hardcoded 16-job experiment: the combination spec belongs
in scripts/sandbox/generate-e2e-matrix.mjs (restored), not copy-pasted
YAML blocks. What survives from the experiment:
* leg names carry everything - 'os: install -> update (tag -> HEAD)' -
generated per entry, since slash-joined names are all the graph
renders;
* pick-releases annotates each tag ({ref, desktop}) and the generator
threads tag_has_desktop onto windows entries, so the windows run
workflow still gates pre-desktop tags without a probe job;
* the per-OS run workflows are untouched: single job, static 'e2e'
name, native skip gates own all capability knowledge.
install-e2e.yml is one generate job + three per-OS matrix fanouts.
Generator shape (4/16/12 legs for 2 tags), annotation threading, and
all six error paths verified; all four workflows pass actionlint;
driver parses clean pure-ASCII.
Graph polish + one structural simplification, after the first render
of the combo-box layout:
* Leg names: every combination job's display name is now
'${{ matrix.tag.ref }} -> HEAD' - the box title (job id) already
carries os+methods, so repeating them per leg was noise. The inner
job renders as a short static 'e2e' tail (dynamic names render
unexpanded on skipped jobs, so it must stay static).
* The windows probe job is gone: pick-releases now annotates each
picked tag with whether its tree ships apps/desktop
({ref, desktop} objects in the matrix), and the windows run
workflow gates on the new tag-has-desktop boolean input directly.
One tree listing at pick time replaces N probe jobs, and the
'probe tag' noise disappears from the graph.
Annotation loop verified against the real tag set (pre/post-desktop
split lands exactly at the app's introduction); 16-combo inventory
re-asserted; all four workflows pass actionlint.
GitHub only draws matrix boxes for the PRIMARY workflow's matrices;
everything inside a called workflow flattens into slash-joined names.
The generator + per-tag sub-workflow therefore bought no structure in
the graph and hid the support matrix in a script.
Invert it: install-e2e.yml now declares one job per {os,
install-method -> update-method} combination (2 linux + 8 windows +
6 macos - same 16 the generator produced, verified by inventory
before/after), each a matrix over the picked release tags. The graph
now renders one titled box per combination whose legs read
'... from vX' - the tag axis inside the combo axis. The per-OS run
workflows are unchanged: they own capability knowledge and natively
skip unimplemented method pairs and pre-desktop tags.
install-e2e-tag.yml and generate-e2e-matrix.mjs are deleted; adding a
method is now adding one job block here, implementing one is flipping
the run workflow's gate.
Run 31530831547 failed the moment old tags hit the windows leg: the
bootstrap install of v2026.4.30 / v2026.5.29.2 succeeded but no app
window ever appeared - those releases predate the desktop app
(#20059, v2026.5.31), so there is nothing to launch and no Update
button to click. Add a probe job that asks the tag's own tree
(git ls-tree apps/desktop) and gate the run job on it, so
desktop-method legs from pre-desktop tags natively skip instead of
failing. Data-driven - no version cutoff list to rot. Probe logic
verified locally against pre- and post-desktop tags plus the auto
sentinel.
Remove all capability knowledge from the combination generator: no
IMPLEMENTED table, no skipped matrix, no per-OS special cases. It now
only declares and expands - every {os, install-method, update-method}
combination is dispatched to its OS's run workflow, and each run
workflow natively skips (grey, job-level if on the method inputs) the
pairs its driver cannot run yet:
* install-e2e-run.yml gains install-method/update-method inputs,
gates on the supported pairs (curl-bash -> hermes-update/curl-bash),
and maps the method id to the sandbox script's --route internally;
* install-e2e-macos-run.yml is new - all pairs skip until a macOS
driver exists, and implementing one flips its job-level if;
* install-e2e-windows-run.yml already worked this way;
* install-e2e-skip.yml is deleted - nothing special-cases macOS
anymore, so the tag workflow is three identical OS fanouts.
Structure is now uniformly matrix(tag) -> matrix(combination) ->
run-or-skip, with capability knowledge living only next to each
driver. Generator shape/route filters/error paths re-verified; all
five workflows pass actionlint.
Run 31530831547 showed skipped windows legs as the literal
'${{ inputs.install-method }} -> ...' - GitHub does not evaluate
name expressions for natively skipped jobs. The caller's job name
already carries the method pair, so name the inner job statically.
Two structural changes to the combination fanout:
1. Tags become the OUTER axis, as a sub-graph per starting version:
install-e2e.yml fans a plain matrix over the picked tags into a new
per-tag reusable workflow (install-e2e-tag.yml), which runs the
combination generator for that one tag and fans out one job per
{os, install-method, update-method}. The Actions graph now reads
'from vX -> windows: install -> update' per leg. Nothing is
hardcoded in the workflows: the tag workflow calls the generator
itself.
2. Native skips move to the point that owns the capability knowledge:
macOS combos (no driving workflow exists) grey out in the tag
workflow via install-e2e-skip.yml, untouched by the tag axis; ALL
windows combos dispatch to install-e2e-windows-run.yml, which takes
install-method/update-method inputs and natively skips the pairs
its driver cannot run yet - so implementing a windows method is a
change in the run workflow + driver only. The driver's -Route ids
now match the generator's method ids verbatim.
Generator output shape, route filters, and all error paths re-verified
locally; all four workflows pass actionlint; driver re-parses clean
pure-ASCII.
Unimplemented combos previously ran as green echo jobs. Make each one
a real GitHub skip (grey, conclusion=skipped, no runner spent) while
keeping one check per combination: matrix context is not available in
job-level if, so the caller cannot natively skip individual legs -
instead each leg calls install-e2e-skip.yml, whose inner job is gated
on an 'implemented' input that defaults to false and is never passed.
Implementing a combo stays a generator-side move into IMPLEMENTED.
Replace the hand-enumerated update/installer/windows-desktop jobs with
a support-matrix generator (scripts/sandbox/generate-e2e-matrix.mjs).
The spec declares every {os, install-method, update-method} combination
a user could be on; generate-matrix expands it and fans out ONE JOB PER
COMBINATION:
* linux combos (curl-bash install x hermes-update/curl-bash rerun)
drive install-e2e-run.yml, still multiplied by the sampled release
tags from pick-releases;
* the windows combo (desktop-installer@latest -> desktop-app) drives
install-e2e-windows-run.yml - the real GUI flow;
* every declared-but-unimplemented combo (all of macOS, the remaining
windows methods) becomes its own visible skipped job, so the
coverage gap is enumerable from the Checks tab and implementing one
is a one-line move into IMPLEMENTED.
Strictness carried into the generator: method ids validate against a
closed set, installer 'versions' arrays only allow 'latest' until a
versioned archive exists, secondUpdate must stay empty until a chained
second-update leg is implemented, and unknown spec keys/routes throw.
Expansion, route filtering, empty-matrix gating, and all seven error
paths verified locally; the dispatch route choice keeps its exact
previous semantics (all/both/update/installer/windows-desktop).
Run 31523695265 went green but both phases logged '(ffmpeg not on
PATH; skipping screen recording)' and the artifact had no
recording.mkv. Restore the winget install + cache + PATH steps from
the retired axis (same pinned actions/cache SHA those green runs
used), scoped to just ffmpeg since AutoHotkey now comes from the
portable zip inside the driver.
Run 31520267702 died in 3s: 'Missing an argument for parameter
InstallRef'. powershell.exe -File drops a "" argument from the command
line entirely, so the parameter binder saw -InstallRef followed by
-SetupExeUrl. Default both the workflow input and the script parameter
to 'auto' (= newest release tag) instead of empty.
Run 31519103491 failed the 'update genuinely available' assert with
the installer landing on HEAD itself. The staging assumed the website
exe installs a baked release pin, but the bootstrap log shows
Pin { commit: None, branch: main } - the published installer installs
whatever main serves, and serve.git's main was parked at HEAD.
Stage the way the linux axis does: park served main at OLD
(-InstallRef, default newest release tag; threaded through the
reusable workflow as install-ref) for the install phase, assert the
install lands exactly there, then advance main to HEAD in the update
phase - an update becomes available the same way it does for a real
user. allowAnySHA1InWant stays as belt-and-braces for installer builds
that DO bake a pin.
Restructure tek's two-job desktop-windows-e2e.yml into the shape the
linux axis already has: install-e2e.yml keeps its update/installer
routes untouched and windows-desktop returns as a route in the same
family, calling a reusable install-e2e-windows-run.yml.
Behind that route is now ONLY the real user flow - the headless
contract job (install.ps1 at HEAD~1, desktop-update.ps1 -NoUi,
BASE/CURRENT/NEXT ref dance) is gone, along with its driver. Every leg
goes through a surface a user touches: website Hermes-Setup.exe headed
with AutoHotkey clicking Install -> Launch, then the installed
Hermes.exe under Playwright's Electron driver clicking Settings ->
About -> 'Update now', through the detached hand-off to a relaunched
window asserted on HEAD.
The driver drops the synthetic-NEXT staging with the contract job:
serve.git just serves HEAD as main and OLD is the release pin baked
into the website exe - the literal starting point of every real GUI
user, same philosophy as the linux axis's release-tag matrix. The
-Route parameter (desktop today) declares the future update mechanisms
as arms: 'update' (hermes update from the installed venv) and
'installer' (re-run the bootstrap exe) raise until implemented, so the
workflow surface is stable when they land.
Reverted before merge, same as tek's pre-merge validation commit on
the upstream PR: workflow_dispatch only works once the file exists on
the default branch, and this fork branch is where the integrated
workflow needs proving.
The cherry-picked desktop-windows-e2e.yml covers everything the
install-e2e-windows-run.yml axis did and more: the contract job drives
the same desktop-update.ps1 hand-off (plus a CURRENT->NEXT forward
leg), and the GUI job replaces AHK-only driving with the full real
user flow - website Hermes-Setup.exe, clicked Install/Launch, then
Playwright clicking Settings -> About -> 'Update now' in the packaged
app, through the detached hand-off to a relaunched window.
Remove the superseded workflow, its driver, and the AHK/button assets
under tests/install/windows/ (the GUI job's e2e-assets carry the
re-captured templates), and drop the windows-desktop route from
install-e2e.yml's dispatch options.
The real-user-flow job passed end to end (run 31492613931, 10m57s):
website Hermes-Setup.exe installed headed (AHK Install+Launch, real app
window), then TWO GUI updates driven by real Settings -> About ->
'Update now' clicks, each carried through the detached hand-off to a
relaunched desktop on the target commit. Every assertion green on both
legs (marker cleanup, checkout on target sha, working hermes, relaunch).
Two finishing touches:
* Foreground the relaunched Hermes window before the 99-relaunched proof
screenshot — the full-desktop grab is z-order dependent and one run
caught VS Code on top. The relaunch ASSERT already passed on the
process signal; this is purely to make the proof image show Hermes.
* Remove the temporary branch push trigger used for pre-merge validation;
back to main + nightly + release tags + manual dispatch only.
Attempt 9 drove the full real update through the hand-off: marker
detected, desktop exited, hermes update fetched from serve.git, found the
commit, pulled, restored -- all correct. It then timed out because the
updater legitimately runs LONG here: the website release we install
(v0.20.0) is weeks of main behind CURRENT, so the update pulls a large
diff AND does a full Electron desktop rebuild (vite + electron-builder)
plus uv sync. The contract job's BASE->CURRENT is a 1-commit tests-only
diff that skips the rebuild, which is why it finishes in ~1 min; the GUI
job's release->CURRENT does not.
* wait window 40 -> 90 min per leg; job timeout 180 -> 240 min
* tail logs/update.log during the wait so the desktop-rebuild phase is
visible in CI output instead of tens of minutes of silence (the rebuild
streams there, not to the handoff log)
The CURRENT->NEXT leg stays fast (NEXT is a same-tree child of CURRENT,
no rebuild), so total stays well within 240 min.
Second job on the Windows E2E workflow covering the surfaces a user
actually touches, per Teknium's requirement:
* INSTALL: downloads the production Hermes-Setup.exe from
hermes-assets.nousresearch.com, launches it HEADED, and AutoHotkey
clicks Install -> waits -> clicks Launch (button templates + ImageSearch
approach from @ethernet8023's #68183, retargeted by process name and
extended to exercise the Launch hand-off). The real Electron Hermes.exe
window must appear.
* UPDATE x2: the installed Hermes.exe is launched under Playwright's
Electron driver and the test CLICKS Settings -> About -> Update now.
The production hand-off chain runs untouched: app quits, detached
updater (repo script or staged binary) runs hermes update, rebuilds
the desktop, relaunches Hermes.exe. Asserts: target sha, marker
cleanup, result JSON when the script path wrote one, working hermes,
and the RELAUNCHED app window. Leg 1 -> CURRENT, leg 2 -> synthetic
NEXT.
Proof artifacts: per-step renderer screenshots (booted app, settings,
About panel, update-available, updating overlay), full-desktop frames
every 3s across the whole run, ahk.log, bootstrap-installer.log,
desktop-update-handoff.log — uploaded on success AND failure.
The website exe runs exactly as shipped (its own pinned install.ps1,
its baked release-pin commit); the only environmental deltas are the
serve.git URL redirect, uploadpack.allowAnySHA1InWant for the commit
pin fetch, and a placeholder provider key so the update legs meet the
app shell instead of onboarding.
The contract job from the previous commits is unchanged and independent
— it remains the rollback position if the GUI job proves flaky.
The Windows E2E ran end-to-end green on this branch (run 31462244593):
install at BASE, update BASE->CURRENT, update CURRENT->NEXT, all asserts
passing. Back to main/nightly/tags/dispatch triggers only.
The push-triggered validation run failed with zero jobs ('workflow file
issue'): job-level env only allows github/inputs/matrix/needs/secrets/
strategy/vars. Use a sibling of github.workspace for the E2E workroot
instead. actionlint now passes clean.
Every commit on main now proves, on a real Windows machine, that:
1. the PRIOR commit (HEAD~1) installs from scratch through its own
scripts/install.ps1 (-IncludeDesktop: uv, managed Python, Node,
venv, packaged Electron Hermes.exe),
2. that install updates TO this commit through the real Desktop GUI
update path (scripts/desktop-update.ps1, the exact hand-off the
Update button spawns -- fail-closed gates, marker lifecycle,
hermes update, result JSON), and
3. this commit updates FORWARD to a synthetic next commit, proving
the updater code shipping in this commit is not the one that
strands users when the next commit lands.
Staging: the driver bare-clones the checkout into serve.git and
redirects the canonical GitHub URLs at it with git insteadOf env
config, then advances the served main ref BASE -> CURRENT -> NEXT
between legs. Installer and updater run byte-for-byte unmodified.
Supersedes the AutoHotkey pixel-driving approach (#68183): the GUI
Update button's entire effect is spawning desktop-update.ps1 with
documented flags, so driving that contract directly tests the same
production code deterministically.
windows sibling of install-e2e-run.yml. no bubblewrap on windows, so the
git proxying is git's own transport rewrite: an isolated GIT_CONFIG_GLOBAL
with multi-valued url.<file://fake.git>.insteadOf for both hardcoded repo
URLs, so the published Hermes-Setup.exe's install.ps1 clone, hermes update's
fetch, and the desktop's ls-remote all land on a local bare repo whose main
the driver controls - installer and updater run verbatim.
one run: seed fake.git from the checkout, force fake main to the newest
release tag, drive the real published bootstrap installer with AutoHotkey
(GUI, no headless mode), promote fake main to HEAD, then apply the desktop
app's builtin update route (scripts/desktop-update.ps1 -NoUi when the
installed base ships it, staged hermes-setup.exe --update otherwise) and
assert HEAD == target with a working hermes.
TODO routes: bare hermes update, and re-running the bootstrap installer
over the existing checkout.
Each test slice uploads an artifact with the same file name,
test_durations.json. The save-durations job downloaded the 12
artifacts with merge-multiple, so all extractions wrote to one
path in parallel. This caused two faults:
- A race between two extractions wrote two JSON documents into
one file. The merge step then failed with 'JSONDecodeError:
Extra data' (run 31382130252).
- On green runs, the last write erased the other 11 slices. The
merged cache held ~230 of ~2760 file durations.
Remove merge-multiple so each artifact extracts into its own
directory, and point the glob at durations/*/test_durations.json.
A local merge of the 12 real artifacts from the failed run gives
2761 durations.
yoniebans