Commit Graph

12 Commits

Author SHA1 Message Date
ethernet 10f99bc15e ci: run the work lanes on larger runners and merge the split jobs
Every Linux lane that does real work ran on a 4-core `ubuntu-latest`. The
Python suite and the JS checks were split into many small jobs to make that
size usable. Each split job repeated the full setup. In most of the JS jobs
the repeated setup cost more than the work.

The work lanes move to larger runners. Then the splits that existed only to
make small runners usable go away.

Python tests: 12 slices become 1 job on a 96-core runner. Slicing cost a
matrix job, a duration cache, a per-slice artifact and a merge job. 96 cores
clear the floor that the slowest single test file sets, which is about 82s. A
second slice divides work that is already at that floor, and adds a second
setup. Duration data from run 32522943054 gives the numbers behind this: 3178
files, 11645s in series.

The worker count is explicit, because `run_tests.sh` defaults to twice the
core count. A later commit sets it from a measurement on this hardware.

JS checks: 14 jobs become 1. The matrix paid about 371s of repeated setup to
spread about 612s of work. One larger runner installs one time. The three UI
shard scripts and `run-ui-shard.mjs` are therefore removed, because the
unsharded `test:ui` covers the same tests.

The unit of parallel work inside that job is a CHECK, and not a workspace.
apps/desktop is most of the payload, and its own `check` is a serial && chain.
A spread across workspaces alone therefore leaves that chain as the long pole.
A package that declares `check:*` sub-scripts gives one unit for each
sub-script. That is the same selection rule the matrix used.

The loop lives in `.github/scripts/run-workspace-checks.mjs`, so the same
sequence runs on a laptop. It runs 11 units together, buffers the output of
each one, and fails at the end with the full list. Children that share one
stdout interleave their lines and make a failure hard to read.
`npm run --ws check` stops at the first workspace that fails.

`check:test:plugins` joins the desktop `check` script. The matrix prefers
`check:*` sub-scripts over the plain `check` script, so `check:test:plugins`
ran only as its own leg. Without this change the merge drops that suite and
the job stays green.

node_modules is cached on the lockfile, and `npm ci` is skipped on an exact
hit. The `cache: npm` option of `setup-node` caches only the ~/.npm tarball
cache, which leaves the extract and the postinstalls to pay again.

The arm64 image build stays on a native arm64 runner. A build of linux/arm64
on an x64 host uses emulation.

The docker test lane caps its workers at the core count. Each of those tests
drives a container, so the docker daemon sets the limit and not the processor.

`.github/actionlint.yaml` declares the runner labels. actionlint knows the
GitHub-hosted labels only, and an undeclared label reads as an error that
hides the real findings.

The `detect` job checks out one file through a sparse checkout, and its
timeout drops to 1 minute. It reads
`scripts/ci/classify_changes.py` and nothing else.

Verification:
- actionlint reports 9 findings across all workflows. An unmodified HEAD with
  the same config reports the same 9. This change adds none.
- A wrong label still fails. actionlint reports `ubuntu-latest-32-cor` and
  `ubuntu-latest-32-arm-cores`.
- Every changed workflow parses, and `name` parses as a string.
- A replay of the `save-durations` merge step against a three-artifact layout
  returns all 3178 entries.
- An expansion of the npm script graph gives the same leaf commands for the
  parallel units and for a plain `npm run check`, in both directions. Against
  the 13-leg matrix the count is 13 to 11, and the whole difference is the
  three UI shards that collapse into one unsharded `check:test:ui`.
- `--list` reports the 11 units, and a full local run completes and reports
  the time of each unit.
- The runner labels cannot be verified here. The first real run is the test.
2026-08-22 02:25:12 -04:00
ethernet 00c3872882 feat(ci): add nix flake check as unrequired job
The workflow owns its triggers and ci.yml does not call it. A
reusable-workflow call holds the caller run in progress for the full
build, and GitHub refuses `gh run rerun` on a run that is still in
progress. A separate run reruns and cancels on its own.

The job restores /nix/store from the GitHub Actions cache and saves from
main only. A cache that a PR writes is visible to that PR alone, so a
save there spends the quota of the repository and helps no later run.
2026-08-18 20:42:06 -04:00
ethernet 9eb0bcd60f change(ci): rip out nix ci for now
to be re-added later when we have more stable ci flows
2026-06-15 12:06:54 -04:00
ethernet a429a2a0bf ci(nix): fold package+devShell builds into flake check
Add build-package and build-devshell as cross-platform check
derivations so nix flake check verifies the default package and
devShell build on every platform (including darwin, which previously
only did eval-only checks).

This lets us drop the separate nix build step from the CI workflow
and removes the macOS-only eval fallback — a single nix flake check
now covers builds + runtime checks on all runners.
2026-06-02 19:14:18 -04:00
dependabot[bot] c4981167e6 chore(actions)(deps): bump actions/checkout from 4.3.1 to 6.0.2
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.3.1 to 6.0.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/34e114876b0b11c390a56381ad16ebd13914f8d5...de0fac2e4500dabe0009e67214ff5f5447ce83dd)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.2
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-19 03:27:54 -07:00
ethernet 2d3c041338 change(nix): dedupe nix lockfile checking scripts in ci (#18000)
* change(nix): dedupe nix lockfile checking scripts in ci

* feat(nix): make .#fix-lockfiles run --apply if no args passed

* fix(nix): use same nodejs version everywhere & small lints

- prevent lockfile thrashing while using nix :3
- use lib.getExe instead of raw /bin/ paths
- use inputs'.self instead of passing system in manually

* fix(nix): update lock files yet again (hopefully for the last time)

* fix(nix): align indentation of collision check echo

---------

Co-authored-by: Hermes Agent <hermes@nousresearch.com>
2026-04-30 22:52:30 +05:30
Siddharth Balyan 9a14540603 fix(nix): replace magic-nix-cache with Cachix (#17928)
* fix(nix): replace magic-nix-cache with Cachix

magic-nix-cache caused recurring CI failures (TwirpErrorResponse
ResourceExhausted) by hitting GitHub Actions Cache's 10 GB limit and
200 req/min rate limit. This was flagged as 'unfixable infra flake' in
#17836 but is actually a fixable architecture choice.

Switch to Cachix (dedicated binary cache, no GHA quota dependency):
- Replace DeterminateSystems/magic-nix-cache-action with cachix/cachix-action
- Add cachix-auth-token input to nix-setup composite action
- Pass CACHIX_AUTH_TOKEN secret through all three nix workflows
- continue-on-error: true so cache failures never block CI

Cache 'hermes-agent' is public at hermes-agent.cachix.org.
Devs can pull locally with: cachix use hermes-agent

* fix: correct cachix-action commit SHA pin

---------

Co-authored-by: Hermes Agent <hermes@nousresearch.com>
2026-04-30 17:38:58 +05:30
Ari Lotter 1d2615b602 dedupe nix cache 2026-04-20 16:52:57 -04:00
Ari Lotter 4dd6d6eeb4 nix: run CI on all lockfile changes 2026-04-20 16:17:15 -04:00
Teknium eed891f1bb security: supply chain hardening — CI pinning, dep pinning, and code fixes (#9801)
CI/CD Hardening:
- Pin all 12 GitHub Actions to full commit SHAs (was mutable @vN tags)
- Add explicit permissions: {contents: read} to 4 workflows
- Pin CI pip installs to exact versions (pyyaml==6.0.2, httpx==0.28.1)
- Extend supply-chain-audit.yml to scan workflow, Dockerfile, dependency
  manifest, and Actions version changes

Dependency Pinning:
- Pin git-based Python deps to commit SHAs (atroposlib, tinker, yc-bench)
- Pin WhatsApp Baileys from mutable branch to commit SHA

Tool Registry:
- Reject tool name shadowing from different tool families (plugins/MCP
  cannot overwrite built-in tools). MCP-to-MCP overwrites still allowed.

MCP Security:
- Add tool description content scanning for prompt injection patterns
- Log detailed change diff on dynamic tool refresh at WARNING level

Skill Manager:
- Fix dangerous verdict bug: agent-created skills with dangerous
  findings were silently allowed (ask->None->allow). Now blocked.
2026-04-14 14:23:37 -07:00
Dilek dbc11abcb6 fix(ci): pin floating GitHub Actions tags and ascii-guard to explicit versions (#3982)
* fix(ci): pin floating GitHub Actions tags and ascii-guard to explicit versions

Actions pinned to @main pull whatever is at that ref at execution time,
so a compromised upstream org could execute arbitrary code in CI.

- Pin DeterminateSystems/nix-installer-action to commit SHA (v22)
- Pin DeterminateSystems/magic-nix-cache-action to commit SHA (v13)
- Pin ascii-guard to 2.3.0 in docs-site-checks workflow

SHA comments include the version tag for human readability; Renovate or
Dependabot can keep these updated automatically.

* Add skill metadata extraction step in workflow

Add step to extract skill metadata for dashboard in CI workflow.

---------

Co-authored-by: Siddharth Balyan <52913345+alt-glitch@users.noreply.github.com>
2026-04-09 21:27:20 +05:30
Siddharth Balyan b6461903ff feat: nix flake — uv2nix build, NixOS module, persistent container mode (#20)
* feat: nix flake, uv2nix build, dev shell and home manager

* fixed nix run, updated docs for setup

* feat(nix): NixOS module with persistent container mode, managed guards, checks

- Replace homeModules.nix with nixosModules.nix (two deployment modes)
- Mode A (native): hardened systemd service with ProtectSystem=strict
- Mode B (container): persistent Ubuntu container with /nix/store bind-mount,
  identity-hash-based recreation, GC root protection, symlink-based updates
- Add HERMES_MANAGED guards blocking CLI config mutation (config set, setup,
  gateway install/uninstall) when running under NixOS module
- Add nix/checks.nix with build-time verification (binary, CLI, managed guard)
- Remove container.nix (no Nix-built OCI image; pulls ubuntu:24.04 at runtime)
- Simplify packages.nix (drop fetchFromGitHub submodules, PYTHONPATH wrappers)
- Rewrite docs/nixos-setup.md with full options reference, container
  architecture, secrets management, and troubleshooting guide

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update config.py

* feat(nix): add CI workflow and enhanced build checks
- GitHub Actions workflow for nix flake check + build on linux/macOS
- Entry point sync check to catch pyproject.toml drift
- Expanded managed-guard check to cover config edit
- Wrap hermes-acp binary in Nix package
- Fix Path type mismatch in is_managed()

* Update MCP server package name; bundled skills support

* fix reading .env. instead have container user a common mounted .env file

* feat(nix): container entrypoint with privilege drop and sudo provisioning

Container was running as non-root via --user, which broke apt/pip installs
and caused crashes when $HOME didn't exist. Replace --user with a Nix-built
entrypoint script that provisions the hermes user, sudo (NOPASSWD), and
/home/hermes inside the container on first boot, then drops privileges via
setpriv. Writable layer persists so setup only runs once.

Also expands MCP server options to support HTTP transport and sampling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix group and user creation in container mode

* feat(nix): persistent /home/hermes and MESSAGING_CWD in container mode

Container mode now bind-mounts ${stateDir}/home to /home/hermes so the
agent's home directory survives container recreation. Previously it lived
in the writable layer and was lost on image/volume/options changes.

Also passes MESSAGING_CWD to the container so the agent finds its
workspace and documents, matching native mode behavior.

Other changes:
- Extract containerDataDir/containerHomeDir bindings (no more magic strings)
- Fix entrypoint chown to run unconditionally (volume mounts always exist)
- Add schema field to container identity hash for auto-recreation
- Add idempotency test (Scenario G) to config-roundtrip check

* docs: add Nix & NixOS setup guide to docs site

Add comprehensive Nix documentation to the Docusaurus site at
website/docs/getting-started/nix-setup.md, covering nix run/profile
install, NixOS module (native + container modes), declarative settings,
secrets management, MCP servers, managed mode, container architecture,
dev shell, flake checks, and full options reference.

- Register nix-setup in sidebar after installation page
- Add Nix callout tip to installation.md linking to new guide
- Add canonical version pointer in docs/nixos-setup.md

* docs: remove docs/nixos-setup.md, consolidate into website docs

Backfill missing details (restart/restartSec in full example,
gateway.pid, 0750 permissions, docker inspect commands) into
the canonical website/docs/getting-started/nix-setup.md and
delete the old standalone file.

* fix(nix): add compression.protect_last_n and target_ratio to config-keys.json

New keys were added to DEFAULT_CONFIG on main, causing the
config-drift check to fail in CI.

* fix(nix): skip checks on aarch64-darwin (onnxruntime wheel missing)

The full Python venv includes onnxruntime (via faster-whisper/STT)
which lacks a compatible uv2nix wheel on aarch64-darwin. Gate all
checks behind stdenv.hostPlatform.isLinux. The package and devShell
still evaluate on macOS.

* fix(nix): skip flake check and build on macOS CI

onnxruntime (transitive dep via faster-whisper) lacks a compatible
uv2nix wheel on aarch64-darwin. Run full checks and build on Linux
only; macOS CI verifies the flake evaluates without building.

* fix(nix): preserve container writable layer across nixos-rebuild

The container identity hash included the entrypoint's Nix store path,
which changes on every nixpkgs update (due to runtimeShell/stdenv
input-addressing). This caused false-positive identity mismatches,
triggering container recreation and losing the persistent writable layer.

- Use stable symlink (current-entrypoint) like current-package already does
- Remove entrypoint from identity hash (only image/volumes/options matter)
- Add GC root for entrypoint so nix-collect-garbage doesn't break it
- Remove global HERMES_HOME env var from addToSystemPackages (conflicted
  with interactive CLI use, service already sets its own)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 01:08:02 +05:30