9 Commits

Author SHA1 Message Date
houren Antony d4f1fbd110 ci: add windows-latest to test matrix + fix 11 cross-platform test bugs (#271)
* ci: add windows-latest to test matrix + fix 11 cross-platform test bugs

The test workflow ran on ``ubuntu-latest`` only. Per the issue's
first bullet — the maintainer's explicit #1 priority — add
``windows-latest`` to the matrix so the manager and related
modules are exercised on Windows on every PR.

The matrix addition surfaces 18 pre-existing Windows-only test
failures. Without fixes the new leg would be 18+ reds from
day one and the matrix would just produce a wall of
``fail-fast`` noise. This PR fixes 11 of them; each fix is
a real (cross-platform) bug, not a Windows-specific hack —
most were already flagged by CodeRabbit on PR #236 but never
acted on. The remaining 4 failures need code refactors
(``os.killpg`` → ``psutil`` in ``background.py``,
``convert_virtual_paths_in_command`` Windows-aware quoting,
tilde expansion) that are documented as out-of-scope
follow-ups below.

## What changed

* ``.github/workflows/test.yml``
  - ``os: [ubuntu-latest, windows-latest]`` → 2 OS × 2 Python
    = 4 cells.
  - ``fail-fast: false`` so one bad cell doesn't cancel the
    rest while the Windows leg is being brought up. Removable
    in a future PR once the suite is fully green.

* ``tests/test_backends.py``
  - Hard-coded ``"python3"`` → ``{sys.executable}`` in 7
    test commands. Windows has no ``python3`` on PATH; using
    ``sys.executable`` is portable and matches what CodeRabbit
    flagged on PR #236.
  - Strict string comparisons → ``shlex.split`` round-trip in
    5 resolver tests. ``shlex.quote`` adds single quotes
    around backslash paths on Windows, which broke the
    direct ``==`` compare.
  - Cross-platform suffix checks in 2 path-resolution tests
    (``Path(resolved).parts[-2:]`` instead of
    ``str(resolved).endswith("src/main.py")``).
  - ``mkdir -p`` → ``sys.executable -c "import os;
    os.makedirs(...)"`` in the cwd-sanitization test.
  - ``skipif(sys.platform == "win32")`` on 3 e2e tests that
    hit the underlying ``shlex.quote`` + ``cmd.exe`` quoting
    bug (real, separate issue).

* ``tests/test_sessions.py``
  - ``test_uses_data_dir``: check ``.evoscientist`` in the
    long path form (via ``Path.resolve()``) rather than the
    short-path form ``get_db_path`` returns on Windows.

* ``tests/test_mcp_client.py``
  - ``endswith("python")`` → ``Path(result).stem.lower()`` so
    ``python.EXE`` matches on Windows.
  - ``endswith("npx")`` also accepts ``npx.cmd`` so the npm
    shim on Windows matches.

## Out of scope (follow-up issues to file)

* ``os.killpg`` doesn't exist on Windows
  (``EvoScientist/background.py:248``) — 3 background tests
  fail. Real fix is the same ``psutil`` walk pattern PR #200
  shipped in ``langgraph_dev/manager.py``.
* Tilde expansion in file mentions.
* Windows-aware shell quoting in
  ``convert_virtual_paths_in_command``.
* Path conventions (``~/.config/evoscientist/`` vs
  ``%APPDATA%\EvoScientist``) — needs design discussion +
  ``platformdirs`` migration.
* Cross-module audit of
  ``EvoScientist/tools/execute.py``,
  ``EvoScientist/ccproxy_manager.py``,
  ``EvoScientist/config/onboard.py``.

Closes #207 (step 1 only — CI matrix + the easy test
fixes; remaining bullets tracked separately).

* fix: cross-platform compatibility for Windows CI runners

- background.py: replace POSIX-only os.killpg/os.getpgid with
  cross-platform _kill_process_tree() helper. On Windows falls back
  to Popen.terminate()/Popen.kill() (TerminateProcess); on POSIX
  keeps existing os.killpg logic.

- test_backends.py: replace mkdir -p shell execution in
  test_literal_workspace_path_replaced with preprocessing-boundary
  assertion (patch LocalShellBackend.execute, capture command,
  assert workspace path was rewritten to ./). Avoids POSIX-only
  mkdir -p on Windows runners.

- test_file_mentions.py: monkeypatch USERPROFILE on Windows so
  ntpath.expanduser() resolves ~ to tmp_path even when HOME is
  unset on CI runners.

* fix(test): cross-platform sleep/true commands for Windows CI

Replace POSIX-only sleep/true with module-level helpers that use
ping -n / cmd /c on Windows. Also fix python3 -> sys.executable
in the non-timeout recovery test.

- test_background.py: 7 sleep/true fixes
- test_background_middleware.py: 6 sleep/true fixes
- test_backends.py: 4 sleep fixes + 1 python3 fix

2318 passed, 0 failed on Windows.

* fix(test): use shell-portable double quotes for python -c on Windows

cmd.exe does not treat single quotes as string delimiters, so
-c 'raise SystemExit(1)' was passed with literal quotes on Windows.
Switch to double quotes which work on both cmd.exe and POSIX sh.

* fix: use psutil for Windows process tree kill + avoid sys.executable under uv

- background.py: replace Popen.terminate()/kill() with psutil-based
  process tree walking on Windows. TerminateProcess does NOT cascade
  to grandchildren; psutil.Process.children(recursive=True) ensures
  the entire tree is signaled.

- test_backends.py: replace sys.executable with 'python' in sandbox
  execute() calls. Under uv, sys.executable is under the workspace
  and gets rewritten to ./ by prepare_sandbox_command, breaking
  Linux CI. The plain 'python' command resolves correctly in any
  activated venv.

* fix: broaden try/except in _kill_process_tree to cover proc.children()

If the process exits between Process(popen.pid) and children(recursive=True),
the children call raises an uncaught exception escaping stop(). Move it inside
the existing try/except block.

* fix: narrow exception to ProcessLookupError in POSIX _kill_process_tree

OSError is too broad — would silently swallow EPERM on SIGKILL, leaving
the process alive when we report it as stopped. Match original behavior
which only caught ProcessLookupError (process already gone).

* style: ruff format test_backends.py

* ci: trigger re-run for flaky prompt_toolkit test

* style: fix ruff check (import order + RUF005 unpacking)

---------

Co-authored-by: Xi Zhang <106144707+X-iZhang@users.noreply.github.com>
2026-06-10 15:43:18 +01:00
dinos 0d8ac4f24b feat(docker): official image with all runtime deps pre-installed (#198)
* feat(docker): official image with all runtime deps pre-installed

Multi-stage build using uv for the EvoScientist core + all messaging-channel
extras, plus Node.js 24 LTS (for npx-based MCP servers) and uv (for runtime
Python MCP installs) in the runtime layer. Runs as non-root user evosci,
with workspace, app data, and config (XDG_CONFIG_HOME) all consolidated
under a single /home/evosci/.evoscientist volume so a single mount
persists everything across container restarts.

Includes a docker-compose.yml starter, a build/push GitHub Actions
workflow targeting ghcr.io with multi-arch (amd64/arm64) and PR-only
build verification, a .dockerignore, and a new Docker section in the
README documenting mounts, derivation recipes for the unbundled stt /
oauth / TinyTeX extras, and proxy/cert handling expectations.

* fix(docker): pin trixie base + drop redundant python image

Switch builder and runtime from `python:3.11-slim-bookworm` to a single
`ghcr.io/astral-sh/uv:python3.11-trixie-slim` base — trixie drops several
CRITICAL vulnerabilities that bookworm carries today, and reusing the uv
image for runtime eliminates the separate `COPY --from=…/uv` line.

* chore(docker): pin GitHub Actions to commit SHAs in workflow

Replace mutable major-version tags with full commit SHAs (with the
corresponding semver tag in a trailing comment) so a compromised /
retagged action release can't silently change what runs in the publish
pipeline.

* chore(deps): enable Dependabot version updates for Dockerfile pins

Adds a weekly `docker` ecosystem that watches the Dockerfile's `FROM` /
`COPY --from=` references — including the ARG-bound `BASE_IMAGE` and
`NODE_IMAGE` digests — and opens one grouped PR per cadence bumping
both the @sha256 digest and the trailing version comment. This keeps
the otherwise-frozen pins flowing with Debian point releases and
upstream patches.

* fix(docker): use nodejs alias stage so NODE_IMAGE ARG actually resolves

`COPY --from=${NODE_IMAGE}` left the dollar-curly literal at parse time
under buildkit 29.x — it expands ARGs in `FROM` but reads `--from=` as a
static stage/image name. Introduce a tiny `FROM ${NODE_IMAGE} AS nodejs`
alias and `COPY --from=nodejs …` against it, which preserves the
ARG-driven Dependabot updates without tripping the parser.

* fix(docker): harden venv ownership and PATH ordering

- Drop `--chown` on the `/opt/venv` COPY so the venv stays root-owned.
  The runtime user only needs read+execute (default Unix perms allow
  that); making it user-owned let the agent rewrite its own
  dependencies, which defeats the sandboxing premise. All persistent
  agent state already lives under /home/evosci/.evoscientist/.
- Reorder PATH so /opt/venv/bin precedes the user-writable
  UV_TOOL_BIN_DIR. Otherwise a stray binary dropped into the latter
  (e.g. via `uv tool install`) could shadow the canonical
  `evosci` / `python` / `pip` shipped with the image.

* docs: update README

* docs(docker): warn about non-root UID and `curl | sh` for derived images

- The image runs as `evosci` (UID 1000), so a host-side `./workspace`
  bind mount fails if the host user has a different UID — same gotcha
  that bites onboarding's `mcp.yaml` write. Add an !IMPORTANT block
  with the two practical fixes (`chown -R 1000:1000` once, or
  `--user "$(id -u):$(id -g)"` on each run).
- The TinyTeX derivation snippet pipes an unpinned remote installer
  into `sh`. Add a one-line pointer to fetching a pinned release
  tarball from `rstudio/tinytex-releases` for users who'd rather not
  trust the upstream script blindly. The official installer is kept
  as the default since that's what TinyTeX itself recommends.

* chore(docker): cancel in-flight workflow runs + flag iMessage as host-only

- Add `concurrency: cancel-in-progress: true` to the docker workflow
  so successive pushes on the same ref supersede the prior run rather
  than queueing in parallel — multi-arch buildx is the slowest job in
  CI, no point burning minutes on superseded builds.
- Spell out that the docker image installs the `all-chanels` extra and
  call out iMessage as a deliberate host-only exclusion: it requires
  the `imsg` CLI bridging to macOS's Messages.app, which no Linux
  container config can satisfy.
2026-05-01 13:24:14 +02:00
Jan Piotrowski 4a3d6c0318 chore: add ruff lint rules and turn on formatting 2026-03-19 17:04:02 +01:00
X-iZhang d226aabe01 update GitHub Actions workflows to use latest action versions 2026-03-12 02:47:51 +00:00
X-iZhang c96d984869 feat(workflows): add cache-dependency-glob for improved dependency caching in build, lint, and test workflows 2026-03-03 01:20:06 +00:00
Dinos Papakostas 6faad4bb98 chore(workflows): update gh workflows to use uv 2026-02-24 20:06:19 +08:00
X-iZhang 7dedf2b426 fix: improve cancellation handling in middleware and mixins for Python 3.12+ 2026-02-22 16:20:55 +00:00
X-iZhang 03f4134190 feat: add UserMessage widget and UI backend selection during onboarding
- Introduced UserMessage widget for displaying user input with a styled prompt.
- Updated onboarding steps to include UI backend selection (Rich CLI or Textual TUI).
- Modified EvoScientistConfig to store selected UI backend.
- Enhanced configuration handling to support UI backend environment variable.
- Updated README with new UI backend options and commands.
- Added tests for new UI backend functionality and UserMessage widget.
- Removed obsolete test files and ensured existing tests are updated accordingly.
2026-02-21 01:18:33 +00:00
X-iZhang e725fb08e2 add CI workflows 2026-02-03 22:24:32 +00:00