From 0d8ac4f24b738c1797a951b6b4f00b3d870037c9 Mon Sep 17 00:00:00 2001 From: dinos Date: Fri, 1 May 2026 13:24:14 +0200 Subject: [PATCH] feat(docker): official image with all runtime deps pre-installed (#198) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(docker): official image with all runtime deps pre-installed Multi-stage build using uv for the EvoScientist core + all messaging-channel extras, plus Node.js 24 LTS (for npx-based MCP servers) and uv (for runtime Python MCP installs) in the runtime layer. Runs as non-root user evosci, with workspace, app data, and config (XDG_CONFIG_HOME) all consolidated under a single /home/evosci/.evoscientist volume so a single mount persists everything across container restarts. Includes a docker-compose.yml starter, a build/push GitHub Actions workflow targeting ghcr.io with multi-arch (amd64/arm64) and PR-only build verification, a .dockerignore, and a new Docker section in the README documenting mounts, derivation recipes for the unbundled stt / oauth / TinyTeX extras, and proxy/cert handling expectations. * fix(docker): pin trixie base + drop redundant python image Switch builder and runtime from `python:3.11-slim-bookworm` to a single `ghcr.io/astral-sh/uv:python3.11-trixie-slim` base — trixie drops several CRITICAL vulnerabilities that bookworm carries today, and reusing the uv image for runtime eliminates the separate `COPY --from=…/uv` line. * chore(docker): pin GitHub Actions to commit SHAs in workflow Replace mutable major-version tags with full commit SHAs (with the corresponding semver tag in a trailing comment) so a compromised / retagged action release can't silently change what runs in the publish pipeline. * chore(deps): enable Dependabot version updates for Dockerfile pins Adds a weekly `docker` ecosystem that watches the Dockerfile's `FROM` / `COPY --from=` references — including the ARG-bound `BASE_IMAGE` and `NODE_IMAGE` digests — and opens one grouped PR per cadence bumping both the @sha256 digest and the trailing version comment. This keeps the otherwise-frozen pins flowing with Debian point releases and upstream patches. * fix(docker): use nodejs alias stage so NODE_IMAGE ARG actually resolves `COPY --from=${NODE_IMAGE}` left the dollar-curly literal at parse time under buildkit 29.x — it expands ARGs in `FROM` but reads `--from=` as a static stage/image name. Introduce a tiny `FROM ${NODE_IMAGE} AS nodejs` alias and `COPY --from=nodejs …` against it, which preserves the ARG-driven Dependabot updates without tripping the parser. * fix(docker): harden venv ownership and PATH ordering - Drop `--chown` on the `/opt/venv` COPY so the venv stays root-owned. The runtime user only needs read+execute (default Unix perms allow that); making it user-owned let the agent rewrite its own dependencies, which defeats the sandboxing premise. All persistent agent state already lives under /home/evosci/.evoscientist/. - Reorder PATH so /opt/venv/bin precedes the user-writable UV_TOOL_BIN_DIR. Otherwise a stray binary dropped into the latter (e.g. via `uv tool install`) could shadow the canonical `evosci` / `python` / `pip` shipped with the image. * docs: update README * docs(docker): warn about non-root UID and `curl | sh` for derived images - The image runs as `evosci` (UID 1000), so a host-side `./workspace` bind mount fails if the host user has a different UID — same gotcha that bites onboarding's `mcp.yaml` write. Add an !IMPORTANT block with the two practical fixes (`chown -R 1000:1000` once, or `--user "$(id -u):$(id -g)"` on each run). - The TinyTeX derivation snippet pipes an unpinned remote installer into `sh`. Add a one-line pointer to fetching a pinned release tarball from `rstudio/tinytex-releases` for users who'd rather not trust the upstream script blindly. The official installer is kept as the default since that's what TinyTeX itself recommends. * chore(docker): cancel in-flight workflow runs + flag iMessage as host-only - Add `concurrency: cancel-in-progress: true` to the docker workflow so successive pushes on the same ref supersede the prior run rather than queueing in parallel — multi-arch buildx is the slowest job in CI, no point burning minutes on superseded builds. - Spell out that the docker image installs the `all-chanels` extra and call out iMessage as a deliberate host-only exclusion: it requires the `imsg` CLI bridging to macOS's Messages.app, which no Linux container config can satisfy. --- .dockerignore | 65 +++++++++++++++++++++++++++++++ .github/dependabot.yml | 20 ++++++++++ .github/workflows/docker.yml | 67 ++++++++++++++++++++++++++++++++ Dockerfile | 75 ++++++++++++++++++++++++++++++++++++ README.md | 67 +++++++++++++++++++++++++++++++- docker-compose.yml | 20 ++++++++++ 6 files changed, 313 insertions(+), 1 deletion(-) create mode 100644 .dockerignore create mode 100644 .github/dependabot.yml create mode 100644 .github/workflows/docker.yml create mode 100644 Dockerfile create mode 100644 docker-compose.yml diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..f90b14d --- /dev/null +++ b/.dockerignore @@ -0,0 +1,65 @@ +# VCS +.git +.gitignore +.gitattributes + +# CI / project meta (image doesn't need these) +.github/ +docs/ +CONTRIBUTING.md +README.zh-CN.md +LICENSE + +# Editor / tooling state +.vscode/ +.idea/ +.cursor/ +.codex/ +.claude/ +.agents/ +.cursorrules +.ruff_cache/ + +# Python build artifacts and caches +__pycache__/ +*.py[cod] +*.egg-info/ +*.egg +build/ +dist/ +.venv/ +venv/ +.pytest_cache/ +.ruff_cache/ +.coverage + +# Tests aren't needed at runtime +tests/ + +# Notebooks +*.ipynb +.ipynb_checkpoints/ + +# Local runtime data (must never leak into the image) +.env +.env.* +!.env.example +runs/ +workspace/ +skills/ +memory/ +memories/ +media/ +conversation_history/ +.deno_cache/ +.langgraph_api/ +large_tool_results/ +*.log +botpy.log + +# Docker outputs themselves +Dockerfile.* +docker-compose*.override.yml + +# OS +.DS_Store diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..cad9f37 --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,20 @@ +version: 2 +updates: + # Base images in Dockerfile (BASE_IMAGE / NODE_IMAGE ARG defaults). + # Dependabot reads `FROM`, `COPY --from=`, and ARG-bound base refs, and + # bumps both the @sha256 digest and the trailing # vX.Y.Z comment. + - package-ecosystem: "docker" + directory: "/" + schedule: + interval: "weekly" + open-pull-requests-limit: 5 + commit-message: + prefix: "chore(docker)" + labels: + - "dependencies" + - "docker" + # Single PR per cadence rather than one per image — keeps reviewer load low + # and lets us validate trixie/uv/node bumps as a coherent set. + groups: + base-images: + patterns: ["*"] diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml new file mode 100644 index 0000000..7de8f67 --- /dev/null +++ b/.github/workflows/docker.yml @@ -0,0 +1,67 @@ +name: Docker + +on: + push: + branches: ["main"] + tags: ["v*"] + pull_request: + paths: + - "Dockerfile" + - ".dockerignore" + - "pyproject.toml" + - "uv.lock" + - "EvoScientist/**" + - ".github/workflows/docker.yml" + workflow_dispatch: + +concurrency: + group: docker-${{ github.ref }} + cancel-in-progress: true + +env: + REGISTRY: ghcr.io + IMAGE_NAME: ${{ github.repository }} + +jobs: + build: + runs-on: ubuntu-latest + timeout-minutes: 45 + permissions: + contents: read + packages: write + steps: + - uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1 + + - uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3.7.0 + - uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0 + + - name: Log in to ${{ env.REGISTRY }} + if: github.event_name != 'pull_request' + uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0 + with: + registry: ${{ env.REGISTRY }} + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Extract image metadata + id: meta + uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0 + with: + images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} + tags: | + type=ref,event=branch + type=ref,event=pr + type=semver,pattern={{version}} + type=semver,pattern={{major}}.{{minor}} + type=raw,value=latest,enable={{is_default_branch}} + + - name: Build and push + uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2 + with: + context: . + platforms: linux/amd64,linux/arm64 + push: ${{ github.event_name != 'pull_request' }} + tags: ${{ steps.meta.outputs.tags }} + labels: ${{ steps.meta.outputs.labels }} + cache-from: type=gha + cache-to: type=gha,mode=max diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..7357d3e --- /dev/null +++ b/Dockerfile @@ -0,0 +1,75 @@ +# syntax=docker/dockerfile:1.7 + +ARG BASE_IMAGE=ghcr.io/astral-sh/uv:python3.11-trixie-slim@sha256:7936cc6625ca04cafa6ecc3c2881ddfe90a747c55c74480cd4ac6ffad6a5af1e +ARG NODE_IMAGE=node:24-trixie-slim@sha256:735dd688da64d22ebd9dd374b3e7e5a874635668fd2a6ec20ca1f99264294086 + +FROM ${NODE_IMAGE} AS nodejs + +# ---------- Builder ---------- +FROM ${BASE_IMAGE} AS builder + +ENV UV_COMPILE_BYTECODE=1 \ + UV_LINK_MODE=copy \ + UV_PYTHON_DOWNLOADS=never \ + UV_PROJECT_ENVIRONMENT=/opt/venv + +WORKDIR /src + +COPY pyproject.toml uv.lock README.md ./ +RUN --mount=type=cache,target=/root/.cache/uv \ + uv sync --frozen --no-install-project --no-dev \ + --extra all-channels + +COPY EvoScientist ./EvoScientist +RUN --mount=type=cache,target=/root/.cache/uv \ + uv sync --frozen --no-dev --no-editable \ + --extra all-channels + +# ---------- Runtime ---------- +FROM ${BASE_IMAGE} AS runtime + +RUN apt-get update \ + && apt-get install -y --no-install-recommends \ + git \ + ca-certificates \ + tini \ + curl \ + && rm -rf /var/lib/apt/lists/* + +COPY --from=nodejs /usr/local/bin/node /usr/local/bin/node +COPY --from=nodejs /usr/local/lib/node_modules /usr/local/lib/node_modules +RUN ln -sf /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm \ + && ln -sf /usr/local/lib/node_modules/npm/bin/npx-cli.js /usr/local/bin/npx + +ARG UID=1000 +ARG GID=1000 +RUN groupadd --gid ${GID} evosci \ + && useradd --uid ${UID} --gid ${GID} --create-home --shell /bin/bash evosci + +COPY --from=builder /opt/venv /opt/venv + +ENV PATH="/opt/venv/bin:/home/evosci/.evoscientist/.local/bin:${PATH}" \ + PYTHONUNBUFFERED=1 \ + PYTHONDONTWRITEBYTECODE=1 \ + EVOSCIENTIST_WORKSPACE_DIR=/workspace \ + EVOSCIENTIST_DATA_DIR=/home/evosci/.evoscientist \ + XDG_CONFIG_HOME=/home/evosci/.evoscientist/.config \ + UV_TOOL_DIR=/home/evosci/.evoscientist/.local/share/uv/tools \ + UV_TOOL_BIN_DIR=/home/evosci/.evoscientist/.local/bin + +RUN mkdir -p /workspace \ + /home/evosci/.evoscientist/.config/evoscientist \ + /home/evosci/.evoscientist/.local/bin \ + /home/evosci/.evoscientist/.local/share/uv/tools \ + && chown -R ${UID}:${GID} /workspace /home/evosci + +USER evosci +WORKDIR /workspace + +LABEL org.opencontainers.image.title="EvoScientist" \ + org.opencontainers.image.description="EvoScientist agent with core + all-channels dependencies pre-installed." \ + org.opencontainers.image.source="https://github.com/EvoScientist/EvoScientist" \ + org.opencontainers.image.documentation="https://github.com/EvoScientist/EvoScientist#-docker" \ + org.opencontainers.image.licenses="Apache-2.0" + +ENTRYPOINT ["tini", "--", "evosci"] diff --git a/README.md b/README.md index 767b8d5..3e3d4db 100644 --- a/README.md +++ b/README.md @@ -148,7 +148,7 @@ Moving beyond traditional human-in-the-loop systems, EvoScientist adopts a human ## 📦 Installation > [!TIP] -> Requires **Python 3.11+** (**< 3.14**). We recommend [**uv**](https://docs.astral.sh/uv/) or **conda** for dependency management and virtual environments. +> Requires **Python 3.11+** (**< 3.14**). We recommend [**uv**](https://docs.astral.sh/uv/) or **conda** for dependency management and virtual environments. Prefer to skip a local Python install entirely? Jump to [🐳 Docker](#-docker).
🪛 Install uv (if you don't have it) @@ -245,6 +245,71 @@ git pull && uv sync --dev
+### 🐳 Docker + +A pre-built image is published to [GitHub Container Registry](https://github.com/EvoScientist/EvoScientist/pkgs/container/evoscientist) with everything `evosci onboard` would otherwise install for you: + +- Python 3.11, EvoScientist, and the cross-platform messaging channels (i.e., `EvoScientist[all-channels]`) +- **`uv`** — used by the MCP registry to install Python MCP servers on demand +- **Node.js 24 LTS + `npx`** — required by the majority of MCP servers + +The **iMessage** channel isn't usable from the container — it requires the `imsg` CLI talking to macOS's Messages.app, which is host-OS-specific. Run EvoScientist directly on macOS if you need iMessage. + +Running EvoScientist in a container also **sandboxes the agent's shell access** — file edits and shell commands stay confined to volumes you explicitly mount. + +```bash +docker run -it --rm \ + --env-file .env \ + -v "$(pwd)/workspace:/workspace" \ + -v evosci-data:/home/evosci/.evoscientist \ + ghcr.io/evoscientist/evoscientist:latest +``` + +What the mounts are for: + +| Mount | Purpose | +| --- | --- | +| `--env-file .env` | API keys (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, …) | +| `./workspace:/workspace` | The agent's working directory | +| `evosci-data:/home/evosci/.evoscientist` | Persistent app state: sessions DB, global skills, memories, and `config.yaml`/`mcp.yaml` | + +> [!IMPORTANT] +> The image runs as a non-root user (`evosci`, UID `1000`). For the `./workspace` bind mount, the host directory must be writable by that UID. If your host user ID differs, either `chown -R 1000:1000 ./workspace` once, or pass `--user "$(id -u):$(id -g)"` on every `docker run` so the container takes on your UID. + +Or use `docker compose` (a starter [`docker-compose.yml`](./docker-compose.yml) is included): + +```bash +docker compose run --rm evoscientist +``` + +To build the image locally instead of pulling: + +```bash +docker build -t evoscientist:dev . +``` + +> [!NOTE] +> Not bundled — install on demand by deriving from the image: +> - **`stt`** (speech-to-text via `faster-whisper`) and **`oauth`** (`ccproxy-api`) +> - **TinyTeX / LaTeX** (`pdflatex`, `latexmk`) for paper-writing skills +> +> ```dockerfile +> FROM ghcr.io/evoscientist/evoscientist:latest +> +> # Python extras +> USER root +> RUN uv pip install --python /opt/venv/bin/python "EvoScientist[stt,oauth]" +> USER evosci +> +> # TinyTeX +> # The official install method is `curl | sh`; if you'd rather not +> # pipe an unpinned remote script into a shell, fetch a specific TinyTeX +> # release tarball from https://github.com/rstudio/tinytex-releases, verify +> # its checksum, and extract to /home/evosci/.TinyTeX instead. +> RUN curl -sL https://yihui.org/tinytex/install-bin-unix.sh | sh \ +> && /home/evosci/.TinyTeX/bin/*/tlmgr install latexmk +> ``` +

🔝Back to top

## 🔑 Configuration diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..dbaa060 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,20 @@ +services: + evoscientist: + image: ghcr.io/evoscientist/evoscientist:latest + # To build the image from this checkout instead of pulling, comment out + # the `image:` line above and uncomment the block below: + # build: + # context: . + container_name: evoscientist + stdin_open: true + tty: true + env_file: + - .env + volumes: + # Working directory for the agent + - ./workspace:/workspace + # Persistent app data: sessions, global skills, memories, config + - evosci-data:/home/evosci/.evoscientist + +volumes: + evosci-data: