docs(webui): implementation plan for workspace selective download
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
m4
@@ -0,0 +1,965 @@
|
||||
# Workspace Selective Download Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add tri-state checkbox multi-selection to the workspace Tree view and download the selected files/folders as one zip via an extended `/api/workspace/download` endpoint.
|
||||
|
||||
**Architecture:** Client holds a collapsed `checked` set + an `excluded` set (pure module, no React) so folders can be checked without loading their subtrees. The existing download route gains repeated `include`/`exclude` query params; includes replace the `.` argument to the spawned `zip`, excludes become extra `-x` patterns (3 per exclude: `rel`, `rel/`, `rel/*`), with glob metacharacters backslash-escaped. A server-side pre-walk rejects selections that would produce a files-empty archive (zip exits 0 with a dirs-only archive in that case — verified experimentally).
|
||||
|
||||
**Tech Stack:** Next.js App Router route handlers, React 19, Vitest, system `zip`/`unzip`.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-08-09-workspace-selective-download-design.md`
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Test runner: `npm test` (vitest run). Single file: `npx vitest run <path>`.
|
||||
- Commit style (from `git log`): `feat(webui): …` / `fix(webui): …` / `test(webui): …`.
|
||||
- Server route tests must realpath the temp workspace dir (macOS `/var` → `/private/var`) before use — `safeResolve` compares canonical paths.
|
||||
- `zip -x` patterns are globs: `*`, `?`, `[`, `]`, `\` must be backslash-escaped when matching literal filenames.
|
||||
- Do not break the existing no-`include` behavior (whole-workspace `workspace.zip`).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Selection state module (`src/lib/workspaceSelection.ts`)
|
||||
|
||||
Pure, dependency-free module modeling the tri-state selection. All later UI work consumes these exact functions.
|
||||
|
||||
**Files:**
|
||||
- Create: `src/lib/workspaceSelection.ts`
|
||||
- Test: `src/lib/workspaceSelection.test.ts`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces (consumed by Task 3):
|
||||
- `interface WorkspaceSelection { readonly checked: ReadonlySet<string>; readonly excluded: ReadonlySet<string> }`
|
||||
- `emptySelection(): WorkspaceSelection`
|
||||
- `isChecked(sel: WorkspaceSelection, path: string): boolean`
|
||||
- `isIndeterminate(sel: WorkspaceSelection, dir: string): boolean`
|
||||
- `check(sel: WorkspaceSelection, path: string): WorkspaceSelection`
|
||||
- `uncheck(sel: WorkspaceSelection, path: string): WorkspaceSelection`
|
||||
- `toParams(sel: WorkspaceSelection): { include: string[]; exclude: string[] }` — both arrays sorted
|
||||
|
||||
Semantics (from the spec):
|
||||
- `checked` is **collapsed**: no entry is a descendant of another. Checking a folder stores only the folder and drops its descendants.
|
||||
- `excluded` holds paths explicitly unchecked underneath a collapsed checked ancestor.
|
||||
- `isChecked(path)` = covered by `checked` and not covered by `excluded`.
|
||||
- `isIndeterminate(dir)` = dir not fully checked, but has a `checked` or `excluded` descendant.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Create `src/lib/workspaceSelection.test.ts`:
|
||||
|
||||
```ts
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
check,
|
||||
emptySelection,
|
||||
isChecked,
|
||||
isIndeterminate,
|
||||
toParams,
|
||||
uncheck,
|
||||
} from "./workspaceSelection";
|
||||
|
||||
describe("workspaceSelection", () => {
|
||||
it("checks a single file", () => {
|
||||
const sel = check(emptySelection(), "a.txt");
|
||||
expect(isChecked(sel, "a.txt")).toBe(true);
|
||||
expect(isChecked(sel, "b.txt")).toBe(false);
|
||||
});
|
||||
|
||||
it("collapses descendants when a folder is checked", () => {
|
||||
let sel = check(emptySelection(), "dir/a.txt");
|
||||
sel = check(sel, "dir/b.txt");
|
||||
sel = check(sel, "dir");
|
||||
expect([...sel.checked]).toEqual(["dir"]);
|
||||
expect(isChecked(sel, "dir/a.txt")).toBe(true);
|
||||
expect(isChecked(sel, "dir/deep/c.txt")).toBe(true);
|
||||
expect(isIndeterminate(sel, "dir")).toBe(false);
|
||||
});
|
||||
|
||||
it("records an exclusion when a child of a checked folder is unchecked", () => {
|
||||
let sel = check(emptySelection(), "dir");
|
||||
sel = uncheck(sel, "dir/a.txt");
|
||||
expect(isChecked(sel, "dir")).toBe(false);
|
||||
expect(isChecked(sel, "dir/a.txt")).toBe(false);
|
||||
expect(isChecked(sel, "dir/b.txt")).toBe(true);
|
||||
expect(isIndeterminate(sel, "dir")).toBe(true);
|
||||
});
|
||||
|
||||
it("excludes a whole subfolder and everything beneath it", () => {
|
||||
let sel = check(emptySelection(), "dir");
|
||||
sel = uncheck(sel, "dir/sub");
|
||||
expect(isChecked(sel, "dir/sub")).toBe(false);
|
||||
expect(isChecked(sel, "dir/sub/c.txt")).toBe(false);
|
||||
expect(isChecked(sel, "dir/a.txt")).toBe(true);
|
||||
});
|
||||
|
||||
it("re-checking an excluded child clears the exclusion", () => {
|
||||
let sel = check(emptySelection(), "dir");
|
||||
sel = uncheck(sel, "dir/a.txt");
|
||||
sel = check(sel, "dir/a.txt");
|
||||
expect([...sel.excluded]).toEqual([]);
|
||||
expect(isChecked(sel, "dir")).toBe(true);
|
||||
expect(isIndeterminate(sel, "dir")).toBe(false);
|
||||
});
|
||||
|
||||
it("unchecking a top-level folder drops its exclusions too", () => {
|
||||
let sel = check(emptySelection(), "dir");
|
||||
sel = uncheck(sel, "dir/a.txt");
|
||||
sel = uncheck(sel, "dir");
|
||||
expect([...sel.checked]).toEqual([]);
|
||||
expect([...sel.excluded]).toEqual([]);
|
||||
expect(isChecked(sel, "dir/a.txt")).toBe(false);
|
||||
});
|
||||
|
||||
it("checking a child of a checked-but-excluded path keeps the collapse", () => {
|
||||
let sel = check(emptySelection(), "dir");
|
||||
sel = uncheck(sel, "dir/sub");
|
||||
sel = check(sel, "dir/sub/c.txt");
|
||||
expect([...sel.checked].sort()).toEqual(["dir"]);
|
||||
expect([...sel.excluded]).toEqual(["dir/sub"]);
|
||||
expect(isChecked(sel, "dir/sub/c.txt")).toBe(false);
|
||||
});
|
||||
|
||||
it("is a no-op when unchecking something not selected", () => {
|
||||
const sel = check(emptySelection(), "dir");
|
||||
expect(uncheck(sel, "other.txt")).toBe(sel);
|
||||
});
|
||||
|
||||
it("produces sorted include/exclude params", () => {
|
||||
let sel = check(emptySelection(), "z.txt");
|
||||
sel = check(sel, "dir");
|
||||
sel = uncheck(sel, "dir/b.txt");
|
||||
sel = uncheck(sel, "dir/a.txt");
|
||||
expect(toParams(sel)).toEqual({
|
||||
include: ["dir", "z.txt"],
|
||||
exclude: ["dir/a.txt", "dir/b.txt"],
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
Note on the 7th test: checking a path whose exclusion lives on a strict *ancestor* (`dir/sub` is excluded; user checks `dir/sub/c.txt`) is a no-op — only a direct `excluded` entry on the path itself can be re-checked away. This matches standard file-manager behavior (a freshly checked child under an excluded folder stays excluded).
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npx vitest run src/lib/workspaceSelection.test.ts`
|
||||
Expected: FAIL — `Cannot find module './workspaceSelection'`
|
||||
|
||||
- [ ] **Step 3: Implement the module**
|
||||
|
||||
Create `src/lib/workspaceSelection.ts`:
|
||||
|
||||
```ts
|
||||
// Tri-state checkbox selection for the workspace tree, modeled as a collapsed
|
||||
// `checked` set (checking a folder stores only the folder, never its
|
||||
// descendants) plus an `excluded` set for paths unchecked underneath a
|
||||
// collapsed ancestor. This lets users select/unselect inside unloaded
|
||||
// subtrees — the server expands folders at zip time.
|
||||
|
||||
export interface WorkspaceSelection {
|
||||
readonly checked: ReadonlySet<string>;
|
||||
readonly excluded: ReadonlySet<string>;
|
||||
}
|
||||
|
||||
export function emptySelection(): WorkspaceSelection {
|
||||
return { checked: new Set(), excluded: new Set() };
|
||||
}
|
||||
|
||||
function isUnder(path: string, ancestor: string): boolean {
|
||||
return path.startsWith(ancestor + "/");
|
||||
}
|
||||
|
||||
function coveredBy(set: ReadonlySet<string>, path: string): boolean {
|
||||
for (const entry of set) {
|
||||
if (entry === path || isUnder(path, entry)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function hasDescendant(set: ReadonlySet<string>, dir: string): boolean {
|
||||
for (const entry of set) {
|
||||
if (isUnder(entry, dir)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/** New set without `path` itself or any of its descendants. */
|
||||
function withoutSubtree(set: ReadonlySet<string>, path: string): Set<string> {
|
||||
const next = new Set<string>();
|
||||
for (const entry of set) {
|
||||
if (entry !== path && !isUnder(entry, path)) next.add(entry);
|
||||
}
|
||||
return next;
|
||||
}
|
||||
|
||||
export function isChecked(sel: WorkspaceSelection, path: string): boolean {
|
||||
return coveredBy(sel.checked, path) && !coveredBy(sel.excluded, path);
|
||||
}
|
||||
|
||||
export function isIndeterminate(sel: WorkspaceSelection, dir: string): boolean {
|
||||
if (isChecked(sel, dir)) return false;
|
||||
return hasDescendant(sel.checked, dir) || hasDescendant(sel.excluded, dir);
|
||||
}
|
||||
|
||||
export function check(
|
||||
sel: WorkspaceSelection,
|
||||
path: string
|
||||
): WorkspaceSelection {
|
||||
if (isChecked(sel, path)) return sel;
|
||||
if (coveredBy(sel.checked, path)) {
|
||||
// Covered by a collapsed ancestor but excluded. Only a DIRECT exclusion
|
||||
// entry on `path` itself can be cleared by re-checking; an exclusion on a
|
||||
// strict ancestor (e.g. `dir/sub` while checking `dir/sub/c.txt`) keeps
|
||||
// the path excluded.
|
||||
if (!sel.excluded.has(path)) return sel;
|
||||
return {
|
||||
checked: sel.checked,
|
||||
excluded: withoutSubtree(sel.excluded, path),
|
||||
};
|
||||
}
|
||||
const checked = withoutSubtree(sel.checked, path);
|
||||
checked.add(path);
|
||||
return { checked, excluded: withoutSubtree(sel.excluded, path) };
|
||||
}
|
||||
|
||||
export function uncheck(
|
||||
sel: WorkspaceSelection,
|
||||
path: string
|
||||
): WorkspaceSelection {
|
||||
if (sel.checked.has(path)) {
|
||||
return {
|
||||
checked: withoutSubtree(sel.checked, path),
|
||||
excluded: withoutSubtree(sel.excluded, path),
|
||||
};
|
||||
}
|
||||
if (isChecked(sel, path)) {
|
||||
// Inside a collapsed checked folder: record an exclusion.
|
||||
const excluded = withoutSubtree(sel.excluded, path);
|
||||
excluded.add(path);
|
||||
return { checked: sel.checked, excluded };
|
||||
}
|
||||
return sel;
|
||||
}
|
||||
|
||||
export function toParams(sel: WorkspaceSelection): {
|
||||
include: string[];
|
||||
exclude: string[];
|
||||
} {
|
||||
return {
|
||||
include: [...sel.checked].sort(),
|
||||
exclude: [...sel.excluded].sort(),
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npx vitest run src/lib/workspaceSelection.test.ts`
|
||||
Expected: 9 tests PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/lib/workspaceSelection.ts src/lib/workspaceSelection.test.ts
|
||||
git commit -m "feat(webui): tri-state workspace selection state module"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Selective download in `GET /api/workspace/download`
|
||||
|
||||
Extend the existing route with repeated `include`/`exclude` query params. No params → byte-identical existing behavior.
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/app/api/workspace/download/route.ts`
|
||||
- Test: `src/app/api/workspace/download/route.test.ts` (new)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `safeResolve`, `resolveInside`, `isHiddenEntry`, `zipExcludeArgs`, `isCrossOrigin` from `@/lib/server/workspace`; `resolveConversationWorkspace`, `ConversationWorkspaceError` from `@/lib/server/conversationWorkspace`.
|
||||
- Produces (consumed by Task 3): `GET /api/workspace/download?threadId=<id>&include=<rel>&exclude=<rel>` (repeatable params) → `application/zip` attachment. Errors: 400 JSON `{ error }`. Archive filename: single include → `<basename>.zip`; multiple → `workspace-selection.zip`; none → `workspace.zip`.
|
||||
|
||||
Verified `zip` behaviors this design relies on (confirmed by experiment on the target machine):
|
||||
- `zip -r out.zip dir -x dir/b.txt` excludes that one file.
|
||||
- Excluding a directory needs three patterns: `dir/sub`, `dir/sub/`, `dir/sub/*`.
|
||||
- `zip -x 'data/we\[ird\].txt'` matches the literal filename (backslash escaping works).
|
||||
- When every file is excluded, zip exits **0** with a dirs-only archive — hence the server-side `selectionHasFiles` pre-check instead of relying on exit code 12.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
Create `src/app/api/workspace/download/route.test.ts`:
|
||||
|
||||
```ts
|
||||
import { afterAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { execFileSync } from "child_process";
|
||||
import { randomUUID } from "crypto";
|
||||
import { promises as fs } from "fs";
|
||||
import { tmpdir } from "os";
|
||||
import { join } from "path";
|
||||
import { NextRequest } from "next/server";
|
||||
|
||||
vi.mock("server-only", () => ({}));
|
||||
|
||||
// Canonicalized: safeResolve requires a realpath'd workspaceDir (macOS
|
||||
// /var -> /private/var would otherwise fail its containment check).
|
||||
const filesDir = await (async () => {
|
||||
const dir = join(tmpdir(), `evosci-download-route-test-${process.pid}`);
|
||||
await fs.mkdir(dir, { recursive: true });
|
||||
return fs.realpath(dir);
|
||||
})();
|
||||
|
||||
vi.mock("@/lib/server/conversationWorkspace", () => ({
|
||||
ConversationWorkspaceError: class ConversationWorkspaceError extends Error {
|
||||
constructor(
|
||||
message: string,
|
||||
readonly status: number
|
||||
) {
|
||||
super(message);
|
||||
}
|
||||
},
|
||||
resolveConversationWorkspace: async () => ({
|
||||
deployment: {},
|
||||
threadId: "thread-1",
|
||||
scopeId: "scope-1",
|
||||
filesDir,
|
||||
runtimeDir: filesDir,
|
||||
}),
|
||||
}));
|
||||
|
||||
const routes = await import("./route");
|
||||
|
||||
function get(query = "") {
|
||||
const qs = query ? `&${query}` : "";
|
||||
return routes.GET(
|
||||
new NextRequest(
|
||||
`http://localhost/api/workspace/download?threadId=thread-1${qs}`
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
async function zipEntries(res: Response): Promise<string[]> {
|
||||
expect(res.status).toBe(200);
|
||||
const tmp = join(tmpdir(), `evosci-dl-test-${randomUUID()}.zip`);
|
||||
await fs.writeFile(tmp, Buffer.from(await res.arrayBuffer()));
|
||||
try {
|
||||
const out = execFileSync("unzip", ["-Z1", tmp], { encoding: "utf-8" });
|
||||
return out.split("\n").filter(Boolean).sort();
|
||||
} finally {
|
||||
await fs.rm(tmp, { force: true });
|
||||
}
|
||||
}
|
||||
|
||||
afterAll(async () => {
|
||||
await fs.rm(filesDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe("GET /api/workspace/download", () => {
|
||||
beforeEach(async () => {
|
||||
await fs.rm(filesDir, { recursive: true, force: true });
|
||||
await fs.mkdir(join(filesDir, "dir", "sub"), { recursive: true });
|
||||
await fs.mkdir(join(filesDir, "data"), { recursive: true });
|
||||
await fs.writeFile(join(filesDir, "dir", "a.txt"), "A");
|
||||
await fs.writeFile(join(filesDir, "dir", "b.txt"), "B");
|
||||
await fs.writeFile(join(filesDir, "dir", "sub", "c.txt"), "C");
|
||||
await fs.writeFile(join(filesDir, "data", "we[ird].txt"), "W");
|
||||
await fs.writeFile(join(filesDir, "data", "ok.txt"), "O");
|
||||
await fs.writeFile(join(filesDir, "notes.md"), "# hi");
|
||||
await fs.writeFile(join(filesDir, ".hidden"), "secret");
|
||||
});
|
||||
|
||||
it("zips the whole workspace when no include is given (regression)", async () => {
|
||||
const res = await get();
|
||||
expect(res.headers.get("content-disposition")).toContain("workspace.zip");
|
||||
const entries = await zipEntries(res);
|
||||
expect(entries).toContain("notes.md");
|
||||
expect(entries).toContain("dir/a.txt");
|
||||
expect(entries).not.toContain(".hidden");
|
||||
});
|
||||
|
||||
it("zips a single included folder and names the archive after it", async () => {
|
||||
const res = await get("include=dir");
|
||||
expect(res.headers.get("content-disposition")).toContain("dir.zip");
|
||||
const entries = await zipEntries(res);
|
||||
expect(entries).toEqual([
|
||||
"dir/",
|
||||
"dir/a.txt",
|
||||
"dir/b.txt",
|
||||
"dir/sub/",
|
||||
"dir/sub/c.txt",
|
||||
]);
|
||||
});
|
||||
|
||||
it("excludes a single file inside an included folder", async () => {
|
||||
const res = await get("include=dir&exclude=dir/b.txt");
|
||||
const entries = await zipEntries(res);
|
||||
expect(entries).toEqual([
|
||||
"dir/",
|
||||
"dir/a.txt",
|
||||
"dir/sub/",
|
||||
"dir/sub/c.txt",
|
||||
]);
|
||||
});
|
||||
|
||||
it("excludes a whole subdirectory", async () => {
|
||||
const res = await get("include=dir&exclude=dir/sub");
|
||||
const entries = await zipEntries(res);
|
||||
expect(entries).toEqual(["dir/", "dir/a.txt", "dir/b.txt"]);
|
||||
});
|
||||
|
||||
it("supports multiple includes with a generic archive name", async () => {
|
||||
const res = await get("include=dir/sub&include=notes.md");
|
||||
expect(res.headers.get("content-disposition")).toContain(
|
||||
"workspace-selection.zip"
|
||||
);
|
||||
const entries = await zipEntries(res);
|
||||
expect(entries).toEqual(["dir/sub/", "dir/sub/c.txt", "notes.md"]);
|
||||
});
|
||||
|
||||
it("matches glob metacharacters in exclude paths literally", async () => {
|
||||
const res = await get(
|
||||
`include=data&exclude=${encodeURIComponent("data/we[ird].txt")}`
|
||||
);
|
||||
const entries = await zipEntries(res);
|
||||
expect(entries).toEqual(["data/", "data/ok.txt"]);
|
||||
});
|
||||
|
||||
it("rejects path traversal in include", async () => {
|
||||
const res = await get(`include=${encodeURIComponent("../outside")}`);
|
||||
expect(res.status).toBe(400);
|
||||
});
|
||||
|
||||
it("rejects hidden entries in include", async () => {
|
||||
const res = await get("include=.hidden");
|
||||
expect(res.status).toBe(400);
|
||||
});
|
||||
|
||||
it("rejects an exclude outside all includes", async () => {
|
||||
const res = await get("include=dir&exclude=notes.md");
|
||||
expect(res.status).toBe(400);
|
||||
});
|
||||
|
||||
it("rejects exclude without any include", async () => {
|
||||
const res = await get("exclude=dir/a.txt");
|
||||
expect(res.status).toBe(400);
|
||||
});
|
||||
|
||||
it("rejects a selection that is empty after excludes", async () => {
|
||||
const res = await get(
|
||||
"include=dir&exclude=dir/a.txt&exclude=dir/b.txt&exclude=dir/sub"
|
||||
);
|
||||
expect(res.status).toBe(400);
|
||||
const body = await res.json();
|
||||
expect(body.error).toBe("The selection is empty.");
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `npx vitest run src/app/api/workspace/download/route.test.ts`
|
||||
Expected: FAIL — the selective tests get a full-workspace zip (or pass trivially for regression); specifically "zips a single included folder" fails because `include` is ignored today.
|
||||
|
||||
- [ ] **Step 3: Implement the route changes**
|
||||
|
||||
Replace the entire contents of `src/app/api/workspace/download/route.ts` with:
|
||||
|
||||
```ts
|
||||
import { createReadStream, promises as fs } from "fs";
|
||||
import { tmpdir } from "os";
|
||||
import { basename, join, relative, sep } from "path";
|
||||
import { randomUUID } from "crypto";
|
||||
import { spawn } from "child_process";
|
||||
import { Readable } from "stream";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import {
|
||||
zipExcludeArgs,
|
||||
isCrossOrigin,
|
||||
isHiddenEntry,
|
||||
resolveInside,
|
||||
safeResolve,
|
||||
} from "@/lib/server/workspace";
|
||||
import {
|
||||
ConversationWorkspaceError,
|
||||
resolveConversationWorkspace,
|
||||
} from "@/lib/server/conversationWorkspace";
|
||||
|
||||
export const runtime = "nodejs";
|
||||
|
||||
/** Zip `includePaths` (relative to `workspaceDir`) into `outFile` using the OS
|
||||
* `zip`. Aborts (and kills the child) if `signal` fires — e.g. the client
|
||||
* disconnects mid-archive. */
|
||||
function zipWorkspace(
|
||||
workspaceDir: string,
|
||||
outFile: string,
|
||||
includePaths: string[],
|
||||
extraExcludeArgs: string[],
|
||||
signal?: AbortSignal
|
||||
): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
// -r recurse, -q quiet, -X drop extra file attributes, -y store symlinks AS
|
||||
// symlinks instead of dereferencing them (so a symlink pointing outside the
|
||||
// workspace can't pull external file *contents* into the archive).
|
||||
// Exclusions come from the shared ignore lists so the archive matches the
|
||||
// tree exactly (dotfiles, large_tool_results/conversation_history, build noise).
|
||||
const child = spawn(
|
||||
"zip",
|
||||
[
|
||||
"-r",
|
||||
"-q",
|
||||
"-X",
|
||||
"-y",
|
||||
outFile,
|
||||
...includePaths,
|
||||
...extraExcludeArgs,
|
||||
...zipExcludeArgs(),
|
||||
],
|
||||
{ cwd: workspaceDir }
|
||||
);
|
||||
|
||||
const onAbort = () => child.kill("SIGKILL");
|
||||
if (signal) {
|
||||
if (signal.aborted) {
|
||||
child.kill("SIGKILL");
|
||||
reject(new Error("Request aborted."));
|
||||
return;
|
||||
}
|
||||
signal.addEventListener("abort", onAbort, { once: true });
|
||||
}
|
||||
|
||||
let stderr = "";
|
||||
child.stderr.on("data", (d) => (stderr += d.toString()));
|
||||
child.on("error", (err) => {
|
||||
signal?.removeEventListener("abort", onAbort);
|
||||
reject(
|
||||
(err as NodeJS.ErrnoException).code === "ENOENT"
|
||||
? new Error(
|
||||
"The `zip` command is not available on this system, so the workspace can't be downloaded as an archive."
|
||||
)
|
||||
: err
|
||||
);
|
||||
});
|
||||
child.on("close", (code) => {
|
||||
signal?.removeEventListener("abort", onAbort);
|
||||
if (signal?.aborted) reject(new Error("Request aborted."));
|
||||
// 12 = "nothing to do" — unreachable for selections (the pre-check below
|
||||
// rejects empty selections up front) unless every include vanished
|
||||
// between the pre-check and the spawn; the whole-workspace case is a
|
||||
// friendly error either way.
|
||||
else if (code === 12) reject(new Error("The workspace is empty."));
|
||||
else if (code !== 0)
|
||||
reject(new Error(stderr.trim() || `zip exited with code ${code}`));
|
||||
else resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/** Escape zip's glob metacharacters so an -x pattern matches one exact path. */
|
||||
function escapeZipPattern(path: string): string {
|
||||
return path.replace(/[\\*?[\]]/g, "\\$&");
|
||||
}
|
||||
|
||||
/** -x args excluding one relative path. A file exclude matches `rel`; a dir
|
||||
* exclude must also drop the dir entry itself (`rel/`) and everything
|
||||
* beneath it (`rel/*`) — zip matches patterns against archive member names. */
|
||||
function excludeArgsFor(rel: string): string[] {
|
||||
const esc = escapeZipPattern(rel);
|
||||
return ["-x", esc, "-x", `${esc}/`, "-x", `${esc}/*`];
|
||||
}
|
||||
|
||||
/** Keep only topmost paths: drop duplicates and entries shadowed by an
|
||||
* already-listed ancestor (sorting guarantees ancestors come first). */
|
||||
function collapseTopmost(rels: string[]): string[] {
|
||||
const sorted = [...new Set(rels)].sort();
|
||||
const out: string[] = [];
|
||||
for (const rel of sorted) {
|
||||
if (!out.some((top) => rel.startsWith(top + "/"))) out.push(rel);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** True if the includes still contain at least one file/symlink after the
|
||||
* hidden-entry policy and the excludes are applied. Runs before spawning
|
||||
* `zip`, which happily produces a dirs-only archive (exit 0) when every file
|
||||
* was excluded — so exit codes can't detect an empty selection. */
|
||||
async function selectionHasFiles(
|
||||
workspaceDir: string,
|
||||
includes: string[],
|
||||
excludes: string[]
|
||||
): Promise<boolean> {
|
||||
const isExcluded = (rel: string) =>
|
||||
excludes.some((e) => rel === e || rel.startsWith(e + "/"));
|
||||
const walk = async (rel: string): Promise<boolean> => {
|
||||
if (isExcluded(rel)) return false;
|
||||
let stat;
|
||||
try {
|
||||
stat = await fs.lstat(join(workspaceDir, rel));
|
||||
} catch {
|
||||
return false; // vanished between listing and download
|
||||
}
|
||||
if (!stat.isDirectory()) return true;
|
||||
const entries = await fs.readdir(join(workspaceDir, rel));
|
||||
for (const name of entries) {
|
||||
if (isHiddenEntry(name)) continue;
|
||||
if (await walk(rel === "" ? name : `${rel}/${name}`)) return true;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
for (const rel of includes) {
|
||||
if (rel === ".") return true; // whole-workspace mode: zip handles emptiness
|
||||
if (await walk(rel)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/** Content-Disposition for an attachment whose name may be non-ASCII. */
|
||||
function attachmentDisposition(filename: string): string {
|
||||
// eslint-disable-next-line no-control-regex
|
||||
const ascii = filename.replace(/[^\x20-\x7e]/g, "_").replace(/["\\]/g, "_");
|
||||
return `attachment; filename="${ascii}"; filename*=UTF-8''${encodeURIComponent(filename)}`;
|
||||
}
|
||||
|
||||
export async function GET(request: NextRequest) {
|
||||
let tmpFile: string | null = null;
|
||||
try {
|
||||
if (isCrossOrigin(request)) {
|
||||
return NextResponse.json(
|
||||
{ error: "Cross-origin workspace access is not allowed." },
|
||||
{ status: 403 }
|
||||
);
|
||||
}
|
||||
|
||||
const { filesDir: workspaceDir } = await resolveConversationWorkspace(
|
||||
request
|
||||
);
|
||||
|
||||
const includes = request.nextUrl.searchParams
|
||||
.getAll("include")
|
||||
.filter((p) => p.trim() !== "");
|
||||
const excludes = request.nextUrl.searchParams
|
||||
.getAll("exclude")
|
||||
.filter((p) => p.trim() !== "");
|
||||
|
||||
// No includes → the original whole-workspace archive.
|
||||
let includeRels: string[] = ["."];
|
||||
let excludeRels: string[] = [];
|
||||
|
||||
if (includes.length === 0) {
|
||||
if (excludes.length > 0) {
|
||||
return NextResponse.json(
|
||||
{ error: "exclude requires at least one include." },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
} else {
|
||||
// Resolve every include to a canonical in-workspace relative path
|
||||
// (safeResolve rejects traversal, hidden entries, symlink escapes, and
|
||||
// paths that don't exist).
|
||||
const rels: string[] = [];
|
||||
for (const p of includes) {
|
||||
const target = await safeResolve(workspaceDir, p);
|
||||
rels.push(relative(workspaceDir, target).split(sep).join("/"));
|
||||
}
|
||||
includeRels = rels.includes("") ? ["."] : collapseTopmost(rels);
|
||||
|
||||
// Excludes only filter what the includes pulled in, so they don't need
|
||||
// to exist (the entry may be gone by now) — a lexical check suffices.
|
||||
// Each must sit strictly inside at least one include.
|
||||
for (const p of excludes) {
|
||||
const target = resolveInside(workspaceDir, p);
|
||||
const rel = relative(workspaceDir, target).split(sep).join("/");
|
||||
const inside = includeRels.some(
|
||||
(inc) => inc === "." || (rel !== "" && rel.startsWith(inc + "/"))
|
||||
);
|
||||
if (!inside) {
|
||||
return NextResponse.json(
|
||||
{ error: `exclude "${p}" is not inside any selected include.` },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
excludeRels.push(rel);
|
||||
}
|
||||
excludeRels = [...new Set(excludeRels)].sort();
|
||||
|
||||
if (
|
||||
includeRels[0] !== "." &&
|
||||
!(await selectionHasFiles(workspaceDir, includeRels, excludeRels))
|
||||
) {
|
||||
return NextResponse.json(
|
||||
{ error: "The selection is empty." },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
tmpFile = join(tmpdir(), `evoscientist-workspace-${randomUUID()}.zip`);
|
||||
const extraExcludeArgs = excludeRels.flatMap(excludeArgsFor);
|
||||
await zipWorkspace(
|
||||
workspaceDir,
|
||||
tmpFile,
|
||||
includeRels,
|
||||
extraExcludeArgs,
|
||||
request.signal
|
||||
);
|
||||
|
||||
const stat = await fs.stat(tmpFile);
|
||||
const nodeStream = createReadStream(tmpFile);
|
||||
// Delete the temp archive once the response has been fully read (or the
|
||||
// client disconnects) — `close` fires in both cases.
|
||||
const cleanup = tmpFile;
|
||||
nodeStream.on("close", () => void fs.rm(cleanup, { force: true }));
|
||||
const webStream = Readable.toWeb(nodeStream) as ReadableStream<Uint8Array>;
|
||||
|
||||
const zipName =
|
||||
includes.length === 0
|
||||
? "workspace.zip"
|
||||
: includeRels.length === 1 && includeRels[0] !== "."
|
||||
? `${basename(includeRels[0])}.zip`
|
||||
: "workspace-selection.zip";
|
||||
|
||||
return new NextResponse(webStream, {
|
||||
headers: {
|
||||
"Content-Type": "application/zip",
|
||||
"Content-Length": String(stat.size),
|
||||
"Content-Disposition": attachmentDisposition(zipName),
|
||||
"Cache-Control": "no-store",
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (tmpFile) await fs.rm(tmpFile, { force: true }).catch(() => {});
|
||||
return NextResponse.json(
|
||||
{
|
||||
error:
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Failed to package the workspace.",
|
||||
},
|
||||
{
|
||||
status:
|
||||
error instanceof ConversationWorkspaceError ? error.status : 400,
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `npx vitest run src/app/api/workspace/download/route.test.ts`
|
||||
Expected: 11 tests PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/app/api/workspace/download/route.ts src/app/api/workspace/download/route.test.ts
|
||||
git commit -m "feat(webui): selective include/exclude download for workspace zips"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Tree-view checkboxes + "Selected (N)" download in `WorkspacePanel`
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/app/components/WorkspacePanel.tsx`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes (from Task 1): `WorkspaceSelection`, `emptySelection`, `isChecked`, `isIndeterminate`, `check`, `uncheck`, `toParams`.
|
||||
- Consumes (from Task 2): `GET /api/workspace/download?threadId=…&include=…&exclude=…`.
|
||||
|
||||
- [ ] **Step 1: Wire selection state into the component**
|
||||
|
||||
In `src/app/components/WorkspacePanel.tsx`:
|
||||
|
||||
a) Add the import at the top with the other imports:
|
||||
|
||||
```ts
|
||||
import {
|
||||
check as checkSelection,
|
||||
emptySelection,
|
||||
isChecked as isSelectionChecked,
|
||||
isIndeterminate as isSelectionIndeterminate,
|
||||
uncheck as uncheckSelection,
|
||||
toParams as selectionToParams,
|
||||
type WorkspaceSelection,
|
||||
} from "@/lib/workspaceSelection";
|
||||
```
|
||||
|
||||
b) Add state next to the other tree-view state (after the `selected` state, ~line 185):
|
||||
|
||||
```ts
|
||||
const [selection, setSelection] = useState<WorkspaceSelection>(
|
||||
emptySelection()
|
||||
);
|
||||
const toggleSelection = useCallback((path: string) => {
|
||||
setSelection((prev) =>
|
||||
isSelectionChecked(prev, path)
|
||||
? uncheckSelection(prev, path)
|
||||
: checkSelection(prev, path)
|
||||
);
|
||||
}, []);
|
||||
```
|
||||
|
||||
c) Reset the selection when the conversation changes — inside the existing `useEffect` that resets `setChildren({})` etc. (~line 241), add one line:
|
||||
|
||||
```ts
|
||||
setSelection(emptySelection());
|
||||
```
|
||||
|
||||
d) Build the download href (place near the `grouped` memo, ~line 332):
|
||||
|
||||
```ts
|
||||
const selectionCount = selection.checked.size;
|
||||
// Native <a href> download so large archives stream straight to disk. The
|
||||
// URL-length guard keeps a huge selection from hitting server/header limits
|
||||
// — the fix is to select fewer, higher-level folders instead.
|
||||
const selectionDownload = useMemo<{
|
||||
href: string | null;
|
||||
tooLarge: boolean;
|
||||
}>(() => {
|
||||
if (!threadId || selectionCount === 0)
|
||||
return { href: null, tooLarge: false };
|
||||
const { include, exclude } = selectionToParams(selection);
|
||||
const params = new URLSearchParams({ threadId });
|
||||
for (const p of include) params.append("include", p);
|
||||
for (const p of exclude) params.append("exclude", p);
|
||||
const href = `/api/workspace/download?${params}`;
|
||||
return href.length <= 7000
|
||||
? { href, tooLarge: false }
|
||||
: { href: null, tooLarge: true };
|
||||
}, [threadId, selection, selectionCount]);
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add the checkbox to tree rows**
|
||||
|
||||
In `renderEntries` (~line 346), the row is currently a single `<button>`. An `<input>` cannot nest inside a `<button>` (invalid HTML, broken events), so restructure each row into a wrapper div holding the checkbox + the existing button. Replace the whole `return entries.map((entry) => …)` block with:
|
||||
|
||||
```tsx
|
||||
return entries.map((entry) => {
|
||||
const isOpen = expanded.has(entry.path);
|
||||
const isLoadingDir = loading.has(entry.path);
|
||||
const entryChecked = isSelectionChecked(selection, entry.path);
|
||||
const entryIndeterminate =
|
||||
entry.type === "dir" && isSelectionIndeterminate(selection, entry.path);
|
||||
return (
|
||||
<div key={entry.path}>
|
||||
<div
|
||||
className="flex w-full items-center gap-1.5 rounded-md py-1 pr-2 text-sm text-foreground transition-colors hover:bg-muted"
|
||||
style={{ paddingLeft: `${depth * 14 + 4}px` }}
|
||||
>
|
||||
<input
|
||||
type="checkbox"
|
||||
className="size-3.5 shrink-0 accent-[var(--brand)]"
|
||||
checked={entryChecked}
|
||||
ref={(el) => {
|
||||
if (el) el.indeterminate = entryIndeterminate;
|
||||
}}
|
||||
onChange={() => toggleSelection(entry.path)}
|
||||
aria-label={`Select ${entry.name}`}
|
||||
/>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() =>
|
||||
entry.type === "dir"
|
||||
? toggleDir(entry.path)
|
||||
: setSelected({ path: entry.path, size: entry.size })
|
||||
}
|
||||
className="flex min-w-0 flex-1 items-center gap-1.5 text-left"
|
||||
title={entry.name}
|
||||
>
|
||||
{entry.type === "dir" ? (
|
||||
<>
|
||||
<span className="flex size-4 shrink-0 items-center justify-center text-muted-foreground">
|
||||
{isLoadingDir ? (
|
||||
<Loader2 className="size-3 animate-spin" />
|
||||
) : isOpen ? (
|
||||
<ChevronDown className="size-3.5" />
|
||||
) : (
|
||||
<ChevronRight className="size-3.5" />
|
||||
)}
|
||||
</span>
|
||||
<Folder className="size-4 shrink-0 text-[var(--brand)]" />
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<span className="size-4 shrink-0" />
|
||||
<FileText className="size-4 shrink-0 text-muted-foreground" />
|
||||
</>
|
||||
)}
|
||||
<span className="truncate">{entry.name}</span>
|
||||
</button>
|
||||
</div>
|
||||
{entry.type === "dir" &&
|
||||
isOpen &&
|
||||
renderEntries(entry.path, depth + 1)}
|
||||
</div>
|
||||
);
|
||||
});
|
||||
```
|
||||
|
||||
Notes:
|
||||
- The hover background moved to the wrapper div so checkbox + label highlight together.
|
||||
- The depth padding moved from the button to the wrapper; the button keeps `min-w-0 flex-1` so long names still truncate.
|
||||
- `indeterminate` is only settable via JS, hence the `ref` callback.
|
||||
- Clicking an indeterminate folder's checkbox fires `onChange` → `toggleSelection` → `check` (fully selects it), the standard file-manager behavior.
|
||||
|
||||
- [ ] **Step 3: Add the "Selected (N)" toolbar button**
|
||||
|
||||
In the toolbar (~line 471, the div wrapping the **All** link and refresh button), insert this immediately before the existing **All** anchor:
|
||||
|
||||
```tsx
|
||||
<a
|
||||
href={selectionDownload.href ?? "#"}
|
||||
download
|
||||
aria-disabled={!selectionDownload.href}
|
||||
onClick={
|
||||
selectionDownload.href ? undefined : (e) => e.preventDefault()
|
||||
}
|
||||
className={cn(
|
||||
"inline-flex items-center gap-1 rounded-md px-1.5 py-1 text-xs transition-colors",
|
||||
selectionDownload.href
|
||||
? "text-muted-foreground hover:bg-muted hover:text-foreground"
|
||||
: "cursor-not-allowed text-muted-foreground/50"
|
||||
)}
|
||||
title={
|
||||
selectionDownload.tooLarge
|
||||
? "Selection is too large to download — select fewer, higher-level folders"
|
||||
: selectionCount === 0
|
||||
? "Select files or folders to download"
|
||||
: "Download the selected items as a zip"
|
||||
}
|
||||
>
|
||||
<Download className="size-3.5" />
|
||||
Selected{selectionCount > 0 ? ` (${selectionCount})` : ""}
|
||||
</a>
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Typecheck, lint, and run the full suite**
|
||||
|
||||
Run: `npx tsc --noEmit && npx eslint src/app/components/WorkspacePanel.tsx src/lib/workspaceSelection.ts src/app/api/workspace/download/route.ts && npm test`
|
||||
Expected: no type errors, no lint errors, all tests PASS (including the 20 new ones from Tasks 1–2).
|
||||
|
||||
- [ ] **Step 5: Manual browser verification**
|
||||
|
||||
Run: `npm run dev` (port 4716), then in the browser with a conversation that has nested workspace files:
|
||||
|
||||
1. Tree view shows a checkbox on every row; checking a folder checks all visible descendants.
|
||||
2. Uncheck one file inside a checked folder → folder shows the indeterminate dash.
|
||||
3. Expand a *different* folder, check it, then click **Selected (2)** → browser downloads `workspace-selection.zip`. Then reduce to exactly one top-level selection → the archive is named `<foldername>.zip`. Verify both.
|
||||
4. Unzip the archive and confirm excluded files are absent and hidden/internal entries (`.langgraph_api`, `large_tool_results`) never appear.
|
||||
5. **All** still downloads the whole workspace as `workspace.zip`.
|
||||
6. Switch conversations → selection clears.
|
||||
7. By-type view has no checkboxes.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src/app/components/WorkspacePanel.tsx
|
||||
git commit -m "feat(webui): checkbox multi-select download in workspace tree"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Notes (already applied)
|
||||
|
||||
- Spec coverage: client state model ✓ (Task 1), API params/validation/naming ✓ (Task 2), Tree-view UI + toolbar + thread-change reset + URL guard ✓ (Task 3), tests for both layers ✓. "By type view unchanged" and "All kept" are preserved by not touching those paths.
|
||||
- Type consistency: Task 3 uses exactly the Task 1 exports (`check`/`uncheck` aliased on import to avoid clashing with local names) and the Task 2 query-param contract.
|
||||
- Deviation from spec, deliberate: excludes are validated with `resolveInside` (lexical) instead of `safeResolve`, because an excluded entry may have been deleted after the user unchecked it — an exclude is only a filter and cannot leak data. Includes still go through `safeResolve`.
|
||||
- Deviation from spec, deliberate: the empty-selection check is a pre-spawn filesystem walk (`selectionHasFiles`) instead of relying on zip exit 12 — experiments show zip exits 0 with a dirs-only archive when everything is excluded.
|
||||
Reference in New Issue
Block a user