test(desktop): make MCP renderer relay regressions executable

This commit is contained in:
Teknium
2026-09-07 03:13:19 -07:00
parent f914c9b070
commit 0d86694fa7
2 changed files with 23 additions and 0 deletions
+19
View File
@@ -0,0 +1,19 @@
# Desktop MCP OAuth integration checks
Run from a checkout with the Python MCP dependencies and Desktop Node dependencies installed.
Outputs are local receipts; keep them outside the checkout. Neither harness uses real provider credentials.
```bash
python3 evals/desktop_mcp_oauth/backend_http_fixture.py --repo . --output /tmp/mcp-backend.json
node evals/desktop_mcp_oauth/renderer_lifecycle.mjs "$PWD" /tmp/mcp-renderer.json approved
node evals/desktop_mcp_oauth/renderer_lifecycle.mjs "$PWD" /tmp/mcp-scope.json cancel
node evals/desktop_mcp_oauth/renderer_lifecycle.mjs "$PWD" /tmp/mcp-unmount.json unmount
```
The backend harness creates disposable HOME/HERMES_HOME directories and a local HTTP OAuth/MCP provider. It exercises production session functions, discovery, dynamic registration, PKCE code exchange, token persistence and fresh-process authenticated MCP access. Wrong state, callback replay, wrong server, wrong profile and cancellation are negative controls. It removes temporary token stores and reports only booleans/paths, never token values.
The renderer harness bundles the real McpTab and its dependencies in Chromium, runs the production native loopback listener with a registration-only Electron IPC adapter, and uses a fixture WebSocket backend. `cancel` changes the component's scope; `unmount` removes it entirely. Both must cancel the original backend session and close its native listener without relaying a callback. `approved` must use the native relay, not REST auth. Set `CHROMIUM_EXECUTABLE` to use an existing Chromium executable; otherwise Playwright's installed browser is used.
These are complementary integration probes, not a single end-to-end Electron/provider test. The renderer backend and Electron registration boundary are fixtures; the Python probe uses production session functions rather than gateway RPC transport. No native Electron application launch or hosted-provider consent is claimed.
For A/B, run the same renderer harness against a baseline checkout by replacing its first positional argument. The approved assertion fails before the shared relay change. For the lifecycle regression, a checkout immediately before the scoped-tab cleanup fails both abandonment assertions. Backend cross-profile callbacks must be rejected even when the session ID and valid state are supplied; the original owner must still be able to finish afterward.
@@ -187,6 +187,10 @@ try {
)
assert(!rpcCalls.some(m => m.method === 'mcp.servers.oauth.callback'), 'no callback after abandonment')
}
if (scenario === 'approved') {
assert(approved && hasScope, 'Skills must complete OAuth through its native listener')
assert(!requests.some(r => r.path.endsWith('/auth')), 'native flow must not use REST auth')
}
assert.deepEqual(errors, [])
} finally {
await browser.close()