Files
EvoScientist/tests/test_mcp_client.py
T
dinos 65828e9666 perf(cli): cut startup latency and defer MCP loading to the background (#171)
* perf(cli): cut startup time of `evosci --help` from ~2.2s to ~0.3s

Module-level imports were eagerly pulling in langchain.chat_models (with
the whole anthropic/openai/google stack), langgraph, textual, and
prompt_toolkit on every invocation — even for `--help` or `config list`.

Defer those with PEP 562 `__getattr__`, using `lazy_loader.attach` (SPEC-1,
the scientific-python standard) where it's a clean attach pattern:

- `EvoScientist/llm/__init__.py`: attach `models` lazily so importing
  `context_window` from this package no longer drags in langchain.
- `EvoScientist/stream/__init__.py`: attach display/events lazily; split
  the shared Rich `Console` singleton into a new lightweight
  `stream/console.py` so callers that only need `console` skip the
  `stream.events` → `langchain_core.messages` chain.
- `EvoScientist/cli/__init__.py`: hand-rolled `__getattr__` (reaches into
  `..stream.state`, which `lazy_loader` doesn't cover) so `commands` and
  `app` are the only eager loads.
- `EvoScientist/cli/commands.py`: move `cmd_interactive`/`cmd_run` to
  in-function imports so prompt_toolkit + textual only load when the
  interactive path actually runs.
- `EvoScientist/cli/_constants.py`: read `AGENT_NAME` on demand so
  `build_metadata` doesn't eagerly import `sessions` (langgraph/aiosqlite).

Adds `lazy-loader>=0.5` as a dependency.

* feat(cli): defer MCP tool loading with live per-server progress

The CLI was blocking ~5 s on MCP tool enumeration before the first
prompt appeared. Move the agent construction off the event loop and
surface per-server progress so the user can interact immediately and see
what's happening.

MCP client:
- Add an `on_progress` callback to `load_mcp_tools` / `aload_mcp_tools`
  / `_load_tools` emitting `start` / `success` / `error` events per
  server.
- Fan connection attempts out with `asyncio.gather` so latency no longer
  scales linearly with server count; cap simultaneous attempts at
  `_MAX_CONCURRENT_CONNECTIONS` (8) via a semaphore so a big stdio fleet
  doesn't spawn every subprocess at once.

Agent wiring:
- Plumb `on_mcp_progress` through `create_cli_agent` / `_load_agent` /
  `load_mcp_and_build_kwargs` so CLI and TUI can plug in collectors.

CLI (`cmd_interactive`):
- Run `_load_agent` in a background thread via `asyncio.to_thread`; the
  prompt and banner render immediately.
- `_await_agent_ready()` awaits the task before each agent-using site
  (first turn, channel messages, `/channel`, `/compact`). Raises if
  called without a prior `_start_agent_load` instead of silently
  reloading without the SQLite checkpointer.
- Pre-prime the progress dict from `load_mcp_config()` so the
  bottom-toolbar's `N/M` denominator is stable from the first render.
- Wrap `session.prompt_async` in `patch_stdout(raw=True)` so
  `console.print` from the worker-thread progress callback lands cleanly
  above the prompt as inline chat messages instead of stomping the
  prompt cursor.

TUI (`EvoTextualInteractiveApp`):
- Same background load + `_await_agent_ready()` gates on every
  `self._agent` read.
- New `MCPLoaderWidget` mounted at the top of `#input-shell` shows a
  header with `N/M` and one live row per server (spinner → ✓ / ✗ with
  tool count or error detail). On completion:
  - all-clean loads auto-dismiss ~2.5 s later;
  - cache hits (no events ever fired) dismiss immediately rather than
    flashing a misleading "0/N loaded";
  - failures keep the widget mounted so the user can read the errors.
  - `dismissed` property lets the app clear its ref so late events from
    slow servers become no-ops. The error branch of `_on_agent_loaded`
    also settles the widget so a load failure can't leave the spinner
    animating forever.
- Chat input is `disabled` while MCP resolves — no placeholder hack, no
"waiting…" system message.

Shared:
- Hoist braille spinner frames to `status_bar.SPINNER_FRAMES` and import
  them in the TUI widget so CLI and TUI animate in sync.

Tests:
- Extend `test_agent_mcp_cache` fakes to accept the new `on_progress`
  kwarg.
- New `TestLoadToolsProgressCallback` in `test_mcp_client` exercises the
  event sequence for success/failure/mixed fleets, verifies a buggy
  callback doesn't break the load, and asserts the semaphore caps
  in-flight connections.

* style: ruff

* chore: update uv.lock

* chore: uv.lock

* fix: coderabbit issues

* style: fmt

* fix: move _await_agent_ready inside try block

* fix(tui): auto-dismiss MCP loader widget on failure

The widget was designed to stay mounted on failure so the user could
read error detail, but since it's pinned above the input it never went
away in practice — just permanent banner clutter.

Auto-dismiss on failure too, with a longer grace (12s vs 2.5s) so the
error summary stays readable.

* fix: address second coderabbit pass

- Channel handlers (CLI + TUI): catch agent-load failures so the
  channel request doesn't hang; CLI moves `_await_agent_ready()`
  inside the existing try/except, TUI catches explicitly and calls
  `_set_channel_response` with the error.

- Stale background loads: `prev.cancel()` only stops the asyncio
  wrapper, not the thread running `_load_agent`. Added a generation
  token (`agent_load_id` / `self._agent_load_id`) and gated both
  progress and completion callbacks on it so a superseded load can't
  clobber the current session's state or UI.

- TUI prompt lifecycle: added `_agent_load_pending()` and gated the
  `_process_channel_message` / `_handle_command` finally blocks on it
  so `/new` or `/resume` invoked from a command keeps the prompt
  disabled until the fresh load settles.

- TUI readiness failures: `_run_turn` and `_handle_command` now
  catch exceptions from `_await_agent_ready()` and surface a
  "Agent failed to load: …" system message instead of letting the
  exception escape into Textual's traceback panel.

* refactor(cli): share background agent loader between CLI and TUI

The CLI and TUI were carrying near-identical copies of the same
background-load state machine: the `agent_task`, the `agent_load_id`
generation token, the gated progress/completion callbacks, and the
per-server progress dict. Every CodeRabbit finding on that lifecycle
had to be fixed in both files.

Extract it into `cli/_agent_loader.py`:

- `MCPProgressTracker` — owns the `server -> (state, detail)` dict;
  exposes `prime`, `record`, `snapshot`, `totals`.

- `BackgroundAgentLoader` — owns `agent`, the in-flight task, and the
  generation token. Exposes `start(**loader_kwargs)`, `await_ready()`,
  `is_pending`. Internally gates all progress/completion callbacks by
  generation so a superseded load can't clobber the current session.
  UI-specific rendering plugs in via `on_progress` / `on_success` /
  `on_failure` callbacks.

Both surfaces now just wire their UI hooks; the loader file holds no
Rich / prompt_toolkit / Textual dependencies. Net -345 lines from
`interactive.py` + `tui_interactive.py`; +20 unit tests pinning the
lifecycle (generation filtering, cache-hit short-circuit, failure
reset, progress ordering).

* refactor(cli): make _on_done the sole authority for agent state transitions

await_ready no longer sets self.agent — it just awaits the task and
reads what _on_done already wrote. Eliminates the dual-write overlap
(asyncio guarantees done-callbacks fire in registration order).

* fix(tui): let users type during MCP load, only block on send

Remove prompt-disabling during background agent load — the TUI now
matches the CLI approach where the input stays enabled and only gates
on await_ready() at submit time. The MCPLoaderWidget still provides
visual feedback that loading is in progress.

* fix(loader): preserve real load error on await_ready; dedup failure message

CodeRabbit flagged two issues with the new loader:

1. After a failed load, `_on_done` nulled `self._task`, so the next
   `await_ready()` hit the "before start()" branch and the CLI wrapper
   remapped it to a misleading "checkpointer not available" message —
   losing the real exception (bad MCP config, network, etc.).

   Keep `_task` set on failure so `await_ready` re-raises the real
   exception. Added `needs_restart` so TUI's auto-retry check stays a
   one-liner and doesn't need to reach into task internals.

2. TUI reported each load failure twice: once from
   `_on_agent_load_failure` (the done-callback) and once from each
   caller of `_await_agent_ready` (`_run_turn`,
   `_process_channel_message`, `_handle_command`) catching the re-raise.

   `_on_agent_load_failure` is now the sole local reporter; callers
   just handle control flow (return cleanly, set channel response to
   unblock remote).

* fix(cli): wire /model handler through the agent loader

The /model command from main (merged via f1f0d7c) still reached for
`state["agent"]` (CLI) and `self._agent` (TUI) — both removed by the
background-loader refactor. CLI raised KeyError on first invocation;
TUI raised AttributeError. Writes to the old fields also had no effect
because every other code path now reads from `agent_loader.agent`, so
the model switch would have silently failed.

Route everything through the loader: `await _await_agent_ready()` up
front so /model doesn't race with the initial background load, build
the `CommandContext` with the current agent, and sync `ctx.agent` back
into `agent_loader.agent` (plus channel globals) when the command
replaces it.

* fix(cli): isolate progress callback, capture awaited agent, gate by requires_agent

Three CodeRabbit findings on the loader + command dispatch path:

- Wrap ``_on_progress`` in try/except inside the loader's gated wrapper
  so a buggy UI adapter can't bubble into ``loader_fn`` and fail the
  whole background load. The MCP client already protects this, but
  defence-in-depth keeps the loader self-contained.

- In CLI channel + main-loop streaming, capture the agent returned by
  ``_await_agent_ready()`` and pass that into ``run_streaming`` rather
  than reading ``agent_loader.agent`` after a subsequent ``await``.
  A concurrent ``/new``/``/resume``/``/model`` could have swapped it.

- Add ``requires_agent: ClassVar[bool] = False`` to ``Command`` and
  mark ``/compact``, ``/model``, ``/channel`` as ``True``. TUI dispatch
  sites (channel and keyboard) now check ``cmd_manager.resolve(...)``
  and only wait for readiness when the command actually needs the
  agent. ``/mcp add``, ``/skills``, ``/new`` etc. no longer deadlock
  behind a failing MCP load they are meant to fix.

* fix(cli): guard sync-back, subcommand-aware gating, /model adopt-path

Three CodeRabbit findings on command dispatch:

- ``_handle_command`` unconditionally synced ``ctx.agent`` back into
  ``agent_loader``.  For non-agent commands ``ctx.agent`` is ``None``,
  so ``/threads`` / ``/mcp`` / ``/skills`` (etc.) could clobber a valid
  loaded agent — and rebind channel globals to ``None``.  Guard the
  sync on ``ctx.agent is not None``.

- ``/channel status`` and ``/channel stop`` don't touch ``ctx.agent``
  but the class-level ``requires_agent = True`` blocked them behind
  agent readiness.  Added ``Command.needs_agent(args)`` (defaults to
  ``requires_agent``) so ``/channel`` can override with subcommand
  awareness; kept the class flag for the common case.

- ``/model`` builds a new agent from scratch, it never reads the
  existing one — gating it on readiness meant a broken provider
  blocked the command that would fix it.  Flipped it to
  ``requires_agent = False`` and added ``BackgroundAgentLoader.adopt``
  so the UI can seat the replacement and supersede any in-flight
  load (the generation token keeps a late completion from clobbering
  the adopted agent).

Bonus cleanup: ``CommandManager.resolve`` now returns
``(command, args)`` so callers can invoke ``needs_agent`` without
re-implementing ``shlex`` parsing.

---------

Co-authored-by: Xi Zhang <106144707+X-iZhang@users.noreply.github.com>
2026-04-22 18:22:33 +02:00

1517 lines
54 KiB
Python

"""Tests for EvoScientist.mcp module."""
import textwrap
from types import SimpleNamespace
import pytest
import yaml
from EvoScientist.mcp.client import (
_build_connections,
_filter_tools,
_interpolate_env,
_resolve_command,
_route_tools,
add_mcp_server,
edit_mcp_server,
load_mcp_config,
parse_mcp_add_args,
parse_mcp_edit_args,
remove_mcp_server,
)
# ---- _interpolate_env ----
class TestInterpolateEnv:
def test_substitutes_env_var(self, monkeypatch):
monkeypatch.setenv("MY_KEY", "secret123")
assert _interpolate_env("Bearer ${MY_KEY}") == "Bearer secret123"
def test_multiple_vars(self, monkeypatch):
monkeypatch.setenv("HOST", "localhost")
monkeypatch.setenv("PORT", "8080")
assert _interpolate_env("${HOST}:${PORT}") == "localhost:8080"
def test_missing_var_returns_empty(self, monkeypatch):
monkeypatch.delenv("NONEXISTENT_VAR_XYZ", raising=False)
assert _interpolate_env("${NONEXISTENT_VAR_XYZ}") == ""
def test_no_vars_unchanged(self):
assert _interpolate_env("plain text") == "plain text"
def test_empty_string(self):
assert _interpolate_env("") == ""
# ---- load_mcp_config ----
@pytest.fixture
def mcp_config_file(monkeypatch, tmp_path):
"""Point USER_MCP_CONFIG to a temp file for isolated testing."""
cfg = tmp_path / "mcp.yaml"
monkeypatch.setattr("EvoScientist.mcp.client.USER_MCP_CONFIG", cfg)
return cfg
class TestLoadMcpConfig:
def test_missing_file_returns_empty(self, mcp_config_file):
# File doesn't exist yet
assert load_mcp_config() == {}
def test_valid_file_parses(self, mcp_config_file):
mcp_config_file.write_text(
textwrap.dedent("""\
my-server:
transport: stdio
command: echo
args: ["hello"]
""")
)
result = load_mcp_config()
assert "my-server" in result
assert result["my-server"]["transport"] == "stdio"
def test_empty_file_returns_empty(self, mcp_config_file):
mcp_config_file.write_text("")
assert load_mcp_config() == {}
def test_comments_only_returns_empty(self, mcp_config_file):
mcp_config_file.write_text("# just a comment\n# another comment\n")
assert load_mcp_config() == {}
def test_env_var_interpolation(self, mcp_config_file, monkeypatch):
monkeypatch.setenv("TEST_TOKEN", "tok_abc")
mcp_config_file.write_text(
textwrap.dedent("""\
my-server:
transport: http
url: "http://localhost:8080/mcp"
headers:
Authorization: "Bearer ${TEST_TOKEN}"
""")
)
result = load_mcp_config()
assert result["my-server"]["headers"]["Authorization"] == "Bearer tok_abc"
# ---- _build_connections ----
# ---- _resolve_command ----
class TestResolveCommand:
def test_absolute_path_returned_as_is(self, tmp_path):
"""Absolute paths are never modified, even if the file doesn't exist."""
fake = str(tmp_path / "mytool")
assert _resolve_command(fake) == fake
def test_found_on_path(self):
"""Commands found via shutil.which are returned as full paths."""
result = _resolve_command("python")
assert result.endswith("python") or result.endswith("python3")
assert result != "python" # resolved, not the bare name
def test_found_in_python_bin(self, tmp_path, monkeypatch):
"""Falls back to sys.executable's directory when not in PATH."""
# Create a fake executable next to sys.executable
bin_dir = tmp_path / "bin"
bin_dir.mkdir()
fake_exe = bin_dir / "my-mcp-tool"
fake_exe.write_text("#!/bin/sh\n")
fake_exe.chmod(0o755)
monkeypatch.setattr("shutil.which", lambda _: None)
monkeypatch.setattr(
"EvoScientist.mcp.client.sys.executable", str(bin_dir / "python")
)
assert _resolve_command("my-mcp-tool") == str(fake_exe)
def test_not_found_returns_original(self, monkeypatch):
"""Returns the original command when not found anywhere (let OS report the error)."""
monkeypatch.setattr("shutil.which", lambda _: None)
monkeypatch.setattr(
"EvoScientist.mcp.client.sys.executable", "/nonexistent/bin/python"
)
assert _resolve_command("unknown-tool-xyz") == "unknown-tool-xyz"
def test_build_connections_resolves_command(self, monkeypatch):
"""_build_connections uses _resolve_command so the full path appears in output."""
monkeypatch.setattr(
"EvoScientist.mcp.client._resolve_command", lambda cmd: f"/resolved/{cmd}"
)
config = {"srv": {"transport": "stdio", "command": "mytool", "args": []}}
conns = _build_connections(config)
assert conns["srv"]["command"] == "/resolved/mytool"
class TestBuildConnections:
def test_stdio_connection(self):
config = {
"fs": {
"transport": "stdio",
"command": "npx",
"args": ["-y", "server"],
}
}
conns = _build_connections(config)
assert "fs" in conns
assert conns["fs"]["transport"] == "stdio"
assert conns["fs"]["command"].endswith("npx")
assert conns["fs"]["args"] == ["-y", "server"]
def test_stdio_with_env(self, monkeypatch):
for var in (
"http_proxy",
"https_proxy",
"HTTP_PROXY",
"HTTPS_PROXY",
"all_proxy",
"ALL_PROXY",
"no_proxy",
"NO_PROXY",
"SSL_CERT_FILE",
"SSL_CERT_DIR",
"REQUESTS_CA_BUNDLE",
"CURL_CA_BUNDLE",
"NODE_EXTRA_CA_CERTS",
):
monkeypatch.delenv(var, raising=False)
config = {
"fs": {
"transport": "stdio",
"command": "npx",
"args": [],
"env": {"FOO": "bar"},
}
}
conns = _build_connections(config)
assert conns["fs"]["env"] == {"FOO": "bar"}
def test_stdio_forwards_proxy_and_cert_env(self, monkeypatch):
"""Proxy and CA bundle env vars are forwarded to stdio subprocesses."""
monkeypatch.setenv("https_proxy", "http://proxy:3128")
monkeypatch.setenv("SSL_CERT_FILE", "/etc/ssl/certs/ca.crt")
config = {"fs": {"transport": "stdio", "command": "npx", "args": []}}
conns = _build_connections(config)
assert conns["fs"]["env"]["https_proxy"] == "http://proxy:3128"
assert conns["fs"]["env"]["SSL_CERT_FILE"] == "/etc/ssl/certs/ca.crt"
def test_stdio_user_env_overrides_forwarded(self, monkeypatch):
"""User-configured env takes precedence over auto-forwarded values."""
monkeypatch.setenv("https_proxy", "http://host-proxy:3128")
config = {
"fs": {
"transport": "stdio",
"command": "npx",
"args": [],
"env": {"https_proxy": "http://user-proxy:9000"},
}
}
conns = _build_connections(config)
assert conns["fs"]["env"]["https_proxy"] == "http://user-proxy:9000"
def test_http_connection(self):
config = {
"api": {
"transport": "http",
"url": "http://localhost:8080/mcp",
"headers": {"Authorization": "Bearer xxx"},
}
}
conns = _build_connections(config)
assert conns["api"]["transport"] == "http"
assert conns["api"]["url"] == "http://localhost:8080/mcp"
assert conns["api"]["headers"]["Authorization"] == "Bearer xxx"
def test_sse_connection(self):
config = {
"sse-srv": {
"transport": "sse",
"url": "http://localhost:9090/sse",
}
}
conns = _build_connections(config)
assert conns["sse-srv"]["transport"] == "sse"
assert conns["sse-srv"]["url"] == "http://localhost:9090/sse"
def test_websocket_connection(self):
config = {
"ws": {
"transport": "websocket",
"url": "ws://localhost:8765",
}
}
conns = _build_connections(config)
assert conns["ws"]["transport"] == "websocket"
def test_unknown_transport_skipped(self):
config = {
"bad": {
"transport": "carrier_pigeon",
"url": "coo://rooftop",
}
}
conns = _build_connections(config)
assert conns == {}
def test_mixed_transports(self):
config = {
"a": {"transport": "stdio", "command": "cmd", "args": []},
"b": {"transport": "http", "url": "http://x"},
"c": {"transport": "unknown"},
}
conns = _build_connections(config)
assert set(conns.keys()) == {"a", "b"}
# ---- _filter_tools ----
def _make_tool(name: str):
"""Create a minimal mock tool with a .name attribute."""
return SimpleNamespace(name=name)
class TestFilterTools:
def test_none_allowlist_passes_all(self):
tools = [_make_tool("a"), _make_tool("b"), _make_tool("c")]
assert _filter_tools(tools, None) == tools
def test_allowlist_filters(self):
tools = [_make_tool("a"), _make_tool("b"), _make_tool("c")]
result = _filter_tools(tools, ["a", "c"])
assert [t.name for t in result] == ["a", "c"]
def test_empty_allowlist_filters_all(self):
tools = [_make_tool("a"), _make_tool("b")]
assert _filter_tools(tools, []) == []
def test_allowlist_with_nonexistent_name(self):
tools = [_make_tool("a")]
result = _filter_tools(tools, ["a", "nonexistent"])
assert [t.name for t in result] == ["a"]
def test_empty_tools_list(self):
assert _filter_tools([], ["a"]) == []
assert _filter_tools([], None) == []
# Wildcard tests
def test_wildcard_star_suffix(self):
"""Test *_exa pattern matching."""
tools = [
_make_tool("web_search_exa"),
_make_tool("get_code_context_exa"),
_make_tool("company_research_exa"),
_make_tool("unrelated_tool"),
]
result = _filter_tools(tools, ["*_exa"])
assert [t.name for t in result] == [
"web_search_exa",
"get_code_context_exa",
"company_research_exa",
]
def test_wildcard_star_prefix(self):
"""Test read_* pattern matching."""
tools = [
_make_tool("read_file"),
_make_tool("read_directory"),
_make_tool("read_link"),
_make_tool("write_file"),
]
result = _filter_tools(tools, ["read_*"])
assert [t.name for t in result] == [
"read_file",
"read_directory",
"read_link",
]
def test_wildcard_star_middle(self):
"""Test pattern with * in the middle."""
tools = [
_make_tool("get_user_data"),
_make_tool("get_admin_data"),
_make_tool("get_file"),
]
result = _filter_tools(tools, ["get_*_data"])
assert [t.name for t in result] == ["get_user_data", "get_admin_data"]
def test_wildcard_star_only(self):
"""Test * matches everything."""
tools = [_make_tool("a"), _make_tool("b"), _make_tool("c")]
result = _filter_tools(tools, ["*"])
assert [t.name for t in result] == ["a", "b", "c"]
def test_wildcard_question_mark(self):
"""Test ? matches single character."""
tools = [
_make_tool("tool_1"),
_make_tool("tool_2"),
_make_tool("tool_10"),
]
result = _filter_tools(tools, ["tool_?"])
assert [t.name for t in result] == ["tool_1", "tool_2"]
def test_wildcard_character_class(self):
"""Test [seq] matches characters in sequence."""
tools = [
_make_tool("tool_a"),
_make_tool("tool_b"),
_make_tool("tool_c"),
_make_tool("tool_d"),
]
result = _filter_tools(tools, ["tool_[abc]"])
assert [t.name for t in result] == ["tool_a", "tool_b", "tool_c"]
def test_wildcard_character_class_range(self):
"""Test [0-9] matches digit range."""
tools = [
_make_tool("tool_0"),
_make_tool("tool_5"),
_make_tool("tool_9"),
_make_tool("tool_a"),
]
result = _filter_tools(tools, ["tool_[0-9]"])
assert [t.name for t in result] == ["tool_0", "tool_5", "tool_9"]
def test_wildcard_negated_character_class(self):
"""Test [!seq] matches characters not in sequence."""
tools = [
_make_tool("tool_a"),
_make_tool("tool_b"),
_make_tool("tool_1"),
_make_tool("tool_2"),
]
result = _filter_tools(tools, ["tool_[!0-9]"])
assert [t.name for t in result] == ["tool_a", "tool_b"]
def test_wildcard_mixed_with_exact(self):
"""Test mixing wildcard and exact patterns."""
tools = [
_make_tool("web_search_exa"),
_make_tool("get_code_context_exa"),
_make_tool("specific_tool"),
_make_tool("another_tool"),
]
result = _filter_tools(tools, ["*_exa", "specific_tool"])
assert [t.name for t in result] == [
"web_search_exa",
"get_code_context_exa",
"specific_tool",
]
def test_wildcard_multiple_patterns(self):
"""Test multiple wildcard patterns."""
tools = [
_make_tool("read_file"),
_make_tool("write_file"),
_make_tool("delete_file"),
_make_tool("search_database"),
]
result = _filter_tools(tools, ["read_*", "write_*"])
assert [t.name for t in result] == ["read_file", "write_file"]
def test_wildcard_no_match(self):
"""Test wildcard pattern that doesn't match anything."""
tools = [_make_tool("foo"), _make_tool("bar")]
result = _filter_tools(tools, ["baz_*"])
assert result == []
def test_wildcard_overlapping_patterns(self):
"""Test overlapping patterns don't duplicate results."""
tools = [_make_tool("tool_abc"), _make_tool("tool_xyz")]
result = _filter_tools(tools, ["tool_*", "*_abc"])
# Should include each tool only once
assert [t.name for t in result] == ["tool_abc", "tool_xyz"]
def test_wildcard_complex_pattern(self):
"""Test complex wildcard pattern."""
tools = [
_make_tool("get_user_info_v1"),
_make_tool("get_user_info_v2"),
_make_tool("get_admin_info_v1"),
_make_tool("set_user_info"),
]
result = _filter_tools(tools, ["get_*_info_v?"])
assert [t.name for t in result] == [
"get_user_info_v1",
"get_user_info_v2",
"get_admin_info_v1",
]
def test_exact_match_performance_path(self):
"""Test exact matching still works (fast path)."""
# This test verifies backward compatibility with exact matching
tools = [_make_tool("a"), _make_tool("b"), _make_tool("c")]
result = _filter_tools(tools, ["a", "c"])
assert [t.name for t in result] == ["a", "c"]
# ---- _route_tools ----
class TestRouteTools:
def test_default_routes_to_main(self):
config = {"srv": {"transport": "stdio"}}
server_tools = {"srv": [_make_tool("x")]}
result = _route_tools(config, server_tools)
assert "main" in result
assert [t.name for t in result["main"]] == ["x"]
def test_expose_to_named_agent(self):
config = {"srv": {"transport": "stdio", "expose_to": ["code-agent"]}}
server_tools = {"srv": [_make_tool("x"), _make_tool("y")]}
result = _route_tools(config, server_tools)
assert "code-agent" in result
assert "main" not in result
assert [t.name for t in result["code-agent"]] == ["x", "y"]
def test_expose_to_multiple_agents(self):
config = {"srv": {"transport": "stdio", "expose_to": ["main", "code-agent"]}}
server_tools = {"srv": [_make_tool("x")]}
result = _route_tools(config, server_tools)
assert [t.name for t in result["main"]] == ["x"]
assert [t.name for t in result["code-agent"]] == ["x"]
def test_tool_filter_applied(self):
config = {"srv": {"transport": "stdio", "tools": ["b"]}}
server_tools = {"srv": [_make_tool("a"), _make_tool("b"), _make_tool("c")]}
result = _route_tools(config, server_tools)
assert [t.name for t in result["main"]] == ["b"]
def test_multiple_servers(self):
config = {
"s1": {"transport": "stdio", "expose_to": ["main"]},
"s2": {"transport": "http", "expose_to": ["research-agent"]},
}
server_tools = {
"s1": [_make_tool("a")],
"s2": [_make_tool("b")],
}
result = _route_tools(config, server_tools)
assert [t.name for t in result["main"]] == ["a"]
assert [t.name for t in result["research-agent"]] == ["b"]
def test_expose_to_string_not_list(self):
config = {"srv": {"transport": "stdio", "expose_to": "debug-agent"}}
server_tools = {"srv": [_make_tool("x")]}
result = _route_tools(config, server_tools)
assert "debug-agent" in result
def test_empty_server_tools(self):
config = {"srv": {"transport": "stdio"}}
server_tools = {"srv": []}
result = _route_tools(config, server_tools)
assert result.get("main", []) == []
# ---- add_mcp_server / remove_mcp_server ----
@pytest.fixture
def user_mcp_dir(tmp_path, monkeypatch):
"""Redirect user MCP config to a temp directory."""
cfg_dir = tmp_path / "config"
cfg_dir.mkdir()
cfg_file = cfg_dir / "mcp.yaml"
monkeypatch.setattr("EvoScientist.mcp.client.USER_CONFIG_DIR", cfg_dir)
monkeypatch.setattr("EvoScientist.mcp.client.USER_MCP_CONFIG", cfg_file)
return cfg_file
class TestAddMcpServer:
def test_add_stdio_server(self, user_mcp_dir):
entry = add_mcp_server(
"fs", "stdio", command="npx", args=["-y", "server", "/tmp"]
)
assert entry["transport"] == "stdio"
assert entry["command"] == "npx"
assert entry["args"] == ["-y", "server", "/tmp"]
# Verify persisted
data = yaml.safe_load(user_mcp_dir.read_text())
assert "fs" in data
def test_add_http_server(self, user_mcp_dir):
entry = add_mcp_server(
"api",
"http",
url="http://localhost:8080/mcp",
headers={"Authorization": "Bearer tok"},
)
assert entry["url"] == "http://localhost:8080/mcp"
assert entry["headers"]["Authorization"] == "Bearer tok"
def test_add_sse_server(self, user_mcp_dir):
entry = add_mcp_server("sse-srv", "sse", url="http://localhost:9090/sse")
assert entry["transport"] == "sse"
def test_add_websocket_server(self, user_mcp_dir):
entry = add_mcp_server("ws", "websocket", url="ws://localhost:8765")
assert entry["transport"] == "websocket"
def test_add_with_tools_and_expose_to(self, user_mcp_dir):
entry = add_mcp_server(
"fs",
"stdio",
command="npx",
args=[],
tools=["read_file"],
expose_to=["main", "code-agent"],
)
assert entry["tools"] == ["read_file"]
assert entry["expose_to"] == ["main", "code-agent"]
def test_add_replaces_existing(self, user_mcp_dir):
add_mcp_server("srv", "stdio", command="old")
add_mcp_server("srv", "http", url="http://new")
data = yaml.safe_load(user_mcp_dir.read_text())
assert data["srv"]["transport"] == "http"
def test_add_invalid_transport_raises(self, user_mcp_dir):
with pytest.raises(ValueError, match="Unknown transport"):
add_mcp_server("bad", "carrier_pigeon", url="coo://rooftop")
def test_stdio_without_command_raises(self, user_mcp_dir):
with pytest.raises(ValueError, match="requires a command"):
add_mcp_server("bad", "stdio")
def test_http_without_url_raises(self, user_mcp_dir):
with pytest.raises(ValueError, match="requires a url"):
add_mcp_server("bad", "http")
def test_add_with_env(self, user_mcp_dir):
entry = add_mcp_server(
"fs", "stdio", command="npx", args=[], env={"FOO": "bar"}
)
assert entry["env"] == {"FOO": "bar"}
def test_add_multiple_servers(self, user_mcp_dir):
add_mcp_server("a", "stdio", command="cmd1")
add_mcp_server("b", "http", url="http://x")
data = yaml.safe_load(user_mcp_dir.read_text())
assert "a" in data
assert "b" in data
class TestRemoveMcpServer:
def test_remove_existing(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx")
assert remove_mcp_server("fs") is True
data = yaml.safe_load(user_mcp_dir.read_text()) or {}
assert "fs" not in data
def test_remove_nonexistent(self, user_mcp_dir):
assert remove_mcp_server("nope") is False
def test_remove_preserves_others(self, user_mcp_dir):
add_mcp_server("a", "stdio", command="cmd1")
add_mcp_server("b", "http", url="http://x")
remove_mcp_server("a")
data = yaml.safe_load(user_mcp_dir.read_text())
assert "a" not in data
assert "b" in data
# ---- _parse_mcp_add_args (CLI arg parser) ----
class TestParseMcpAddArgs:
def test_stdio_basic(self):
r = parse_mcp_add_args(["fs", "npx", "-y", "server", "/tmp"])
assert r["name"] == "fs"
assert r["transport"] == "stdio"
assert r["command"] == "npx"
assert r["args"] == ["-y", "server", "/tmp"]
def test_http_auto_detected(self):
r = parse_mcp_add_args(["api", "http://localhost:8080/mcp"])
assert r["transport"] == "http"
assert r["url"] == "http://localhost:8080/mcp"
def test_https_auto_detected(self):
r = parse_mcp_add_args(["api", "https://example.com/mcp"])
assert r["transport"] == "http"
assert r["url"] == "https://example.com/mcp"
def test_ws_auto_detected(self):
r = parse_mcp_add_args(["ws", "ws://localhost:9090"])
assert r["transport"] == "websocket"
def test_explicit_transport_override(self):
r = parse_mcp_add_args(["srv", "https://example.com/sse", "--transport", "sse"])
assert r["transport"] == "sse"
assert r["url"] == "https://example.com/sse"
def test_explicit_transport_short_flag(self):
r = parse_mcp_add_args(["srv", "https://x", "-T", "websocket"])
assert r["transport"] == "websocket"
def test_tools_flag(self):
r = parse_mcp_add_args(["srv", "http://x", "--tools", "a,b"])
assert r["tools"] == ["a", "b"]
def test_expose_to_flag(self):
r = parse_mcp_add_args(["srv", "http://x", "--expose-to", "main,code-agent"])
assert r["expose_to"] == ["main", "code-agent"]
def test_header_flag(self):
r = parse_mcp_add_args(
["srv", "http://x", "--header", "Authorization:Bearer tok"]
)
assert r["headers"] == {"Authorization": "Bearer tok"}
def test_env_flag(self):
r = parse_mcp_add_args(["srv", "cmd", "--env", "FOO=bar"])
assert r["env"] == {"FOO": "bar"}
def test_too_few_tokens_raises(self):
with pytest.raises(ValueError, match="Usage"):
parse_mcp_add_args(["fs"])
def test_double_dash_ignored(self):
r = parse_mcp_add_args(["srv", "npx", "--", "-y", "pkg"])
assert r["command"] == "npx"
assert r["args"] == ["-y", "pkg"]
assert "--" not in r["args"]
def test_env_ref_flag(self):
r = parse_mcp_add_args(["srv", "cmd", "--env-ref", "FOO"])
assert r["env"] == {"FOO": "${FOO}"}
def test_env_ref_and_env_combined(self):
r = parse_mcp_add_args(
["srv", "cmd", "--env", "DEBUG=true", "--env-ref", "API_KEY"]
)
assert r["env"] == {"DEBUG": "true", "API_KEY": "${API_KEY}"}
def test_missing_command_or_url_raises(self):
with pytest.raises(ValueError, match="command or URL is required"):
parse_mcp_add_args(["fs", "--tools", "a"])
# ---- edit_mcp_server ----
class TestEditMcpServer:
def test_edit_expose_to(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx", args=[])
entry = edit_mcp_server("fs", expose_to=["main", "code-agent"])
assert entry["expose_to"] == ["main", "code-agent"]
assert entry["command"] == "npx" # unchanged
def test_edit_tools(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx", args=[])
entry = edit_mcp_server("fs", tools=["read_file"])
assert entry["tools"] == ["read_file"]
def test_edit_clear_tools(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx", tools=["read_file"])
entry = edit_mcp_server("fs", tools=None)
assert "tools" not in entry
def test_edit_url(self, user_mcp_dir):
add_mcp_server("api", "http", url="http://old:8080/mcp")
entry = edit_mcp_server("api", url="http://new:9090/mcp")
assert entry["url"] == "http://new:9090/mcp"
assert entry["transport"] == "http" # unchanged
def test_edit_nonexistent_raises(self, user_mcp_dir):
with pytest.raises(KeyError, match="not found"):
edit_mcp_server("nope", tools=["a"])
def test_edit_invalid_transport_raises(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx")
with pytest.raises(ValueError, match="Unknown transport"):
edit_mcp_server("fs", transport="carrier_pigeon")
def test_edit_removes_required_field_raises(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx")
with pytest.raises(ValueError, match="requires a command"):
edit_mcp_server("fs", command=None)
def test_edit_preserves_unrelated_fields(self, user_mcp_dir):
add_mcp_server(
"fs",
"stdio",
command="npx",
args=["-y", "srv"],
tools=["a"],
expose_to=["main"],
)
entry = edit_mcp_server("fs", expose_to=["code-agent"])
assert entry["tools"] == ["a"]
assert entry["args"] == ["-y", "srv"]
assert entry["expose_to"] == ["code-agent"]
# ---- parse_mcp_edit_args ----
class TestParseMcpEditArgs:
def test_basic_field(self):
name, fields = parse_mcp_edit_args(["srv", "--url", "http://new"])
assert name == "srv"
assert fields["url"] == "http://new"
def test_tools_none_clears(self):
_, fields = parse_mcp_edit_args(["srv", "--tools", "none"])
assert fields["tools"] is None
def test_expose_to_csv(self):
_, fields = parse_mcp_edit_args(["srv", "--expose-to", "main,code-agent"])
assert fields["expose_to"] == ["main", "code-agent"]
def test_multiple_fields(self):
_, fields = parse_mcp_edit_args(["srv", "--url", "http://x", "--tools", "a,b"])
assert fields["url"] == "http://x"
assert fields["tools"] == ["a", "b"]
def test_empty_raises(self):
with pytest.raises(ValueError, match="Usage"):
parse_mcp_edit_args([])
def test_no_fields_raises(self):
with pytest.raises(ValueError, match="No fields"):
parse_mcp_edit_args(["srv"])
# ---- uv tool compatibility ----
class TestUvToolCompat:
"""Tests for uv tool environment detection and compatible install helpers."""
# -- _is_uv_tool_env --
def test_is_uv_tool_env_false_when_no_virtual_env(self, monkeypatch):
from EvoScientist.mcp.registry import _is_uv_tool_env
monkeypatch.delenv("VIRTUAL_ENV", raising=False)
assert _is_uv_tool_env() is False
def test_is_uv_tool_env_false_for_regular_venv(self, monkeypatch):
from EvoScientist.mcp.registry import _is_uv_tool_env
monkeypatch.setenv("VIRTUAL_ENV", "/home/user/projects/myapp/.venv")
assert _is_uv_tool_env() is False
def test_is_uv_tool_env_true_unix(self, monkeypatch):
from EvoScientist.mcp.registry import _is_uv_tool_env
monkeypatch.setenv(
"VIRTUAL_ENV", "/home/user/.local/share/uv/tools/evoscientist"
)
assert _is_uv_tool_env() is True
def test_is_uv_tool_env_true_windows_backslashes(self, monkeypatch):
from EvoScientist.mcp.registry import _is_uv_tool_env
monkeypatch.setenv(
"VIRTUAL_ENV", r"C:\Users\user\AppData\Local\uv\tools\evoscientist"
)
assert _is_uv_tool_env() is True
# -- _uv_tool_name --
def test_uv_tool_name_returns_name(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.setenv(
"VIRTUAL_ENV", "/home/user/.local/share/uv/tools/evoscientist"
)
assert reg._uv_tool_name() == "evoscientist"
def test_uv_tool_name_returns_none_when_not_uv(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.setenv("VIRTUAL_ENV", "/home/user/projects/myapp/.venv")
assert reg._uv_tool_name() is None
def test_uv_tool_name_returns_none_when_no_virtual_env(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.delenv("VIRTUAL_ENV", raising=False)
assert reg._uv_tool_name() is None
# -- _uv_tool_existing_requirements --
def test_existing_requirements_from_receipt(self, monkeypatch, tmp_path):
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "arxiv-mcp-server" },\n'
' { name = "rich" },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
result = reg._uv_tool_existing_requirements()
assert result == {"arxiv-mcp-server": "arxiv-mcp-server", "rich": "rich"}
def test_existing_requirements_preserves_specifiers_and_extras(
self, monkeypatch, tmp_path
):
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "rich", specifier = ">=13.0" },\n'
' { name = "requests", extras = ["socks"] },\n'
' { name = "lark-oapi", specifier = ">=1.4.0", extras = ["oauth"] },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
result = reg._uv_tool_existing_requirements()
assert result == {
"rich": "rich>=13.0",
"requests": "requests[socks]",
"lark-oapi": "lark-oapi[oauth]>=1.4.0",
}
def test_existing_requirements_excludes_tool_name(self, monkeypatch, tmp_path):
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
'[tool]\nrequirements = [\n { name = "evoscientist" },\n]\n'
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
assert reg._uv_tool_existing_requirements() == {}
def test_existing_requirements_no_receipt(self, monkeypatch, tmp_path):
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
assert reg._uv_tool_existing_requirements() == {}
# -- pip_install_hint --
def test_pip_install_hint_uv_tool(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: True)
hint = reg.pip_install_hint()
assert "uv tool install --reinstall evoscientist --with" in hint
def test_pip_install_hint_uv_no_tool(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
assert reg.pip_install_hint() == "uv pip install"
def test_pip_install_hint_plain_pip(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(reg.shutil, "which", lambda x: None)
assert reg.pip_install_hint() == "pip install"
# -- install_library / install_cli_tool --
def test_install_library_uv_tool_env_uses_uv_tool_install(
self, monkeypatch, tmp_path
):
"""In a uv tool env, should use ``uv tool install --with`` for durability."""
import EvoScientist.mcp.registry as reg
# Set up a fake uv tool env with receipt
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "existing-pkg" },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(list(cmd))
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
result = reg.install_library("new-mcp-server")
assert result is True
assert len(captured) == 1
cmd = captured[0]
assert cmd[:3] == ["uv", "tool", "install"]
assert "evoscientist" in cmd
# Must preserve existing --with requirement
assert "--with" in cmd
with_args = [cmd[i + 1] for i, v in enumerate(cmd) if v == "--with"]
assert "existing-pkg" in with_args
assert "new-mcp-server" in with_args
def test_install_library_uv_tool_env_no_duplicate_with(self, monkeypatch, tmp_path):
"""If the package is already in the receipt, don't add it twice."""
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "arxiv-mcp-server" },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(list(cmd))
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
reg.install_library("arxiv-mcp-server")
cmd = captured[0]
with_args = [cmd[i + 1] for i, v in enumerate(cmd) if v == "--with"]
assert with_args.count("arxiv-mcp-server") == 1
def test_install_library_uv_tool_preserves_specifiers(self, monkeypatch, tmp_path):
"""Existing --with specs with extras/versions must be preserved."""
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "rich", specifier = ">=13.0" },\n'
' { name = "requests", extras = ["socks"] },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(list(cmd))
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
reg.install_library("new-pkg")
cmd = captured[0]
with_args = [cmd[i + 1] for i, v in enumerate(cmd) if v == "--with"]
assert "rich>=13.0" in with_args
assert "requests[socks]" in with_args
assert "new-pkg" in with_args
def test_install_library_uv_tool_dedup_with_version_spec(
self, monkeypatch, tmp_path
):
"""Dedup must match bare name even if package arg has version spec."""
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "rich", specifier = ">=13.0" },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(list(cmd))
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
# package arg has version constraint — should still dedup against "rich"
reg.install_library("rich>=14.0")
cmd = captured[0]
with_args = [cmd[i + 1] for i, v in enumerate(cmd) if v == "--with"]
# Should keep the existing spec, not add a duplicate
assert with_args.count("rich>=13.0") == 1
assert "rich>=14.0" not in with_args
def test_install_library_uv_tool_falls_back_on_failure(self, monkeypatch, tmp_path):
"""install_library: if ``uv tool install --with`` fails, fall back
to ``uv pip install`` — standalone uv tool install is NEVER tried."""
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
'[tool]\nrequirements = [\n { name = "evoscientist" },\n]\n'
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
commands: list[list[str]] = []
def fake_run(cmd, **kwargs):
commands.append(cmd)
if cmd[:3] == ["uv", "tool", "install"]:
return type("R", (), {"returncode": 1})()
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
result = reg.install_library("some-package")
assert result is True
# Order: uv tool install --with (fail), uv pip install (ok).
assert len(commands) == 2
assert commands[0][:3] == ["uv", "tool", "install"]
assert "--with" in commands[0]
assert commands[1][:3] == ["uv", "pip", "install"]
def test_install_cli_tool_uv_tool_env_tries_standalone_on_failure(
self, monkeypatch, tmp_path
):
"""install_cli_tool: uv-tool env, --with fails → standalone uv tool
install is tried, then pip fallback."""
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
'[tool]\nrequirements = [\n { name = "evoscientist" },\n]\n'
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
commands: list[list[str]] = []
def fake_run(cmd, **kwargs):
commands.append(cmd)
if cmd[:3] == ["uv", "tool", "install"]:
return type("R", (), {"returncode": 1})()
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
result = reg.install_cli_tool("some-cli", verify_command="some-cli")
assert result is True
assert len(commands) == 3
assert commands[0][:3] == ["uv", "tool", "install"]
assert "--with" in commands[0]
assert commands[1][:3] == ["uv", "tool", "install"]
assert "--with" not in commands[1]
assert commands[2][:3] == ["uv", "pip", "install"]
def test_install_library_goes_straight_to_pip_outside_uv_tool(self, monkeypatch):
"""install_library outside a uv-tool env must skip ``uv tool install
<pkg>`` entirely — standalone uv tools aren't importable."""
import sys
import EvoScientist.mcp.registry as reg
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(cmd)
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
result = reg.install_library("some-library")
assert result is True
assert len(captured) == 1
assert captured[0][:3] == ["uv", "pip", "install"]
assert sys.executable in captured[0]
def test_install_cli_tool_prefers_standalone_uv_tool_install(
self, monkeypatch, tmp_path
):
"""install_cli_tool outside a uv-tool env: standalone ``uv tool
install <pkg>`` is preferred so the binary survives uv sync."""
import EvoScientist.mcp.registry as reg
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(cmd)
return type("R", (), {"returncode": 0})()
fake_bin = tmp_path / "some-cli"
fake_bin.write_text("#!/bin/sh\n")
fake_bin.chmod(0o755)
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
monkeypatch.setattr(
reg, "_uv_tool_bin", lambda cmd: fake_bin if cmd == "some-cli" else None
)
result = reg.install_cli_tool("some-cli", verify_command="some-cli")
assert result is True
assert len(captured) == 1
assert captured[0][:3] == ["uv", "tool", "install"]
assert "some-cli" in captured[0]
def test_install_cli_tool_missing_bin_triggers_pip_fallback(self, monkeypatch):
"""install_cli_tool: if the binary isn't in uv's tool bin dir after
``uv tool install`` (e.g. package has no console-script), fall
through to ``uv pip install``."""
import sys
import EvoScientist.mcp.registry as reg
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(cmd)
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
monkeypatch.setattr(reg, "_uv_tool_bin", lambda cmd: None)
result = reg.install_cli_tool(
"lib-without-entrypoint", verify_command="ghost-cli"
)
assert result is True
assert len(captured) == 2
assert captured[0][:3] == ["uv", "tool", "install"]
assert "--python" in captured[1]
assert sys.executable in captured[1]
def test_install_cli_tool_bin_present_short_circuits(self, monkeypatch, tmp_path):
"""install_cli_tool: ``uv tool install`` success + binary present in
uv-tool bin dir ⇒ no pip fallback, even if a stale copy exists in
the venv on PATH."""
import EvoScientist.mcp.registry as reg
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(cmd)
return type("R", (), {"returncode": 0})()
fake_bin = tmp_path / "arxiv-mcp-server"
fake_bin.write_text("#!/bin/sh\n")
fake_bin.chmod(0o755)
def fake_which(x):
if x == "uv":
return "/usr/bin/uv"
if x == "arxiv-mcp-server":
return "/venv/bin/arxiv-mcp-server" # stale, should NOT be trusted
return None
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(reg.shutil, "which", fake_which)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
monkeypatch.setattr(
reg,
"_uv_tool_bin",
lambda cmd: fake_bin if cmd == "arxiv-mcp-server" else None,
)
result = reg.install_cli_tool(
"arxiv-mcp-server", verify_command="arxiv-mcp-server"
)
assert result is True
assert len(captured) == 1
assert captured[0][:3] == ["uv", "tool", "install"]
def test_install_library_falls_back_to_pip_when_no_uv(self, monkeypatch):
import sys
import EvoScientist.mcp.registry as reg
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(cmd)
ns = type("R", (), {"returncode": 0})()
return ns
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(reg.shutil, "which", lambda x: None)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
reg.install_library("some-package")
assert len(captured) == 1
assert sys.executable in captured[0]
assert "-m" in captured[0]
assert "pip" in captured[0]
# -- _resolve_command_path --
def test_resolve_command_path_absolute_passthrough(self):
from EvoScientist.mcp.registry import _resolve_command_path
assert _resolve_command_path("/usr/bin/my-tool") == "/usr/bin/my-tool"
def test_resolve_command_path_found_in_bin_dir(self, monkeypatch, tmp_path):
import sys
import EvoScientist.mcp.registry as reg
# Create a fake executable in a temp bin dir
bin_dir = tmp_path / "bin"
bin_dir.mkdir()
fake_exe = bin_dir / "my-mcp-server"
fake_exe.touch()
fake_exe.chmod(0o755)
# Point sys.executable to something in that bin dir
fake_python = bin_dir / "python"
fake_python.touch()
monkeypatch.setattr(sys, "executable", str(fake_python))
# Ensure shutil.which won't find it on PATH
monkeypatch.setattr(reg.shutil, "which", lambda x: None)
result = reg._resolve_command_path("my-mcp-server")
assert result == str(fake_exe)
def test_resolve_command_path_windows_exe_suffix(self, monkeypatch, tmp_path):
import os
import sys
import EvoScientist.mcp.registry as reg
if os.name != "nt":
pytest.skip("Windows-only behaviour")
bin_dir = tmp_path / "bin"
bin_dir.mkdir()
fake_exe = bin_dir / "my-mcp-server.exe"
fake_exe.touch()
monkeypatch.setattr(sys, "executable", str(bin_dir / "python.exe"))
monkeypatch.setattr(reg.shutil, "which", lambda x: None)
result = reg._resolve_command_path("my-mcp-server")
assert result == str(fake_exe)
def test_resolve_command_path_returns_bare_when_not_found(
self, monkeypatch, tmp_path
):
import sys
import EvoScientist.mcp.registry as reg
monkeypatch.setattr(reg.shutil, "which", lambda x: None)
monkeypatch.setattr(reg, "_uv_tool_bin", lambda cmd: None)
monkeypatch.setattr(sys, "executable", str(tmp_path / "bin" / "python"))
result = reg._resolve_command_path("nonexistent-tool")
assert result == "nonexistent-tool"
def test_resolve_command_path_prefers_uv_tool_over_venv_shadow(
self, monkeypatch, tmp_path
):
"""When both ``uv tool dir --bin`` and a venv's ``bin/`` contain the
command, prefer the uv-tool location so the path written to mcp.yaml
survives ``uv sync``."""
import EvoScientist.mcp.registry as reg
uv_bin_dir = tmp_path / "uv-bin"
uv_bin_dir.mkdir()
uv_copy = uv_bin_dir / "arxiv-mcp-server"
uv_copy.write_text("#!/bin/sh\n")
uv_copy.chmod(0o755)
# Stale venv copy that `uv run` would surface first via PATH.
venv_copy = tmp_path / "venv-bin" / "arxiv-mcp-server"
venv_copy.parent.mkdir()
venv_copy.write_text("#!/bin/sh\n")
venv_copy.chmod(0o755)
monkeypatch.setattr(reg, "_uv_tool_bin_dir", lambda: uv_bin_dir)
monkeypatch.setattr(reg.shutil, "which", lambda x: str(venv_copy))
result = reg._resolve_command_path("arxiv-mcp-server")
assert result == str(uv_copy)
# ---- _load_tools progress callback ----
class TestLoadToolsProgressCallback:
"""Verify per-server ``on_progress`` events fire in the expected order."""
@staticmethod
def _patch_client(monkeypatch, behavior):
"""Install a fake MultiServerMCPClient whose ``get_tools`` delegates
to *behavior* — a ``dict[server_name, list | Exception]``. Each
success value is returned; Exception values are raised.
"""
class _FakeClient:
def __init__(self, connections):
self.connections = connections
async def get_tools(self, server_name):
outcome = behavior[server_name]
if isinstance(outcome, Exception):
raise outcome
return outcome
import langchain_mcp_adapters.client as lc_client
monkeypatch.setattr(lc_client, "MultiServerMCPClient", _FakeClient)
def test_success_emits_start_then_success_with_tool_count(self, monkeypatch):
import asyncio
from EvoScientist.mcp.client import _load_tools
events: list[tuple[str, str, str]] = []
self._patch_client(
monkeypatch,
{"srv": ["tool1", "tool2", "tool3"]},
)
config = {"srv": {"transport": "stdio", "command": "demo"}}
def record(event, name, detail):
events.append((event, name, detail))
asyncio.run(_load_tools(config, on_progress=record))
assert events == [
("start", "srv", ""),
("success", "srv", "3"),
]
def test_failure_emits_start_then_error_with_detail(self, monkeypatch):
import asyncio
from EvoScientist.mcp.client import _load_tools
events: list[tuple[str, str, str]] = []
self._patch_client(monkeypatch, {"srv": RuntimeError("boom")})
config = {"srv": {"transport": "stdio", "command": "demo"}}
def record(event, name, detail):
events.append((event, name, detail))
asyncio.run(_load_tools(config, on_progress=record))
assert events == [
("start", "srv", ""),
("error", "srv", "boom"),
]
def test_mixed_fleet_reports_each_server_independently(self, monkeypatch):
import asyncio
from EvoScientist.mcp.client import _load_tools
events: list[tuple[str, str, str]] = []
self._patch_client(
monkeypatch,
{
"ok_srv": ["a"],
"bad_srv": ConnectionError("refused"),
},
)
config = {
"ok_srv": {"transport": "stdio", "command": "demo"},
"bad_srv": {"transport": "stdio", "command": "demo"},
}
def record(event, name, detail):
events.append((event, name, detail))
asyncio.run(_load_tools(config, on_progress=record))
by_server = {}
for ev, name, detail in events:
by_server.setdefault(name, []).append((ev, detail))
assert by_server["ok_srv"] == [("start", ""), ("success", "1")]
assert by_server["bad_srv"] == [("start", ""), ("error", "refused")]
def test_callback_errors_do_not_break_the_load(self, monkeypatch):
import asyncio
from EvoScientist.mcp.client import _load_tools
self._patch_client(monkeypatch, {"srv": ["tool1"]})
config = {"srv": {"transport": "stdio", "command": "demo"}}
def bad_callback(event, name, detail):
raise RuntimeError("callback bug")
result = asyncio.run(_load_tools(config, on_progress=bad_callback))
assert result == {"srv": ["tool1"]}
def test_semaphore_caps_concurrent_connections(self, monkeypatch):
"""Many configured servers must not all spawn at once."""
import asyncio
from EvoScientist.mcp import client as mcp_client
inflight = {"count": 0, "peak": 0}
class _TrackingClient:
def __init__(self, connections):
self.connections = connections
async def get_tools(self, server_name):
inflight["count"] += 1
inflight["peak"] = max(inflight["peak"], inflight["count"])
# Yield so sibling tasks can try to enter concurrently.
await asyncio.sleep(0)
inflight["count"] -= 1
return [f"tool-of-{server_name}"]
import langchain_mcp_adapters.client as lc_client
monkeypatch.setattr(lc_client, "MultiServerMCPClient", _TrackingClient)
# Force a small cap so the test doesn't depend on the default.
monkeypatch.setattr(mcp_client, "_MAX_CONCURRENT_CONNECTIONS", 3)
config = {
f"srv{i}": {"transport": "stdio", "command": "demo"} for i in range(10)
}
asyncio.run(mcp_client._load_tools(config))
assert inflight["peak"] <= 3
assert inflight["peak"] > 1 # sanity: we *are* parallelizing