Files
EvoScientist/tests/test_mcp_client.py
T
houren Antony d4f1fbd110 ci: add windows-latest to test matrix + fix 11 cross-platform test bugs (#271)
* ci: add windows-latest to test matrix + fix 11 cross-platform test bugs

The test workflow ran on ``ubuntu-latest`` only. Per the issue's
first bullet — the maintainer's explicit #1 priority — add
``windows-latest`` to the matrix so the manager and related
modules are exercised on Windows on every PR.

The matrix addition surfaces 18 pre-existing Windows-only test
failures. Without fixes the new leg would be 18+ reds from
day one and the matrix would just produce a wall of
``fail-fast`` noise. This PR fixes 11 of them; each fix is
a real (cross-platform) bug, not a Windows-specific hack —
most were already flagged by CodeRabbit on PR #236 but never
acted on. The remaining 4 failures need code refactors
(``os.killpg`` → ``psutil`` in ``background.py``,
``convert_virtual_paths_in_command`` Windows-aware quoting,
tilde expansion) that are documented as out-of-scope
follow-ups below.

## What changed

* ``.github/workflows/test.yml``
  - ``os: [ubuntu-latest, windows-latest]`` → 2 OS × 2 Python
    = 4 cells.
  - ``fail-fast: false`` so one bad cell doesn't cancel the
    rest while the Windows leg is being brought up. Removable
    in a future PR once the suite is fully green.

* ``tests/test_backends.py``
  - Hard-coded ``"python3"`` → ``{sys.executable}`` in 7
    test commands. Windows has no ``python3`` on PATH; using
    ``sys.executable`` is portable and matches what CodeRabbit
    flagged on PR #236.
  - Strict string comparisons → ``shlex.split`` round-trip in
    5 resolver tests. ``shlex.quote`` adds single quotes
    around backslash paths on Windows, which broke the
    direct ``==`` compare.
  - Cross-platform suffix checks in 2 path-resolution tests
    (``Path(resolved).parts[-2:]`` instead of
    ``str(resolved).endswith("src/main.py")``).
  - ``mkdir -p`` → ``sys.executable -c "import os;
    os.makedirs(...)"`` in the cwd-sanitization test.
  - ``skipif(sys.platform == "win32")`` on 3 e2e tests that
    hit the underlying ``shlex.quote`` + ``cmd.exe`` quoting
    bug (real, separate issue).

* ``tests/test_sessions.py``
  - ``test_uses_data_dir``: check ``.evoscientist`` in the
    long path form (via ``Path.resolve()``) rather than the
    short-path form ``get_db_path`` returns on Windows.

* ``tests/test_mcp_client.py``
  - ``endswith("python")`` → ``Path(result).stem.lower()`` so
    ``python.EXE`` matches on Windows.
  - ``endswith("npx")`` also accepts ``npx.cmd`` so the npm
    shim on Windows matches.

## Out of scope (follow-up issues to file)

* ``os.killpg`` doesn't exist on Windows
  (``EvoScientist/background.py:248``) — 3 background tests
  fail. Real fix is the same ``psutil`` walk pattern PR #200
  shipped in ``langgraph_dev/manager.py``.
* Tilde expansion in file mentions.
* Windows-aware shell quoting in
  ``convert_virtual_paths_in_command``.
* Path conventions (``~/.config/evoscientist/`` vs
  ``%APPDATA%\EvoScientist``) — needs design discussion +
  ``platformdirs`` migration.
* Cross-module audit of
  ``EvoScientist/tools/execute.py``,
  ``EvoScientist/ccproxy_manager.py``,
  ``EvoScientist/config/onboard.py``.

Closes #207 (step 1 only — CI matrix + the easy test
fixes; remaining bullets tracked separately).

* fix: cross-platform compatibility for Windows CI runners

- background.py: replace POSIX-only os.killpg/os.getpgid with
  cross-platform _kill_process_tree() helper. On Windows falls back
  to Popen.terminate()/Popen.kill() (TerminateProcess); on POSIX
  keeps existing os.killpg logic.

- test_backends.py: replace mkdir -p shell execution in
  test_literal_workspace_path_replaced with preprocessing-boundary
  assertion (patch LocalShellBackend.execute, capture command,
  assert workspace path was rewritten to ./). Avoids POSIX-only
  mkdir -p on Windows runners.

- test_file_mentions.py: monkeypatch USERPROFILE on Windows so
  ntpath.expanduser() resolves ~ to tmp_path even when HOME is
  unset on CI runners.

* fix(test): cross-platform sleep/true commands for Windows CI

Replace POSIX-only sleep/true with module-level helpers that use
ping -n / cmd /c on Windows. Also fix python3 -> sys.executable
in the non-timeout recovery test.

- test_background.py: 7 sleep/true fixes
- test_background_middleware.py: 6 sleep/true fixes
- test_backends.py: 4 sleep fixes + 1 python3 fix

2318 passed, 0 failed on Windows.

* fix(test): use shell-portable double quotes for python -c on Windows

cmd.exe does not treat single quotes as string delimiters, so
-c 'raise SystemExit(1)' was passed with literal quotes on Windows.
Switch to double quotes which work on both cmd.exe and POSIX sh.

* fix: use psutil for Windows process tree kill + avoid sys.executable under uv

- background.py: replace Popen.terminate()/kill() with psutil-based
  process tree walking on Windows. TerminateProcess does NOT cascade
  to grandchildren; psutil.Process.children(recursive=True) ensures
  the entire tree is signaled.

- test_backends.py: replace sys.executable with 'python' in sandbox
  execute() calls. Under uv, sys.executable is under the workspace
  and gets rewritten to ./ by prepare_sandbox_command, breaking
  Linux CI. The plain 'python' command resolves correctly in any
  activated venv.

* fix: broaden try/except in _kill_process_tree to cover proc.children()

If the process exits between Process(popen.pid) and children(recursive=True),
the children call raises an uncaught exception escaping stop(). Move it inside
the existing try/except block.

* fix: narrow exception to ProcessLookupError in POSIX _kill_process_tree

OSError is too broad — would silently swallow EPERM on SIGKILL, leaving
the process alive when we report it as stopped. Match original behavior
which only caught ProcessLookupError (process already gone).

* style: ruff format test_backends.py

* ci: trigger re-run for flaky prompt_toolkit test

* style: fix ruff check (import order + RUF005 unpacking)

---------

Co-authored-by: Xi Zhang <106144707+X-iZhang@users.noreply.github.com>
2026-06-10 15:43:18 +01:00

1523 lines
54 KiB
Python

"""Tests for EvoScientist.mcp module."""
import textwrap
from pathlib import Path
from types import SimpleNamespace
import pytest
import yaml
from EvoScientist.mcp.client import (
_build_connections,
_filter_tools,
_interpolate_env,
_resolve_command,
_route_tools,
add_mcp_server,
edit_mcp_server,
load_mcp_config,
parse_mcp_add_args,
parse_mcp_edit_args,
remove_mcp_server,
)
# ---- _interpolate_env ----
class TestInterpolateEnv:
def test_substitutes_env_var(self, monkeypatch):
monkeypatch.setenv("MY_KEY", "secret123")
assert _interpolate_env("Bearer ${MY_KEY}") == "Bearer secret123"
def test_multiple_vars(self, monkeypatch):
monkeypatch.setenv("HOST", "localhost")
monkeypatch.setenv("PORT", "8080")
assert _interpolate_env("${HOST}:${PORT}") == "localhost:8080"
def test_missing_var_returns_empty(self, monkeypatch):
monkeypatch.delenv("NONEXISTENT_VAR_XYZ", raising=False)
assert _interpolate_env("${NONEXISTENT_VAR_XYZ}") == ""
def test_no_vars_unchanged(self):
assert _interpolate_env("plain text") == "plain text"
def test_empty_string(self):
assert _interpolate_env("") == ""
# ---- load_mcp_config ----
@pytest.fixture
def mcp_config_file(monkeypatch, tmp_path):
"""Point USER_MCP_CONFIG to a temp file for isolated testing."""
cfg = tmp_path / "mcp.yaml"
monkeypatch.setattr("EvoScientist.mcp.client.USER_MCP_CONFIG", cfg)
return cfg
class TestLoadMcpConfig:
def test_missing_file_returns_empty(self, mcp_config_file):
# File doesn't exist yet
assert load_mcp_config() == {}
def test_valid_file_parses(self, mcp_config_file):
mcp_config_file.write_text(
textwrap.dedent("""\
my-server:
transport: stdio
command: echo
args: ["hello"]
""")
)
result = load_mcp_config()
assert "my-server" in result
assert result["my-server"]["transport"] == "stdio"
def test_empty_file_returns_empty(self, mcp_config_file):
mcp_config_file.write_text("")
assert load_mcp_config() == {}
def test_comments_only_returns_empty(self, mcp_config_file):
mcp_config_file.write_text("# just a comment\n# another comment\n")
assert load_mcp_config() == {}
def test_env_var_interpolation(self, mcp_config_file, monkeypatch):
monkeypatch.setenv("TEST_TOKEN", "tok_abc")
mcp_config_file.write_text(
textwrap.dedent("""\
my-server:
transport: http
url: "http://localhost:8080/mcp"
headers:
Authorization: "Bearer ${TEST_TOKEN}"
""")
)
result = load_mcp_config()
assert result["my-server"]["headers"]["Authorization"] == "Bearer tok_abc"
# ---- _build_connections ----
# ---- _resolve_command ----
class TestResolveCommand:
def test_absolute_path_returned_as_is(self, tmp_path):
"""Absolute paths are never modified, even if the file doesn't exist."""
fake = str(tmp_path / "mytool")
assert _resolve_command(fake) == fake
def test_found_on_path(self):
"""Commands found via shutil.which are returned as full paths."""
result = _resolve_command("python")
# Cross-platform: ``shutil.which`` may return ``python.exe`` /
# ``python3.exe`` (case may differ). Match the basename stem
# case-insensitively rather than pinning the suffix.
assert Path(result).stem.lower() in ("python", "python3")
assert result != "python" # resolved, not the bare name
def test_found_in_python_bin(self, tmp_path, monkeypatch):
"""Falls back to sys.executable's directory when not in PATH."""
# Create a fake executable next to sys.executable
bin_dir = tmp_path / "bin"
bin_dir.mkdir()
fake_exe = bin_dir / "my-mcp-tool"
fake_exe.write_text("#!/bin/sh\n")
fake_exe.chmod(0o755)
monkeypatch.setattr("shutil.which", lambda _: None)
monkeypatch.setattr(
"EvoScientist.mcp.client.sys.executable", str(bin_dir / "python")
)
assert _resolve_command("my-mcp-tool") == str(fake_exe)
def test_not_found_returns_original(self, monkeypatch):
"""Returns the original command when not found anywhere (let OS report the error)."""
monkeypatch.setattr("shutil.which", lambda _: None)
monkeypatch.setattr(
"EvoScientist.mcp.client.sys.executable", "/nonexistent/bin/python"
)
assert _resolve_command("unknown-tool-xyz") == "unknown-tool-xyz"
def test_build_connections_resolves_command(self, monkeypatch):
"""_build_connections uses _resolve_command so the full path appears in output."""
monkeypatch.setattr(
"EvoScientist.mcp.client._resolve_command", lambda cmd: f"/resolved/{cmd}"
)
config = {"srv": {"transport": "stdio", "command": "mytool", "args": []}}
conns = _build_connections(config)
assert conns["srv"]["command"] == "/resolved/mytool"
class TestBuildConnections:
def test_stdio_connection(self):
config = {
"fs": {
"transport": "stdio",
"command": "npx",
"args": ["-y", "server"],
}
}
conns = _build_connections(config)
assert "fs" in conns
assert conns["fs"]["transport"] == "stdio"
assert conns["fs"]["command"].endswith("npx") or conns["fs"][
"command"
].lower().endswith("npx.cmd")
assert conns["fs"]["args"] == ["-y", "server"]
def test_stdio_with_env(self, monkeypatch):
for var in (
"http_proxy",
"https_proxy",
"HTTP_PROXY",
"HTTPS_PROXY",
"all_proxy",
"ALL_PROXY",
"no_proxy",
"NO_PROXY",
"SSL_CERT_FILE",
"SSL_CERT_DIR",
"REQUESTS_CA_BUNDLE",
"CURL_CA_BUNDLE",
"NODE_EXTRA_CA_CERTS",
):
monkeypatch.delenv(var, raising=False)
config = {
"fs": {
"transport": "stdio",
"command": "npx",
"args": [],
"env": {"FOO": "bar"},
}
}
conns = _build_connections(config)
assert conns["fs"]["env"] == {"FOO": "bar"}
def test_stdio_forwards_proxy_and_cert_env(self, monkeypatch):
"""Proxy and CA bundle env vars are forwarded to stdio subprocesses."""
monkeypatch.setenv("https_proxy", "http://proxy:3128")
monkeypatch.setenv("SSL_CERT_FILE", "/etc/ssl/certs/ca.crt")
config = {"fs": {"transport": "stdio", "command": "npx", "args": []}}
conns = _build_connections(config)
assert conns["fs"]["env"]["https_proxy"] == "http://proxy:3128"
assert conns["fs"]["env"]["SSL_CERT_FILE"] == "/etc/ssl/certs/ca.crt"
def test_stdio_user_env_overrides_forwarded(self, monkeypatch):
"""User-configured env takes precedence over auto-forwarded values."""
monkeypatch.setenv("https_proxy", "http://host-proxy:3128")
config = {
"fs": {
"transport": "stdio",
"command": "npx",
"args": [],
"env": {"https_proxy": "http://user-proxy:9000"},
}
}
conns = _build_connections(config)
assert conns["fs"]["env"]["https_proxy"] == "http://user-proxy:9000"
def test_http_connection(self):
config = {
"api": {
"transport": "http",
"url": "http://localhost:8080/mcp",
"headers": {"Authorization": "Bearer xxx"},
}
}
conns = _build_connections(config)
assert conns["api"]["transport"] == "http"
assert conns["api"]["url"] == "http://localhost:8080/mcp"
assert conns["api"]["headers"]["Authorization"] == "Bearer xxx"
def test_sse_connection(self):
config = {
"sse-srv": {
"transport": "sse",
"url": "http://localhost:9090/sse",
}
}
conns = _build_connections(config)
assert conns["sse-srv"]["transport"] == "sse"
assert conns["sse-srv"]["url"] == "http://localhost:9090/sse"
def test_websocket_connection(self):
config = {
"ws": {
"transport": "websocket",
"url": "ws://localhost:8765",
}
}
conns = _build_connections(config)
assert conns["ws"]["transport"] == "websocket"
def test_unknown_transport_skipped(self):
config = {
"bad": {
"transport": "carrier_pigeon",
"url": "coo://rooftop",
}
}
conns = _build_connections(config)
assert conns == {}
def test_mixed_transports(self):
config = {
"a": {"transport": "stdio", "command": "cmd", "args": []},
"b": {"transport": "http", "url": "http://x"},
"c": {"transport": "unknown"},
}
conns = _build_connections(config)
assert set(conns.keys()) == {"a", "b"}
# ---- _filter_tools ----
def _make_tool(name: str):
"""Create a minimal mock tool with a .name attribute."""
return SimpleNamespace(name=name)
class TestFilterTools:
def test_none_allowlist_passes_all(self):
tools = [_make_tool("a"), _make_tool("b"), _make_tool("c")]
assert _filter_tools(tools, None) == tools
def test_allowlist_filters(self):
tools = [_make_tool("a"), _make_tool("b"), _make_tool("c")]
result = _filter_tools(tools, ["a", "c"])
assert [t.name for t in result] == ["a", "c"]
def test_empty_allowlist_filters_all(self):
tools = [_make_tool("a"), _make_tool("b")]
assert _filter_tools(tools, []) == []
def test_allowlist_with_nonexistent_name(self):
tools = [_make_tool("a")]
result = _filter_tools(tools, ["a", "nonexistent"])
assert [t.name for t in result] == ["a"]
def test_empty_tools_list(self):
assert _filter_tools([], ["a"]) == []
assert _filter_tools([], None) == []
# Wildcard tests
def test_wildcard_star_suffix(self):
"""Test *_exa pattern matching."""
tools = [
_make_tool("web_search_exa"),
_make_tool("get_code_context_exa"),
_make_tool("company_research_exa"),
_make_tool("unrelated_tool"),
]
result = _filter_tools(tools, ["*_exa"])
assert [t.name for t in result] == [
"web_search_exa",
"get_code_context_exa",
"company_research_exa",
]
def test_wildcard_star_prefix(self):
"""Test read_* pattern matching."""
tools = [
_make_tool("read_file"),
_make_tool("read_directory"),
_make_tool("read_link"),
_make_tool("write_file"),
]
result = _filter_tools(tools, ["read_*"])
assert [t.name for t in result] == [
"read_file",
"read_directory",
"read_link",
]
def test_wildcard_star_middle(self):
"""Test pattern with * in the middle."""
tools = [
_make_tool("get_user_data"),
_make_tool("get_admin_data"),
_make_tool("get_file"),
]
result = _filter_tools(tools, ["get_*_data"])
assert [t.name for t in result] == ["get_user_data", "get_admin_data"]
def test_wildcard_star_only(self):
"""Test * matches everything."""
tools = [_make_tool("a"), _make_tool("b"), _make_tool("c")]
result = _filter_tools(tools, ["*"])
assert [t.name for t in result] == ["a", "b", "c"]
def test_wildcard_question_mark(self):
"""Test ? matches single character."""
tools = [
_make_tool("tool_1"),
_make_tool("tool_2"),
_make_tool("tool_10"),
]
result = _filter_tools(tools, ["tool_?"])
assert [t.name for t in result] == ["tool_1", "tool_2"]
def test_wildcard_character_class(self):
"""Test [seq] matches characters in sequence."""
tools = [
_make_tool("tool_a"),
_make_tool("tool_b"),
_make_tool("tool_c"),
_make_tool("tool_d"),
]
result = _filter_tools(tools, ["tool_[abc]"])
assert [t.name for t in result] == ["tool_a", "tool_b", "tool_c"]
def test_wildcard_character_class_range(self):
"""Test [0-9] matches digit range."""
tools = [
_make_tool("tool_0"),
_make_tool("tool_5"),
_make_tool("tool_9"),
_make_tool("tool_a"),
]
result = _filter_tools(tools, ["tool_[0-9]"])
assert [t.name for t in result] == ["tool_0", "tool_5", "tool_9"]
def test_wildcard_negated_character_class(self):
"""Test [!seq] matches characters not in sequence."""
tools = [
_make_tool("tool_a"),
_make_tool("tool_b"),
_make_tool("tool_1"),
_make_tool("tool_2"),
]
result = _filter_tools(tools, ["tool_[!0-9]"])
assert [t.name for t in result] == ["tool_a", "tool_b"]
def test_wildcard_mixed_with_exact(self):
"""Test mixing wildcard and exact patterns."""
tools = [
_make_tool("web_search_exa"),
_make_tool("get_code_context_exa"),
_make_tool("specific_tool"),
_make_tool("another_tool"),
]
result = _filter_tools(tools, ["*_exa", "specific_tool"])
assert [t.name for t in result] == [
"web_search_exa",
"get_code_context_exa",
"specific_tool",
]
def test_wildcard_multiple_patterns(self):
"""Test multiple wildcard patterns."""
tools = [
_make_tool("read_file"),
_make_tool("write_file"),
_make_tool("delete_file"),
_make_tool("search_database"),
]
result = _filter_tools(tools, ["read_*", "write_*"])
assert [t.name for t in result] == ["read_file", "write_file"]
def test_wildcard_no_match(self):
"""Test wildcard pattern that doesn't match anything."""
tools = [_make_tool("foo"), _make_tool("bar")]
result = _filter_tools(tools, ["baz_*"])
assert result == []
def test_wildcard_overlapping_patterns(self):
"""Test overlapping patterns don't duplicate results."""
tools = [_make_tool("tool_abc"), _make_tool("tool_xyz")]
result = _filter_tools(tools, ["tool_*", "*_abc"])
# Should include each tool only once
assert [t.name for t in result] == ["tool_abc", "tool_xyz"]
def test_wildcard_complex_pattern(self):
"""Test complex wildcard pattern."""
tools = [
_make_tool("get_user_info_v1"),
_make_tool("get_user_info_v2"),
_make_tool("get_admin_info_v1"),
_make_tool("set_user_info"),
]
result = _filter_tools(tools, ["get_*_info_v?"])
assert [t.name for t in result] == [
"get_user_info_v1",
"get_user_info_v2",
"get_admin_info_v1",
]
def test_exact_match_performance_path(self):
"""Test exact matching still works (fast path)."""
# This test verifies backward compatibility with exact matching
tools = [_make_tool("a"), _make_tool("b"), _make_tool("c")]
result = _filter_tools(tools, ["a", "c"])
assert [t.name for t in result] == ["a", "c"]
# ---- _route_tools ----
class TestRouteTools:
def test_default_routes_to_main(self):
config = {"srv": {"transport": "stdio"}}
server_tools = {"srv": [_make_tool("x")]}
result = _route_tools(config, server_tools)
assert "main" in result
assert [t.name for t in result["main"]] == ["x"]
def test_expose_to_named_agent(self):
config = {"srv": {"transport": "stdio", "expose_to": ["code-agent"]}}
server_tools = {"srv": [_make_tool("x"), _make_tool("y")]}
result = _route_tools(config, server_tools)
assert "code-agent" in result
assert "main" not in result
assert [t.name for t in result["code-agent"]] == ["x", "y"]
def test_expose_to_multiple_agents(self):
config = {"srv": {"transport": "stdio", "expose_to": ["main", "code-agent"]}}
server_tools = {"srv": [_make_tool("x")]}
result = _route_tools(config, server_tools)
assert [t.name for t in result["main"]] == ["x"]
assert [t.name for t in result["code-agent"]] == ["x"]
def test_tool_filter_applied(self):
config = {"srv": {"transport": "stdio", "tools": ["b"]}}
server_tools = {"srv": [_make_tool("a"), _make_tool("b"), _make_tool("c")]}
result = _route_tools(config, server_tools)
assert [t.name for t in result["main"]] == ["b"]
def test_multiple_servers(self):
config = {
"s1": {"transport": "stdio", "expose_to": ["main"]},
"s2": {"transport": "http", "expose_to": ["research-agent"]},
}
server_tools = {
"s1": [_make_tool("a")],
"s2": [_make_tool("b")],
}
result = _route_tools(config, server_tools)
assert [t.name for t in result["main"]] == ["a"]
assert [t.name for t in result["research-agent"]] == ["b"]
def test_expose_to_string_not_list(self):
config = {"srv": {"transport": "stdio", "expose_to": "debug-agent"}}
server_tools = {"srv": [_make_tool("x")]}
result = _route_tools(config, server_tools)
assert "debug-agent" in result
def test_empty_server_tools(self):
config = {"srv": {"transport": "stdio"}}
server_tools = {"srv": []}
result = _route_tools(config, server_tools)
assert result.get("main", []) == []
# ---- add_mcp_server / remove_mcp_server ----
@pytest.fixture
def user_mcp_dir(tmp_path, monkeypatch):
"""Redirect user MCP config to a temp directory."""
cfg_dir = tmp_path / "config"
cfg_dir.mkdir()
cfg_file = cfg_dir / "mcp.yaml"
monkeypatch.setattr("EvoScientist.mcp.client.USER_CONFIG_DIR", cfg_dir)
monkeypatch.setattr("EvoScientist.mcp.client.USER_MCP_CONFIG", cfg_file)
return cfg_file
class TestAddMcpServer:
def test_add_stdio_server(self, user_mcp_dir):
entry = add_mcp_server(
"fs", "stdio", command="npx", args=["-y", "server", "/tmp"]
)
assert entry["transport"] == "stdio"
assert entry["command"] == "npx"
assert entry["args"] == ["-y", "server", "/tmp"]
# Verify persisted
data = yaml.safe_load(user_mcp_dir.read_text())
assert "fs" in data
def test_add_http_server(self, user_mcp_dir):
entry = add_mcp_server(
"api",
"http",
url="http://localhost:8080/mcp",
headers={"Authorization": "Bearer tok"},
)
assert entry["url"] == "http://localhost:8080/mcp"
assert entry["headers"]["Authorization"] == "Bearer tok"
def test_add_sse_server(self, user_mcp_dir):
entry = add_mcp_server("sse-srv", "sse", url="http://localhost:9090/sse")
assert entry["transport"] == "sse"
def test_add_websocket_server(self, user_mcp_dir):
entry = add_mcp_server("ws", "websocket", url="ws://localhost:8765")
assert entry["transport"] == "websocket"
def test_add_with_tools_and_expose_to(self, user_mcp_dir):
entry = add_mcp_server(
"fs",
"stdio",
command="npx",
args=[],
tools=["read_file"],
expose_to=["main", "code-agent"],
)
assert entry["tools"] == ["read_file"]
assert entry["expose_to"] == ["main", "code-agent"]
def test_add_replaces_existing(self, user_mcp_dir):
add_mcp_server("srv", "stdio", command="old")
add_mcp_server("srv", "http", url="http://new")
data = yaml.safe_load(user_mcp_dir.read_text())
assert data["srv"]["transport"] == "http"
def test_add_invalid_transport_raises(self, user_mcp_dir):
with pytest.raises(ValueError, match="Unknown transport"):
add_mcp_server("bad", "carrier_pigeon", url="coo://rooftop")
def test_stdio_without_command_raises(self, user_mcp_dir):
with pytest.raises(ValueError, match="requires a command"):
add_mcp_server("bad", "stdio")
def test_http_without_url_raises(self, user_mcp_dir):
with pytest.raises(ValueError, match="requires a url"):
add_mcp_server("bad", "http")
def test_add_with_env(self, user_mcp_dir):
entry = add_mcp_server(
"fs", "stdio", command="npx", args=[], env={"FOO": "bar"}
)
assert entry["env"] == {"FOO": "bar"}
def test_add_multiple_servers(self, user_mcp_dir):
add_mcp_server("a", "stdio", command="cmd1")
add_mcp_server("b", "http", url="http://x")
data = yaml.safe_load(user_mcp_dir.read_text())
assert "a" in data
assert "b" in data
class TestRemoveMcpServer:
def test_remove_existing(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx")
assert remove_mcp_server("fs") is True
data = yaml.safe_load(user_mcp_dir.read_text()) or {}
assert "fs" not in data
def test_remove_nonexistent(self, user_mcp_dir):
assert remove_mcp_server("nope") is False
def test_remove_preserves_others(self, user_mcp_dir):
add_mcp_server("a", "stdio", command="cmd1")
add_mcp_server("b", "http", url="http://x")
remove_mcp_server("a")
data = yaml.safe_load(user_mcp_dir.read_text())
assert "a" not in data
assert "b" in data
# ---- _parse_mcp_add_args (CLI arg parser) ----
class TestParseMcpAddArgs:
def test_stdio_basic(self):
r = parse_mcp_add_args(["fs", "npx", "-y", "server", "/tmp"])
assert r["name"] == "fs"
assert r["transport"] == "stdio"
assert r["command"] == "npx"
assert r["args"] == ["-y", "server", "/tmp"]
def test_http_auto_detected(self):
r = parse_mcp_add_args(["api", "http://localhost:8080/mcp"])
assert r["transport"] == "http"
assert r["url"] == "http://localhost:8080/mcp"
def test_https_auto_detected(self):
r = parse_mcp_add_args(["api", "https://example.com/mcp"])
assert r["transport"] == "http"
assert r["url"] == "https://example.com/mcp"
def test_ws_auto_detected(self):
r = parse_mcp_add_args(["ws", "ws://localhost:9090"])
assert r["transport"] == "websocket"
def test_explicit_transport_override(self):
r = parse_mcp_add_args(["srv", "https://example.com/sse", "--transport", "sse"])
assert r["transport"] == "sse"
assert r["url"] == "https://example.com/sse"
def test_explicit_transport_short_flag(self):
r = parse_mcp_add_args(["srv", "https://x", "-T", "websocket"])
assert r["transport"] == "websocket"
def test_tools_flag(self):
r = parse_mcp_add_args(["srv", "http://x", "--tools", "a,b"])
assert r["tools"] == ["a", "b"]
def test_expose_to_flag(self):
r = parse_mcp_add_args(["srv", "http://x", "--expose-to", "main,code-agent"])
assert r["expose_to"] == ["main", "code-agent"]
def test_header_flag(self):
r = parse_mcp_add_args(
["srv", "http://x", "--header", "Authorization:Bearer tok"]
)
assert r["headers"] == {"Authorization": "Bearer tok"}
def test_env_flag(self):
r = parse_mcp_add_args(["srv", "cmd", "--env", "FOO=bar"])
assert r["env"] == {"FOO": "bar"}
def test_too_few_tokens_raises(self):
with pytest.raises(ValueError, match="Usage"):
parse_mcp_add_args(["fs"])
def test_double_dash_ignored(self):
r = parse_mcp_add_args(["srv", "npx", "--", "-y", "pkg"])
assert r["command"] == "npx"
assert r["args"] == ["-y", "pkg"]
assert "--" not in r["args"]
def test_env_ref_flag(self):
r = parse_mcp_add_args(["srv", "cmd", "--env-ref", "FOO"])
assert r["env"] == {"FOO": "${FOO}"}
def test_env_ref_and_env_combined(self):
r = parse_mcp_add_args(
["srv", "cmd", "--env", "DEBUG=true", "--env-ref", "API_KEY"]
)
assert r["env"] == {"DEBUG": "true", "API_KEY": "${API_KEY}"}
def test_missing_command_or_url_raises(self):
with pytest.raises(ValueError, match="command or URL is required"):
parse_mcp_add_args(["fs", "--tools", "a"])
# ---- edit_mcp_server ----
class TestEditMcpServer:
def test_edit_expose_to(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx", args=[])
entry = edit_mcp_server("fs", expose_to=["main", "code-agent"])
assert entry["expose_to"] == ["main", "code-agent"]
assert entry["command"] == "npx" # unchanged
def test_edit_tools(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx", args=[])
entry = edit_mcp_server("fs", tools=["read_file"])
assert entry["tools"] == ["read_file"]
def test_edit_clear_tools(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx", tools=["read_file"])
entry = edit_mcp_server("fs", tools=None)
assert "tools" not in entry
def test_edit_url(self, user_mcp_dir):
add_mcp_server("api", "http", url="http://old:8080/mcp")
entry = edit_mcp_server("api", url="http://new:9090/mcp")
assert entry["url"] == "http://new:9090/mcp"
assert entry["transport"] == "http" # unchanged
def test_edit_nonexistent_raises(self, user_mcp_dir):
with pytest.raises(KeyError, match="not found"):
edit_mcp_server("nope", tools=["a"])
def test_edit_invalid_transport_raises(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx")
with pytest.raises(ValueError, match="Unknown transport"):
edit_mcp_server("fs", transport="carrier_pigeon")
def test_edit_removes_required_field_raises(self, user_mcp_dir):
add_mcp_server("fs", "stdio", command="npx")
with pytest.raises(ValueError, match="requires a command"):
edit_mcp_server("fs", command=None)
def test_edit_preserves_unrelated_fields(self, user_mcp_dir):
add_mcp_server(
"fs",
"stdio",
command="npx",
args=["-y", "srv"],
tools=["a"],
expose_to=["main"],
)
entry = edit_mcp_server("fs", expose_to=["code-agent"])
assert entry["tools"] == ["a"]
assert entry["args"] == ["-y", "srv"]
assert entry["expose_to"] == ["code-agent"]
# ---- parse_mcp_edit_args ----
class TestParseMcpEditArgs:
def test_basic_field(self):
name, fields = parse_mcp_edit_args(["srv", "--url", "http://new"])
assert name == "srv"
assert fields["url"] == "http://new"
def test_tools_none_clears(self):
_, fields = parse_mcp_edit_args(["srv", "--tools", "none"])
assert fields["tools"] is None
def test_expose_to_csv(self):
_, fields = parse_mcp_edit_args(["srv", "--expose-to", "main,code-agent"])
assert fields["expose_to"] == ["main", "code-agent"]
def test_multiple_fields(self):
_, fields = parse_mcp_edit_args(["srv", "--url", "http://x", "--tools", "a,b"])
assert fields["url"] == "http://x"
assert fields["tools"] == ["a", "b"]
def test_empty_raises(self):
with pytest.raises(ValueError, match="Usage"):
parse_mcp_edit_args([])
def test_no_fields_raises(self):
with pytest.raises(ValueError, match="No fields"):
parse_mcp_edit_args(["srv"])
# ---- uv tool compatibility ----
class TestUvToolCompat:
"""Tests for uv tool environment detection and compatible install helpers."""
# -- _is_uv_tool_env --
def test_is_uv_tool_env_false_when_no_virtual_env(self, monkeypatch):
from EvoScientist.mcp.registry import _is_uv_tool_env
monkeypatch.delenv("VIRTUAL_ENV", raising=False)
assert _is_uv_tool_env() is False
def test_is_uv_tool_env_false_for_regular_venv(self, monkeypatch):
from EvoScientist.mcp.registry import _is_uv_tool_env
monkeypatch.setenv("VIRTUAL_ENV", "/home/user/projects/myapp/.venv")
assert _is_uv_tool_env() is False
def test_is_uv_tool_env_true_unix(self, monkeypatch):
from EvoScientist.mcp.registry import _is_uv_tool_env
monkeypatch.setenv(
"VIRTUAL_ENV", "/home/user/.local/share/uv/tools/evoscientist"
)
assert _is_uv_tool_env() is True
def test_is_uv_tool_env_true_windows_backslashes(self, monkeypatch):
from EvoScientist.mcp.registry import _is_uv_tool_env
monkeypatch.setenv(
"VIRTUAL_ENV", r"C:\Users\user\AppData\Local\uv\tools\evoscientist"
)
assert _is_uv_tool_env() is True
# -- _uv_tool_name --
def test_uv_tool_name_returns_name(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.setenv(
"VIRTUAL_ENV", "/home/user/.local/share/uv/tools/evoscientist"
)
assert reg._uv_tool_name() == "evoscientist"
def test_uv_tool_name_returns_none_when_not_uv(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.setenv("VIRTUAL_ENV", "/home/user/projects/myapp/.venv")
assert reg._uv_tool_name() is None
def test_uv_tool_name_returns_none_when_no_virtual_env(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.delenv("VIRTUAL_ENV", raising=False)
assert reg._uv_tool_name() is None
# -- _uv_tool_existing_requirements --
def test_existing_requirements_from_receipt(self, monkeypatch, tmp_path):
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "arxiv-mcp-server" },\n'
' { name = "rich" },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
result = reg._uv_tool_existing_requirements()
assert result == {"arxiv-mcp-server": "arxiv-mcp-server", "rich": "rich"}
def test_existing_requirements_preserves_specifiers_and_extras(
self, monkeypatch, tmp_path
):
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "rich", specifier = ">=13.0" },\n'
' { name = "requests", extras = ["socks"] },\n'
' { name = "lark-oapi", specifier = ">=1.4.0", extras = ["oauth"] },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
result = reg._uv_tool_existing_requirements()
assert result == {
"rich": "rich>=13.0",
"requests": "requests[socks]",
"lark-oapi": "lark-oapi[oauth]>=1.4.0",
}
def test_existing_requirements_excludes_tool_name(self, monkeypatch, tmp_path):
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
'[tool]\nrequirements = [\n { name = "evoscientist" },\n]\n'
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
assert reg._uv_tool_existing_requirements() == {}
def test_existing_requirements_no_receipt(self, monkeypatch, tmp_path):
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
assert reg._uv_tool_existing_requirements() == {}
# -- pip_install_hint --
def test_pip_install_hint_uv_tool(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: True)
hint = reg.pip_install_hint()
assert "uv tool install --reinstall evoscientist --with" in hint
def test_pip_install_hint_uv_no_tool(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
assert reg.pip_install_hint() == "uv pip install"
def test_pip_install_hint_plain_pip(self, monkeypatch):
import EvoScientist.mcp.registry as reg
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(reg.shutil, "which", lambda x: None)
assert reg.pip_install_hint() == "pip install"
# -- install_library / install_cli_tool --
def test_install_library_uv_tool_env_uses_uv_tool_install(
self, monkeypatch, tmp_path
):
"""In a uv tool env, should use ``uv tool install --with`` for durability."""
import EvoScientist.mcp.registry as reg
# Set up a fake uv tool env with receipt
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "existing-pkg" },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(list(cmd))
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
result = reg.install_library("new-mcp-server")
assert result is True
assert len(captured) == 1
cmd = captured[0]
assert cmd[:3] == ["uv", "tool", "install"]
assert "evoscientist" in cmd
# Must preserve existing --with requirement
assert "--with" in cmd
with_args = [cmd[i + 1] for i, v in enumerate(cmd) if v == "--with"]
assert "existing-pkg" in with_args
assert "new-mcp-server" in with_args
def test_install_library_uv_tool_env_no_duplicate_with(self, monkeypatch, tmp_path):
"""If the package is already in the receipt, don't add it twice."""
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "arxiv-mcp-server" },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(list(cmd))
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
reg.install_library("arxiv-mcp-server")
cmd = captured[0]
with_args = [cmd[i + 1] for i, v in enumerate(cmd) if v == "--with"]
assert with_args.count("arxiv-mcp-server") == 1
def test_install_library_uv_tool_preserves_specifiers(self, monkeypatch, tmp_path):
"""Existing --with specs with extras/versions must be preserved."""
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "rich", specifier = ">=13.0" },\n'
' { name = "requests", extras = ["socks"] },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(list(cmd))
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
reg.install_library("new-pkg")
cmd = captured[0]
with_args = [cmd[i + 1] for i, v in enumerate(cmd) if v == "--with"]
assert "rich>=13.0" in with_args
assert "requests[socks]" in with_args
assert "new-pkg" in with_args
def test_install_library_uv_tool_dedup_with_version_spec(
self, monkeypatch, tmp_path
):
"""Dedup must match bare name even if package arg has version spec."""
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
"[tool]\nrequirements = [\n"
' { name = "evoscientist" },\n'
' { name = "rich", specifier = ">=13.0" },\n'
"]\n"
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(list(cmd))
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
# package arg has version constraint — should still dedup against "rich"
reg.install_library("rich>=14.0")
cmd = captured[0]
with_args = [cmd[i + 1] for i, v in enumerate(cmd) if v == "--with"]
# Should keep the existing spec, not add a duplicate
assert with_args.count("rich>=13.0") == 1
assert "rich>=14.0" not in with_args
def test_install_library_uv_tool_falls_back_on_failure(self, monkeypatch, tmp_path):
"""install_library: if ``uv tool install --with`` fails, fall back
to ``uv pip install`` — standalone uv tool install is NEVER tried."""
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
'[tool]\nrequirements = [\n { name = "evoscientist" },\n]\n'
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
commands: list[list[str]] = []
def fake_run(cmd, **kwargs):
commands.append(cmd)
if cmd[:3] == ["uv", "tool", "install"]:
return type("R", (), {"returncode": 1})()
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
result = reg.install_library("some-package")
assert result is True
# Order: uv tool install --with (fail), uv pip install (ok).
assert len(commands) == 2
assert commands[0][:3] == ["uv", "tool", "install"]
assert "--with" in commands[0]
assert commands[1][:3] == ["uv", "pip", "install"]
def test_install_cli_tool_uv_tool_env_tries_standalone_on_failure(
self, monkeypatch, tmp_path
):
"""install_cli_tool: uv-tool env, --with fails → standalone uv tool
install is tried, then pip fallback."""
import EvoScientist.mcp.registry as reg
venv = tmp_path / "uv" / "tools" / "evoscientist"
venv.mkdir(parents=True)
receipt = venv / "uv-receipt.toml"
receipt.write_text(
'[tool]\nrequirements = [\n { name = "evoscientist" },\n]\n'
)
monkeypatch.setenv("VIRTUAL_ENV", str(venv))
commands: list[list[str]] = []
def fake_run(cmd, **kwargs):
commands.append(cmd)
if cmd[:3] == ["uv", "tool", "install"]:
return type("R", (), {"returncode": 1})()
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
result = reg.install_cli_tool("some-cli", verify_command="some-cli")
assert result is True
assert len(commands) == 3
assert commands[0][:3] == ["uv", "tool", "install"]
assert "--with" in commands[0]
assert commands[1][:3] == ["uv", "tool", "install"]
assert "--with" not in commands[1]
assert commands[2][:3] == ["uv", "pip", "install"]
def test_install_library_goes_straight_to_pip_outside_uv_tool(self, monkeypatch):
"""install_library outside a uv-tool env must skip ``uv tool install
<pkg>`` entirely — standalone uv tools aren't importable."""
import sys
import EvoScientist.mcp.registry as reg
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(cmd)
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
result = reg.install_library("some-library")
assert result is True
assert len(captured) == 1
assert captured[0][:3] == ["uv", "pip", "install"]
assert sys.executable in captured[0]
def test_install_cli_tool_prefers_standalone_uv_tool_install(
self, monkeypatch, tmp_path
):
"""install_cli_tool outside a uv-tool env: standalone ``uv tool
install <pkg>`` is preferred so the binary survives uv sync."""
import EvoScientist.mcp.registry as reg
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(cmd)
return type("R", (), {"returncode": 0})()
fake_bin = tmp_path / "some-cli"
fake_bin.write_text("#!/bin/sh\n")
fake_bin.chmod(0o755)
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
monkeypatch.setattr(
reg, "_uv_tool_bin", lambda cmd: fake_bin if cmd == "some-cli" else None
)
result = reg.install_cli_tool("some-cli", verify_command="some-cli")
assert result is True
assert len(captured) == 1
assert captured[0][:3] == ["uv", "tool", "install"]
assert "some-cli" in captured[0]
def test_install_cli_tool_missing_bin_triggers_pip_fallback(self, monkeypatch):
"""install_cli_tool: if the binary isn't in uv's tool bin dir after
``uv tool install`` (e.g. package has no console-script), fall
through to ``uv pip install``."""
import sys
import EvoScientist.mcp.registry as reg
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(cmd)
return type("R", (), {"returncode": 0})()
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(
reg.shutil, "which", lambda x: "/usr/bin/uv" if x == "uv" else None
)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
monkeypatch.setattr(reg, "_uv_tool_bin", lambda cmd: None)
result = reg.install_cli_tool(
"lib-without-entrypoint", verify_command="ghost-cli"
)
assert result is True
assert len(captured) == 2
assert captured[0][:3] == ["uv", "tool", "install"]
assert "--python" in captured[1]
assert sys.executable in captured[1]
def test_install_cli_tool_bin_present_short_circuits(self, monkeypatch, tmp_path):
"""install_cli_tool: ``uv tool install`` success + binary present in
uv-tool bin dir ⇒ no pip fallback, even if a stale copy exists in
the venv on PATH."""
import EvoScientist.mcp.registry as reg
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(cmd)
return type("R", (), {"returncode": 0})()
fake_bin = tmp_path / "arxiv-mcp-server"
fake_bin.write_text("#!/bin/sh\n")
fake_bin.chmod(0o755)
def fake_which(x):
if x == "uv":
return "/usr/bin/uv"
if x == "arxiv-mcp-server":
return "/venv/bin/arxiv-mcp-server" # stale, should NOT be trusted
return None
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(reg.shutil, "which", fake_which)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
monkeypatch.setattr(
reg,
"_uv_tool_bin",
lambda cmd: fake_bin if cmd == "arxiv-mcp-server" else None,
)
result = reg.install_cli_tool(
"arxiv-mcp-server", verify_command="arxiv-mcp-server"
)
assert result is True
assert len(captured) == 1
assert captured[0][:3] == ["uv", "tool", "install"]
def test_install_library_falls_back_to_pip_when_no_uv(self, monkeypatch):
import sys
import EvoScientist.mcp.registry as reg
captured: list[list[str]] = []
def fake_run(cmd, **kwargs):
captured.append(cmd)
ns = type("R", (), {"returncode": 0})()
return ns
monkeypatch.setattr(reg, "_is_uv_tool_env", lambda: False)
monkeypatch.setattr(reg.shutil, "which", lambda x: None)
monkeypatch.setattr(reg.subprocess, "run", fake_run)
reg.install_library("some-package")
assert len(captured) == 1
assert sys.executable in captured[0]
assert "-m" in captured[0]
assert "pip" in captured[0]
# -- _resolve_command_path --
def test_resolve_command_path_absolute_passthrough(self):
from EvoScientist.mcp.registry import _resolve_command_path
assert _resolve_command_path("/usr/bin/my-tool") == "/usr/bin/my-tool"
def test_resolve_command_path_found_in_bin_dir(self, monkeypatch, tmp_path):
import sys
import EvoScientist.mcp.registry as reg
# Create a fake executable in a temp bin dir
bin_dir = tmp_path / "bin"
bin_dir.mkdir()
fake_exe = bin_dir / "my-mcp-server"
fake_exe.touch()
fake_exe.chmod(0o755)
# Point sys.executable to something in that bin dir
fake_python = bin_dir / "python"
fake_python.touch()
monkeypatch.setattr(sys, "executable", str(fake_python))
# Ensure shutil.which won't find it on PATH
monkeypatch.setattr(reg.shutil, "which", lambda x: None)
result = reg._resolve_command_path("my-mcp-server")
assert result == str(fake_exe)
def test_resolve_command_path_windows_exe_suffix(self, monkeypatch, tmp_path):
import os
import sys
import EvoScientist.mcp.registry as reg
if os.name != "nt":
pytest.skip("Windows-only behaviour")
bin_dir = tmp_path / "bin"
bin_dir.mkdir()
fake_exe = bin_dir / "my-mcp-server.exe"
fake_exe.touch()
monkeypatch.setattr(sys, "executable", str(bin_dir / "python.exe"))
monkeypatch.setattr(reg.shutil, "which", lambda x: None)
result = reg._resolve_command_path("my-mcp-server")
assert result == str(fake_exe)
def test_resolve_command_path_returns_bare_when_not_found(
self, monkeypatch, tmp_path
):
import sys
import EvoScientist.mcp.registry as reg
monkeypatch.setattr(reg.shutil, "which", lambda x: None)
monkeypatch.setattr(reg, "_uv_tool_bin", lambda cmd: None)
monkeypatch.setattr(sys, "executable", str(tmp_path / "bin" / "python"))
result = reg._resolve_command_path("nonexistent-tool")
assert result == "nonexistent-tool"
def test_resolve_command_path_prefers_uv_tool_over_venv_shadow(
self, monkeypatch, tmp_path
):
"""When both ``uv tool dir --bin`` and a venv's ``bin/`` contain the
command, prefer the uv-tool location so the path written to mcp.yaml
survives ``uv sync``."""
import EvoScientist.mcp.registry as reg
uv_bin_dir = tmp_path / "uv-bin"
uv_bin_dir.mkdir()
uv_copy = uv_bin_dir / "arxiv-mcp-server"
uv_copy.write_text("#!/bin/sh\n")
uv_copy.chmod(0o755)
# Stale venv copy that `uv run` would surface first via PATH.
venv_copy = tmp_path / "venv-bin" / "arxiv-mcp-server"
venv_copy.parent.mkdir()
venv_copy.write_text("#!/bin/sh\n")
venv_copy.chmod(0o755)
monkeypatch.setattr(reg, "_uv_tool_bin_dir", lambda: uv_bin_dir)
monkeypatch.setattr(reg.shutil, "which", lambda x: str(venv_copy))
result = reg._resolve_command_path("arxiv-mcp-server")
assert result == str(uv_copy)
# ---- _load_tools progress callback ----
class TestLoadToolsProgressCallback:
"""Verify per-server ``on_progress`` events fire in the expected order."""
@staticmethod
def _patch_client(monkeypatch, behavior):
"""Install a fake MultiServerMCPClient whose ``get_tools`` delegates
to *behavior* — a ``dict[server_name, list | Exception]``. Each
success value is returned; Exception values are raised.
"""
class _FakeClient:
def __init__(self, connections):
self.connections = connections
async def get_tools(self, server_name):
outcome = behavior[server_name]
if isinstance(outcome, Exception):
raise outcome
return outcome
import langchain_mcp_adapters.client as lc_client
monkeypatch.setattr(lc_client, "MultiServerMCPClient", _FakeClient)
def test_success_emits_start_then_success_with_tool_count(self, monkeypatch):
import asyncio
from EvoScientist.mcp.client import _load_tools
events: list[tuple[str, str, str]] = []
self._patch_client(
monkeypatch,
{"srv": ["tool1", "tool2", "tool3"]},
)
config = {"srv": {"transport": "stdio", "command": "demo"}}
def record(event, name, detail):
events.append((event, name, detail))
asyncio.run(_load_tools(config, on_progress=record))
assert events == [
("start", "srv", ""),
("success", "srv", "3"),
]
def test_failure_emits_start_then_error_with_detail(self, monkeypatch):
import asyncio
from EvoScientist.mcp.client import _load_tools
events: list[tuple[str, str, str]] = []
self._patch_client(monkeypatch, {"srv": RuntimeError("boom")})
config = {"srv": {"transport": "stdio", "command": "demo"}}
def record(event, name, detail):
events.append((event, name, detail))
asyncio.run(_load_tools(config, on_progress=record))
assert events == [
("start", "srv", ""),
("error", "srv", "boom"),
]
def test_mixed_fleet_reports_each_server_independently(self, monkeypatch):
import asyncio
from EvoScientist.mcp.client import _load_tools
events: list[tuple[str, str, str]] = []
self._patch_client(
monkeypatch,
{
"ok_srv": ["a"],
"bad_srv": ConnectionError("refused"),
},
)
config = {
"ok_srv": {"transport": "stdio", "command": "demo"},
"bad_srv": {"transport": "stdio", "command": "demo"},
}
def record(event, name, detail):
events.append((event, name, detail))
asyncio.run(_load_tools(config, on_progress=record))
by_server = {}
for ev, name, detail in events:
by_server.setdefault(name, []).append((ev, detail))
assert by_server["ok_srv"] == [("start", ""), ("success", "1")]
assert by_server["bad_srv"] == [("start", ""), ("error", "refused")]
def test_callback_errors_do_not_break_the_load(self, monkeypatch):
import asyncio
from EvoScientist.mcp.client import _load_tools
self._patch_client(monkeypatch, {"srv": ["tool1"]})
config = {"srv": {"transport": "stdio", "command": "demo"}}
def bad_callback(event, name, detail):
raise RuntimeError("callback bug")
result = asyncio.run(_load_tools(config, on_progress=bad_callback))
assert result == {"srv": ["tool1"]}
def test_semaphore_caps_concurrent_connections(self, monkeypatch):
"""Many configured servers must not all spawn at once."""
import asyncio
from EvoScientist.mcp import client as mcp_client
inflight = {"count": 0, "peak": 0}
class _TrackingClient:
def __init__(self, connections):
self.connections = connections
async def get_tools(self, server_name):
inflight["count"] += 1
inflight["peak"] = max(inflight["peak"], inflight["count"])
# Yield so sibling tasks can try to enter concurrently.
await asyncio.sleep(0)
inflight["count"] -= 1
return [f"tool-of-{server_name}"]
import langchain_mcp_adapters.client as lc_client
monkeypatch.setattr(lc_client, "MultiServerMCPClient", _TrackingClient)
# Force a small cap so the test doesn't depend on the default.
monkeypatch.setattr(mcp_client, "_MAX_CONCURRENT_CONNECTIONS", 3)
config = {
f"srv{i}": {"transport": "stdio", "command": "demo"} for i in range(10)
}
asyncio.run(mcp_client._load_tools(config))
assert inflight["peak"] <= 3
assert inflight["peak"] > 1 # sanity: we *are* parallelizing