Files
EvoScientist-Multi/EvoScientist/cli/widgets/mcp_loader_widget.py
T
dinos 65828e9666 perf(cli): cut startup latency and defer MCP loading to the background (#171)
* perf(cli): cut startup time of `evosci --help` from ~2.2s to ~0.3s

Module-level imports were eagerly pulling in langchain.chat_models (with
the whole anthropic/openai/google stack), langgraph, textual, and
prompt_toolkit on every invocation — even for `--help` or `config list`.

Defer those with PEP 562 `__getattr__`, using `lazy_loader.attach` (SPEC-1,
the scientific-python standard) where it's a clean attach pattern:

- `EvoScientist/llm/__init__.py`: attach `models` lazily so importing
  `context_window` from this package no longer drags in langchain.
- `EvoScientist/stream/__init__.py`: attach display/events lazily; split
  the shared Rich `Console` singleton into a new lightweight
  `stream/console.py` so callers that only need `console` skip the
  `stream.events` → `langchain_core.messages` chain.
- `EvoScientist/cli/__init__.py`: hand-rolled `__getattr__` (reaches into
  `..stream.state`, which `lazy_loader` doesn't cover) so `commands` and
  `app` are the only eager loads.
- `EvoScientist/cli/commands.py`: move `cmd_interactive`/`cmd_run` to
  in-function imports so prompt_toolkit + textual only load when the
  interactive path actually runs.
- `EvoScientist/cli/_constants.py`: read `AGENT_NAME` on demand so
  `build_metadata` doesn't eagerly import `sessions` (langgraph/aiosqlite).

Adds `lazy-loader>=0.5` as a dependency.

* feat(cli): defer MCP tool loading with live per-server progress

The CLI was blocking ~5 s on MCP tool enumeration before the first
prompt appeared. Move the agent construction off the event loop and
surface per-server progress so the user can interact immediately and see
what's happening.

MCP client:
- Add an `on_progress` callback to `load_mcp_tools` / `aload_mcp_tools`
  / `_load_tools` emitting `start` / `success` / `error` events per
  server.
- Fan connection attempts out with `asyncio.gather` so latency no longer
  scales linearly with server count; cap simultaneous attempts at
  `_MAX_CONCURRENT_CONNECTIONS` (8) via a semaphore so a big stdio fleet
  doesn't spawn every subprocess at once.

Agent wiring:
- Plumb `on_mcp_progress` through `create_cli_agent` / `_load_agent` /
  `load_mcp_and_build_kwargs` so CLI and TUI can plug in collectors.

CLI (`cmd_interactive`):
- Run `_load_agent` in a background thread via `asyncio.to_thread`; the
  prompt and banner render immediately.
- `_await_agent_ready()` awaits the task before each agent-using site
  (first turn, channel messages, `/channel`, `/compact`). Raises if
  called without a prior `_start_agent_load` instead of silently
  reloading without the SQLite checkpointer.
- Pre-prime the progress dict from `load_mcp_config()` so the
  bottom-toolbar's `N/M` denominator is stable from the first render.
- Wrap `session.prompt_async` in `patch_stdout(raw=True)` so
  `console.print` from the worker-thread progress callback lands cleanly
  above the prompt as inline chat messages instead of stomping the
  prompt cursor.

TUI (`EvoTextualInteractiveApp`):
- Same background load + `_await_agent_ready()` gates on every
  `self._agent` read.
- New `MCPLoaderWidget` mounted at the top of `#input-shell` shows a
  header with `N/M` and one live row per server (spinner → ✓ / ✗ with
  tool count or error detail). On completion:
  - all-clean loads auto-dismiss ~2.5 s later;
  - cache hits (no events ever fired) dismiss immediately rather than
    flashing a misleading "0/N loaded";
  - failures keep the widget mounted so the user can read the errors.
  - `dismissed` property lets the app clear its ref so late events from
    slow servers become no-ops. The error branch of `_on_agent_loaded`
    also settles the widget so a load failure can't leave the spinner
    animating forever.
- Chat input is `disabled` while MCP resolves — no placeholder hack, no
"waiting…" system message.

Shared:
- Hoist braille spinner frames to `status_bar.SPINNER_FRAMES` and import
  them in the TUI widget so CLI and TUI animate in sync.

Tests:
- Extend `test_agent_mcp_cache` fakes to accept the new `on_progress`
  kwarg.
- New `TestLoadToolsProgressCallback` in `test_mcp_client` exercises the
  event sequence for success/failure/mixed fleets, verifies a buggy
  callback doesn't break the load, and asserts the semaphore caps
  in-flight connections.

* style: ruff

* chore: update uv.lock

* chore: uv.lock

* fix: coderabbit issues

* style: fmt

* fix: move _await_agent_ready inside try block

* fix(tui): auto-dismiss MCP loader widget on failure

The widget was designed to stay mounted on failure so the user could
read error detail, but since it's pinned above the input it never went
away in practice — just permanent banner clutter.

Auto-dismiss on failure too, with a longer grace (12s vs 2.5s) so the
error summary stays readable.

* fix: address second coderabbit pass

- Channel handlers (CLI + TUI): catch agent-load failures so the
  channel request doesn't hang; CLI moves `_await_agent_ready()`
  inside the existing try/except, TUI catches explicitly and calls
  `_set_channel_response` with the error.

- Stale background loads: `prev.cancel()` only stops the asyncio
  wrapper, not the thread running `_load_agent`. Added a generation
  token (`agent_load_id` / `self._agent_load_id`) and gated both
  progress and completion callbacks on it so a superseded load can't
  clobber the current session's state or UI.

- TUI prompt lifecycle: added `_agent_load_pending()` and gated the
  `_process_channel_message` / `_handle_command` finally blocks on it
  so `/new` or `/resume` invoked from a command keeps the prompt
  disabled until the fresh load settles.

- TUI readiness failures: `_run_turn` and `_handle_command` now
  catch exceptions from `_await_agent_ready()` and surface a
  "Agent failed to load: …" system message instead of letting the
  exception escape into Textual's traceback panel.

* refactor(cli): share background agent loader between CLI and TUI

The CLI and TUI were carrying near-identical copies of the same
background-load state machine: the `agent_task`, the `agent_load_id`
generation token, the gated progress/completion callbacks, and the
per-server progress dict. Every CodeRabbit finding on that lifecycle
had to be fixed in both files.

Extract it into `cli/_agent_loader.py`:

- `MCPProgressTracker` — owns the `server -> (state, detail)` dict;
  exposes `prime`, `record`, `snapshot`, `totals`.

- `BackgroundAgentLoader` — owns `agent`, the in-flight task, and the
  generation token. Exposes `start(**loader_kwargs)`, `await_ready()`,
  `is_pending`. Internally gates all progress/completion callbacks by
  generation so a superseded load can't clobber the current session.
  UI-specific rendering plugs in via `on_progress` / `on_success` /
  `on_failure` callbacks.

Both surfaces now just wire their UI hooks; the loader file holds no
Rich / prompt_toolkit / Textual dependencies. Net -345 lines from
`interactive.py` + `tui_interactive.py`; +20 unit tests pinning the
lifecycle (generation filtering, cache-hit short-circuit, failure
reset, progress ordering).

* refactor(cli): make _on_done the sole authority for agent state transitions

await_ready no longer sets self.agent — it just awaits the task and
reads what _on_done already wrote. Eliminates the dual-write overlap
(asyncio guarantees done-callbacks fire in registration order).

* fix(tui): let users type during MCP load, only block on send

Remove prompt-disabling during background agent load — the TUI now
matches the CLI approach where the input stays enabled and only gates
on await_ready() at submit time. The MCPLoaderWidget still provides
visual feedback that loading is in progress.

* fix(loader): preserve real load error on await_ready; dedup failure message

CodeRabbit flagged two issues with the new loader:

1. After a failed load, `_on_done` nulled `self._task`, so the next
   `await_ready()` hit the "before start()" branch and the CLI wrapper
   remapped it to a misleading "checkpointer not available" message —
   losing the real exception (bad MCP config, network, etc.).

   Keep `_task` set on failure so `await_ready` re-raises the real
   exception. Added `needs_restart` so TUI's auto-retry check stays a
   one-liner and doesn't need to reach into task internals.

2. TUI reported each load failure twice: once from
   `_on_agent_load_failure` (the done-callback) and once from each
   caller of `_await_agent_ready` (`_run_turn`,
   `_process_channel_message`, `_handle_command`) catching the re-raise.

   `_on_agent_load_failure` is now the sole local reporter; callers
   just handle control flow (return cleanly, set channel response to
   unblock remote).

* fix(cli): wire /model handler through the agent loader

The /model command from main (merged via f1f0d7c) still reached for
`state["agent"]` (CLI) and `self._agent` (TUI) — both removed by the
background-loader refactor. CLI raised KeyError on first invocation;
TUI raised AttributeError. Writes to the old fields also had no effect
because every other code path now reads from `agent_loader.agent`, so
the model switch would have silently failed.

Route everything through the loader: `await _await_agent_ready()` up
front so /model doesn't race with the initial background load, build
the `CommandContext` with the current agent, and sync `ctx.agent` back
into `agent_loader.agent` (plus channel globals) when the command
replaces it.

* fix(cli): isolate progress callback, capture awaited agent, gate by requires_agent

Three CodeRabbit findings on the loader + command dispatch path:

- Wrap ``_on_progress`` in try/except inside the loader's gated wrapper
  so a buggy UI adapter can't bubble into ``loader_fn`` and fail the
  whole background load. The MCP client already protects this, but
  defence-in-depth keeps the loader self-contained.

- In CLI channel + main-loop streaming, capture the agent returned by
  ``_await_agent_ready()`` and pass that into ``run_streaming`` rather
  than reading ``agent_loader.agent`` after a subsequent ``await``.
  A concurrent ``/new``/``/resume``/``/model`` could have swapped it.

- Add ``requires_agent: ClassVar[bool] = False`` to ``Command`` and
  mark ``/compact``, ``/model``, ``/channel`` as ``True``. TUI dispatch
  sites (channel and keyboard) now check ``cmd_manager.resolve(...)``
  and only wait for readiness when the command actually needs the
  agent. ``/mcp add``, ``/skills``, ``/new`` etc. no longer deadlock
  behind a failing MCP load they are meant to fix.

* fix(cli): guard sync-back, subcommand-aware gating, /model adopt-path

Three CodeRabbit findings on command dispatch:

- ``_handle_command`` unconditionally synced ``ctx.agent`` back into
  ``agent_loader``.  For non-agent commands ``ctx.agent`` is ``None``,
  so ``/threads`` / ``/mcp`` / ``/skills`` (etc.) could clobber a valid
  loaded agent — and rebind channel globals to ``None``.  Guard the
  sync on ``ctx.agent is not None``.

- ``/channel status`` and ``/channel stop`` don't touch ``ctx.agent``
  but the class-level ``requires_agent = True`` blocked them behind
  agent readiness.  Added ``Command.needs_agent(args)`` (defaults to
  ``requires_agent``) so ``/channel`` can override with subcommand
  awareness; kept the class flag for the common case.

- ``/model`` builds a new agent from scratch, it never reads the
  existing one — gating it on readiness meant a broken provider
  blocked the command that would fix it.  Flipped it to
  ``requires_agent = False`` and added ``BackgroundAgentLoader.adopt``
  so the UI can seat the replacement and supersede any in-flight
  load (the generation token keeps a late completion from clobbering
  the adopted agent).

Bonus cleanup: ``CommandManager.resolve`` now returns
``(command, args)`` so callers can invoke ``needs_agent`` without
re-implementing ``shlex`` parsing.

---------

Co-authored-by: Xi Zhang <106144707+X-iZhang@users.noreply.github.com>
2026-04-22 18:22:33 +02:00

192 lines
7.2 KiB
Python

"""Live-updating widget that shows per-server MCP load progress.
Mounted above the chat input while MCP tools are being fetched in the
background. Re-renders on a 100 ms tick so the spinner animates and the
per-server states transition smoothly from pending → ok/error.
When the load finishes:
- All-success runs auto-dismiss after a short grace period so the chat
area isn't permanently crowded.
- Failures stick around longer so the user has time to read the error
detail, then auto-dismiss — otherwise the widget pins itself above
the input forever.
"""
from __future__ import annotations
import time
from rich.text import Text
from textual.widgets import Static
from ..status_bar import SPINNER_FRAMES
_DIM = "#7c8594"
_STRONG = "#e5e7eb"
_GOOD = "#5fcf8b"
_WARN = "#d7b45a"
_BAD = "#d86f6f"
# How long to wait after an all-success load before auto-dismissing.
_AUTO_DISMISS_SECONDS = 2.5
# Longer grace on failure so the user has time to read error detail.
_AUTO_DISMISS_ON_ERROR_SECONDS = 12.0
class MCPLoaderWidget(Static):
"""Shows a header line + one line per MCP server with its live status."""
DEFAULT_CSS = """
MCPLoaderWidget {
height: auto;
padding: 0 1;
margin: 0 0 1 0;
}
"""
TICK_SECONDS = 0.1
def __init__(self, servers: list[str]) -> None:
# server_name -> (state, detail); state ∈ {"pending","ok","error"}.
self._progress: dict[str, tuple[str, str]] = dict.fromkeys(
servers, ("pending", "")
)
self._frame = 0
self._tick_handle = None
self._finished = False
self._dismissed = False
self._auto_dismiss_at: float | None = None
# Seed with real content so Textual can measure us before the
# first tick; ``self.update()`` during ``__init__`` is unsafe
# (widget isn't attached yet), but we can pass the renderable
# straight into ``Static.__init__``.
super().__init__(self._build_renderable())
def on_mount(self) -> None:
self._tick_handle = self.set_interval(self.TICK_SECONDS, self._tick)
def on_unmount(self) -> None:
if self._tick_handle is not None:
self._tick_handle.stop()
self._tick_handle = None
# ── Public API ───────────────────────────────────────────────────
@property
def dismissed(self) -> bool:
"""Whether the widget has already removed itself from the DOM."""
return self._dismissed
def update_server(self, name: str, state: str, detail: str = "") -> None:
"""Record a progress event for one server and re-render."""
if self._dismissed or state not in ("pending", "ok", "error"):
return
# First-time-seen servers (e.g., ones missing from the initial
# prime set because the config file changed mid-load) just get
# appended — order stays stable for already-known entries.
self._progress[name] = (state, detail)
self._refresh_content()
def mark_finished(self) -> None:
"""Call once the background load task resolves (success or error).
If nothing ever progressed past ``pending``, the load was served
from cache (no events emitted) — drop the widget immediately
instead of flashing a misleading "0/N loaded" header.
Otherwise schedule an auto-dismiss: short on full success so the
chat area isn't cluttered, longer on failure so the user has
time to read the error detail before it goes away.
"""
if self._finished:
return
self._finished = True
progressed = any(state != "pending" for state, _ in self._progress.values())
if not progressed:
self._dismiss()
return
has_errors = any(state == "error" for state, _ in self._progress.values())
delay = _AUTO_DISMISS_ON_ERROR_SECONDS if has_errors else _AUTO_DISMISS_SECONDS
self._auto_dismiss_at = time.monotonic() + delay
self._refresh_content()
# ── Internal ─────────────────────────────────────────────────────
def _tick(self) -> None:
self._frame = (self._frame + 1) % len(SPINNER_FRAMES)
if (
self._auto_dismiss_at is not None
and time.monotonic() >= self._auto_dismiss_at
):
self._auto_dismiss_at = None
self._dismiss()
return
if not self._finished:
self._refresh_content()
def _dismiss(self) -> None:
"""Stop the tick timer and detach from the DOM.
Sets :attr:`dismissed` so the app can clear its widget reference
and late progress events become no-ops.
"""
if self._dismissed:
return
self._dismissed = True
if self._tick_handle is not None:
self._tick_handle.stop()
self._tick_handle = None
# Fire-and-forget remove — nothing awaits us.
self.remove()
def _build_renderable(self) -> Text:
spinner = SPINNER_FRAMES[self._frame]
pending = sum(1 for state, _ in self._progress.values() if state == "pending")
total = len(self._progress)
done = total - pending
header = Text()
if self._finished:
errors = sum(1 for state, _ in self._progress.values() if state == "error")
if errors:
header.append("✗ MCP ", style=f"{_BAD} bold")
header.append(
f"{done - errors}/{total} loaded, {errors} failed",
style=_STRONG,
)
else:
header.append("✓ MCP ", style=f"{_GOOD} bold")
header.append(f"{done}/{total} servers loaded", style=_STRONG)
else:
header.append(f"{spinner} ", style=f"{_WARN} bold")
header.append("Loading MCP tools ", style=_STRONG)
header.append(f"{done}/{total}", style=_DIM)
lines: list[Text] = [header]
for name, (state, detail) in self._progress.items():
line = Text(" ")
if state == "pending":
line.append(f"{spinner} ", style=_WARN)
line.append(name, style=_DIM)
elif state == "ok":
line.append("✓ ", style=_GOOD)
line.append(name, style=_STRONG)
if detail:
line.append(f" {detail} tools", style=_DIM)
else: # error
line.append("✗ ", style=_BAD)
line.append(name, style=_STRONG)
if detail:
summary = detail if len(detail) <= 80 else detail[:77] + "…"
line.append(f" {summary}", style=_BAD)
lines.append(line)
return Text("\n").join(lines)
def _refresh_content(self) -> None:
# NB: don't name this ``_render`` — that shadows Textual's internal
# ``Widget._render`` which must return a ``Visual``. Silently
# breaking that contract triggers ``'NoneType' object has no
# attribute 'get_height'`` during layout.
self.update(self._build_renderable())