* feat(middleware): reposition code interpreter middleware in the stack
* feat(models): add qwen3.7-plus model entry and update context window comment
* feat(models): add qwen3.7-max and qwen3.7-plus model entries for DashScope
* feat(auxiliary): implement auxiliary model support for background tasks and tool selection
- Added auxiliary model configuration to EvoScientistConfig.
- Introduced _ensure_auxiliary_chat_model function to manage auxiliary model instances.
- Updated onboarding steps to include auxiliary model selection.
- Modified middleware to route tool selection to the auxiliary model when applicable.
- Enhanced tests to cover auxiliary model functionality and configuration.
* feat(steps): update UI backend selection options and descriptions
* Refactor code structure for improved readability and maintainability
* feat(patches): implement OpenRouter response reasoning item stripping to prevent multi-turn errors
* feat: update version to v0.1.4 in badges, README, and pyproject.toml; adjust skill counts in steps.py
* feat(config): add auxiliary model and provider environment variables to test setup
* feat(memory): add observation memory lifecycle
Add file-backed observation memory with deterministic markdown records,
structured record_observation tooling, startup indexing, and
profile/observation prompt guidance.
Launch post-turn and post-subagent EvoMemory workers through LangGraph
dev so completed runs can update profile memory, save durable
observations, and write subagent execution summaries without blocking
the active agent.
Wire memory middleware into the main agent, subagents, async graphs, TUI
status reporting, worker activity accounting, and observation-aware
research prompts, with regression coverage for storage, lifecycle
scheduling, graph registration, status display, and stream reset
behavior.
* fix(cli): sync background agent server on resume
Resume flows now need to keep the LangGraph dev background server
aligned with the active workspace even when async subagents are
disabled. EvoMemory workers use that server too, so gating resume-time
sync on enable_async_subagents could leave workers pinned to the launch
workspace after resuming a thread from another workspace.
Run workspace sync unconditionally for Rich CLI and Textual resume
paths, while preserving WorkspaceMismatchError handling so failed sync
aborts the resume before mutating the active thread or workspace.
Propagate aborted resume callbacks through the command UI so
channel-issued /resume commands do not send false success or history
output. Channel slash dispatch now treats CommandManager-caught command
errors as command errors and skips completion hooks for those failed
commands.
Add regression coverage for disabled async subagents, callback aborts,
and channel command error reporting.
* fix(cli): prepare serve resume workspace before adopting
Load the resumed workspace agent and sync the background server as a
single pre-adoption step. Restore the previous active workspace if
preparation fails so serve mode keeps using the old session
consistently.
* fix(memory): untrack abandoned worker status watches
Stop treating watcher shutdown as confirmed worker completion. Terminal
worker statuses still count memory deltas, while poll failures or
watcher setup failures now remove the active run without crediting
partial outputs.
* fix(cli): report channel command failures accurately
Treat command_error as a None sentinel so empty error strings still
fail, and let TUI resumes continue only on non-mismatch
background-server sync failures while reporting degraded mode.
* fix(stream): clear memory counters for resume streams
Reset completed-memory counters for every new agent stream, including
Command-based HITL and resume streams, so saved-memory indicators do not
leak across turns.
* docs(tools): make observation recording guidance conditional
Clarify that agents should call record_observation only when the
observation tool is available, preserving the existing durability and
usefulness criteria.
* feat(config): add controls for profile and observation memory
Add config flags for profile memory, observation memory, observation
writer placement, and background memory workers.
Wire the controls through main agents, subagents, EvoMemory middleware,
and memory lifecycle workers so observation writes can be assigned to
the live agent, subagent worker, both, or neither. Keep turn memory
workers profile-only and make prompts reflect the available observation
read/write paths. Skip langgraph dev startup when neither async
subagents nor memory workers need the background server.
Add coverage for config parsing, prompt gating, middleware wiring, and
worker tool availability.
* test(cli): include memory defaults in serve config stubs
* fix(memory): offload async worker launch blocking calls
Run the langgraph-dev health check and memory-output snapshot in worker
threads from the async EvoMemory launcher so it does not block the event
loop.
* chore(memory): harden turn worker subagent guardrail
* chore(memory): refresh profile context per request
* fix(memory): offload async profile file reads
* fix(memory): offload async worker completion accounting
* Enhance multimodal handling in LLM model
- Updated `_flatten_message_content` to preserve media blocks (images, files) while flattening text content.
- Introduced `_sanitize_messages` to manage media hoisting for tool messages, ensuring compatibility with OpenAI APIs.
- Modified `_patch_openai_compat_content` to accommodate new media handling logic, including retry mechanisms for media errors.
- Added comprehensive tests for media preservation, including various scenarios with images, files, and unsupported media types.
* fix: preserve order of text and media blocks in message flattening
* test: add tests for _strip_media_types to ensure position preservation and deduplication
* feat(memory): migrate to profile memory files
* chore(stream): read profile headings from templates
* fix(display): keep assistant responses if response_text has started
* fix(memory): do not treat failed bootstraps as profile creation
* chore(memory): unlink blank legacy memory
* fix(memory): resolve project_id once
* fix(memory): preserve unreadable profile files
* chore(tui): render streamed narration inline with tool timeline
Update the TUI streaming timeline so assistant text emitted before or
between tool calls is rendered inline where it occurs, rather than being
kept as a single answer bubble above or below the tools.
If the model begins an assistant response and then emits another tool
call, the provisional response is converted into inline narration before
that tool. The final assistant message then renders only the remaining
response suffix, avoiding duplicate text in the completed transcript.
Stop/cancel handling now preserves any active inline narration, appends
the visible stopped marker only to the remaining displayed segment, and
still returns the full normalized stopped response for channel callers.
Completed tools continue to collapse while long runs are active, but
expand again when the turn reaches a final state so the completed
transcript shows the full tool timeline.
* fix(stream): preserve narration around tool timelines
Keep assistant narration attached to the tool call that follows it
instead of folding all streamed text into the final answer block.
Track narrated response segments in stream state, render them before
their corresponding regular or task tool entries, and keep final answers
limited to the response suffix that has not already been shown inline.
Preserve narration across normal completion, stop/error final frames,
sub-agent task calls, and collapsed live tool summaries.
Add regression coverage for pending tools, completed tools, sub-agent
task delegations, collapsed completed/running tool summaries, and final
stop frames.
* fix(tui): finalize inline narration transitions
* test(memory): use canonical project id helper
* feat: add WebUI mode support with related configuration and onboarding steps
* feat: enhance WebUI port configuration to prevent conflicts with backend port
* feat: add support for fresh interactive session detection in WebUI
* fix: update deepagents and langchain versions; enhance _reduce_messages_delta handling for None state
* fix: update langchain version constraint to >=1.3 in pyproject.toml and uv.lock
* feat: implement configurable sandbox execute timeout and enhance recovery instructions
* feat: add background process management tools and middleware for sandbox execution
* feat: enhance background process management with completion notifications and deduplication
* feat: enhance sandbox execution timeout validation and update related messages
* feat: enhance background process management with thread-specific completion notifications and HITL approval handling
* test: assert completion notification waits for process finish timestamp
* fix(cli): forward async-notifier replies back to originating channel
When PR #214's auto-notifier fires a synthetic agent turn after a
channel-originated conversation, the synthesized response only rendered
to the local CLI/TUI — the channel user (iMessage etc.) saw nothing
and had to manually re-prompt to find out what happened.
Adds a per-thread channel-origin registry in cli/channel.py and wires
the three notifier paths (Rich CLI / TUI / serve) to publish the final
response back via bus.publish_outbound when the originating thread was
started by a channel turn. Publish is fire-and-forget (scheduled on the
bus loop + done-callback for failure logging) so the notifier turn
doesn't block on the asyncio / textual event loop.
The registry is cleared on /new and /resume rotation so stale entries
don't accumulate.
* fix(cli): address review feedback on channel-origin forwarding
Follow-up to the review on #244 (din0s, X-iZhang):
- Guard the /resume origin cleanup on a real thread change in Rich CLI
and TUI (serve mode already did via thread_changed). Resuming the
already-active thread no longer wipes its still-live origin, which
would otherwise silently drop a later async-notifier forward — the
exact gap this PR closes.
- Re-bind the now-current thread to its channel after a channel-issued
/new or /resume slash command (which rotates the thread inside the
dispatch), so notifier turns on the rotated thread still forward.
- Guard the publish done-callback against a cancelled future, whose
.exception() raises CancelledError (rather than returning it) on
bus-loop teardown, so the intended warning still logs.
- Mirror the normal reply path's manager.record_message(channel, "sent")
for forwarded notifications so per-channel stats stay accurate.
- Print the closing "[channel: Replied to ...]" line in all three
notifier paths (Rich CLI / TUI / serve) when a forward actually
happened, so the forwarded block reads as terminated on screen.
Adds test_publish_records_sent_metric. ruff clean; notification-origin
suite (10) + related channel/CLI/serve suites (728) pass.
* fix(cli): store sender information separately from chat_id in channel origin
---------
Co-authored-by: X-iZhang <zacharyzhang2022@gmail.com>
* Add onboarding wizard with style and validation components
- Introduced `style.py` for shared visual elements used in the onboarding wizard.
- Created `validators.py` for input validation, including integer and choice validators, and API key validation functions for various providers.
- Implemented `wizard.py` as the entry point for the onboarding process, managing user prompts and configuration steps.
- Added progress rendering and autosave functionality to enhance user experience during the onboarding process.
* feat(onboarding): enhance validation and configuration for onboarding wizard
- Added validation for UI backends, workspace modes, and providers in the onboarding command.
- Updated channel definitions to include secret field handling for sensitive tokens.
- Improved user prompts for required fields, ensuring sensitive data is masked.
- Introduced constants for valid providers, UI backends, and workspace modes to maintain consistency.
- Implemented tests to ensure alignment between constants and interactive choices in onboarding steps.
* feat(onboarding): improve WeChat account ID prompt and validation for newly enabled channels
* feat(onboarding): enhance WeChat backend credential prompts and validation
* feat(onboarding): refine WeChat backend credential prompts for wecom and wechatmp
* Refactor onboarding package for improved structure and clarity
- Simplified the onboarding package by removing unnecessary re-exports and consolidating public API to only include `run_onboard`.
- Updated `install_back_keys` to `install_navigation_keys` for clarity and consistency in the prompter module.
- Enhanced the `NonInteractivePrompter` class to support strict mode, allowing for better handling of non-interactive prompts.
- Adjusted the onboarding steps to utilize the new navigation keys installation method.
- Improved the `run_onboard` function to handle section implications based on user flags, enhancing the onboarding experience.
- Updated tests to reflect changes in imports and ensure compatibility with the new structure.
* feat(onboarding): enhance validation logic for non-interactive prompts
* refactor(onboarding): streamline onboarding module structure and enhance validation error handling
* refactor(onboarding): enhance config revert logic to preserve original file state
* refactor(onboarding): enhance tavily key validation and error handling in onboarding process
* feat(deploy): implement standalone LangGraph server and CLI command for deployment
* feat(deploy): enhance port validation and environment variable management for deployment
* Refactor langgraph dev deployment and introduce workspace sidecar protocol
- Updated the deployment mode handling in `server.py` to use a single environment variable `EVOSCIENTIST_DEPLOY_MODE` with values `full` and `stripped`.
- Enhanced the `manager.py` to implement a workspace fingerprint sidecar, allowing cross-process reuse of langgraph dev instances while ensuring workspace consistency.
- Introduced functions to write and read the workspace sidecar, with error handling for missing or corrupt data.
- Added tests for the workspace sidecar functionality, including validation of the JSON schema and ensuring proper error handling for workspace mismatches.
- Updated existing tests to reflect changes in deployment mode handling and added new tests for signal handling during shutdown.
- Ensured that cleanup routines remove the workspace sidecar alongside the PID file during shutdown.
* fix(langgraph): improve workspace sidecar checks for process ownership and stale handles
* feat(middleware): add CodeInterpreterMiddleware with project-specific configuration
chore(config): increase checkpoint retention limit for runaway conversations
fix(tests): update database schema references from 'blob' to 'value'
chore(deps): update deepagents dependency to include quickjs support
* feat(deepagents): update to version 0.6.1 and add optional dependencies for quickjs
* feat(sessions): improve error handling for message deltas and update Overwrite type check
* Enhance PruningCheckpointer with DeltaChannel Awareness
- Introduced a new pruning strategy in `_prune_after_put` to preserve the `_DeltaSnapshot` chain during checkpoint pruning.
- Implemented methods to fetch recent checkpoint IDs and walk to snapshot ancestors, ensuring that necessary checkpoints are retained.
- Updated SQL queries to handle checkpoint and write deletions more efficiently.
- Added comprehensive tests for DeltaChannel-aware pruning, ensuring that the pruning logic correctly handles various checkpoint scenarios, including those with and without snapshot seeds.
- Refactored `_load_checkpoint_messages` to utilize the new saver interface, improving message reconstruction from checkpoints.
* feat(tests): add migration sweep test to preserve snapshot ancestor
* feat(sessions): enhance checkpoint retrieval to prevent transcript leakage in multi-agent scenarios
* feat(middleware): enhance CodeInterpreterMiddleware with configurable timeout and result character limit
feat(config): add CodeInterpreterMiddleware tuning parameters to EvoScientistConfig
feat(sessions): implement inline message delta reducer for improved message handling
* feat(dependencies): update deepagents version to 0.6.2 in pyproject.toml and uv.lock
* feat(backends): implement tier-aware virtual mount resolution for skills and memories
* test: add end-to-end test for workspace tier shadowing global tier in CustomSandboxBackend
* feat(backends): enhance virtual mount resolution for skills and memories with tier paths and quoting
* fix(tests): update Python command in virtual mount resolution tests to use python3
* fix(cli): handle spaces in @file mentions
The @file parser truncated at the first space, so dragging or pasting a
filename like `@PREPING_ Building Agent.pdf` only matched `@PREPING_`
and warned "file not found". Now supports `@"..."` / `@'...'` quoted
form for explicit paths, plus a greedy expansion fallback that walks
across whitespace until an existing file resolves (bounded by newlines,
the next `@`, and a 20-token cap). Autocomplete also returns quoted
mentions for any candidate containing a space.
* style: apply ruff format to file_mentions
* feat(llm): add dashscope-code provider for Alibaba Coding Plan keys
Alibaba Cloud Bailian "Coding Plan" subscription keys (sk-sp-*) route
through a separate endpoint (coding.dashscope.aliyuncs.com/v1) that the
standard `dashscope` provider can't reach. Add a sibling provider entry
matching the zhipu/zhipu-code and moonshot/kimi-coding precedents, with
its own validator (the coding endpoint returns 404 on /models, so probe
via chat.completions instead).
Closes#224
* fix(llm): keep dashscope as default provider for qwen3-coder shortcut
The MODELS dict is built from _MODEL_ENTRIES via a last-write-wins dict
comprehension. The initial commit listed dashscope-code AFTER dashscope,
which silently flipped the bare `get_chat_model("qwen3-coder")` shortcut
to the coding endpoint — breaking standard sk-* keys.
Reorder to match the zhipu-code / zhipu precedent: coding endpoint first,
general endpoint last so the general endpoint wins the collision and
remains the default for the shared "qwen3-coder" short name.
* feat(qq): add inline keyboard buttons for C2C HITL approval
QQ Bot supports inline buttons via `markdown + keyboard` payloads. Clicks
arrive as `interaction_create` events through the existing botpy
WebSocket gateway — no extra subscription needed beyond enabling the
`interaction` intent. Group-scope clicks are out of scope here (DM only).
Send path
- `_build_qq_keyboard(buttons)` mirrors the Feishu helper, mapping the
generic `{text, value, type}` shape to QQ's `{render_data, action}`
with action.type=1 (callback). One button per row for mobile clarity.
- `_send_chunk` extracts `metadata["buttons"]` and threads a `keyboard`
payload into `_post_markdown_message` for C2C only.
- Markdown→plain fallback can't carry a keyboard, so when buttons were
attached the fallback content gets a textual `Reply: 1=Approve, …`
hint built from the button list. `_parse_approval_reply` accepts
the same values typed manually, so the user is never stuck.
Receive path
- `on_interaction_create` is registered on the bot class.
- `_on_interaction` extracts `data.resolved.button_data`, builds an
InboundMessage, runs it through inbound middleware (Dedup suppresses
retry callbacks), and publishes directly to the bus — bypassing the
per-sender debounce buffer so the click value isn't merged with any
text typed in the same window.
- Always ACKs via `api.on_interaction_result(id, 0)` in `finally` so
QQ doesn't show the button as "expired", even if middleware drops
the click or something throws downstream.
`QQ.inline_buttons=True`; `_approval_prompt_metadata` now auto-attaches
the Approve/Reject/Approve-all button row for QQ HITL prompts.
* fix(qq): button-value coercion, ACK timing, HITL consumer wiring
Fixes 6 bugs found in the inline-keyboard commit and consolidates the
button helpers so the keyboard builder, plain-text fallback hint, and
interaction handler share one coercion path.
- Plain-text fallback no longer crashes on non-string `value` (e.g.
`{"text": "OK", "value": 42}`). Extracted `_normalize_button` is now
the single place that resolves `(label, value)` and coerces non-strings.
- `metadata["button_value"]` is the coerced string instead of the raw
payload, matching `content` and downstream string comparisons.
- `_on_interaction` ACKs first, before publishing to the bus, so the
QQ button UI never shows "expired" if middleware is slow.
- Wire `_approval_prompt_metadata` + `_format_approval_prompt(with_buttons=)`
into `InboundConsumer._stream_with_hitl` and `cli.channel.channel_hitl_prompt`
so the QQ `inline_buttons=True` capability is actually used end-to-end
(HITL prompts auto-attach Approve/Reject/Approve-all buttons when the
channel advertises the capability).
- Trim contradictory `_QQ_DEFAULT_PERMISSION` comment.
- Fix `test_group_interaction_ignored` docstring (ACK runs first now,
not in `finally` after a `return`).
Tests: `_normalize_button` covered indirectly via existing keyboard tests;
new regressions for non-string fallback hint, ACK-on-handler-throw, and
string-coerced `button_value` metadata.
* refactor(qq): slim button helpers and explicit has_buttons flag
Inline single-use _button_hint and the _QQ_BUTTON_STYLE/_QQ_DEFAULT_PERMISSION
constants in qq/channel.py; tighten _on_interaction (drop unreachable
"[button click]" sentinel and unused triggering_message_id metadata; collapse
"if resolved else" ternaries via `or ""`).
Replace the metadata round-trip ("buttons" in metadata) used to detect button
support in consumer.py and cli/channel.py with an explicit has_buttons bool
threaded through both the prompt formatter and metadata builder.
Apply ruff format to the previously unformatted blocks introduced earlier on
this branch so CI lint passes.
* feat(qq): send post-decision confirmation after HITL approval
Send a visible confirmation message ("✅ 已批准" / "❌ 已拒绝") right after
the user resolves a HITL approval — QQ Bot has no message-recall or edit API
for C2C, so a follow-up message is the only way to give the click/reply
strong feedback.
Bus consumer (consumer.py): only sends the confirmation when the user
actually responded (event was set), to avoid pretending the user approved
when the request really timed out and auto-approved.
CLI HITL prompt (cli/channel.py): mirrors the same set of confirmation
strings. Timeout and unrecognized-reply paths keep their existing English
text since their semantics differ (auto-reject vs auto-approve, plus a
hint about the unparsed input).
* feat(middleware): add ConfigurableModelMiddleware for dynamic model resolution
- Introduced ConfigurableModelMiddleware to resolve chat models from RunnableConfig.configurable on each call.
- Updated middleware initialization to include ConfigurableModelMiddleware.
- Enhanced context editing middleware tests to verify presence of ConfigurableModelMiddleware.
- Implemented tests for ConfigurableModelMiddleware to ensure correct model overriding and caching behavior.
- Added tests for deepagents model-passthrough patch to verify configuration injection in async tasks.
* feat(async-subagent): update middleware handling to prevent deadlocks in async sub-agents
* style: Refactor code formatting for improved readability in patches and test files
* refactor: streamline middleware construction and improve async handling in ConfigurableModelMiddleware
* fix: remove unused request parameter from _read_model_override function
* refactor: improve async handling in _ClientProxy and enhance logging in ConfigurableModelMiddleware
test: add behavior test to ensure AskUserMiddleware is excluded in async subagent mode
* fix: Improve watcher logic to prevent false-positive notifications on clean stream exits
* fix: Update watcher logic to drop notifications on persistent runs.get failures
* fix: Refactor test for watcher persistent failure notification handling
* fix: Enhance watcher test to validate all notification queues are empty after reconnect budget exhaustion
* feat: Implement async sub-agent auto-notification system
- Added async notifier functionality to handle notifications for sub-agents reaching terminal states.
- Introduced `AsyncTaskNotification` dataclass for structured notification data.
- Implemented `watch_run_and_notify` to monitor agent runs and enqueue notifications.
- Created `spawn_watcher` to manage watcher tasks and ensure proper cancellation of previous watchers.
- Developed `consume_notifications` to process notifications, deduplicate them, and format messages for LLM.
- Added tests for notification handling, including draining, deduplication, and formatting.
- Patched deepagents to integrate the new watcher functionality into start and update tools.
* Enhance async notifier with per-thread notification routing and error handling
- Introduced `origin_cli_thread_id` to `AsyncTaskNotification` for routing notifications back to the originating CLI session.
- Implemented per-thread notification queues to handle notifications based on the originating thread.
- Updated `has_pending_notifications` and `drain_notifications` to respect thread-specific queues.
- Enhanced `watch_run_and_notify` to detect in-band error events from the SSE stream and handle clean exits.
- Modified tests to verify the new notification routing behavior and ensure proper handling of notifications across threads.
- Added a fixture to restore the async watcher patch state in tests to prevent state leakage.
- Updated deepagents patching to capture the main agent's CLI thread ID for notification routing.
* feat: Enhance async notifier with thread-specific watcher management and notification filtering
* test: Enhance notification draining logic for cleaner test setup
* refactor: Remove summary field from AsyncTaskNotification and update related tests
* feat: Enhance async notification handling with target thread ID support
* Refactor async notifier and middleware for improved task management
- Removed the no-op shutdown watcher loop from async_notifier.py as it is no longer needed.
- Updated watch_run_and_notify to clarify notification handling and race conditions.
- Cleaned up shutdown handling in commands.py, interactive.py, and tui_interactive.py by removing obsolete shutdown watcher calls.
- Deleted the deepagents async watcher patch from patches.py, transitioning to a new middleware approach.
- Introduced AsyncWatcherMiddleware to handle async task notifications directly during tool calls.
- Updated tests to validate the new middleware functionality and ensure proper watcher spawning and cancellation.
- Enhanced test coverage for async watcher middleware, including edge cases and error handling.
* feat(tests): add fixture to reset notifier state before each test
* refactor(channels): remove dead MessageBus dispatcher
Outbound routing has two implementations: ``MessageBus.dispatch_outbound``
(subscriber-based) and ``ChannelManager._dispatch_outbound`` (registry
lookup). Only the latter is ever started in production — the former
is reachable solely from tests, yet both consume from the same
``bus.outbound`` queue. If anyone followed the bus's own API surface
they would silently steal messages from the real dispatcher.
Drop the unused machinery to leave a single, obvious outbound path:
- ``MessageBus.subscribe_outbound`` / ``dispatch_outbound`` / ``stop``
- ``_running`` flag and ``_outbound_subscribers`` map
- ``OutboundCallback`` type alias
- The lone ``bus.stop()`` call in ``cli/channel.py`` (was no-op)
- Four tests covering the removed code paths
* test(channels): drop empty MessageBus stubs after dispatcher removal
---------
Co-authored-by: Xi Zhang <106144707+X-iZhang@users.noreply.github.com>
* Refactor sub-agent architecture and introduce async support
- Removed the legacy subagent.yaml file and replaced it with individual YAML files for each sub-agent in the subagents directory.
- Updated the load_subagents function to support both directory and single file layouts for loading sub-agent configurations.
- Added new langgraph_dev module for managing async sub-agent lifecycle and deployment.
- Created graphs for async sub-agents (writing-agent, data-analysis-agent) and updated langgraph.json for deployment.
- Introduced new sub-agent definitions for planner, research, debug, code, and writing agents with appropriate system prompts and configurations.
- Enhanced package data inclusion in pyproject.toml to accommodate new sub-agent YAML files.
* Refactor code for improved readability by consolidating conditional statements and formatting
* feat: enhance async sub-agent support with workspace synchronization and user feedback
- Added console status messages during async sub-agent server startup and workspace synchronization to improve user experience.
- Implemented a new WorkspaceSyncWidget for live feedback during workspace sync operations.
- Updated onboarding to reject occupied ports and ensure proper workspace handling for async sub-agents.
- Introduced locking mechanisms to manage concurrent access to langgraph dev processes and workspace states.
* feat: add async sub-agent configuration and server management functions
* feat: improve port occupation handling and log file management in start_langgraph_dev
* feat: enhance async sub-agent handling and introduce comprehensive tests
- Updated `_maybe_swap_async_subagents` to improve async sub-agent management, ensuring internal flags are stripped before handoff.
- Enhanced port management in `onboard.py` to allow reuse of occupied ports if already running by the same service.
- Introduced file locking in `manager.py` to prevent race conditions during concurrent CLI invocations.
- Added new tests for async sub-agent swapping and langgraph manager functionalities to ensure reliability and correctness.
- Updated dependencies in `pyproject.toml` to include `psutil` and `filelock`.
* fix(docs): clarify sub-agent configuration in README
* test(manager): isolate _PID_DIR + tighten reuse-path assertion
Addresses CodeRabbit review on tests/test_langgraph_manager.py:
- Patch _PID_DIR to tmp_path so the FileLock setup in
ensure_langgraph_dev doesn't mkdir the user's real
~/.config/evoscientist/ dir as a test side-effect.
- Tighten "result is None or hasattr(result, 'poll')" to a strict
"result is None" — the reuse path returns None unconditionally,
so the OR clause was hiding potential regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(manager): clean up stale PID file when unrelated process reuses PID
* feat(tests): add validation tests for async flag in load_subagents
* fix(load_subagents): restrict to .yaml files and clarify configuration handling
* fix(load_subagents): improve error handling for non-dict specifications in YAML
* feat(onboard): add "LangGraph Port" step to onboarding process
* feat(langgraph): add concurrency configuration for langgraph dev workers
* feat(async-subagents): enhance MCP tool routing for async sub-agents
* fix(manager): update exception handling for connection errors and prevent zombie processes
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(onboard): detect installed skill packs via install manifest
Onboarding's _step_skills only inspected USER_SKILLS_DIR and matched
recommended entries by directory-name hint, so a pack like
EvoScientist/EvoSkills@skills (which explodes into paper-writing/,
evo-memory/, etc. under GLOBAL_SKILLS_DIR) was never detected and kept
appearing as not-yet-installed.
skills_manager now writes a per-tier .installed.yaml mapping skill
directory name -> original install source on every install, removes the
entry on uninstall, and exposes installed_sources(). _step_skills checks
both tiers and treats a recommended source as installed when present in
any manifest -- so packs are recognized regardless of how their child
dirs are named.
* fix(onboard): write install manifest atomically
Stage to a sibling temp file, fsync, then os.replace into place. A crash
mid-write can no longer leave a half-written .installed.yaml behind,
which would otherwise wipe out pack detection until the next reinstall.
* style: fmt
* fix(onboard): catch decode errors when loading install manifest
read_text() can raise UnicodeDecodeError on a hand-edited or corrupt .installed.yaml; pin encoding="utf-8" and add UnicodeError to the except clause so the function honors its "returns {} on any error" contract.
* refactor(cli): replace channel module globals with ChannelRuntime
Removes _cli_agent / _cli_thread_id from EvoScientist/cli/channel.py
and threads a ChannelRuntime via CommandContext.channel_runtime so
/model and /channel rebind without poking module-level state.
* fix(cli): address coderabbit review
- _auto_start_channel: bind ChannelRuntime only after
_start_channels_bus_mode succeeds, so a startup failure no longer
leaves a stale binding pointing at channels that never started.
- _sync_tui_command_completion (TUI) and the Rich CLI command-completion
paths: rebind the runtime on thread rotation, not just agent swap, so
/new and /resume keep ChannelRuntime in sync with the running thread
(matches the serve-mode hook contract).
- test_hook_syncs_channel_runtime: pin ctx.thread_id explicitly so a
bare MagicMock attribute can't silently mutate runtime.thread_id.
- New regression test covering the rebind-on-thread-rotation contract.
* fix(sessions): ensure migration sweep runs before yielding checkpointer to prevent race conditions
* fix(sessions): enhance migration sweep with progress indication and ETA estimation
* Implement PruningCheckpointer for efficient checkpoint management and add comprehensive tests
- Introduced `PruningCheckpointer` to manage checkpoint pruning after each `aput()`, ensuring only the latest checkpoints are retained based on a configurable limit.
- Added migration sweep functionality to clean up legacy checkpoints and prevent database bloat.
- Enhanced `get_checkpointer()` to utilize the new `PruningCheckpointer` and trigger migration sweeps when necessary.
- Developed a suite of integration tests for `PruningCheckpointer`, covering various scenarios including pruning behavior, concurrent writes, and retention policies.
- Implemented tests for migration sweep functionality, ensuring proper partitioning and user version management.
- Added diagnostic helper `db_stats` to provide insights into the database state, including thread and checkpoint counts.
* feat(sessions): enhance pruning logic to handle legacy DBs without writes table
* fix(tests): prevent atexit hook leakage in TestMigrationSweep
* feat(tests): enhance TestPruningCheckpointer to validate put+prune serialization
* feat(tests): refactor mock path implementation for get_db_path in test cases
* feat(context-window): add model context window patch table and apply function
* fix(tests): clean up formatting in context window tests
* fix(tests): update context window tests for Claude model exceptions
* feat: Refactor system prompt structure and enhance documentation for clarity
* refactor: Improve clarity and consistency in prompt documentation
* refactor(tests): Improve readability of first-person avoidance test assertion
* refactor: Remove redundant datetime imports and enhance prompt documentation
* feat: Enhance ModelPickerWidget for Ollama integration
- Implemented a sentinel row for "Custom Ollama model..." in ModelPickerWidget, allowing users to input arbitrary model names.
- Updated action handling in ModelPickerWidget to manage transitions between list and input modes.
- Added async model discovery for Ollama models, integrating with the /model command to fetch locally installed models.
- Created tests for Ollama model discovery and ModelPickerWidget behavior, ensuring proper functionality and user experience.
- Refactored validate_ollama_connection and discover_ollama_models for improved error handling and response management.
* fix: Simplify code by removing unnecessary line breaks in ModelPickerWidget and test cases
* fix: Restore globals on set_chat_model failure to prevent half-switched session
* fix: Improve error handling in ModelCommand by restoring globals on failure
* fix(chat): resolve model switch lag by tracking model/provider key for cache invalidation
* style: format set_chat_model function for improved readability
* fix(chat): improve model switching logic to prevent unnecessary cache rebuilds
* fix(model): ensure globals are restored on agent load failure to prevent model switch issues
* feat(cli): migrate command handling to CommandManager and enhance UI interactions
* feat(cli): add /clear and /help commands to enhance user experience
* feat(cli): implement /new and /resume commands with interactive session management
* Refactor MCP and Skills Command Handling
- Moved the interactive picker style to a centralized widget for consistency across MCP and Skills commands.
- Updated the MCP command to remove the old command dispatch logic, delegating to the new InstallMCPCommand.
- Enhanced the Skills command to utilize a new interactive picker for skill selection, improving user experience.
- Implemented cancellation handling in the picker to differentiate between user cancellations and empty selections.
- Added comprehensive tests for the new command structures and picker functionalities to ensure reliability.
* refactor(cli): streamline CommandManager dispatch and remove deprecated command set
* refactor(cli): enhance error handling and state management in ChannelCommand and RichCLICommandUI
* refactor(cli): update lifecycle callback terminology and improve async prompt handling in RichCLICommandUI
* refactor(cli): enhance SlashCommandCompleter to dynamically fetch workspace directory for autocompletion
* refactor(cli): unify quit handling in RichCLICommandUI with shared _stop helper
* refactor(cli): remove hardcoded slash commands and utilize command manager for dynamic completion
* refactor(cli): update MCP and skills command files for improved clarity and organization
* refactor(mcp_ui): remove unnecessary newline in _show_mcp_config function
* perf(cli): cut startup time of `evosci --help` from ~2.2s to ~0.3s
Module-level imports were eagerly pulling in langchain.chat_models (with
the whole anthropic/openai/google stack), langgraph, textual, and
prompt_toolkit on every invocation — even for `--help` or `config list`.
Defer those with PEP 562 `__getattr__`, using `lazy_loader.attach` (SPEC-1,
the scientific-python standard) where it's a clean attach pattern:
- `EvoScientist/llm/__init__.py`: attach `models` lazily so importing
`context_window` from this package no longer drags in langchain.
- `EvoScientist/stream/__init__.py`: attach display/events lazily; split
the shared Rich `Console` singleton into a new lightweight
`stream/console.py` so callers that only need `console` skip the
`stream.events` → `langchain_core.messages` chain.
- `EvoScientist/cli/__init__.py`: hand-rolled `__getattr__` (reaches into
`..stream.state`, which `lazy_loader` doesn't cover) so `commands` and
`app` are the only eager loads.
- `EvoScientist/cli/commands.py`: move `cmd_interactive`/`cmd_run` to
in-function imports so prompt_toolkit + textual only load when the
interactive path actually runs.
- `EvoScientist/cli/_constants.py`: read `AGENT_NAME` on demand so
`build_metadata` doesn't eagerly import `sessions` (langgraph/aiosqlite).
Adds `lazy-loader>=0.5` as a dependency.
* feat(cli): defer MCP tool loading with live per-server progress
The CLI was blocking ~5 s on MCP tool enumeration before the first
prompt appeared. Move the agent construction off the event loop and
surface per-server progress so the user can interact immediately and see
what's happening.
MCP client:
- Add an `on_progress` callback to `load_mcp_tools` / `aload_mcp_tools`
/ `_load_tools` emitting `start` / `success` / `error` events per
server.
- Fan connection attempts out with `asyncio.gather` so latency no longer
scales linearly with server count; cap simultaneous attempts at
`_MAX_CONCURRENT_CONNECTIONS` (8) via a semaphore so a big stdio fleet
doesn't spawn every subprocess at once.
Agent wiring:
- Plumb `on_mcp_progress` through `create_cli_agent` / `_load_agent` /
`load_mcp_and_build_kwargs` so CLI and TUI can plug in collectors.
CLI (`cmd_interactive`):
- Run `_load_agent` in a background thread via `asyncio.to_thread`; the
prompt and banner render immediately.
- `_await_agent_ready()` awaits the task before each agent-using site
(first turn, channel messages, `/channel`, `/compact`). Raises if
called without a prior `_start_agent_load` instead of silently
reloading without the SQLite checkpointer.
- Pre-prime the progress dict from `load_mcp_config()` so the
bottom-toolbar's `N/M` denominator is stable from the first render.
- Wrap `session.prompt_async` in `patch_stdout(raw=True)` so
`console.print` from the worker-thread progress callback lands cleanly
above the prompt as inline chat messages instead of stomping the
prompt cursor.
TUI (`EvoTextualInteractiveApp`):
- Same background load + `_await_agent_ready()` gates on every
`self._agent` read.
- New `MCPLoaderWidget` mounted at the top of `#input-shell` shows a
header with `N/M` and one live row per server (spinner → ✓ / ✗ with
tool count or error detail). On completion:
- all-clean loads auto-dismiss ~2.5 s later;
- cache hits (no events ever fired) dismiss immediately rather than
flashing a misleading "0/N loaded";
- failures keep the widget mounted so the user can read the errors.
- `dismissed` property lets the app clear its ref so late events from
slow servers become no-ops. The error branch of `_on_agent_loaded`
also settles the widget so a load failure can't leave the spinner
animating forever.
- Chat input is `disabled` while MCP resolves — no placeholder hack, no
"waiting…" system message.
Shared:
- Hoist braille spinner frames to `status_bar.SPINNER_FRAMES` and import
them in the TUI widget so CLI and TUI animate in sync.
Tests:
- Extend `test_agent_mcp_cache` fakes to accept the new `on_progress`
kwarg.
- New `TestLoadToolsProgressCallback` in `test_mcp_client` exercises the
event sequence for success/failure/mixed fleets, verifies a buggy
callback doesn't break the load, and asserts the semaphore caps
in-flight connections.
* style: ruff
* chore: update uv.lock
* chore: uv.lock
* fix: coderabbit issues
* style: fmt
* fix: move _await_agent_ready inside try block
* fix(tui): auto-dismiss MCP loader widget on failure
The widget was designed to stay mounted on failure so the user could
read error detail, but since it's pinned above the input it never went
away in practice — just permanent banner clutter.
Auto-dismiss on failure too, with a longer grace (12s vs 2.5s) so the
error summary stays readable.
* fix: address second coderabbit pass
- Channel handlers (CLI + TUI): catch agent-load failures so the
channel request doesn't hang; CLI moves `_await_agent_ready()`
inside the existing try/except, TUI catches explicitly and calls
`_set_channel_response` with the error.
- Stale background loads: `prev.cancel()` only stops the asyncio
wrapper, not the thread running `_load_agent`. Added a generation
token (`agent_load_id` / `self._agent_load_id`) and gated both
progress and completion callbacks on it so a superseded load can't
clobber the current session's state or UI.
- TUI prompt lifecycle: added `_agent_load_pending()` and gated the
`_process_channel_message` / `_handle_command` finally blocks on it
so `/new` or `/resume` invoked from a command keeps the prompt
disabled until the fresh load settles.
- TUI readiness failures: `_run_turn` and `_handle_command` now
catch exceptions from `_await_agent_ready()` and surface a
"Agent failed to load: …" system message instead of letting the
exception escape into Textual's traceback panel.
* refactor(cli): share background agent loader between CLI and TUI
The CLI and TUI were carrying near-identical copies of the same
background-load state machine: the `agent_task`, the `agent_load_id`
generation token, the gated progress/completion callbacks, and the
per-server progress dict. Every CodeRabbit finding on that lifecycle
had to be fixed in both files.
Extract it into `cli/_agent_loader.py`:
- `MCPProgressTracker` — owns the `server -> (state, detail)` dict;
exposes `prime`, `record`, `snapshot`, `totals`.
- `BackgroundAgentLoader` — owns `agent`, the in-flight task, and the
generation token. Exposes `start(**loader_kwargs)`, `await_ready()`,
`is_pending`. Internally gates all progress/completion callbacks by
generation so a superseded load can't clobber the current session.
UI-specific rendering plugs in via `on_progress` / `on_success` /
`on_failure` callbacks.
Both surfaces now just wire their UI hooks; the loader file holds no
Rich / prompt_toolkit / Textual dependencies. Net -345 lines from
`interactive.py` + `tui_interactive.py`; +20 unit tests pinning the
lifecycle (generation filtering, cache-hit short-circuit, failure
reset, progress ordering).
* refactor(cli): make _on_done the sole authority for agent state transitions
await_ready no longer sets self.agent — it just awaits the task and
reads what _on_done already wrote. Eliminates the dual-write overlap
(asyncio guarantees done-callbacks fire in registration order).
* fix(tui): let users type during MCP load, only block on send
Remove prompt-disabling during background agent load — the TUI now
matches the CLI approach where the input stays enabled and only gates
on await_ready() at submit time. The MCPLoaderWidget still provides
visual feedback that loading is in progress.
* fix(loader): preserve real load error on await_ready; dedup failure message
CodeRabbit flagged two issues with the new loader:
1. After a failed load, `_on_done` nulled `self._task`, so the next
`await_ready()` hit the "before start()" branch and the CLI wrapper
remapped it to a misleading "checkpointer not available" message —
losing the real exception (bad MCP config, network, etc.).
Keep `_task` set on failure so `await_ready` re-raises the real
exception. Added `needs_restart` so TUI's auto-retry check stays a
one-liner and doesn't need to reach into task internals.
2. TUI reported each load failure twice: once from
`_on_agent_load_failure` (the done-callback) and once from each
caller of `_await_agent_ready` (`_run_turn`,
`_process_channel_message`, `_handle_command`) catching the re-raise.
`_on_agent_load_failure` is now the sole local reporter; callers
just handle control flow (return cleanly, set channel response to
unblock remote).
* fix(cli): wire /model handler through the agent loader
The /model command from main (merged via f1f0d7c) still reached for
`state["agent"]` (CLI) and `self._agent` (TUI) — both removed by the
background-loader refactor. CLI raised KeyError on first invocation;
TUI raised AttributeError. Writes to the old fields also had no effect
because every other code path now reads from `agent_loader.agent`, so
the model switch would have silently failed.
Route everything through the loader: `await _await_agent_ready()` up
front so /model doesn't race with the initial background load, build
the `CommandContext` with the current agent, and sync `ctx.agent` back
into `agent_loader.agent` (plus channel globals) when the command
replaces it.
* fix(cli): isolate progress callback, capture awaited agent, gate by requires_agent
Three CodeRabbit findings on the loader + command dispatch path:
- Wrap ``_on_progress`` in try/except inside the loader's gated wrapper
so a buggy UI adapter can't bubble into ``loader_fn`` and fail the
whole background load. The MCP client already protects this, but
defence-in-depth keeps the loader self-contained.
- In CLI channel + main-loop streaming, capture the agent returned by
``_await_agent_ready()`` and pass that into ``run_streaming`` rather
than reading ``agent_loader.agent`` after a subsequent ``await``.
A concurrent ``/new``/``/resume``/``/model`` could have swapped it.
- Add ``requires_agent: ClassVar[bool] = False`` to ``Command`` and
mark ``/compact``, ``/model``, ``/channel`` as ``True``. TUI dispatch
sites (channel and keyboard) now check ``cmd_manager.resolve(...)``
and only wait for readiness when the command actually needs the
agent. ``/mcp add``, ``/skills``, ``/new`` etc. no longer deadlock
behind a failing MCP load they are meant to fix.
* fix(cli): guard sync-back, subcommand-aware gating, /model adopt-path
Three CodeRabbit findings on command dispatch:
- ``_handle_command`` unconditionally synced ``ctx.agent`` back into
``agent_loader``. For non-agent commands ``ctx.agent`` is ``None``,
so ``/threads`` / ``/mcp`` / ``/skills`` (etc.) could clobber a valid
loaded agent — and rebind channel globals to ``None``. Guard the
sync on ``ctx.agent is not None``.
- ``/channel status`` and ``/channel stop`` don't touch ``ctx.agent``
but the class-level ``requires_agent = True`` blocked them behind
agent readiness. Added ``Command.needs_agent(args)`` (defaults to
``requires_agent``) so ``/channel`` can override with subcommand
awareness; kept the class flag for the common case.
- ``/model`` builds a new agent from scratch, it never reads the
existing one — gating it on readiness meant a broken provider
blocked the command that would fix it. Flipped it to
``requires_agent = False`` and added ``BackgroundAgentLoader.adopt``
so the UI can seat the replacement and supersede any in-flight
load (the generation token keeps a late completion from clobbering
the adopted agent).
Bonus cleanup: ``CommandManager.resolve`` now returns
``(command, args)`` so callers can invoke ``needs_agent`` without
re-implementing ``shlex`` parsing.
---------
Co-authored-by: Xi Zhang <106144707+X-iZhang@users.noreply.github.com>
* feat: add support for session resumption with --resume flag and enhance thread ID resolution
* refactor(tests): streamline help output testing for --resume flag
* feat: enhance session resume functionality with improved thread ID resolution and SQL wildcard handling
* feat: improve error handling for resume hint retrieval in interactive modes
* refactor: streamline logging for print_resume_hint failure in interactive mode
* feat: implement deferred scrolling for Markdown-heavy content in interactive mode
* fix(mcp): forward proxy and CA bundle env vars to stdio subprocesses
The MCP SDK's stdio transport inherits only a minimal allowlist (HOME,
PATH, USER, …) from the parent, stripping http_proxy/https_proxy and
SSL_CERT_FILE/REQUESTS_CA_BUNDLE/etc. Behind a proxy or with a custom CA
bundle, stdio MCP servers silently hang on outbound requests while the
same server over HTTP transport works. Auto-forward the proxy and cert
vars when present; user-configured env still takes precedence.
* fix(mcp): use `uv tool install` so MCP packages survive uv sync
Source installs previously used `uv pip install --python $VENV <pkg>`,
which lands in the evosci venv but is not recorded in pyproject.toml or
uv.lock. A subsequent `uv sync` (typical after `git pull`) reconciles
the venv to the lockfile and removes the MCP package, forcing users to
re-run onboard.
Prefer `uv tool install <pkg>` for the non-uv-tool install path: the
binary symlink in ~/.local/bin survives uv sync and evosci upgrades,
and the MCP server gets its own isolated env (no dep conflicts).
Verify the expected CLI entry point resolves afterward; if not (package
has no console-script), fall through to the old uv-pip path so
command-less packages still work.
The uv-tool-env path (`uv tool install evoscientist --with <pkg>`) is
unchanged — it was already durable via uv's receipt.
* fix(mcp): gate standalone uv tool install on verify_command
Previously `install_pip_package` would route every install through
`uv tool install <pkg>` when `verify_command` was None, returning
success as long as the uv subprocess exited 0. Library callers
(`evoscientist[oauth]`, `lark-oapi`, etc.) expect the package to land
in the active venv so they can import it — a standalone uv tool env
is not importable, so the import fails at the next line.
Gate the `uv tool install <pkg>` branch on `verify_command` being
set: that signals the caller wants a durable CLI binary, which is
what `uv tool install` produces. Library callers omit it and go
straight to the pip-install-into-venv path.
Also: log info messages on every fall-through so stale-binary and
entry-point-missing failure modes are debuggable, and document the
--with → standalone recovery path.
* fix(mcp): resolve MCP binaries to `uv tool dir --bin`, not `.venv/bin`
Under `uv run`, the project venv's `bin/` comes first on PATH, so
`shutil.which("arxiv-mcp-server")` returns a stale `.venv/bin/` copy
left over from an earlier install instead of the fresh symlink that
`uv tool install` just placed in `~/.local/bin`. The venv copy gets
written to mcp.yaml and is then wiped by the next `uv sync` — exactly
the failure mode the durability fix was meant to prevent.
Query `uv tool dir --bin` directly and prefer binaries found there
over `shutil.which`. Same change to the post-install verify in
`install_pip_package` so a venv shadow can't falsely short-circuit
the fallback.
* refactor(mcp): split install_pip_package into install_library + install_cli_tool
`verify_command` was doing double duty: naming the CLI binary to check
*and* signaling "this is a CLI install, use the standalone `uv tool
install` path." Callers routed library installs through the CLI branch
any time they forgot to pass it, and the resulting standalone uv tool
env wasn't importable from the active venv.
Separate the two use cases into distinct functions, each with one
install strategy per environment shape. Shared logic lives in private
`_install_with_uv_tool_env` / `_install_via_pip` helpers.
- install_library(pkg): uv-tool-env --with → pip. Never uses standalone
`uv tool install <pkg>` (not importable from active venv).
- install_cli_tool(pkg, *, verify_command): uv-tool-env --with →
standalone `uv tool install` → pip. `verify_command` is now required.
Callers pick the right function at the call site: registry.py picks
based on whether `entry.command` is set; onboard.py call sites all
install libraries.
* fix(channel/qq): deliver HITL approval prompts reliably
QQ approval prompts were silently dropped when the markdown send hit
a QQ server-side error (e.g. template not configured, content audit)
because the fallback path only matched TypeError / specific string
patterns, and the plain-text retry reused the already-consumed
msg_seq which QQ then rejects as duplicate.
- Consume a fresh msg_seq for the plain-text fallback send
- Recognize QQ server error codes (304014/304023/304003/40034059)
and CN fragments ("模版"/"审核") as markdown-fallback triggers
- Promote send failure logs from debug to warning/error with
chat_id/msg_id/seq so real-world errors can be diagnosed
- Extend test_qq_channel with a server-error-code fallback case
* style(channel/qq): apply ruff formatter to approval-delivery fix
* Fix
* refactor(paths): unify global data directory to ~/.evoscientist and update related paths
* refactor(paths): update legacy session migration to respect XDG_CONFIG_HOME
* refactor(tests): clear XDG_CONFIG_HOME in legacy session migration tests for deterministic behavior
* feat(prompt): enhance user interaction with multiple-choice and free-text questions
* refactor(paths): rename MEMORY_DIR to MEMORIES_DIR for consistency
* style(tests): format code for better readability in test cases
* feat(prompt): add validation for 'other' option in user prompt
* feat(prompt): refactor validation logic for user prompts and add skip option
* feat(style): refactor to use shared _PICKER_STYLE from interactive module
Xi Zhang