Commit Graph

21 Commits

Author SHA1 Message Date
Teknium 9b669ca507 refactor(agent/relay): _ManagedAttempt error-recording ctx manager; managed_callback_guard as contextmanager; plugin cleanup steps table; close_session via _operation; anthropic normalize predicates collapsed 2026-09-02 22:35:41 -07:00
Teknium c6b813e9d6 refactor(agent/relay): hoist relay_tools guard closure; suppress/tuple-swap micro-collapses 2026-09-02 20:47:01 -07:00
Teknium c378c7f632 refactor(agent/relay): shared canonical JSON encoder; ManagedLlmStream startup teardown collapsed to one guard 2026-09-02 20:44:09 -07:00
Teknium ec3ad0c7e6 refactor(agent/relay_llm): drop dead _relay_observes_chunks branch; _ManagedAttempt._complete; fold stream_current fast paths 2026-09-02 20:14:42 -07:00
Teknium c83de74cb7 refactor(agent/relay): pack call/signature spans to 120 cols 2026-09-02 19:55:45 -07:00
Teknium e58175139d refactor(agent/relay): drop _safe in favour of _warn_on_error/suppress; pack spans to 110 cols; strip intra-function separator blanks 2026-09-02 19:24:48 -07:00
Teknium be3a659b14 refactor(agent/relay): compact relay/transport docstrings and comments (keep every invariant); reflow boolean spans; drop no-op helpers 2026-09-02 18:53:45 -07:00
Teknium a560fd840c refactor(agent/relay): fold execute/execute_async invoke into _ManagedAttempt; stream() = ManagedLlmStream; codec tool normalizer table; plugin acquire preflight/activate split; pack call/signature spans 2026-09-02 18:35:48 -07:00
Teknium 00cd12f7e4 refactor(agent/relay): fold relay_llm attempt scaffolding into _ManagedAttempt; dedupe runtime session/scope paths; drop dead relay_runtime module-level shims 2026-09-02 13:29:48 -07:00
Teknium 961f7481a7 fix(relay): bypass managed execution for nested calls inside managed callbacks
The native Relay pipeline binds its Futures to the event loop that
entered run_in_session_async. While a managed tool callback executes,
that loop is blocked until the callback returns — so a nested managed
relay call made from inside the callback (vision_analyze's auxiliary
LLM call on a worker-thread loop) awaits a Future that can never
complete: 'RuntimeError: Future attached to a different loop', or a
deadlock, plus 'Event loop is closed' at shutdown when the orphaned
future completes late. (#77244)

Fix: managed_callback_guard, a ContextVar depth marker set around every
Hermes callback the relay adapters hand to the native pipeline
(relay_tools.execute invoke, relay_llm execute/execute_async invoke,
ManagedLlmStream run_callback). resolve_execution_context returns the
no-relay triple while the marker is set, so nested calls run unmanaged.
The marker propagates through contextvars.copy_context() into the
worker threads tools use for their internal async work.

Top-level turn LLM calls and tool wraps stay fully managed — verified
live: vision_analyze works under active shared metrics while the main
turn still records managed llm.execute events.

Alternative fixes considered and rejected: removing
retain_managed_execution (kills the shared-metrics managed pipeline)
and gating on main-thread identity (managed tool wraps legitimately
run on the run_agent thread, so that gate disables relay everywhere).
2026-08-08 23:46:17 -07:00
Teknium ceebb21dd7 fix: suppress pydantic serializer warnings leaking to the terminal
The Anthropic SDK's streaming accumulator builds ParsedMessage snapshots
whose ParsedTextBlock content doesn't match the generic union pydantic
expects, so model_dump() on stream events (message_stop) emits
PydanticSerializationUnexpectedValue UserWarnings straight into the
user's CLI output mid-response.

Pass warnings=False at every helper that dumps arbitrary SDK models
(relay_llm/_jsonable, relay_tools/_jsonable, anthropic_adapter
_to_plain_data, run_agent _hook_jsonable, chat_completion_helpers
extra_content/reasoning_details sites, chat_completions transport),
with a TypeError fallback for duck-typed model_dump implementations.

Adds regression tests including a precondition test that proves the
fixture still trips the warning without suppression.
2026-08-08 23:14:30 -07:00
Alex Fournier 14bed44c8c Reapply "feat(observability): integrate NeMo Relay runtime and shared metrics"
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-27 21:10:51 -07:00
Jeffrey Quesnelle 841a5a744a Revert "feat(observability): integrate NeMo Relay runtime and shared metrics" 2026-07-27 22:28:08 -04:00
Alex Fournier 06ed41aa1f fix(relay): preserve managed cancellation signals
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-23 14:21:37 -07:00
Alex Fournier 4788994bd2 fix(relay): preserve managed callback context
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-23 13:53:59 -07:00
Alex Fournier c16f86dc4c fix(relay): serialize rewritten tool results
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-23 13:13:58 -07:00
Alex Fournier 1c6a96032c fix(relay): preserve successful managed results
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-23 13:12:50 -07:00
Alex Fournier 6efc5fe0af fix(runtime): preserve provider payloads through Relay
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-21 09:25:22 -07:00
Alex Fournier 55b7903cf7 fix(observability): harden Relay terminal lifecycle metrics
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-19 08:59:42 -04:00
Alex Fournier 4dedaa4237 refactor(runtime): consolidate Relay lifecycle ownership
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-19 08:59:06 -04:00
Alex Fournier 9bc521b138 refactor(runtime): manage Relay LLM tool and subagent execution
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-19 08:57:38 -04:00