zabih-sudo
7098862dea
perf(gateway): replace SSE poll loop with call_soon_threadsafe-fed asyncio.Queue
...
_write_sse_chat_completion and _write_sse_responses bridged their
stream_delta_callback queue into the event loop via
`await loop.run_in_executor(None, lambda: stream_q.get(timeout=0.5))`
in a while-True poll — a thread-pool round trip on every 0.5s tick even
when idle, plus up to 500ms of tail latency between a delta landing in
the queue and it reaching the SSE response.
Add ThreadSafeAsyncQueue (asyncio.Queue + a put_threadsafe() that wraps
call_soon_threadsafe), used by both streaming producer closures
(_on_delta, tool start/complete callbacks — all invoked from the worker
thread running run_conversation via loop.run_in_executor). Consumers
now do a plain `await asyncio.wait_for(stream_q.get(), timeout=0.5)` —
woken immediately when a delta arrives, no executor hop, no poll
interval.
Updated tests/gateway/test_sse_agent_cancel.py's 7 call sites to
construct ThreadSafeAsyncQueue inside the running loop (required, since
it captures asyncio.get_running_loop() at construction) instead of a
bare queue.Queue() at test-method scope.
2026-08-04 13:07:44 +05:30
..
2026-08-02 20:21:59 -07:00
2026-08-01 13:27:58 +05:30
2026-08-04 11:08:32 +05:30
2026-08-01 14:59:14 -07:00
2026-08-02 20:30:06 -07:00
2026-07-29 14:12:25 -07:00
2026-07-29 13:39:40 -07:00
2026-08-03 17:18:51 +05:30
2026-07-29 13:39:40 -07:00
2026-08-01 10:52:34 -07:00
2026-07-28 08:25:25 -07:00
2026-07-29 12:19:37 -07:00
2026-08-04 13:07:44 +05:30
2026-08-04 12:28:39 +05:30
2026-08-03 20:43:38 +05:30
2026-08-03 17:53:55 +05:30
2026-07-29 13:39:40 -07:00
2026-08-03 20:35:47 +05:30
2026-08-03 20:35:47 +05:30
2026-08-03 17:16:34 +05:30
2026-08-04 11:26:39 +05:30
2026-07-31 22:33:11 -07:00
2026-08-02 00:11:50 -07:00
2026-08-02 16:18:28 -07:00
2026-07-31 23:18:12 -07:00
2026-08-03 22:53:32 +05:30
2026-08-04 12:28:05 +05:30
2026-07-31 22:33:11 -07:00
2026-08-03 22:43:15 +05:30
2026-07-29 13:10:23 -07:00
2026-07-28 14:07:21 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-08-01 15:05:05 +05:30
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-08-02 00:11:50 -07:00
2026-07-25 07:17:52 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 15:12:28 -07:00
2026-08-03 17:19:49 -07:00
2026-07-29 13:10:23 -07:00
2026-08-01 23:28:28 -05:00
2026-07-29 13:10:23 -07:00
2026-08-02 00:11:50 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-08-03 20:30:32 +05:30
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-08-01 22:56:52 -05:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-31 22:35:07 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 18:13:09 -07:00
2026-08-03 20:28:35 +05:30
2026-07-29 18:55:10 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 17:35:16 -05:00
2026-07-26 19:30:32 -07:00
2026-07-31 22:39:11 -07:00
2026-07-26 16:06:50 -05:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 18:13:09 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 21:30:53 -07:00
2026-07-29 19:53:17 -07:00
2026-08-01 21:17:51 -04:00
2026-08-01 13:13:30 +05:30
2026-07-30 00:08:28 -05:00
2026-07-29 13:10:23 -07:00
2026-08-03 17:38:13 +05:30
2026-07-29 13:10:23 -07:00
2026-07-29 15:12:28 -07:00
2026-07-29 13:10:23 -07:00
2026-07-31 22:35:16 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 19:48:37 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 21:30:53 -07:00
2026-07-29 21:30:53 -07:00
2026-07-29 13:10:23 -07:00
2026-08-02 10:04:48 -07:00
2026-08-03 21:02:11 +05:30
2026-07-29 13:10:23 -07:00
2026-08-03 20:37:17 +05:30
2026-08-02 21:32:13 +05:30
2026-07-31 23:18:12 -07:00
2026-07-29 13:10:23 -07:00
2026-08-04 11:26:15 +05:30
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-08-01 15:58:42 +05:30
2026-08-03 20:43:38 +05:30
2026-07-29 13:10:23 -07:00
2026-07-28 11:57:37 -07:00
2026-08-02 21:30:54 +05:30
2026-08-02 23:10:28 +05:30
2026-07-28 19:20:14 -05:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-29 13:10:23 -07:00
2026-07-24 23:06:05 -07:00