Commit Graph

2169 Commits

Author SHA1 Message Date
Teknium 0bc8d4e2d3 refactor(ntfy): fold fatal-status handling into one helper, dedupe headers/env seeding, compact layout 2026-09-02 21:17:47 -07:00
Teknium bb25c36306 refactor(image_gen): compact krea/openai-codex/xai providers; extract krea payload builder 2026-09-02 21:17:25 -07:00
Teknium e02bb9e437 refactor(memory/holographic): lambda action table, _harvestable_text helper, entity lookup loop, hrr blob decoders 2026-09-02 21:17:08 -07:00
Teknium 38a5c86729 refactor(memory): compact retaindb queue bootstrap/overlay and byterover config loader/schemas 2026-09-02 21:16:10 -07:00
Teknium 034078214c refactor(memory/mem0): _setup probe helper, single TOOL_SCHEMAS table, closing() in backend, drop banners/restating docstrings 2026-09-02 21:15:59 -07:00
Teknium e86d3fab86 refactor(plugins/kanban): AST-neutral bracket hugging in dashboard plugin_api 2026-09-02 21:15:25 -07:00
Teknium 8e35634acc refactor(hindsight): compact __init__ state setup, header docs and helper docstrings 2026-09-02 21:13:55 -07:00
Teknium cfa88e0104 refactor(memory/byterover): tool dispatch as one data table (arg, fail msg, runner, formatter) 2026-09-02 21:13:20 -07:00
Teknium 9e3fbba4d2 refactor(memory/supermemory): unify client build, tool dispatch with error prefixes, config loading; drop dead prefetch state 2026-09-02 21:12:37 -07:00
Teknium b0b8e39862 refactor(memory/retaindb): compact queue connection handling, prefetch assembly, dispatch and shutdown 2026-09-02 21:12:01 -07:00
Teknium b0f7d07773 refactor(memory/holographic): store _one/_write SQL helpers, retrieval atom helper, compact docstrings 2026-09-02 21:09:39 -07:00
Teknium e986577dca refactor(plugins/kanban,dashboard_auth): split update_task/bulk_update into phase helpers, dict-driven orchestration writes; unify provider register() via _shared.register_provider/SkipRegistration 2026-09-02 21:09:26 -07:00
Teknium a2811e03f1 refactor(memory/retaindb): inline single-use prefetch fetchers into jobs table, fold config strip, merge queue close helpers 2026-09-02 21:06:38 -07:00
Teknium 968cbef157 refactor(memory/mem0): table-driven tool dispatch, shared _oss_hint/_truthy/_add helpers, packed provider tables 2026-09-02 21:05:29 -07:00
Teknium b54d21697e refactor(image_gen/openrouter): split generate into chat/image-api phase helpers, compact docs 2026-09-02 21:04:50 -07:00
Teknium 8c83b89dd4 refactor(model-providers): actual reuses base fetch_models + hermes_cli normalizer; commandcode anthropic subclasses chat profile; compact router/openrouter docstrings 2026-09-02 21:04:38 -07:00
Teknium c829298476 refactor(memory/byterover): flatten _run_brv error handling, compact provider init/hooks/dispatch 2026-09-02 21:01:39 -07:00
Teknium e501ee1465 refactor(langfuse): unify tool-result capture, root-close and flush helpers; compact docstrings 2026-09-02 20:57:44 -07:00
Teknium 42291693ab refactor(memory/retaindb): fold HTTP calls into one _http helper, unify debug-logged failures via _quiet, compact schemas/client 2026-09-02 20:57:29 -07:00
Teknium fc46eabcf5 refactor(plugins): AST-neutral layout compaction of loader, chronos, disk-cleanup, google_meet 2026-09-02 20:47:48 -07:00
Teknium d9beaff938 refactor(langfuse): drop orphaned _capture/_trace modules (never imported; superseded by __init__) 2026-09-02 20:34:44 -07:00
Teknium 4d1880e0bf fix(integration): restore subprocess encoding/stdin guards dropped in simplification
Simplification workers collapsed subprocess call sites into shared kwargs
helpers and dropped the Windows/TUI safety kwargs on the way:

- encoding='utf-8', errors='replace' restored on text=True runs in
  copilot_acp_client, hermes_cli/setup (vercel install), managed_uv
  (codesign steps), local_runtime/hardware._stdout, a2a adapter.
- stdin=subprocess.DEVNULL restored on copilot probe, verify/runner
  _SUBPROCESS_KW, iron_proxy._run, google_meet playwright/system_profiler,
  simplex convert, whatsapp _RUN_TEXT, mem0 ollama serve Popen.
  google_meet sudo/brew install keeps inherited stdin (user-confirmed,
  may prompt) — marked noqa: subprocess-stdin.
- Windows-safe SIGKILL: getattr(signal, 'SIGKILL', SIGTERM) in
  verify/runner; photon _kill call re-marked windows-footgun: ok
  (unreachable on win32).
- scripts/check_subprocess_stdin.py now recognizes **kwargs splats
  (**_KW / **_kw(...)) ONLY when the same-file definition provably sets
  stdin= — covers tui_gateway _capture_run_kwargs/run_kw. Parity test added.
2026-09-02 16:36:10 -07:00
Teknium ea6644e1dc fix(integration): telegram dm-topic persist reads config.yaml via read_user_config_raw()
Raw yaml.safe_load of config.yaml is only legal in owner modules; the
write-back round-trip uses the sanctioned raw primitive. Behavior-identical
(same file, same parse, {} on missing).
2026-09-02 15:41:16 -07:00
kshitijk4poor 60ee3790e0 fix(gateway): publish degraded status when polling dies mid-session, via a public seam
Follow-ups on the #101406 salvage (#101391):

- The issue's measured incident was a healthy connect followed by silent
  polling loss; nothing republished for 11 h. `_schedule_polling_recovery`
  (the single entry to the recovery ladder) now publishes "retrying" while
  the adapter is running; `_record_polling_progress` already flips it back.
- `BasePlatformAdapter.send_path_degraded` property (default False) replaces
  the runner's `getattr(adapter, "_send_path_degraded")` reach into a
  plugin-private attribute; Telegram overrides it. `_mark_connected()` reads
  the property instead of taking a `degraded=` kwarg, and `_mark_degraded()`
  + `DEGRADED_STATUS_MESSAGE` replace three copies of the same write/string.
- The startup connect stamp (sibling of the reconnect stamp) honours the
  same flag.
- Tests: mid-session death publishes retrying; pre-connect death does not;
  progress never flips while not running / fatal; runner reconnect stamp
  honours the adapter flag.
2026-09-03 03:47:38 +05:30
chelsealong ce52ae21a7 fix(telegram): do not publish connected for a degraded, unconfirmed polling path
connect() intentionally returns True when Telegram polling starts in
degraded mode (or a reconnect's require_progress=False skips the strict
readiness gate), so background recovery can retry without failing
gateway startup. But _mark_connected() published platform_state
"connected" unconditionally, and the reconnect watcher in gateway/run.py
stamped "connected" again right after -- so gateway_state.json was
indistinguishable from a healthy adapter for as long as recovery took
(observed ~11h on one seat).

_send_path_degraded already tracks exactly this (set at polling
generation start, cleared on the first confirmed getUpdates round-trip),
so:
- BasePlatformAdapter._mark_connected() takes a `degraded` flag and
  publishes "retrying" instead of "connected" when set.
- TelegramAdapter passes its current _send_path_degraded into
  _mark_connected().
- The gateway/run.py reconnect watcher checks the same flag before
  overwriting the adapter's own status write.
- _record_polling_progress() republishes "connected" the moment
  polling actually proves progress, so the state does not stay wedged
  at "retrying" until the next disconnect.

Fixes #101391
2026-09-03 03:47:38 +05:30
Teknium ae70536485 Merge branch 'simp/web' into simp/integration 2026-09-02 15:01:45 -07:00
Teknium 01b523ee96 Merge branch 'simp/adapters' into simp/integration 2026-09-02 14:19:41 -07:00
Teknium a07dceb01f refactor(adapters/small_group): 5147->3565; line/email/ntfy/homeassistant/sms send-family and standalone-send dedupe, god-method extraction, dead find_pending_for_chat removed 2026-09-02 14:06:48 -07:00
Teknium c4f96565d0 refactor(adapters/whatsapp_webhook): 6449->4465; whatsapp_cloud/common/plugin send + media unification, webhook route/filter dispatch tables 2026-09-02 14:06:38 -07:00
Teknium 791417590d refactor(adapters/matrix_group): 9272->6311; matrix dead fetch_history/_send_simple_message/HTMLParser stubs removed, simplex/mattermost/irc one-verb API helpers and send builders 2026-09-02 14:06:37 -07:00
Teknium 8f1ef66b1f refactor(adapters/p2p_group): 12752->8794; buzz/photon/a2a/raft dedupe (sidecar paths unified in photon package, JSON-RPC helpers, Nostr event builders), dead a2a security getters + stream_message removed 2026-09-02 14:06:36 -07:00
Teknium b8bea7199c refactor(adapters/cn_group): 12368->8549; wecom split into streaming/media/send_queue mixins, dingtalk inbound parsers -> inbound.py, google_chat cards.py + setup_files.py, teams summary_writer.py; dedupe token/URL/credential helpers 2026-09-02 14:06:34 -07:00
Teknium 513ba5b846 refactor(adapters/feishu): 8095->5665; unify request/response wrappers, card builders, comment timeline selectors; dead symbols dropped; compact docs 2026-09-02 14:06:33 -07:00
Teknium 33c74a26c6 refactor(adapters/slack): 10667->7456; drop F811 _ssrf_redirect_guard shadow, split _handle_slack_message_impl (1135->~500) and connect(), unify Block Kit handlers, client/DM lookups, ts-eviction, upload retry; compact docs 2026-09-02 14:06:33 -07:00
Teknium 6e5b084b8b refactor(adapters/discord): 11250->7733; _HermesView base for 6 component views (1074-line factory -> 634), _send_prompt/_resolve_channel/_send_url_media/_send_local_file unification, opus loading and setup helpers extracted, dead _discord_allow_any_attachment removed 2026-09-02 14:06:32 -07:00
Teknium cc11c6bfdd refactor(adapters/telegram): 11957->8819; unify send/edit retry classification, media caching, auth gates, model-picker branches, polling recovery; compact docs 2026-09-02 14:06:32 -07:00
kshitijk4poor 9954445fea fix(feishu): create the dedup persist lock lazily for bare-constructed adapters
tests/gateway/test_feishu_bot_admission.py builds FeishuAdapter via
object.__new__ and installs dedup state by hand (install_dedup_state), so
the asyncio.Lock added in __init__ was missing -> AttributeError in CI.
Resolve the lock through a getattr-with-create helper and add it to the
shared test helper.
2026-09-03 02:17:01 +05:30
kshitijk4poor 1730a2c32a fix(gateway): serialize offloaded persists so concurrent flushes land in order
Offloading the fsync to a worker thread (previous commit) lets two inbound
messages have two persists in flight. Without ordering, a slow first flush
lands after the second and the disk file loses the newer token/id/seen-id
(in-memory state stayed correct, so the loss only surfaced after restart).
The loop-side dict could also be mutated while the worker iterated it.

- weixin ContextTokenStore / discord non-conversational tracker: snapshot
  the payload on the loop, hand it to the worker, serialize with a per-store
  asyncio.Lock.
- feishu dedup: same asyncio.Lock around the offloaded persist (its snapshot
  was already taken under _dedup_lock; the write was not).
- Regression tests: two concurrent persists with a slow first write must
  leave the union on disk. All three fail on the previous commit.
2026-09-03 02:17:01 +05:30
pierrenode cf6547da47 fix(gateway): offload remaining atomic_json_write calls on platform-adapter hot paths
atomic_json_write() calls os.fsync(), which blocks until the write
reaches stable storage. #83906 offloaded this in channel_directory.py,
gateway/run.py, and gateway/slash_commands.py, but the same blocking
pattern was still present on three platform-adapter message hot paths
that were never touched by that pass:

- plugins/platforms/feishu/adapter.py: _is_duplicate() persists the
  dedup cache on every new inbound message (shared by both websocket
  and webhook transports). The in-memory bookkeeping stays under
  _dedup_lock; the fsync now runs off it via asyncio.to_thread so
  concurrent callers on the ws-thread loop aren't blocked on disk I/O.

- gateway/platforms/weixin.py: ContextTokenStore.set() persists the
  context-token cache on every inbound message carrying a
  context_token, called from the async poll-loop handler
  _process_message.

- plugins/platforms/discord/adapter.py: mark_many() persists the
  non-conversational message-id tracker, called from both send() and
  send_update_prompt() when a message is tagged nonconversational.

Each fix mirrors the existing asyncio.to_thread(atomic_json_write, ...)
pattern from #83906. Regression tests assert the persist call runs on
a different thread than the event loop, mirroring
test_directory_write_runs_off_event_loop_thread in
test_channel_directory.py.
2026-09-03 02:17:01 +05:30
kshitijk4poor e0567770f3 fix(gateway): off-loop review follow-ups for #101603 / #101605
Findings from the efficiency review pass on the two salvages, applied as one
small follow-up:

- copilot_auth: check the negative cache BEFORE taking the per-fingerprint
  exchange lock. During the 60 s post-failure window, dashboard polls now
  raise immediately instead of parking an executor thread behind the
  in-flight holder (up to ~50 s) to learn the same answer. Test hangs
  without the check (timeout 124), passes with it.
- buzz _localize_inbound_media: download_path.read_bytes() was still
  evaluated on the loop as the argument to the offloaded cache call — up to
  the 128 MiB inbound cap. Read off the loop too.
- test_list_credential_pool_keeps_loop_responsive: 0.5 s block / 0.25 s
  threshold (2x margin) so runner descheduling cannot false-fail it while a
  real regression still trips it.
2026-09-03 02:07:05 +05:30
Teknium 0af4d3682f refactor(web): kanban plugin_api — unify status dispatch, run/board helpers, _errors_to_500 2026-09-02 13:33:24 -07:00
Teknium 800648ddc0 refactor(web): achievements plugin_api — table-driven catalog builder, shared json/session helpers 2026-09-02 13:33:24 -07:00
Teknium 313b244e8f refactor(plugins/model-providers): reuse agent.reasoning_effort clamps, per-model dict tables, compact profiles 2026-09-02 13:30:28 -07:00
Teknium 5d78bd817d refactor(plugins): disk-cleanup and security-guidance — table-driven scans/patterns, safety lists unchanged 2026-09-02 13:30:10 -07:00
Teknium 517f593af9 refactor(plugins/google_meet): run_bot extraction, subcommand tables, shared json-file helper, node protocol dedupe 2026-09-02 13:30:10 -07:00
Teknium 74edfe04e9 refactor(plugins/spotify): action dispatch table, single request helper 2026-09-02 13:30:10 -07:00
Teknium eb60dd519e refactor(plugins/teams_pipeline): subcommand dispatch, unified download/Graph helpers, compact models 2026-09-02 13:30:10 -07:00
Teknium 238260e6aa refactor(plugins/dashboard_auth): shared session/JWT/config helpers in _shared; compact basic/drain/nous/self_hosted 2026-09-02 13:30:10 -07:00
Teknium 191cc8d30a refactor(plugins/web): BaseWebSearchProvider + _common helpers across all search providers; keyless_mcp failover cleanup 2026-09-02 13:30:10 -07:00
Teknium e43f381b4e refactor(plugins/browser): shared BaseCloudBrowserProvider for browser_use/browserbase/firecrawl 2026-09-02 13:30:10 -07:00