Commit Graph

11 Commits

Author SHA1 Message Date
teknium1 579dbe0c71 fix(cron): sqlite_util imports are late so a running scheduler survives an on-disk upgrade
cron/ledger.py (e24c8499) existed so a long-running scheduler that lazily imports
notepad/incidents AFTER `hermes update` never needs new names from a module it already has
cached. The dedup deleted it and imported open_db/transaction from hermes_cli.sqlite_util at
module level; a pre-upgrade daemon has the OLD sqlite_util cached (executions imported
add_column_if_missing from it), so the first job tick after an upgrade would ImportError in
scheduler_prompt._build_job_prompt until restart.

- cron/{notepad,incidents,executions,delivery_queue}: import open_db/transaction/
  add_column_if_missing and cron.jobs._ensure_cron_dir inside _connect/_transaction/
  _initialize_schema. This also stops the 3.8k-line cron.jobs being pulled eagerly by
  importing a store (it was lazy in cron/ledger.open_ledger).
- gateway/hosted_rooms_common, hosted_room_policy_checkpoint: same treatment; the gateway
  imports hosted_rooms lazily from request handlers, so it has the same skew exposure.
- tests/cron/test_upgrade_module_skew.py: simulate the real skew (delete open_db/transaction
  from the cached sqlite_util, then import each store). The previous repoint deleted names
  from cron.executions, which notepad/incidents do not import from, so it passed regardless.
  Sabotage: a module-level `from hermes_cli.sqlite_util import open_db` in notepad fails it
  with "cannot import name 'open_db'".
2026-09-13 05:08:29 -07:00
teknium1 576accd92b refactor(sqlite): one open_db/transaction layer for every small store; plugin DBs use the WAL fallback
Twelve modules each carried their own sqlite3.connect + PRAGMA + `with conn:`
stack. The #69567 fd-leak fix (a `with conn:` commits but never closes, so each
call leaked a connection and its WAL/SHM fds until GC) was pasted as code plus
docstring into six of them and hosted_room_policy_checkpoint never received
it; plugins/plugin_storage.plugin_db was the only production caller issuing a
raw `PRAGMA journal_mode=WAL`, bypassing the network-FS fallback, the
WAL-reset-bug gate and the never-live-downgrade invariant that
hermes_state_wal.apply_wal_with_fallback carries.

hermes_cli/sqlite_util.py (already home to add_column_if_missing/write_txn,
imported by cron, gateway and hermes_cli alike) gains `open_db(path, *,
db_label, busy_timeout_ms, wal, foreign_keys, synchronous_full, row_factory,
check_same_thread, wal_lock_retries, initialize)` and `transaction(conn,
immediate=)`; cron/ledger.py is deleted and hosted_rooms_common's
open_sqlite/connect/transaction become 1-3 line forwarders. Migrated:
agent/verification_evidence, cron/{executions,incidents,notepad,
delivery_queue}, gateway/{delivery_ledger,hosted_room_policy_checkpoint,
hosted_rooms_common (-> hosted_rooms, hosted_room_driver)}, hermes_cli/
projects_db, tools/async_delegation, plugins/plugin_storage.

Behavior changes (each module keeps its effective PRAGMA set otherwise):
- hosted_room_policy_checkpoint: connection now closed after every use and
  on init failure (was leaked per call), busy_timeout PRAGMA set explicitly.
- projects_db: gains busy_timeout=5000 (was the sqlite3 default 5 s connect
  timeout with no PRAGMA); explicit and observable.
- delivery_ledger / async_delegation: busy_timeout PRAGMA now mirrors the
  10 s connect timeout they already had.
- plugin_storage.plugin_db: WAL through apply_wal_with_fallback (DELETE on
  network filesystems / WAL-reset-vulnerable builds instead of raw WAL);
  busy_timeout=5000.
- cron/incidents._redact_error: redact_sensitive_text(force=True) — the
  error text is persisted to disk.
- delivery_ledger's private duplicate-column guard and the unguarded
  `ALTER TABLE ADD COLUMN` sites (shared_metrics, api_server_run_idempotency,
  holographic store, kanban model_override) go through add_column_if_missing.
- hermes_state.py::_scrub_surrogates: dead byte-copy of
  hermes_state_messages._scrub_surrogates (0 callers) deleted.
2026-09-13 05:08:29 -07:00
fangliquanflq e24c8499f2 fix(cron): isolate ledger helpers from stale execution modules 2026-09-07 06:01:31 -07:00
Teknium e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium b385c0198f refactor(cron): rewrap long comment/docstring lines to 100 cols 2026-09-02 19:37:58 -07:00
Teknium b3d9b20a3b refactor(cron): share ledger connect/prepare/transaction across executions, incidents, notepad; compact suggestions 2026-09-02 19:18:21 -07:00
Teknium 064bcaf3c1 refactor(cron): compact scheduler_provider and small cron modules
- scheduler_provider.py: _profile_entry/_profile_cron_scope replace four hand-rolled
  home-override+store blocks in the multiplex ticker; comments compacted to the WHY.
- incidents/executions/notepad/monitor/suggestions/blueprint_catalog/__init__: docstrings
  and comments compacted; no code change (AST-identical).
2026-09-02 13:29:30 -07:00
wanliqin 9da8842585 fix(cron): resolve profile store paths per call
Resolve notepad and suggestion paths at transaction time so multiplexed profile ticks cannot write into the import-time home. Preserve explicit test overrides and cover writes after a profile context switch.

Co-authored-by: 이민재 <19909783+honor2030@users.noreply.github.com>
2026-09-02 06:27:24 -07:00
kshitijk4poor 0dc9367163 fix(cron): widen deleted-profile protection to all cron mkdir sites
Replace #96637's inline active_profile_homes() closure with #96508's
module-level _existing_profile_homes() filter (testable in isolation).
Widen _ensure_cron_dir from 3 to 12 mkdir sites across cron/ so every
directory creation fails closed for deleted named profiles, not just
the 3 originally protected. Add _is_named_profile_path() that checks
'profiles' in path parts (works for subdirs like cron/output/<job> and
scripts/ that the original parent.name heuristic couldn't reach).

Co-authored-by: misterdas <das7514@gmail.com>
2026-08-28 13:39:19 +05:30
Teknium 5dc0fa3889 fix: post-merge audit follow-ups for #81138/#81139/#81141/#81148
Four fix-forwards from the adversarial post-merge audit of the Aug 7
unreviewed merge batch:

- estop (#81148): is_engaged() now fails SAFE (engaged) on stat errors;
  the gateway estop gate lets recognized slash commands and replies owned
  by in-flight work (update prompts, clarify, slash-confirm, tool
  approvals, running sessions) through instead of consuming them; new
  gateway /pause [reason|off] command gives messaging-only operators an
  in-band engage/resume path (busy_policy=dispatch so it works mid-run).
- cron monitor mode (#81138): execution-mode invariants (monitor x
  no_agent, monitor_script x monitor_url, no_agent-requires-script) now
  have ONE owner (_validate_job_mode_invariants) called from BOTH
  create_job and update_job, so the create-time invariant can no longer
  be silently violated through the update door.
- cron notepad (#81139): remove_job now clears the job's notepad rows
  (clear_notepad was dead code -> orphaned KV state forever); clear is
  best-effort and no-ops without creating notepad.db.
- delegation batch gate (#81141): template-marker regex narrowed to
  multi-word placeholder shapes only (<feature name>, {file_path}) so
  generics (Vec<T>), HTML tags, JSON snippets, glob braces and f-string
  style no longer reject legitimate batches; duplicate-goal rejection
  removed (best-of-N fan-outs are legitimate).
2026-08-08 05:21:09 -07:00
Teknium 04e8a661f2 feat(cron): per-job durable notepad — KV scratchpad surviving scheduled runs
- cron/notepad.py: SQLite-backed cron_notepad(job_id, key, value,
  updated_at) store in its own profile-local db (cron/notepad.db),
  following the executions.py connection/transaction pattern. APIs:
  set_note/get_note/delete_note/list_notes/clear_notepad +
  render_notepad_section. Documented size caps: 16KB per value,
  128-char keys, 64KB per job total; oversized writes raise ValueError.
- cron/scheduler.py: inject non-empty notepads into the job prompt at
  the context_from data-injection seam as a clearly-labeled
  "Job notepad (persistent across runs)" section that also documents
  the CLI write path. Empty notepad renders "" — byte-stable prompts
  for jobs that never use the feature.
- hermes_cli/cron.py + hermes_cli/subcommands/cron.py:
  `hermes cron notepad <job_id> [get|set|delete|list]` under the
  existing cron subcommand tree (no new top-level command, no new
  model tool — the agent writes via terminal + CLI).
- tests/cron/test_notepad.py: CRUD, durability, cap enforcement,
  prompt injection, byte-stable empty case, read-failure resilience,
  CLI handler + dispatch (TDD; watched fail first).

Inspired by: Amp (Sourcegraph) cron notepad (idea-level, proprietary —
zero code).
2026-08-07 08:57:48 -07:00