Commit Graph

36 Commits

Author SHA1 Message Date
Teknium a1682376ca feat(profiles): rename any agent — the default profile gets a display name (#45624)
`hermes profile rename default <name>` (and the Desktop/dashboard rename
flows) now set a presentation-only `display_name` in profile.yaml instead
of erroring. The canonical id stays "default"; resolution, comparison,
and spawn paths are untouched. Named profiles keep real renames and their
display_name survives the move.

Surfaces: profile list/show/status, /profile (text only — data.profile
stays canonical), dashboard ProfilesPage, TUI-gateway profiles.list, and
Desktop (rail, switcher, Manage page, and the Bot Mode roster via a
displayName fallback so a renamed default shows its name, not "default").

Slimmer redo of the direction in PR #87760 by @yxssxn — thanks; see PR
body for what changed vs that approach.
2026-08-18 02:27:18 -07:00
Teknium 2c2697b52e feat: widen Tier 1 advisory scan to license + security checks
Review feedback from NVIDIA (Nir Paz): run the full deterministic
Tier 1 surface, not just pii,unicode,lint.

- TIER1_CHECKS now pii,unicode,lint,license,security. License is pure
  static (no measurable cost); security invokes NVIDIA SkillSpector in
  its keyless static-rules mode (~+1.2s per install). schema/quality
  stay excluded: hygiene signal ("author not specified" is
  high-severity upstream), wrong noise for an install prompt.
- SkillSpector is a second optional binary, pinned separately. Absent
  or failing, the security check reports status="incomplete" and the
  adapter treats it as "no opinion" — surfaced as a dim "(not run: ...)"
  note, never as a failure.
- _parse_report derives the verdict from COMPLETED validators only.
  This also absorbs a live upstream inconsistency: SkillEvaluator's
  anti-tamper cross-check on SkillSpector's risk score currently trips
  on moderate-finding skills (fail verdict with zero findings, e.g.
  github-pr-workflow at 15 MEDIUM issues / score 35). Reported to
  NVIDIA separately; either way an evidence-free fail must not render
  as an unexplained failure at install time.
- Dashboard tier1 block gains incomplete_checks.
- Docs: SkillSpector install command + not-run semantics.
- Tests: 28 (was 24) — incomplete-status exclusion, verdict derivation,
  not-run formatting.

E2E against real binaries: clean skill (no findings), skill tripping
the upstream consistency check (passed, "(not run: Security Scan)"),
seeded dirty skill (2 findings, SECRETS row). Full scan cost measured
at ~1.4-1.5s per skill, install-time only.
2026-08-17 17:04:40 -07:00
Teknium 183f18d530 feat: advisory NVIDIA SkillEvaluator Tier 1 scan on skill installs
Adds an optional, advisory second-opinion scan to the skills hub install
path using NVIDIA SkillEvaluator's deterministic, keyless Tier 1 checks
(PII, unicode smuggling, script lint).

- tools/skillevaluator_scan.py: subprocess adapter — runs the scanner
  over the quarantined bundle, parses the JSON report, classifies
  secrets-class findings (private keys, tokens, credentialed connection
  strings) apart from advisory PII findings. Every failure mode
  (binary missing, timeout, crash, bad JSON) degrades to a no-op.
- hermes_cli/skills_hub.py: prints the advisory panel after the built-in
  guard's policy decision and before the install confirmation. Findings
  are shown with file:line; secrets-class findings render red with a
  loud warning. Warn-and-continue by design — the built-in skills guard
  remains the only enforcement layer, because the upstream PII scanner
  has known false-positive classes (git@github.com, docs example
  emails, op:// references).
- hermes_cli/web_routers/skills.py: the dashboard Browse-hub scan
  endpoint returns the same advisory data in a new `tier1` field.
- config: skills.tier1_advisory (default true; no-op without the
  optional scanner binary on PATH).
- docs: user-guide/features/skills.md section with install command and
  config toggle.

Scanner install (optional):
  uv tool install --python 3.13 \
    "skillevaluator @ git+https://github.com/NVIDIA/SkillEvaluator.git"

E2E-validated against the real scanner binary: clean bundled skill (no
findings, "no findings" line), seeded dirty skill (email + credentialed
connection string -> yellow/red panel, install continues), config
disable via real config.yaml (silence). Real scan cost: ~0.2s per skill.
2026-08-17 17:04:40 -07:00
Teknium cb1b1da219 fix: surface missed cron fires as last_fire_error on the job record
On hosted deployments a scheduled fire that cannot be forwarded to the
gateway api_server (dead 8642 listener, gateway down) was invisible
outside gui.log: no execution row is created because the claim never
happens, so `cronjob list` showed a healthy job that silently missed
days of scheduled runs (4 consecutive nightly misses in the field,
diagnosed only by log grep).

Changes:
- cron/jobs.py: note_fire_forward_failure() durably stamps
  last_fire_error ({at, detail}) on the job record; mark_job_run clears
  it on the next successful run so it always describes current
  auto-fire health (mirrors preflight_alerted/drift_alerted).
- hermes_cli/web_routers/cron.py: the dashboard fire webhook stamps the
  job on the gateway-unreachable path, best-effort (never disturbs the
  503/Retry-After retry contract or the OOF-266 intentional-stop drop).
- tools/cronjob_tools.py: _format_job carries last_fire_error so the
  agent-facing cronjob list surfaces it.
- hermes_cli/cron.py: `hermes cron list` prints a red
  "Missed scheduled fire" line.
- web/: dashboard CronPage renders the miss; api.ts type updated.
- gateway/run.py: one-time startup warning when an external cron
  provider is active but the api_server adapter is not running (the
  fire path is dead-on-arrival; most common cause is API_SERVER_KEY
  missing from an unsupervised gateway relaunch).
- website/docs: cron doc section on missed fires.
2026-08-17 11:29:10 -07:00
Shannon Sands 4822156923 fix(cron): stop retry storms when the gateway is deliberately stopped (OOF-266)
Since the managed-cron redesign (#84339, v2026.8.13) the dashboard fire
webhook forwards fires to the gateway process and returns 503 when it is
unreachable so NAS/QStash retries. Correct for transient windows — but an
operator-STOPPED gateway can never be fixed by retrying: every fire on
every job burns the full scheduler retry budget, NAS converts each 503 to
a retryable 502, and the resulting storms page on-call for a non-incident
(OOF-266 and its five duplicate tickets; +93% relay callback failures as
the fleet adopted v2026.8.13).

Split the unreachable path by durable operator intent:

- desired_state == "stopped" (written only by the s6 lifecycle commands;
  the same intent signal container-boot reconciliation trusts) -> drop
  the fire with 200 + a structured log line, mirroring NAS's own
  instance_stopped drop. Jobs are not lost: the Chronos provider
  reconciles and re-arms every job on the next gateway start.
- Anything else (crash loop, scale-to-zero wake, restart, legacy state
  file without desired_state) -> keep the retryable 503, now stamped
  with Retry-After: 60 so a scheduler that honors it spaces retries
  past the wake/restart window instead of exhausting them inside it.
  The gateway's own pass-through 503s (draining) get the same hint.

The intent check fails open (any parse/resolution error -> retryable
path) and is only consulted when the gateway is actually unreachable, so
a stale state file can never shadow a live gateway.
2026-08-16 19:50:07 -07:00
Teknium e22fa90769 feat(desktop): MCP fleet cost/usage overlay with schema token estimates and 30-day usage
Each configured server row on the MCP Capabilities page now shows what it
costs and whether it earns its keep:

- ~per-call token estimate of the server's tool schemas, summed over ENABLED
  tools only (ceil(schema_chars/4) via the existing include/exclude filter)
- 30-day usage count from getUsageAnalytics(30), cached per scope profile
  like the Toolsets tab's toolCallsCache, mapped to servers via the
  mcp__<server>__<tool> registry-name convention (tools/mcp_tool.py)
- a subtle muted "unused" pill on enabled, probed-ok servers with nonzero
  schema cost and zero 30-day uses — never a dialog

Backend: the /api/mcp/servers/{name}/test probe now fills an additive
per-tool `schema_chars` (length of the SAME converted registry schema the
agent registers). Older backends omit it → renderer shows counts only;
older renderers ignore the extra key. Display-only: nothing changes what
schemas are sent to models, no config knobs.

i18n keys (costTokens/usage30d/unusedPill) added to types/en/zh/zh-hant/ja
(ar inherits en via defineLocale overrides). Pure math lives in
lib/mcp-cost.ts with unit tests; Python wire shape pinned in
tests/hermes_cli/test_web_server_profile_unification.py.
2026-08-16 02:24:32 -07:00
Teknium fc8ebff6d8 feat(mcp): unify the desktop MCP suggestion directory into the catalog
The desktop app carried its own hardcoded list of 17 vendor MCP endpoints
(apps/desktop/src/lib/mcp-directory.ts) powering the composer suggestion
pills — a second PR-reviewed vendor list, overlapping and drifting from the
Nous-approved MCP catalog (optional-mcps/).

This makes the catalog the single source of truth:

- manifest schema: optional `suggest:` block (keywords + hosts), parsed,
  validated, and normalized in mcp_catalog.py
- 15 new URL-only hosted-remote catalog entries (atlassian, sentry, datadog,
  notion, stripe, vercel, supabase, netlify, hugging_face, asana, intercom,
  airtable, webflow, paypal, square); figma + linear manifests gain suggest
  blocks
- GET /api/mcp/catalog now serves the suggest metadata
- desktop suggestion provider builds its match index from the catalog;
  the static directory remains only as a compatibility rung for older
  backends without suggest metadata
- setup card source line prefers the catalog entry's transport URL

GitHub stays out of the catalog on purpose: its hosted MCP rejects generic
DCR and the bundled github/* skills (gh CLI) are the stronger integration.
New desktop `github` suggestion provider offers the github-auth skill
instead — gated on a new cached GET /api/git/gh-auth probe so already-
authenticated users never see the pill.
2026-08-15 02:03:24 -07:00
Nicolas Formenton c75d835555 feat(desktop): mark a session as unread/read with a persisted watermark 2026-08-15 01:21:40 -07:00
Teknium 67a1c1ed1a fix(dashboard): use a fixed sidebar cache TTL (no HERMES_* env var for non-secret config) 2026-08-15 00:32:53 -07:00
Lucas Oliveira f0cfe5a56f perf(dashboard): bound multi-profile sidebar polling 2026-08-15 00:32:53 -07:00
zuowen7 f71f91a39b fix(desktop): surface compaction-archived messages in transcript reads (#80680) 2026-08-14 21:08:14 -07:00
Evgenii 1a8625abee fix(cron): harden gateway fire admission and provider compatibility
- The gateway api_server fire webhook acknowledges 202 only after a
  durable claim + execution row exist (admission failure stays retryable
  as 503; a live claim answers 200 duplicate), then dispatches the
  claimed snapshot with the live runner adapters (delivery parity with
  the built-in ticker, including relay-fronted and E2EE platforms).
- Legacy single-phase providers (a documented fire_due override without
  split hooks) keep being driven through their own hook. Capability
  detection now credits claim_fire AND fire_claimed overrides, so
  Chronos is correctly classified split-aware (its re-arm lives in
  fire_claimed; the redundant fire_due passthrough override is removed).
- Multi-profile dashboards fail closed for external providers: an
  unscoped reconcile would disarm other profiles' armed one-shots in the
  shared NAS registry.
- Manual runs (cronjob run) carry the owner-bearing claimed snapshot
  through every entry point, composing with upstream's manual-run
  heartbeat (#76502) and background dispatch.

Note: current main moved the dashboard NAS webhook to a pure
forward-to-gateway design (the gateway owns execution and live
adapters), so the dashboard-side claim/tracking machinery from earlier
revisions of this PR is dropped; the durable admission contract lives in
the gateway webhook path.
2026-08-14 20:46:50 -07:00
Teknium 7060ac7bed feat(computer-use): provision cua-driver at install time and on toolset enable
Choosing Computer Use should be a config flip, not a hunt for
'hermes computer-use install'. Three provisioning rungs:

- install.sh / install.ps1 pre-install cua-driver (best-effort,
  non-fatal, time-boxed at 660s above the upstream installer's 600s
  lock window; --skip-computer-use / -SkipComputerUse to opt out;
  Termux and unwritable-/Applications skipped cleanly)
- PUT /api/tools/toolsets/{name} (dashboard + desktop toggle) spawns
  the background 'hermes tools post-setup cua_driver' action when the
  toolset is enabled while the binary is missing — previously the
  toggle 'saved' but the tool never appeared in the schema because
  check_computer_use_requirements() couldn't find the binary
- hermes tools interactive flow already installed via
  _toolset_needs_configuration_prompt/_POST_SETUP_INSTALLED (unchanged)

Docs: computer-use.md enabling section rewritten around the new flow;
installation.md documents --skip-computer-use.
2026-08-13 02:44:48 -07:00
Brooklyn Nicholson 3efce9b98c feat(desktop): suggest MCP servers from the composer draft as brand pills
A renderer-local directory of official hosted MCP remotes (URL-only,
vendor-documented endpoints — deliberately not the reviewed install
catalog) powers keyword and pasted-link suggestions: typing jira or
pasting a *.atlassian.net URL floats an 'Add Atlassian' pill in the
composer's micro-action strip. Matching is whole-word/phrase (unicode
boundaries) plus strict host-suffix on links, host hits outrank
keywords, capped at two, debounced 600ms, and excludes servers already
in mcp_servers. Pills are session-scoped like the micro-action badges
and self-limiting rather than dismissible — they exist only while a
trigger is in the draft. A click drafts the setup request; the agent's
setup_mcp card carries the consent. Brand glyphs extracted from the
mcp-tab into lib/mcp-brands (shared, monochrome marks follow the theme
so GitHub/Notion/Vercel survive dark mode).
2026-08-13 01:06:51 -05:00
Ben Barclay bb597e1c02 fix(cron): managed-cron fires execute in the gateway process (live adapters + dashboard forwarder) (#84339)
* fix(gateway): pass live adapters to cron fire webhook's fire_due

The Chronos fire webhook (/api/cron/fire) called
provider.fire_due(job_id, adapters=None, loop=loop), so every
externally-triggered fire delivered through the standalone path even
with a live gateway in-process. E2EE platforms and relay-fronted
logical platforms (whose ONLY send path is the live relay adapter — no
native credential exists on the box) failed every external fire with
"platform 'X' not configured/enabled", while the same job delivered
fine under the built-in ticker (gateway/run.py passes runner.adapters).

Resolve the runner (self.gateway_runner → app['gateway_runner'] →
_gateway_runner_ref(), the same chain the drain check uses) and forward
its adapters. No runner → adapters=None, preserving the historical
standalone path byte-identically.

Note: does not by itself fix Fly-hosted scale-to-zero deployments where
NAS's callback lands on the DASHBOARD process (internal_port 9119) —
_fire_cron_job_for_profile there has no gateway runner in-process. That
topology needs a separate fire handoff (design pending).

* fix(cron): dashboard forwards Chronos fires to the gateway (503 when unreachable)

The dashboard's /api/cron/fire executed cron jobs in the DASHBOARD
process via _fire_cron_job_for_profile with adapters=None. On hosted
deployments (Fly proxy exposes only the dashboard's port) that made
every managed-cron fire deliver through the standalone send path, which
cannot serve relay-fronted logical platforms (their only sender is the
live relay adapter in the gateway process — no native credential exists
on the box) or E2EE rooms. It also ran the whole agent turn inside the
dashboard: wrong process for memory/session ownership and fire-claim
attribution.

Restore the invariant that the GATEWAY owns cron execution:

- Dashboard route: after verifying the NAS JWT and resolving the job's
  profile, FORWARD the fire to the gateway api_server's own
  /api/cron/fire on loopback, NAS bearer preserved (the gateway
  re-verifies the JWT — defense in depth, no new trust link), and pass
  the gateway's response through. Gateway unreachable → 503 so NAS
  retries per the Chronos contract (non-2xx = retryable; the store CAS
  de-dupes the eventual double fire). Deliberately NO local-execution
  fallback.
- Endpoint resolution mirrors gateway/config.py's api_server load order
  per target profile (config.yaml extra.port → API_SERVER_PORT from
  process env or the profile's .env → 8642), with /p/<profile>/ prefix
  routing under multiplex.
- docker/stage2-hook.sh: generate a strong API_SERVER_KEY into .env on
  first boot when absent (never overwrites an operator value), so the
  loopback api_server passes its startup guard on hosted images. The
  fire route itself is NAS-JWT-authed; the key gates the rest of the
  api_server surface. The listener binds 127.0.0.1 by default and the
  Fly service exposes only the dashboard port.
- _fire_cron_job_for_profile kept but deprecated (late-binding seam
  compatibility); no route calls it.
- docs/chronos-managed-cron-contract.md: document the two-hop inbound
  topology and the 503-retry semantics.

Depends on the previous commit (fire webhook passes live adapters to
fire_due) — together they make NAS→dashboard→gateway fires deliver over
relay end to end.

* fix(cron): read the profile api_server port via the canonical config loader

CI guard test_config_read_guard flagged the new _gateway_fire_endpoint
for a raw yaml.safe_load of the profile's config.yaml — the exact drift
class the guard exists to kill (raw reads miss the managed-scope
overlay, ${ENV_VAR} expansion, and root-model normalization).

Read through load_config() under a HERMES_HOME override scoped to the
target profile instead (the same pattern the deprecated
_fire_cron_job_for_profile uses for its store scope), and pull the port
with cfg_get. Test updated to stub load_config rather than write a raw
config.yaml.

* fix(gateway): only messaging platforms count for the scale-to-zero arm gate

The stage2 hook now generates API_SERVER_KEY for every Docker container,
and key presence force-enables the api_server platform. The scale-to-zero
arm gate counted every enabled platform, so the loopback api_server
listener made messaging_is_relay_only_or_absent False on every hosted
instance — silently disarming the feature (the not-armed log would show
enabled platforms=['relay','api_server']).

The arm gate and the not-armed logger now share one helper that filters
to enabled MESSAGING platforms, excluding LOCAL/API_SERVER/WEBHOOK —
the same non-messaging exclusion set _connect_platforms already uses.
A genuinely enabled direct-socket platform (Discord/Telegram) still
disarms. Two of the three new tests fail without this fix.
2026-08-12 17:04:44 +10:00
Brooklyn Nicholson 5b68d2271b feat(profiles): serve a cross-profile project tree and per-profile usage totals
`projects.tree` answers for the backend's own profile, so the grouped
sidebar had nothing to draw once the user asked to see every profile.
Run the same authoritative builder once per profile against that
profile's state.db and merge the results by folder, so one checkout is
one group no matter how many profiles work in it, and the owning profile
rides on each session row where the badge and filter can read it.

Group totals are summed in SQL rather than over the loaded page — a
number that shrank as you scrolled would be worse than no number.

Scope the batched sidebar slices while we're here: cron and messaging
came back cross-profile unconditionally, which is why a concrete profile
showed another profile's Telegram threads and cronjobs.

Closes #65710
Closes #42651
Closes #70629
2026-08-10 03:13:08 -05:00
Brooklyn Nicholson 21aaa8b4f8 feat(desktop): resolve a session's pull request
A session row can say whether its work is open, merged or closed, and link
to it. The join is the session's own repo + branch, asked of GitHub in one
batched GraphQL request per repo (branch aliases, not a `gh pr list` page
that a busy repo crowds ours out of), through the remote-aware git facade so
a desktop on a remote gateway asks the backend's `gh`.

Two ways a session's branch can't answer, both covered:

- It ran on trunk. Fork PRs share our branch namespace, so asking about
  `main` badges a stranger's PR onto it — trunk is never asked about, and
  cross-repository PRs are dropped server-side either way.
- It worked in a worktree, so the branch it recorded at start isn't where
  the PR came from. Creating a PR from the review pane binds the session to
  the branch it actually used, and for sessions that predate that, the PR is
  recovered from the transcript: `gh pr create` prints a bare PR url and
  nothing else, so a tool result whose whole output is one is a claim rather
  than a mention. Scanned read-only across profiles, once per session ever.
2026-08-09 06:21:04 -05:00
kshitij e8b05dc6c2 perf(dashboard): keyset pagination for streaming session export
OFFSET paging made the streaming export O(n^2) on huge transcripts;
after_id keyset paging keeps each page seek O(1). Adds after_id to
SessionDB.get_messages (ascending-only, guarded against latest/offset
combos).
2026-08-08 13:36:08 +05:30
kinsolee c750d5354a fix(sessions): prevent oversized transcripts from exhausting memory 2026-08-08 13:36:08 +05:30
kshitij 4ecdee38a6 fix(dashboard): close config-RMW gaps left by the off-loop sweep 2026-08-08 13:28:28 +05:30
Royalaid 965a548788 fix(gateway): serialize config mutations and finish the router off-loop sweep
Two follow-ups to the off-loop move, from external review (both verified,
the second larger than reported):

- Config read-modify-write handlers moved to worker threads could now
  interleave — _CONFIG_LOCK covers each load/save individually, never the
  span between them; the event loop used to serialize these accidentally.
  New _CONFIG_MUTATION_LOCK (worker-threads only, so it can never block
  the loop) held across the whole load→mutate→save span in all seven RMW
  handlers. update_config_raw skipped: it's a full-document replace with
  no server-side read, so a lock cannot close its client-side window.

- The review flagged two skills routes still taking _SKILLS_PROFILE_LOCK
  on the event loop; a systematic audit of hermes_cli/web_routers/ found
  24 on-loop routes (skills 5, mcp 9, tools 10, cron 1). All moved to the
  same inner-_run + asyncio.to_thread pattern, mutating ones under the
  mutation lock, uniform lock order (_SKILLS_PROFILE_LOCK →
  _CONFIG_MUTATION_LOCK). Await-safe _config_profile_scope routes, plain
  def routes, and already-threaded routes unchanged.

Regression tests: concurrent theme+font updates both survive (fails with
the lock nulled: "theme write lost to a concurrent font write"); event
loop stays responsive while the profile lock is held during GET
/api/skills. 214 tests passing across the touched suites.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 13:28:28 +05:30
kshitij afb46fdab4 refactor(cron): polish registration partial-failure surfaces
Follow-up to the salvaged registration contract:
- share one _raise_if_cron_registration_error() helper for the two
  byte-identical dashboard 424 except-blocks (web_server + cron router,
  via the existing late() seam)
- add endpoint-level 424 coverage for /api/cron/blueprints/instantiate
  (previously only the sync worker was tested)
- give chat/CLI surfaces a human-facing user_message() (job name, no
  exception class name) and add a recovery hint (pause/resume or update
  re-registers via provider reconcile) to the model/REST message
- consolidate five inline provider test doubles into one ABC-subclassing
  make_cron_provider conftest factory; the web_server test double now
  subclasses CronScheduler so an ABC rename fails loudly
- narrow the wrapper facade to keyword-only (**kwargs) and route the
  tool's partial-failure return through tool_error()
2026-08-07 17:45:06 +05:30
Gille f346458f29 fix(cron): surface initial scheduler registration failures 2026-08-07 17:45:06 +05:30
emozilla bdee48928f fix(dashboard): derive the stale-schema read probe from SCHEMA_SQL
After `hermes update`, the desktop sidebar showed "No sessions yet" until
the user's first message. #72424 added sessions.last_activity_at, which
list_sessions_rich now selects — but column adds only land through
_reconcile_columns() in the writable _init_schema, and read-only opens
skip that by design. Every sidebar read path opens state.db read-only, so
each poll raised "no such column: s.last_activity_at" until the first
prompt's lazy session-row persist forced a writable open and reconciled.

A heal for exactly this class already existed (_open_session_db_for_profile
probes the read-only handle and does a one-time writable reopen on
staleness), but its probe was a hand-written four-column list that never
learned last_activity_at — it went stale three days after shipping. And the
batched sidebar route (/api/profiles/sessions/sidebar) bypassed the helper
entirely, swallowing per-profile failures into an errors array the desktop
never surfaces, so the incident produced an empty sidebar with clean logs.

The fix removes the maintenance burden instead of paying it once more:

- hermes_state_schema.schema_read_probe_statements() derives one
  `SELECT <every declared column> FROM <table> LIMIT 0` per table from
  SCHEMA_SQL via the existing _parse_schema_columns() — the same source of
  truth the writable reconciler diffs against, so any future ADD COLUMN is
  probed with no list to update. Column references are table-qualified:
  an unqualified double-quoted identifier that fails to resolve silently
  degrades to a string literal (SQLite's double-quoted-string misfeature)
  and would make the probe pass on exactly the store it exists to catch.

- web_server splits the heal into a path-level _open_session_db_at_path
  (semantics unchanged) so the cross-profile session routes can share it;
  both profiles.py loops and _count_status_active_sessions (the remaining
  raw read-only sibling) now open through it. The heal stays a helper
  rather than a SessionDB classmethod on purpose: escalation-to-writable
  must remain an explicit caller decision — update_cmd.py opens read-only
  mid-update and must never write.

- Exhaustion guard: if the writable heal SUCCEEDS and the re-probe still
  fails (a schema problem ADD COLUMN cannot express), the store is marked
  exhausted — warn once, skip the probe, serve reads probe-less — instead
  of re-running the full writable init on every poll against a possibly
  live DB. A FAILED writable open (transient lock) is deliberately not
  recorded, so the next poll retries the heal.

- The per-profile swallow sites in profiles.py now also log a deduplicated
  warning, so a persistent read failure is loud in errors.log even though
  the response errors array stays invisible to the sidebar.

Tests: probe/SCHEMA_SQL coverage invariants (tests/test_schema_read_probe.py),
last_activity_at added to the /api/sessions heal parametrize, a sidebar-route
heal test reproducing the shipped symptom (errors == [] and the session
returned against a store missing the column), and an exhaustion test pinning
exactly one writable open. The sidebar and last_activity_at tests fail on
main.
2026-08-07 00:41:58 -04:00
kshitij 3556728a54 refactor(utils): move mode+owner preservation into atomic_write_text
Follow-up to the salvaged #79323 commits. The three hand-rolled
stat -> atomic_write_text -> chmod blocks (xai migration, uninstaller
shell-rc rewrite, dashboard SOUL.md editor) collapse into an opt-in
preserve_mode=True kwarg on utils.atomic_write_text, plus create_mode=
on both atomic_write_text and atomic_yaml_write for first-create paths
(SOUL.md first save, write_manifest's allowlist create path).

Beyond deduplication this closes two gaps the hand-rolled copies had:

- Owner preservation: the old in-place writes kept the inode, so file
  ownership survived root-run rewrites for free. atomic_write_text
  swaps in a new inode owned by the writing user, and the hand-rolled
  blocks restored only the mode -- a root-run 'hermes migrate xai' or
  sudo uninstall on a user-owned Docker/NAS volume would flip
  config.yaml / ~/.zshrc ownership to root. preserve_mode now routes
  through the same _preserve_file_owner/_restore_file_owner helpers
  atomic_yaml_write and atomic_json_write already use.

- chmod-after-replace window: the mode is applied to the temp fd via
  fchmod BEFORE the replace (mirroring atomic_json_write's mode= param),
  so the target never transits through mkstemp's 0600.

Also removes write_manifest's caller-side existed/chmod block (and its
small TOCTOU) in favor of atomic_yaml_write(create_mode=0o644), and
corrects the SOUL.md mode comment (the default profile's runtime seeder
does run it through _secure_file; named profiles do not).

preserve_mode defaults to False so the existing callers (memory store,
skill manager, cron, agent importer) keep their current semantics.

New tests in tests/test_atomic_write_text_metadata.py cover mode
preservation, owner restore through symlinks, fchmod-before-replace,
create_mode on both writers, and no-behavior-change without opt-in;
all mutation-checked.
2026-08-06 05:00:17 +05:30
briandevans 4541d30181 fix(cli): keep newly created SOUL.md and distribution.yaml at 0644
Both files were routed through the shared atomic writers earlier in this
branch. tempfile.mkstemp creates the temp file 0600 and the atomic swap
carries that mode onto the target, so the *create* paths silently tightened
two files that previously landed at the umask default:

- web_routers/profiles.py: the dashboard persona editor's first-ever Save has
  no prior SOUL.md to copy permissions from, so the existing guard skipped the
  chmod entirely -- contradicting the comment directly below it, which states
  profile SOUL.md is created 0644 and is not run through _secure_file.
- profile_distribution.py: atomic_yaml_write only restores a mode it captured
  from a file that already existed. _materialize() calls write_manifest() with
  no manifest on disk whenever a distribution declares an explicit
  distribution_owned allowlist that omits distribution.yaml, so the staged
  copy is never placed in the profile.

Both are fixed with a local chmod at the two sites this branch regressed;
utils.py's public mode semantics are left alone. profiles.py now also
distinguishes "no file yet" (FileNotFoundError -> 0644) from "stat failed for
some other reason" (-> leave the mode alone rather than guess at it).

uninstall.py and xai_retirement.py have no create path and are unchanged: the
former captures prior_mode unconditionally after a successful read_text(), and
the latter runs require_readable_config_before_write() first.
2026-08-06 05:00:17 +05:30
briandevans 67827dd99e fix(cli): route the remaining destructive user-file rewrites through atomic writes
`utils.atomic_write_text`'s docstring states the invariant: it exists "so that
every destructive file rewrite in the codebase shares one implementation."
Four full-file rewrites of *existing user-authored files* still bypass it and
use a bare truncating `open(path, "w")` / `Path.write_text()`, which truncates
the target before the new content is produced. A crash, SIGINT, or ENOSPC
mid-write therefore leaves the file empty or half-written.

In all four cases the read half degrades silently to a default rather than
erroring, so the damage is invisible and the next write cements it:

* `xai_retirement.apply_migration()` rewrites the user's config.yaml. Merged
  commit beaa1a08e added a readability guard here and noted the writer "lives
  outside the atomic_yaml_write path, so the chokepoint didn't cover it"; this
  closes the durability half it left open. `--no-backup` is a documented flag,
  so on that path the truncated file is the only copy that exists, and the
  loader returns early on `doc is None` — the next run reports nothing to
  migrate rather than surfacing the damage.
* `uninstall.remove_path_from_shell_configs()` rewrites the user's shell rc
  (~/.bashrc, ~/.zshrc, ...). Hermes does not own these files and this function
  takes no backup; the enclosing `except Exception` downgrades a partial write
  to a warning, so the next login just starts a bare shell.
* `web_routers.profiles.update_profile_soul()` replaces SOUL.md from the
  dashboard editor. The paired GET reports an unreadable file as
  `{"content": "", "exists": False}`, so an interrupted save presents as "your
  persona was never set" and the editor's next Save persists the empty document.
* `profile_distribution.write_manifest()` rewrites distribution.yaml on every
  install/update. `read_manifest` treats an unparseable manifest as "not a
  distribution", silently dropping update tracking and env_requires.

The xAI migration keeps its ruamel round-trip dumper (comments, key order and
quoting must survive) and now serializes to a string before handing the bytes
to the shared writer. `write_manifest` moves to `atomic_yaml_write`, whose
SafeDumper output the manifest already round-trips through, retiring the local
`_dump_yaml` helper.

`atomic_write_text` recreates the target from a 0600 temp file, so each of its
call sites re-applies the file's previous permission bits: `_secure_file`
deliberately leaves config.yaml alone under managed (NixOS 0640) and container
installs, shell rc files are normally 0644, and profile SOUL.md is created 0644
and never secured. `atomic_yaml_write` already preserves mode and owner itself.
Routing through `atomic_replace` also keeps a symlinked config.yaml or ~/.zshrc
(dotfiles repo, managed deployment) pointing at the real file.

Tests: one regression test per site fails on clean main (the interrupted write
completes there and destroys the file) and passes here; the remaining cases are
behaviour guards covering symlink survival, permission preservation, comment
round-tripping, and the existing happy paths.
2026-08-06 05:00:17 +05:30
Brooklyn Nicholson d1196750c0 feat(profiles): REST export/import + extra_files overlay hook
export_profile() accepts extra_files (root-relative filename -> text) so a
caller can stage companion files into the archive; the desktop uses it for
desktop.json, its appearance/interface overlay, now part of the default
profile's export allow-list.

New routes wrapping the existing hermes profile export/import machinery:
- POST /api/profiles/{name}/export  (extra_files + optional output path)
- POST /api/profiles/import         (returns the bundled desktop overlay)
- GET  /api/profiles/{name}/desktop-overlay

Paths cross the API, not bytes - the desktop's native dialogs and its
local/pooled backends share a filesystem.
2026-08-04 12:07:29 -06:00
kshitij 2000278874 fix(clamps): raise profile fan-out limit to le=500 (simplify-pass finding)
le=100 would 422 real desktop callers: sessions-settings fetches
archived at limit=200, the command palette lists at 200, and the
electron remote-merge over-fetches limit+offset (exceeds 100 at
offset>=81, and its .catch(()=>null) silently drops remote sessions).
Clamp must sit above real client maxima. New test pins limit=200 w/
offset.
2026-08-03 18:46:58 +05:30
aydnOktay 105aba6705 fix(web): clamp dashboard pagination and analytics-days params (#39200 + #74778 salvage)
Re-derivation of aydnOktay's twin clamp PRs onto current main (the
session-list endpoints moved into web_routers/; the analytics endpoints
gained asyncio.to_thread wrappers since the originals):

- limit le=100 on /api/sessions, /api/sessions/search and the
  /api/profiles/sessions fan-out (one unbounded request could drag every
  session row + correlated-subquery preview work out of SQLite, times
  every profile's state.db on the fan-out).
- days ge=1 le=365 on /api/analytics/usage + /api/analytics/models
  (huge or non-positive values force full-history InsightsEngine work or
  inverted windows; the UI only offers 7/30/90 presets).

FastAPI Query bounds reject at the validation layer (422). 8 new tests;
both clamp classes mutation-checked (clamp removed -> its tests fail).
2026-08-03 18:46:58 +05:30
Jakub Wolniewicz ffb54305c4 perf(session-search): project fields before enrichment 2026-08-03 17:50:58 +05:30
szzhoujiarui 16bd5d23b4 fix(tools): reuse subscription features for toolset listing 2026-08-03 16:07:16 +05:30
joelbrilliant 57197cd48d fix(dashboard): preserve maintenance writes on read polling
Signed-off-by: joelbrilliant <joelbrilliant1@gmail.com>
2026-08-02 21:30:54 +05:30
joaomarcos 0d87e5d71b fix(api): enforce pagination bounds on session/message list routes
Ports the negative limit/offset fix onto the current router modules
(hermes_cli/web_routers/sessions.py, profiles.py) since the handlers
moved out of web_server.py in 011ec4513e after this PR was opened.

Per review feedback: only add Query(..., ge=0) — no le=500. The
messages route already clamps oversized requests with min(limit, 500)
and must keep that behavior (succeed + cap) rather than reject them;
the two session-list routes never had a public 500 cap and shouldn't
gain a new rejecting one as a side effect of this fix.
2026-07-31 22:35:49 -07:00
teknium1 011ec4513e refactor(web): extract sessions/mcp/skills/tools routes to APIRouter modules (wave 2; route-table equality verified)
- hermes_cli/web_routers/sessions.py: 14 routes across 3 routers
  (list_router, search_router, manage_router) mounted at the three original
  registration points so global route order is preserved exactly.
- hermes_cli/web_routers/mcp.py: 11 routes; OAuth flow registry
  (_mcp_oauth_flows/lock/cap) stays in web_server, reached via new
  web_deps.LateState live proxies so tests mutating web_server._mcp_oauth_flows
  keep working.
- hermes_cli/web_routers/skills.py: 12 routes across hub_router + router
  (two original registration points straddle the profiles router include).
- hermes_cli/web_routers/tools.py: 12 routes; toolset/terminal catalogs stay
  in web_server (some are defined after the mount point), reached via LateState.
- web_deps.py: add LateState — operation-time proxy for web_server-owned
  module state (getattr/item/iter/len/contains/context-manager/comparisons).
- Handler bodies byte-identical; legacy re-exports keep
  web_server.<handler> importable for tests.
- Verified: ordered route table (method, path) identical to pre-refactor app
  (291 routes); import smoke; ruff; windows-footguns clean.
- test_web_server_sessiondb_eventloop.py: structural AST scan now reads both
  web_server.py and web_routers/sessions.py (handlers moved; helpers stayed).
2026-07-29 16:21:54 -07:00
teknium1 27b1377b4c refactor(web): extract git/profiles/cron routes to APIRouter modules (web_deps seam; route-table equality verified) 2026-07-29 12:23:28 -07:00