Repointing the two _load_config copies at load_config_readonly() dropped the
except-Exception guards the old copies had. load_config_readonly runs
ensure_hermes_home(), which can raise FileNotFoundError / HomeInitializationError,
so decompose_task (promises ok=False) and /api/local-models/status (garnish
that must render degraded) would raise / 500 instead. The guard is restored
at both sites with the same breadth the old code had.
doctor_live and kanban_decompose carried byte-identical
`try: load_config() or {}` wrappers; local_models wrapped load_config in
_quiet; each is now a direct load_config_readonly() call (read-only callers;
the canonical already fails open and returns a mapping). Tests that patched the
local wrappers patch hermes_cli.config.load_config_readonly instead.
tools/code_execution_tool._load_config read the RAW file, so a managed-pinned
`code_execution.mode` and the DEFAULT_CONFIG keys were invisible at tool
discovery — it now reads load_config_readonly() (behavior change: the managed
overlay applies to execute_code's mode/timeout). onboarding.mark_seen and
credential_lifecycle's config mirror scrub parsed config.yaml with a bare
safe_load; both are read→mutate→write round-trips and use read_user_config_raw,
the documented write-back primitive.
Stop recommending a system-RAM spill when no curated model fits resident.
Preserve explicit model selection and the existing resident quality/speed
ranking, including the separate unified-memory policy.
Require a recommendation for automatic quickstart, expose Browse when
none exists, and rename Configure to Let me choose. Keep policy copy and
reason keys consistent across the four translated local-model sections.
Cover automatic refusal and explicit spilled setup against one budget,
plus the Browse, Download and Use interactions in the desktop pane.
Price weights, context, runtime, projector and batch overhead consistently
across catalog admission, initial launch, growth and restored windows.
Keep MTP and the larger window when lean batches avoid unnecessary spill.
Admit optional external drafts only when their complete footprint fits.
Use preset-only model discovery so refused files cannot autoload, and
preserve refusal/spill decisions atomically for desktop status read-back.
Add regression coverage for complete-footprint boundaries, MTP restarts,
growth admission, draft budgets and placement status transitions.
Builds on the overhead-accounting contribution in #102993 and the
restored-window MTP contribution in #106897. Does not adopt the 40%
host-RAM reserve or resolve the remaining requests in #102865/#106895.
Co-authored-by: infinitycrew39 <infinitycrew39@gmail.com>
Co-authored-by: KoNit-K <124019182+KoNit-K@users.noreply.github.com>
Follow-up to the salvaged fix from #102994. The quickstart leg tests
stubbed web_deps.late itself, so the resolution path that regressed
(late() -> web_server facade -> getattr) was never exercised; the PR's
proposed tests pinned the module string instead, which fails on any
future move of the symbol even when the call site is updated.
Stub only the leaf (web_server_config._apply_model_assignment_sync) so
the real late() runs, and add one invariant test: _assign_default must
call it with ("main", "llamacpp", model_id, "", "", ""). Red on base
(AttributeError), green with the fix; not tied to where the symbol lives.
Drop the docstring parenthetical that described test mechanics.
The whole-codebase refactor moved _apply_model_assignment_sync from the
web_server facade into web_server_config, but _assign_default in
web_routers/local_models.py still resolved it through the default late()
module (web_server), which no longer carries the symbol. A local-model
quickstart therefore failed its final 'making it your default' step with
AttributeError: module 'hermes_cli.web_server' has no attribute
'_apply_model_assignment_sync'.
Pass the defining module explicitly to web_deps.late() so the call
resolves against hermes_cli.web_server_config, matching how the sibling
models.py router imports the same helper. Update the two quickstart test
stubs to accept the module argument the real late(name, module=...) takes.
Run models locally as a first-class provider. The CLI grows a managed
llama.cpp runtime (engine install, model download, server supervision);
the desktop app grows the full setup and management story on top of it.
GUI surfaces ship behind the desktop --local launch flag (hermes desktop
--local, or the flag on the packaged app); backend routes and the CLI
are always live.
Runtime (hermes_cli/local_runtime/):
- curated GGUF catalog with per-machine variant selection: hardware
probe (VRAM/RAM/UMA), fit planning with spill accounting, quant choice
by context window
- derived recommendation: quality-ranked picks gated by a predicted
decode-speed floor, bandwidth-aware on unified memory; the decision
table is pinned as a test (pick AND reason per memory class), and the
Recommended badge explains its pick in a tooltip fed by the resolver's
actual branch
- engine install + model download with resumable split parts, cumulative
plan-level progress, and staged-model integrity (a split GGUF counts
only when every part is present)
- server supervision: spawn/adopt/stop, router mode with per-model load
progress relayed over SSE, abandoned-request cleanup
Desktop:
- Settings -> Providers -> Local models: one-click quickstart (install
engine, download the recommended model, boot) plus per-model download/
activate/eject, fit-ranked catalog with context pills
- model pickers (composer dropdown + Cmd+K) show staged local models,
in-flight downloads as live progress rows, and load-into-memory bars
- local-setup campaign tip for eligible hardware; System resources
statusbar widget (GPU/VRAM/RAM); in-chat load progress during sends
- friendly dead-server errors, and failed agent builds retry on the next
send instead of wedging the session
Co-developed with NVIDIA field feedback on RTX 5090 and DGX Spark.
teknium1