model_switch.py (4405 -> 3712):
- list_authenticated_providers (1367 lines) is a thin driver over _PickerBuild
state and one builder per section (_lap_builtin_rows, _lap_overlay_rows,
_lap_canonical_rows, _lap_user_provider_rows, _lap_bare_custom_row,
_lap_custom_provider_rows). Credential checks shared with
_collect_authed_provider_slugs are single-copy helpers
(_iter_builtin_candidates, _auth_store_has_provider, _raw_pool_usable,
_pool_usable, _overlay_has_env_creds, _has_aws_sdk_creds_for_listing);
live/curated lookup (_live_or_curated_ids, _aws_live_or_curated_ids,
_nous_picker_model_ids), endpoint discovery (_discover_endpoint_models) and
config-entry parsing are deduped across sections 1/2/2b/3/3b/4.
- switch_model: _switch_fail, _runtime_creds, _entry_configured_key,
_ollama_configured_base, _unknown_provider_message, _aggregator_alias_error,
_aggregator_catalog_match, _config_declares_model,
_apply_direct_alias_endpoint replace inline blocks; four copies of the
resolve_runtime_provider unpack collapse to one.
- _model_sort_key: one _flush() for the four repeated version-flush blocks
(identical on 400k random ids).
Row dicts, ordering, error strings and lazy-import patch points unchanged;
picker rows and the interactive hermes model screen byte-identical vs base.
The overlay loop in list_authenticated_providers() checks every way a
provider might be authenticated — env keys, the auth store, the
credential pool, even Claude Code's external token files — but never
asks the one question that matters for an external_process provider:
does the executable resolve? copilot-acp has no key or token by design
(the spawned `copilot --acp --stdio` brings its own auth), so has_creds
stayed False and the filter dropped it from every picker. Funny enough,
five lines further down the same loop has a dedicated copilot-acp
branch for fetching its model ids — it just never got a chance to run.
Availability now comes from get_auth_status(), the same source
`hermes model` and the auth status endpoints already use, so the CLI
and GUI agree on what 'configured' means for external-process
providers.
Fixes#63662
Symptom: picking a model in the Desktop composer for the primary chat
silently rewrote config.yaml (model.default + model.provider) as the
profile default, ignoring model.persist_switch_by_default. A throwaway
pick that resolved to e.g. openai-api (no key) left the profile with an
unusable default on the next launch (#90235).
Root cause: 7d96537bc8 (#86414) made use-model-controls.ts send --global
for every primary-tile pick so a fresh profile would get a persisted
provider instead of falling through to a leftover OPENAI_API_KEY env var.
That put a persistence policy in the client, contradicting the
server-side rule /model uses (resolve_persist_behavior).
Fix:
- resolve_persist_behavior gains one rule, ahead of the --provider
session-only rule: when neither model.default nor model.provider is
configured yet, persist. This preserves #86414's first-pick motivation
for CLI, gateway and Desktop alike. With a default configured, a plain
pick is session-only unless --global / persist_switch_by_default.
- Desktop primary-tile picks send no scope flag and let the gateway decide.
Secondary tiles and MoA presets still send --session.
- /model help text in cli.py said "(persists)"; it now matches reality and
lists --global.
- Docs: desktop.md picker note + slash-commands /model row.
Tests: test_first_pick_persists_then_session_only (fails on main), and the
existing use-model-controls vitest updated to assert the flag-less request.
- alibaba-coding-plan-cn / alibaba-token-plan-cn keep the shared intl key vars
as ordered fallbacks after their dedicated *_CN_API_KEY, so users who set
ALIBABA_CODING_PLAN_API_KEY / ALIBABA_TOKEN_PLAN_API_KEY for the CN endpoint
keep working (the PR as filed dropped them).
- list_authenticated_providers hides a '-cn' row whose only lit key vars are
ones it shares with its non-CN sibling, unless that CN provider is the
configured model.provider. With only the shared key: one row, not two;
DASHSCOPE_API_KEY alone: 3 alibaba rows, not 4.
- Docs: environment-variables.md, providers.md.
Follow-ups on top of #87210 (@liuhao1024) and #87246 (@JoaoMarcos44):
- resolve_startup_model_route: aggregator-native slugs stay on the current
routing aggregator (bare vendor slugs resolve WITHIN the aggregator first);
URL-bearing aliases resolve via direct_alias_runtime_request so a foreign
provider label never carries the vendor token to the alias host (#28660);
route carries the alias's own api_key.
- cli.py: pass current_provider; explicit --api-key wins over alias key.
- Drop #87246's oneshot double-handling (main's oneshot alias+detection path
already covers it once #87210's detection fix is in) and the PR-body SVG.
- Rewrote/extended startup-route tests for the hardened semantics.
Salvaged from PR #84199 by @RickyYii. DirectAlias gains api_key/key_env; the direct-alias override re-resolves credentials against the alias endpoint (host-gated, #28660) and reuses the pre-alias key only on an origin match; oneshot -m <alias> passes the alias key as explicit_api_key; direct-alias branch gains the OLLAMA_API_KEY host gate. Fixes#83612.
Use a distinct runtime_capabilities field on agents, preserve compatibility with earlier snapshots, and resolve the canonical direct OpenAI endpoint when a cross-provider switch omits base_url. Keep ambiguous proxy routes fail-closed.
Stage destination native-compaction capabilities until the complete runtime and context setup succeeds, and restore them with primary and fallback runtimes. Keep native compaction default-deny across live switches and session reconstruction.\n\nVerification: uv run --with pytest --with pyyaml python -m pytest tests/run_agent/test_switch_model_context.py tests/run_agent/test_native_compaction.py tests/run_agent/test_native_compaction_switch_capabilities.py tests/run_agent/test_switch_model_rollback.py tests/run_agent/test_fallback_reasoning_override.py tests/run_agent/test_primary_runtime_restore.py tests/run_agent/test_provider_fallback.py -q -o 'addopts='; uv run --with ruff ruff check <touched files>; git diff --check
Follow-up to the previous commit (which fixed the default/fallback
provider path). A mid-session `/model` switch stores a per-session
override bundle in `_session_model_overrides` that omitted
`request_overrides`, and the two consumers
(`_resolve_session_agent_runtime` fast path and
`_apply_session_model_override`) only copied
provider/api_key/base_url/api_mode. So switching *to* a custom provider
via `/model` did not apply its `extra_body`.
- `ModelSwitchResult` gains a `request_overrides` field, derived for the
switched provider via `_get_named_custom_provider` /
`_custom_provider_request_overrides` (the same overrides
`resolve_runtime_provider` surfaces for the default path).
- Both `/model` override-storage sites in slash_commands.py persist it.
- Both consumers apply it; `_apply_session_model_override` also clears a
stale value when switching to a provider that has none.
Extends tests/gateway/test_turn_request_overrides.py (3 new cases).
Carry provider-derived request_overrides through runtime resolution,
fallback projection, session /model state, restart rehydration, and
turn-route merge so named custom providers keep extra_body and related
overrides.
The fallback also ran on unavailable_models, which is legitimately empty on a
paid tier, filling the picker with the whole reachable set. Make it opt-in.
The `/model` picker warms `provider_models_cache.json` in parallel before
its serial build loop, and Nous was collected into that prefetch because
the credential scan treats any auth.json providers entry as credentials
regardless of auth type.
Nothing reads the result. The picker's nous branch builds from the curated
list rather than `cached_provider_model_ids`, and Nous cannot reach the
api_key-only unified pathway that would call it. Because the prefetch
forces a refresh it also skips the cache read, so the entry is written and
never read — a live authenticated /v1/models round trip per picker open
for nothing.
Exclude it. Also add the plan this and the preceding commits implement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four surfaces list Nous models, and none of them was filtered. All four
seed from the docs-hosted curated manifest and union the Portal's
`recommended-models` endpoint; neither source is authenticated, so org
policy had no effect on the model a user picks — which is the model they
then use. The Portal endpoint compounds it, serving one globally
CDN-cached payload for the whole platform, invalidated only by admin
pricing edits and never by a policy change, so it can put a hidden model
straight back into a list.
Narrow all four against the authenticated catalog:
- `_login_nous`, which chooses the model the session starts on
- `_model_flow_nous`, the `hermes model` picker
- `list_authenticated_providers`, the `/model` picker
- `/api/model/recommended-default`, dashboard onboarding
The list stays curated and curated-ordered — the policy set only ever
subtracts. Replacing a list with the catalog's keys would swap a curated
agentic list for a large alphabetical dump of vendor-prefixed models,
which is the regression the picker's nous branch already exists to avoid.
The `/model` picker's filter sits outside the try that wraps the Portal
union, so a Portal outage still yields a policy-filtered curated list.
`_login_nous` and `_model_flow_nous` also narrow their unavailable lists,
so a policy-hidden model is not offered as a free-tier upsell either.
For an org with no policy — the common case — the filter is a no-op and
every list is what it was.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PyYAML loads unquoted names like provider: 2070 as int. GET /api/model/options
then called .lower() on the providers dict key and 500ed, and activate/delete
looked up "2070" and missed. Stringify identity fields so Desktop can assign
and remove those endpoints.
Co-authored-by: xxxigm <tuancanhnguyen706@gmail.com>
A keyless provider has no credential to lack, but every auth-gated
surface treated 'no key' as 'not authenticated', so opencode-free was
invisible in /model, provider:model listing, and the desktop model
pickers unless a user had unrelated OpenCode env vars set.
One policy, three gates, all derived from the HermesOverlay keyless
flag (#91358):
- auth.py get_api_key_provider_status: keyless providers report
configured/logged_in=True with key_source 'keyless' — flows through
get_auth_status to every status consumer (hermes status, dashboards,
list_available_providers).
- model_switch.py list_authenticated_providers: keyless overlay rows
get has_creds=True before any env/pool/auth-store checks — this is
the source for /model, the TUI picker, and the desktop
/api/model/options payload.
- inventory.py explicit-only filter (desktop chat pickers): keyless
providers are kept — there is nothing to 'explicitly configure', and
hiding a zero-setup provider defeats its purpose.
E2E (temp HERMES_HOME, all keys stripped): get_auth_status logged_in,
list_available_providers authenticated, picker row with 6 models,
desktop payload default AND explicit_only both include the provider,
and the full switch pipeline (parse free:x-preview-f-free →
switch_model) resolves to the keyless runtime. 4 new tests.
Builds on @Lesnak1's #85619 (issue #85589):
- New opencode_provider_family() single-owner predicate in
hermes_cli/models.py — resolves built-in AND custom family providers
(opencode-go-bridge, OpenCode-Zen-Custom, ...) case-insensitively.
Migrated all 8 inlined family checks (models.py x3, runtime_provider.py
x4 from the salvaged commits) plus 4 sibling sites the PR missed:
cli.py api_mode sync, agent_runtime_helpers.py double-/v1 guard,
model_normalize.py flat-namespace strip, model_switch.py base_url
normalization.
- Responses transport: alias OpenCode-reserved function names
(web_search, search_files -> hermes_*) on the wire and map them back on
dispatch — same pattern as the xAI web_search collision fix. Matches
family providers and any base_url on opencode.ai. Fixes the HTTP 400
'custom function name X is reserved' half of #85589.
- Tests: custom-provider routing assertions + 5 new transport alias tests.
PR #67934 marked auto-discovered catalogs by writing two sentinel keys
INSIDE the user-facing ``models`` mapping of custom provider entries:
``__discovered_model_catalog__`` (written by
_save_discovered_models_to_config) and ``__explicit_model_allowlist__``
(injected by _normalize_custom_provider_entry). Every consumer of that
mapping — pickers, selectors, gateway/agent readers, and the user's own
config.yaml — had to know to filter those keys, and any site that
didn't listed them as phantom model IDs (``__discovered_model_catalog__``
showing up as a selectable "model"). The v11→v12 config migration and
the ACP session-state test caught exactly that leak on main.
Replace the in-mapping sentinels with a single entry-level flag:
- ``models_discovered: true`` now sits next to ``models``/``base_url``
on the provider entry; the models mapping stays a clean
``{model_id: metadata}`` dict with no reserved keys.
- _save_discovered_models_to_config writes the new shape and refreshes
catalogs it previously discovered (entry-level flag or legacy
sentinel) instead of treating them as user-curated metadata.
- _normalize_custom_provider_entry no longer injects
``__explicit_model_allowlist__``; a dict-shaped models mapping counts
as an explicit allowlist exactly when the entry is NOT marked
models_discovered.
- _models_config_is_allowlist takes the discovered flag as a parameter
(new helper _entry_models_discovered resolves it, including the
legacy in-mapping sentinel); all call sites updated
(model_switch.py, model_setup_flows.py, acp_adapter/server.py).
- Backward compat, no config version bump: configs written by a
pre-fix Hermes (sentinels inside models) still read correctly —
``__discovered_model_catalog__: true`` is treated as
models_discovered, both sentinel keys are stripped from model
listings, and the next discovery save migrates the entry to the
clean shape. Covered by a new regression test.
Also restore ``except Exception:`` on the pre-existing guards this PR
had narrowed to specific exception tuples (the resolve_runtime_provider
fallback in switch_model, the picker discovery/cache guards in
list_authenticated_providers, _get_model_config_dict, and
_credential_fingerprint). Those guards were intentionally broad on
main — a failed resolution or probe must degrade to the fallback path,
never crash the model switch. Guards the PR introduced for its own new
probe code keep their authored tuples.
The ACP new_session payload also goes back to
probe_current_custom_provider=False, matching the contract main's
test_new_session_returns_authenticated_cross_provider_model_state pins
(session opens must not block on live-probing the current custom
endpoint).
When the 1h provider_models_cache.json TTL lapses, the model picker
serially fetches /v1/models for each authenticated provider. With 10+
providers this stacks to 15-30s of blocking before the picker renders.
Add a parallel prefetch step before the serial picker build loops:
- _collect_authed_provider_slugs(): lightweight credential pre-scan
that mirrors sections 1/2/2b without fetching model lists
- _prefetch_provider_models_parallel(): ThreadPoolExecutor-based
concurrent fetch of stale/missing cache entries (max 8 workers)
- update_provider_cache_entry(): thread-safe single-entry cache writer
with threading.Lock to prevent concurrent write races
Guardrails:
- Skipped when <=3 authed providers (overhead not worth it)
- Skipped when refresh=True (serial path force-refreshes)
- Exception-isolated (falls back to serial path on any failure)
- No behavioral change (same model lists, same picker output)
Closes#80413
(cherry picked from commit 89dddd6cb5d53d73278e0518c375fb5b878e5c6b)
Harden the models.dev catalog refresh path (#35838) with three missing
pieces:
1. ETag conditional GET — every network request sends If-None-Match
with the last-known ETag (persisted alongside the cache file). A 304
Not Modified re-confirms the existing cache without re-downloading
the full ~2 MB registry. This makes the 4-hour TTL effectively free
to maintain.
2. No-network-on-hot-paths invariant — allow_network=False is now the
default for every query function called on the conversation hot path:
get_model_capabilities, get_model_info, lookup_models_dev_context,
_get_provider_models. These are called during vision routing, image
routing, cost-guard checks, and context-length resolution on every
turn — they must never block on the network. Interactive flows
(model picker, model switch) explicitly pass allow_network=True.
3. Mirror URL override — models_dev.url in config.yaml lets deployments
point at a self-hosted mirror without code changes. Follows the same
pattern as model_catalog.url.
Additional hardening:
- Cache TTL bumped from 1h to 4h (ETag makes refresh cheap)
- Corrupt/empty disk cache is rejected with a warning instead of being
served as {} and silently breaking provider/model resolution
- _validate_registry() guards against non-dict and empty-dict payloads
Fixes#35838
Extends the picker fix to the read that actually uses the key: switch_model's
user-provider credential resolution (the ${VAR} api_key expansion and the
key_env fallback at the resolve-credentials step) still read os.environ raw
and passed the result to resolve_runtime_provider as explicit_api_key — so
under multiplex_profiles the actual switch, not just the picker listing,
could adopt another profile's key. Same _scoped_key_env helper, same
fail-closed semantics; identical behavior when multiplexing is off.
Adds end-to-end switch_model tests pinning that an installed scope wins over
the process environment for both read shapes.
854007d1c routed the remaining main-agent fallback key reads through
agent.secret_scope so the multiplexed gateway's per-profile scope applies.
list_authenticated_providers - which gateway/slash_commands.py calls
directly for /model - still resolved custom-endpoint and fallback-entry
credentials with raw os.environ.get(key_env), so under multiplex_profiles
one profile's picker reads whatever key the process environment happens to
hold, i.e. another profile's.
no multiplexing : profileA-key (unchanged)
scope installed : profileB-key (was profileA-key)
Route both reads through a _scoped_key_env() helper over
secret_scope.get_secret(). get_secret is identical to os.getenv when
multiplexing is off, so single-profile deployments are byte-for-byte
unchanged; a fail-closed UnscopedSecretError is treated as "no credential
visible for this profile", which is how the picker already handles a
missing key.
Scope: only the two key_env credential reads. The other environment reads
in that function are provider-presence probes (AWS creds, LM_BASE_URL),
a separate concern.
An alias that family-matches multiple catalog models (/model opus) used to
silently pick one via _model_sort_key heuristics. The heuristics have
guessed wrong repeatedly — dated snapshots like claude-opus-4-20250514
parsed as version 20,250,514 and outranked claude-opus-4-8; suffix
tiebreaks landed on the cheapest tier — and every wrong guess silently
switches the user to a model they did not ask for.
resolve_alias now raises AmbiguousAliasError whenever more than one model
matches the alias family; switch_model catches it at all three call sites
(explicit-provider path, current-provider path, authenticated-provider
fallback) and returns a failure result listing the candidates
(best-guess-first ordering, capped at 10) with instructions to pick an
exact name. A single match still resolves automatically, and DIRECT_ALIASES
exact mappings are unaffected.
The date-stamp split from #67571 is kept, demoted from selection logic to
display ordering of the candidate list.
Supersedes the auto-pick approach of #67571; credit to @Sahaun and @GottZ
for the date-stamp parser analysis that this builds on.
_model_sort_key treated YYYYMMDD snapshot stamps (e.g.
claude-opus-4-20250514) as version components, so 20250514 > 8
and resolve_alias("opus", "anthropic") returned the wrong model.
Fix: split components ≥ 19_000_101 (smallest plausible date stamp)
out of the version tuple, keeping them as a trailing tiebreaker so
bare IDs sort before their dated snapshots and newer snapshots
before older ones. Shorter numeric components (mistral-large-2411,
gpt-4-0613) keep their current behavior. No models.dev dependency
in the sort path.
Teknium