feat(model-catalog): picker catalogs refresh every 20 minutes, gateway keeps them warm

The /model picker's remote catalogs (curated manifest, OpenRouter live
filter, Nous Portal recommendations) only refreshed when someone opened
the picker on a stale cache, with a 1h TTL. A delisted model (tencent/hy3:free
after the free promo ended) or a newly published one could sit stale for
an hour after the manifest deploy, and indefinitely in a gateway nobody
opened /model in.

- model_catalog.ttl_minutes: 20 replaces ttl_hours: 1 as the default;
  an explicitly set legacy ttl_hours is still honoured.
- model_catalog.refresh_catalogs() force-refreshes all three sources to
  disk; refresh_interval_seconds() exposes the cadence.
- Gateway spawns a supervised _model_catalog_refresh_watcher that calls
  it off-thread every TTL window, so every surface on the machine reads
  a cache no older than 20 minutes.
- Config migration v39→v40 drops the old ttl_hours: 1 default only.
- Docs: reference/model-catalog.md updated.
This commit is contained in:
Teknium
2026-09-02 06:04:25 -07:00
parent 11f932c935
commit c2954c8934
8 changed files with 147 additions and 13 deletions
+23
View File
@@ -863,6 +863,28 @@ def _migrate_to_39(results: Dict[str, Any], quiet: bool) -> None:
)
def _migrate_to_40(results: Dict[str, Any], quiet: bool) -> None:
# ── Version 39 → 40: model_catalog.ttl_hours → ttl_minutes (default 20) ──
# The picker catalogs now refresh every 20 minutes (and the gateway
# refreshes them in the background on that cadence). Only the OLD default
# (ttl_hours: 1, written by the v25 migration) is dropped so the new
# default applies; any other explicit ttl_hours is a deliberate choice
# and stays honoured by the loader.
_c = _cfg()
read_raw_config = _c.read_raw_config
_persist_migration = _c._persist_migration
config = read_raw_config()
raw_mc = config.get("model_catalog")
if isinstance(raw_mc, dict) and raw_mc.get("ttl_hours") == 1 and "ttl_minutes" not in raw_mc:
del raw_mc["ttl_hours"]
config["model_catalog"] = raw_mc
_persist_migration(config)
results["config_added"].append("model_catalog.ttl_hours 1 → ttl_minutes 20 (default)")
if not quiet:
print(" ✓ Model catalog now refreshes every 20 minutes (model_catalog.ttl_minutes)")
#: Registry of (target_version, migration_fn), strictly ascending. The driver
#: applies every entry whose target version is greater than the on-disk
#: observe earlier steps' writes via read_raw_config() (filesystem state).
@@ -890,6 +912,7 @@ MIGRATIONS: Tuple[Tuple[int, Callable[[Dict[str, Any], bool], None]], ...] = (
(37, _migrate_to_37),
(38, _migrate_to_38),
(39, _migrate_to_39),
(40, _migrate_to_40),
)