refactor(nous): trim comments and drop an unused field

This commit is contained in:
Mariano Nicolini
2026-08-28 15:39:01 -03:00
parent da3c2435e2
commit a51df3864e
-286
View File
@@ -1,286 +0,0 @@
# Honouring the Nous org model policy in the pickers
> **Audience:** Contributors touching Nous model selection
> **Source files:** `hermes_cli/auth.py` (`_login_nous`, `fetch_nous_models`,
> `_prompt_model_selection`), `hermes_cli/models.py` (`fetch_models_with_pricing`,
> `get_pricing_for_provider`, `union_with_portal_*`, `partition_nous_models_by_tier`),
> `hermes_cli/model_setup_flows.py` (`_model_flow_nous`),
> `hermes_cli/model_switch.py` (`list_authenticated_providers`),
> `hermes_cli/web_server.py` (`/api/model/recommended-default`),
> `hermes_cli/nous_account.py` (`_info_from_valid_jwt`)
> **Related:** Inference gateway PR #164 (filters `GET /v1/models` by org policy),
> NAS #941 (team admins restrict providers), NAS `openrouter-provider-map`
> (publishes the model→providers map the gateway filter needs)
## What changed upstream
A Nous team admin can restrict which models and which serving providers their
org may use. The inference gateway applies that policy to `GET /v1/models`, so
an authenticated catalog read returns only what the caller may actually reach.
Blocked models are **omitted** — the row is skipped, no marker field is added
(`api/src/handlers/models.ts:99-138`). An anonymous read is still allowed and
still returns the full catalog (`api/src/app.ts:309-313` — no auth middleware
on the route).
Two things bound how urgent this is.
**The gateway is authoritative and this is cosmetic.** Asking for a hidden
model is refused at request time with `403 model_blocked_by_org_policy`
(`api/src/middleware/model_entitlement_gate.ts:337-356`). The listing fails
open; the request gate fails closed. Nothing here is a security boundary — the
cost of a wrong list is a predictable 403, and the gateway PR states that
tradeoff deliberately. This document is only about the client showing the
right list.
**It is inert today.** PR #164 is merged, but is switched off until NAS
publishes the policy fields and the provider map, and the admin surface sits
behind the `org-model-policy` Vercel flag. The `openrouter-provider-map` branch
is the publisher half (a daily cron writing `openrouter_model_providers` to the
entitlement Redis). Until that lands, every caller — anonymous and
authenticated — gets the same unfiltered list. **No change here is verifiable
end to end yet; every test mocks the filtered response.**
## Where we stand
Four surfaces list Nous models. **None of them is filtered.**
| surface | builds its list from | filtered |
| --- | --- | --- |
| Login (`_login_nous`, `auth.py:9383`) | `get_curated_nous_model_ids()` ∪ Portal recommendations | no |
| `hermes model` (`_model_flow_nous`, `model_setup_flows.py:399`) | same | no |
| `/model` picker (`list_authenticated_providers`, `model_switch.py:3062`) | same | no |
| Dashboard onboarding (`web_server.py:7486`) | same | no |
All four seed from the docs-hosted manifest and union the Portal's
`recommended-models` endpoint. Neither source is authenticated, so org policy
has no effect on any list a user picks from.
`cached_provider_model_ids("nous")` — which *does* reach the authenticated
`fetch_nous_models` — is not consulted by any of them. The `/model` picker
handles nous in its own branch that deliberately bypasses it, and nous cannot
reach the generic pathway at `model_switch.py:2898` because line 2861 skips
every non-`api_key` provider. Its only caller for nous is the background
prefetch (`model_switch.py:2390`), which writes an entry nothing reads.
Two things that are already fine, and should stay that way:
- `nous` is **not** in `_MODELS_DEV_PREFERRED`, so no models.dev entries are
merged on top of the live list.
- The nous fallback ladder in `provider_model_ids` is a *chain* (live →
manifest → in-repo snapshot), not a merge, so a successful live fetch is
used exclusively.
---
## Fix 0 — put auth state in the pricing cache key
**This is a prerequisite for fix 1, and worth landing on its own merits.**
**Problem.** `fetch_models_with_pricing` caches on the base URL alone, and the
cache check happens *above* the point where the `Authorization` header is built
(`models.py:2404`):
```python
cache_key = (base_url or "").rstrip("/")
if not force_refresh:
cached = _cached_catalog(cache_key)
if cached is not None:
return cached
...
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
```
`_pricing_cache` is process-lifetime with no expiry for a non-empty result
(`models.py:2231-2253`). So whichever read of a given base URL lands first —
authenticated or anonymous — answers every later read in that process,
whatever key it passes. An anonymous read landing first (the auxiliary-model
path in fix 2 is one) makes a later authenticated read return an unfiltered
list without touching the network. A fix built on this cache looks like it
works and does not.
**Do.** Fold auth state into the cache key. Distinguishing authenticated from
anonymous is enough — the token value need not be in the key, and keeping it
out avoids hashing a secret.
**Do** update `agent/credits_tracker.py:257`, which reaches into the private
`_pricing_cache` dict assuming one entry per base URL.
**Test.** An anonymous read followed by an authenticated read of the same base
URL issues two requests and returns two different lists. Independently
testable today, unlike everything below.
## Fix 1 — narrow each list to the org's policy
**Problem.** All four surfaces build their list from
`get_curated_nous_model_ids()` unioned with the Portal's `recommended-models`
endpoint. Neither is authenticated, so org policy has no effect on the model a
user picks — which is the model they then use. The Portal endpoint compounds
it: it takes no auth and no parameters, returns one globally CDN-cached payload
for the whole platform, and is invalidated only by admin pricing edits — never
by a policy change. It can put a hidden model straight back into a list. There
is no policy-aware variant of it and no parameter that would make one.
Each surface, however, already fetches `/v1/models`.
`get_pricing_for_provider("nous")` calls `fetch_models_with_pricing`, which
reads that endpoint and returns `{model_id: {...}}`, and already resolves
credentials (`_resolve_nous_pricing_credentials`), so it is already the
authenticated read. Its keys are the reachable set.
**Do.** Use that set to *narrow* each list, keeping the curated order.
`nous_policy_allowed_ids()` obtains the set; `restrict_to_nous_policy()`
applies it. Both live in `models.py`, and the fetch reuses the pricing cache
entry the surface already populates, so no surface makes an extra request.
**Do not** replace a list with the response's keys. Every surface shows the
curated agentic list in curated order deliberately — the live catalog is a
large alphabetical dump of vendor-prefixed models, and swapping it in is the
regression `model_switch.py:3070` records.
**Do not** treat the policy set as subtract-only either. An allowlist can name
only models the curated manifest has never heard of, and intersecting alone
then empties the picker — strictly worse than an unfiltered list, because the
models the org may use are the ones dropped. Fall back to the reachable set
itself in exactly that case.
Only when the intersection is empty, only when the set is small enough to be
an allowlist rather than a whole catalog, and **only for the list a user picks
from**. Callers opt in per list. Any list whose emptiness carries meaning must
not get the rescue: a paid-tier user's unavailable list is legitimately empty,
and rescuing it reads that as "nothing survived" and fills the picker's
unavailable block with the entire reachable set.
A size cap alone does not make this safe — a jurisdiction filter can leave few
enough models to pass it — and neither does gating on an empty intersection,
because an intentionally empty input is indistinguishable from a fully filtered
one. The opt-in is what separates them.
**Do not** narrow a list on evidence that cannot support it.
`nous_policy_allowed_ids()` returns `None` — meaning "leave the list alone" —
in three cases, and each matters:
- **The org has no policy, or the token is too old to say.** Gated on the
`policy_present` claim (fix 4). For an unrestricted org — the common case —
filtering buys nothing and risks dropping a Portal recommendation the
gateway catalog has not caught up on yet. This keeps the change a no-op for
everyone the policy does not apply to.
- **Credential resolution failed**, so the read was anonymous and therefore
unfiltered. A full catalog must not be mistaken for a filtered one. A stated
degradation, not a silent one.
- **The read came back empty**, which is a fetch failure, not an org that may
reach nothing.
A `:free` sibling is kept when its base model is reachable, mirroring the
gateway, which admits a row when any of its requestable ids passes and treats
anything unknown as a keep — "over-listing costs a 403 from the authoritative
gate, while hiding a row the gate would serve is unrecoverable from the client"
(`api/src/libs/catalog_policy.ts:74-78`). Prefer over-listing here too.
**Test.** With a policy hiding model X: X is absent from each of the four
lists, and no surface makes more Nous requests than it does today. With no
policy, with credentials broken, or with an empty read, every list is byte-for-
byte what it is today. A model the Portal flags as free but the org hides stays
out; curated ordering survives filtering.
## Fix 2 — audit the other readers of the pricing map
**Problem.** `fetch_models_with_pricing` is shared, so any caller that treats
its keys as "the models that exist" inherits whatever authentication the first
caller happened to have. Fix 0 stops the *authentication* from leaking between
callers; this fix is about which callers may treat the map as a source of ids
at all.
**Do.** Make the map a lookup *for* ids already in the list, never a source of
ids. Two consumers are already correct and should stay that way:
`partition_nous_models_by_tier` only looks up ids it was given, and the
`union_with_portal_*` pair only ever writes into the map — their id-widening
comes from the Portal endpoint (fix 2), not from the map.
The one that is wrong is `agent/auxiliary_client.py:869-908`
(`_fast_model_from_catalog`), which iterates the map's keys directly as its
candidate list off an anonymous read. Reachable for nous on the titling path,
where it can select a policy-hidden model that then 403s at request time.
**Test.** With credentials broken so the read falls back to anonymous, no
list grows.
## Fix 3 — stop prefetching the nous catalog
**Problem.** The background prefetch calls
`cached_provider_model_ids("nous", force_refresh=True)`
(`model_switch.py:2390`); nous is collected into it because
`_collect_authed_provider_slugs` treats any `auth.json` providers entry as
credentials regardless of `auth_type` (`model_switch.py:2519-2526`). Because
`force_refresh=True` skips the cache read and no nous surface reads the entry,
this is a live authenticated `/v1/models` round trip per picker open written to
a location nothing consults.
**Do.** Exclude nous from the prefetch and delete the write-only entry.
This replaces what an earlier draft proposed here — folding `org_id` into
`_credential_fingerprint` and shortening `_PROVIDER_MODELS_STALE_SERVE_MAX`
for nous (a single global constant, `models.py:4204`, with no per-provider
branching today). Both would have hardened a cache that, after fix 1, has no
nous readers to protect. If a future surface routes nous through
`cached_provider_model_ids` again, revisit the fingerprint then: it hashes
env-var values and `auth.json` mtime and carries no org signal
(`models.py:4277`), so two orgs on one machine can serve each other's list.
**Test.** Opening the `/model` picker makes no Nous `/v1/models` request beyond
the one the displayed list is built from.
## Fix 4 — the `policy_present` claim
**Problem.** Under omission a blocked model simply vanishes, which reads as
"Hermes does not support this" rather than "your org disallows it".
**Do.** Read the `policy_present` claim off the Nous OAuth access token and,
when it is `true`, show a single line stating that the org restricts which
models are available. No enumeration, no per-model marking.
The claim rides the same JWT as `org_id`
(`access-token-issuer.ts:552,595`, `token_use: "access"`) — the token the
client already decodes — and `_info_from_valid_jwt` already retains every
claim in `raw_claims` (`nous_account.py:600-647`), so surfacing it is one
typed field on `NousPortalAccountInfo` and no new request.
It is already widened to cover provider-only restrictions, not just model
allowlists (`nous-account-service/src/server/entitlement-snapshot.ts:478-480`).
Two NAS docs still describe it as allowlist-only and list the widening as
pending — they are stale; trust that expression.
**Do not** enumerate the blocked set. Model policy is allowlist-only —
`denyModels` is a dead column (`nous-account-service/src/server/model-policy.ts:230`)
— so an org that allows five models blocks the entire rest of the catalog.
Graying hundreds of rows is a worse UI than omitting them. An earlier draft
proposed deriving the blocked set by diffing the anonymous and authenticated
reads and feeding it to `_prompt_model_selection`'s `unavailable_models`; that
is the wrong shape twice over, because that picker carries one
`unavailable_message` for the whole list and cannot say "free-tier-gated" and
"policy-hidden" at once.
**Do not** report the absence of the claim as the absence of a policy. It is
tri-state: `true`, `false`, and absent, where absent means unknown — an older
mint, not an unrestricted org. The gateway rejects a corrupt (non-boolean)
claim outright rather than reading it as "no policy"
(`api/src/middleware/nas_jwt_auth.ts:179`). Show the line only on `true`.
**Known bound:** the claim is stamped at mint time, so it goes stale until the
next token refresh — the line can lag a policy change by up to the access
token's lifetime. Acceptable, and worth stating rather than rediscovering.
**Test.** With `policy_present` true the line shows; with it false or absent it
does not.
---
## Order
Fix 0 first: fix 1 is silently wrong without it, and it is the only piece
testable before NAS switches the feature on. Fix 4's claim gates fix 1, so the
two land together. Fix 1 is the correctness work — without it the policy is
bypassed on every surface a user picks from. Fix 2 keeps the pricing map from
becoming another way to widen a list. Fix 3 is a deletion that fix 1 makes
safe.
Run tests with `scripts/run_tests.sh` — not bare `pytest`.