Commit Graph

25 Commits

Author SHA1 Message Date
Teknium 208bd0b65a fix(multiplex): per-profile plugin caches and the Yuanbao active adapter
Ramp Router efforts cache + warm/disk flags, xAI and OpenRouter image catalogs,
Hindsight append-capability verdict, memory-provider skill registry, OpenViking
atexit provider, Honcho loopback flow status, Langfuse client (os.environ-only
credentials) and disk-cleanup's protected cron paths held one profile's
credential- or home-derived value process-wide; YuanbaoAdapter._active_instance
was last-connected-wins across profiles.

Keyed by home key / credential fingerprint under an override, credentials read
through the secret scope, warm threads run under copy_context(); unscoped module
slots stay for the single-profile path and the existing monkeypatch tests.
2026-09-12 01:35:05 -07:00
Teknium d63e380324 compat(plugins): warn once per name when a plugin resolves an old import path; lint step restored in CI
Every PLUGIN-COMPAT __getattr__ now calls hermes_cli.plugin_compat.warn_once(facade, name, target) before
resolving, emitting a HermesPluginCompatWarning (FutureWarning) once per process per name: old path, new
path, removal target. Importing a facade for its live API stays silent; only resolving a moved name warns.
COMPAT_MANIFEST.md documents the warning and how to silence it during migration.

Verified the runtime never routes through a pointer: every entry point (run_agent, cli, hermes_cli.main,
gateway.run, tui_gateway.server, web_server, model_tools + tool discovery, hermes_state, cron.scheduler,
browser_tool, mcp_tool, kanban, auth) imports clean and `hermes doctor` runs end to end with the warning
promoted to an error.

Also restores the check_compat_pointers CI step to .github/workflows/lint.yml, which a0be177aac dropped
when the compat layer was regenerated (the lint script itself was present; the workflow step was not).

hermes_cli/plugin_compat.py, tests/test_plugin_compat_warning.py and the two-line insert per facade are
part of the compat layer and go away with it.
2026-09-04 00:15:16 -07:00
Teknium 2776813df3 compat(plugins): temporary import-path shims for external plugins — ONE commit, revert on schedule
The Sep 2026 decomposition (PR #102117) makes internal import paths a non-API: names now live in
the focused modules that define them. This commit is the ONLY thing keeping the old paths alive,
so external plugins have time to update. It is deliberately a single, unsquashed commit:

    git revert <this sha>

removes every shim, stub and manifest at once on the announced date. Nothing in-tree may depend on
these pointers: scripts/check_compat_pointers.py (wired into lint.yml) fails CI if it does.

What it adds (see COMPAT_MANIFEST.md, compat_manifest.json):
- 332 facade modules get one delimited `PLUGIN-COMPAT` block appended at the end of the file
- 1,172 moved names resolved lazily via a module `__getattr__` (PEP 562) — never a top-level import,
  so no import cycles; facades that already had `__getattr__` get a chained one
- 592 third-party/stdlib names the old modules used to expose, with their original import statements
- 266 public definitions that had been deleted as unused, restored byte-for-byte from the pre-decomposition
  tree (+40 private helpers and 16 imports pulled in only because a restored definition needs them)
- 3 deleted modules recreated as re-export stubs (gateway/startup_watchdog, hermes_cli/observability/
  relay_runtime, tools/environments/modal_utils)
- private names (`_x`) get no pointer: they were never API (3,792 skipped)

Verified: all 335 touched modules import under a fresh HERMES_HOME and every manifest name resolves;
the lint reports zero in-tree uses; ruff clean; targeted suites unchanged.
2026-09-03 17:13:22 -07:00
Teknium 2c4db54a31 refactor(image_gen,browser): AST-neutral closer hugging after edits 2026-09-03 00:08:53 -07:00
Teknium d1a0cdae2c refactor(image_gen,browser): compact module/function docstrings and comments across providers (WHYs kept) 2026-09-03 00:06:34 -07:00
Teknium c2af7b2a85 refactor(image_gen/xai): extract _check_source_images phase, drop config/time wrappers, compact tables 2026-09-02 23:41:58 -07:00
Teknium 9adcfb4abd refactor(image_gen,browser): AST-neutral layout compaction — hug call/import closers 2026-09-02 23:33:58 -07:00
Teknium 95ed2667f7 refactor(image_gen,browser): AST-neutral layout compaction (imports, signatures, call args) 2026-09-02 22:18:05 -07:00
Teknium f21cb136aa refactor(image_gen): StaticImageGenProvider base — identity/catalog/setup boilerplate unified across 7 providers 2026-09-02 21:57:51 -07:00
Teknium 354b580fda refactor(image_gen,browser): krea submit via shared post_json, poll-error table, catalog fetch helper; compact providers 2026-09-02 21:52:06 -07:00
Teknium bb25c36306 refactor(image_gen): compact krea/openai-codex/xai providers; extract krea payload builder 2026-09-02 21:17:25 -07:00
Teknium b5b62aa83d refactor(plugins/image_gen): shared _common (config/key/save/error/picker helpers), convert all 8 providers, per-model tables 2026-09-02 13:30:10 -07:00
Teknium 9c6ecf2ca7 feat(image-gen): OpenRouter image picker lists every live image-output model; xAI edits honor dispatched model
- plugins/image_gen/openrouter: list_models() now queries the endpoint's
  /models catalog filtered to output_modalities containing "image"
  (per-backend 5-min cache, 10s timeout, static 2-model chain as offline
  fallback; openrouter/auto* router pseudo-models excluded). Every image
  model OpenRouter serves — including future releases — is selectable in
  `hermes tools` with no code change. Applies to Nous Portal too via the
  shared provider class.
- plugins/image_gen/xai: forward the dispatched model kwarg into
  _resolve_edit_model() so an explicitly selected edit-capable model is
  honored on /images/edits (extends the salvaged #55893 fix to the edit
  path; text-only models still fall back to quality).
- Tests: OpenRouter live-catalog filtering/exclusions/order, offline
  fallback, cache single-fetch; xAI edit-kwarg forwarding incl. the
  text-only-hijack negative case.

Live-verified against openrouter.ai: 9 image-output models returned and
rendered, matching the public models?output_modalities=image listing.
2026-08-19 02:04:29 -07:00
srojk34 008d469991 fix(xai): forward image_gen.model kwarg to _resolve_model in generate() 2026-08-19 02:04:29 -07:00
Teknium ac0a8cd281 feat(image-gen): xAI Grok image catalog goes live-driven; grok-imagine-image-2.0 selectable
- plugins/image_gen/xai: merge the live /v1/image-generation-models catalog
  (5-min cache, 10s timeout, static-table fallback when offline/unauth)
  into the picker so new xAI Imagine models appear automatically the day
  they launch, with generic metadata until curated text is added.
- Add grok-imagine-image-2.0 to the curated static table (typography/
  layout-aware model, API-available since Aug 8 2026).
- Edits honor an explicitly selected image-input-capable model
  (e.g. grok-imagine-image-2.0) instead of always forcing
  grok-imagine-image-quality; quality remains the default edit baseline.
- Tests: hermetic autouse fixture keeps unit runs offline; new coverage
  for live-merge, unknown-future-model selection, offline fallback, and
  edit-model resolution. Docs model table updated (en + zh-Hans).

Live-verified: /image-generation-models returns grok-imagine-image,
grok-imagine-image-2.0, grok-imagine-image-quality; real generation with
2.0 succeeded end to end.
2026-08-19 01:19:37 -07:00
kshitijk4poor c1826e2690 fix(image-gen): route local-input credential guard through one shared chokepoint + cover xai (#57698)
Follow-up to the per-provider guards. Three improvements from review:

1. Extract agent.file_safety.raise_if_read_blocked() as a single shared
   chokepoint and route the OpenAI, OpenRouter, and (newly) xAI image
   providers through it, replacing the 3x-duplicated inline try/except.
   Fixes the whole bug class: xai/_xai_image_field read a model-supplied
   local path via open() with no guard — the same vulnerability the PR
   fixed for OpenAI/OpenRouter, in a sibling provider it missed.
2. Strengthen the regression tests from pass-on-any-ValueError to true
   security invariants: spy open()/read_bytes() and assert the blocked
   credential is NEVER read; add negative controls (legit local image
   still loads; remote/data: URIs pass through unguarded) so a
   block-everything regression can't pass.
3. Guard is best-effort by design (defense-in-depth, not a security
   boundary) — documented on the shared helper.

- agent/file_safety.py: raise_if_read_blocked()
- plugins/image_gen/{openai,openrouter,xai}: route through helper
- tests: no-read spies + negative controls across all three providers
2026-07-03 18:47:53 +05:30
teknium1 5a3d7fb99d fix(xai): suppress false-positive windows-footgun on binary image read
open(..., "rb") is binary mode and needs no encoding=; the checker's
regex doesn't recognize the mode. Add the documented suppression comment.
2026-06-29 21:11:58 -07:00
Jaaneek 9ce79cd642 feat(xai): Imagine public-URL storage, chaining & video edit/extend
Add durable public-URL output and URL-based chaining to xAI Grok Imagine:

- Store generated media on files-cdn with permanent public HTTPS URLs
  (public_url: true, no expiry by default).
- Chain by URL: generate -> edit -> extend each take a prior result's
  public HTTPS URL (or a data URI / local file for inputs).
- Add provider-specific xai_video_edit and xai_video_extend tools.
- Image generation: public-URL/storage output, multi-reference edits,
  and ~/ local-path support for image edits.

Credentials use xAI Grok device-code OAuth (separate PR).
2026-06-29 21:11:58 -07:00
Teknium c02192ff6a feat(image-gen): add image-to-image / editing to image_generate (#48705)
* feat(image-gen): add image-to-image / editing to image_generate

Brings image generation to parity with video generation: the unified
image_generate tool now edits/transforms a source image (image-to-image)
when given image_url / reference_image_urls, routing to each backend's
edit endpoint, exactly as video_generate routes to image-to-video.

- ImageGenProvider ABC: generate() gains keyword-only image_url +
  reference_image_urls; new capabilities() declares modalities +
  max_reference_images (defaults to text-only, backward compatible).
  success_response gains a modality field; adds normalize_reference_images.
- image_generate tool: schema exposes image_url + reference_image_urls;
  dynamic schema reflects the active model's actual edit capability so the
  agent knows when image_url is honored. Handler + plugin dispatch forward
  the new inputs; legacy/text-only providers get a clear modality_unsupported
  error instead of silently dropping the source image.
- In-tree FAL: 7 models gain edit endpoints (flux-2-klein, flux-2-pro,
  nano-banana-pro, gpt-image-1.5, gpt-image-2, ideogram/v3, qwen-image)
  with per-model edit_supports whitelists + reference caps; routes to the
  /edit endpoint and skips the upscaler for edits.
- Plugins: openai (images.edit, 16 refs), xai (/v1/images/edits via
  grok-imagine-image-quality, JSON body per xAI docs), krea
  (image_style_references, 10 refs). openai-codex stays text-only and
  rejects edits with an actionable error.
- Tests: 15 new (payload, routing, dispatch forwarding, dynamic schema,
  capabilities); updated 2 change-detector/lambda tests for the new schema.
- Docs: image-generation feature page, image-gen provider plugin guide,
  tools reference.

* fix(image-gen): preserve legacy passthrough in fal/krea plugin tests

Two existing plugin tests asserted pre-image-to-image behavior:
- fal: forward image_url/reference_image_urls only when supplied, so a
  text-to-image delegation stays byte-identical (no None kwargs).
- krea: keep dict-shaped image_style_references refs verbatim (the unified
  string refs go through normalize_reference_images; legacy non-string ref
  objects pass through unchanged) — fixes KeyError when callers pass the
  richer Krea ref-object shape.

* fix(image-gen): clearer not-capable message for text-to-image-only models

When a text-to-image-only model (incl. gpt-image-2 on the Codex OAuth path,
which can't do editing through the Responses image_generation tool) gets a
source image, say 'this model is not capable of image-to-image / editing —
provide a text-only prompt' rather than sending the user shopping for other
backends. Applies to the openai-codex guard, the in-tree FAL no-edit-endpoint
error, and the dynamic tool-schema text-only line.
2026-06-18 22:13:07 -07:00
Teknium 031f9c9edc fix(image_gen): cache xAI ephemeral URL responses to disk (#26942) (#31759)
xAI's grok-imagine-image API returns ephemeral imgen.x.ai/xai-tmp-* URLs
that 404 within minutes — long before downstream consumers (Telegram
send_photo, browser preview, multi-tier delivery fallback) get a chance
to fetch them.  The xAI image_gen provider was passing those URLs
through unchanged on the elif url: branch; b64 responses were already
cached locally via save_b64_image.  Result: every image_generate call
on a Telegram-routed xai-oauth profile delivered no image, falling
through to text-only.

Adds agent.image_gen_provider.save_url_image() — a sibling helper to
save_b64_image that downloads URL bytes to $HERMES_HOME/cache/images/.
Content-type-aware extension inference with URL-suffix fallback;
oversize cap (25MB default) with partial-write cleanup; empty-body
refusal.  Mirrors the audio_cache pattern used by text_to_speech.

Wires save_url_image into both the xAI and OpenAI providers' URL
branches.  When the download fails (network blip, 404 in-flight) we
log a warning and fall back to the bare URL rather than turning the
tool call into a hard error — the gateway's existing URL-send fallback
then gets a chance to surface the original error legibly.

Test plan:
- tests/agent/test_save_url_image.py — 8 direct tests against a real
  in-process HTTP server: bytes round-trip, content-type → extension,
  URL-suffix fallback, default-to-png, 404 propagation, empty-body
  refusal, oversize cap + cleanup, filename uniqueness.
- tests/plugins/image_gen/test_xai_provider.py — flip
  test_successful_url_response (was asserting the bug), add
  test_url_response_falls_back_to_bare_url_when_download_fails.
- tests/plugins/image_gen/test_openai_provider.py — symmetric pair.

160/160 in the broader image_gen test surface.
2026-05-24 18:10:47 -07:00
Jaaneek b62c997973 feat(xai-oauth): add xAI Grok OAuth (SuperGrok Subscription) provider
Adds a new authentication provider that lets SuperGrok subscribers sign
in to Hermes with their xAI account via the standard OAuth 2.0 PKCE
loopback flow, instead of pasting a raw API key from console.x.ai.

Highlights
----------
* OAuth 2.0 PKCE loopback login against accounts.x.ai with discovery,
  state/nonce, and a strict CORS-origin allowlist on the callback.
* Authorize URL carries `plan=generic` (required for non-allowlisted
  loopback clients) and `referrer=hermes-agent` for best-effort
  attribution in xAI's OAuth server logs.
* Token storage in `auth.json` with file-locked atomic writes; JWT
  `exp`-based expiry detection with skew; refresh-token rotation
  synced both ways between the singleton store and the credential
  pool so multi-process / multi-profile setups don't tear each other's
  refresh tokens.
* Reactive 401 retry: on a 401 from the xAI Responses API, the agent
  refreshes the token, swaps it back into `self.api_key`, and retries
  the call once. Guarded against silent account swaps when the active
  key was sourced from a different (manual) pool entry.
* Auxiliary tasks (curator, vision, embeddings, etc.) route through a
  dedicated xAI Responses-mode auxiliary client instead of falling back
  to OpenRouter billing.
* Direct HTTP tools (`tools/xai_http.py`, transcription, TTS, image-gen
  plugin) resolve credentials through a unified runtime → singleton →
  env-var fallback chain so xai-oauth users get them for free.
* `hermes auth add xai-oauth` and `hermes auth remove xai-oauth N` are
  wired through the standard auth-commands surface; remove cleans up
  the singleton loopback_pkce entry so it doesn't silently reinstate.
* `hermes model` provider picker shows
  "xAI Grok OAuth (SuperGrok Subscription)" and the model-flow falls
  back to pool credentials when the singleton is missing.

Hardening
---------
* Discovery and refresh responses validate the returned
  `token_endpoint` host against the same `*.x.ai` allowlist as the
  authorization endpoint, blocking MITM persistence of a hostile
  endpoint.
* Discovery / refresh / token-exchange `response.json()` calls are
  wrapped to raise typed `AuthError` on malformed bodies (captive
  portals, proxy error pages) instead of leaking JSONDecodeError
  tracebacks.
* `prompt_cache_key` is routed through `extra_body` on the codex
  transport (sending it as a top-level kwarg trips xAI's SDK with a
  TypeError).
* Credential-pool sync-back preserves `active_provider` so refreshing
  an OAuth entry doesn't silently flip the active provider out from
  under the running agent.

Testing
-------
* New `tests/hermes_cli/test_auth_xai_oauth_provider.py` (~63 tests)
  covers JWT expiry, OAuth URL params (plan + referrer), CORS origins,
  redirect URI validation, singleton↔pool sync, concurrency races,
  refresh error paths, runtime resolution, and malformed-JSON guards.
* Extended `test_credential_pool.py`, `test_codex_transport.py`, and
  `test_run_agent_codex_responses.py` cover the pool sync-back,
  `extra_body` routing, and 401 reactive refresh paths.
* 165 tests passing on this branch via `scripts/run_tests.sh`.
2026-05-15 12:11:32 -07:00
Ayman Kamal 13b474c56e fix: send correct resolution param to xAI image generation API
The xAI /v1/images/generations endpoint expects resolution as a
literal string ('1k' or '2k'), not the numeric value ('1024').

- Change _XAI_RESOLUTIONS from a dict mapping to a validation set
- Use the resolution key directly instead of the mapped value
- Fall back to DEFAULT_RESOLUTION on invalid config values

Fixes 422 Unprocessable Entity errors when resolution was sent.
2026-05-09 13:07:46 -07:00
Cameron Aragon 239ea1bdea fix(image-gen): preserve xAI API error status 2026-05-04 04:43:07 -07:00
teknium1 9599271180 fix(xai-image): drop unreachable editing code path
The agent-facing image_generate tool only passes prompt + aspect_ratio to
provider.generate() (see tools/image_generation_tool.py:953). The editing
block (reference_images / edit_image kwargs) could never fire from the
tool surface, and the xAI edits endpoint is /images/edits with a
different payload shape anyway — not /images/generations as submitted.

- Remove reference_images / edit_image kwargs handling from generate()
- Remove matching test_with_reference_images case
- Update docstring + plugin.yaml description to text-to-image only
- Surface resolution in the success extras

Follow-up to PR #14547. Tests: 18/18 pass.
2026-04-23 15:13:34 -07:00
Julien Talbot a5e4a86ebe feat(xai): add xAI image generation provider (grok-imagine-image)
Add xAI as a plugin-based image generation backend using grok-imagine-image.
Follows the existing ImageGenProvider ABC pattern used by OpenAI and FAL.

Changes:
- plugins/image_gen/xai/__init__.py: xAI provider implementation
  - Uses xAI /images/generations endpoint
  - Supports text-to-image and image editing with reference images
  - Multiple aspect ratios (1:1, 16:9, 9:16, 4:3, 3:4, 3:2, 2:3)
  - Multiple resolutions (1K, 2K)
  - Base64 output saved to cache
  - Config via config.yaml image_gen.xai section
- plugins/image_gen/xai/plugin.yaml: plugin metadata
- tests/plugins/image_gen/test_xai_provider.py: 19 unit tests
  - Provider class (name, display_name, is_available, list_models, setup_schema)
  - Config (default model, resolution, custom model)
  - Generate (missing key, success b64/url, API error, timeout, empty response, reference images, auth header)
  - Registration

Requires XAI_API_KEY in ~/.hermes/.env.
To use: set image_gen.provider: xai in config.yaml.
2026-04-23 15:13:34 -07:00