hbentel 2673d5f5bd fix(gemini): embed images in Gemini 3.x functionResponse.parts for multimodal tool results ...

_translate_tool_result_to_gemini called _coerce_content_to_text unconditionally,
silently dropping image_url parts from multimodal tool results (e.g. vision_analyze
responses). Gemini 3.x supports a functionResponse.parts field for embedding
inlineData images directly inside the function response; Gemini 2.x does not.

Thread is_gemini3 through _build_gemini_contents → _translate_tool_result_to_gemini
and gate image embedding on _gemini_major_version >= 3. Reuses the existing
_extract_multimodal_parts helper (no duplicate code). Non-3.x path unchanged.

Original PR #32352 by @hbentel, salvaged onto current main.

Co-authored-by: kshitijk4poor <82637225+kshitijk4poor@users.noreply.github.com>

2026-08-27 21:07:21 +05:30

..

lsp

fix(lsp): abort diagnostics waits after transport death

2026-08-25 01:48:43 +05:30

monitoring

feat(cron): acked failure signatures stop re-pinging — durable incidents + ack CLI (salvage #94692) (#95017)

2026-08-25 14:02:40 -07:00

pet

fix(pets): remove the non-PNG base draft after hardening

2026-08-15 00:33:01 -07:00

proxy_sources

fix(windows): sweep remaining unguarded text-mode subprocess sites codebase-wide

2026-07-24 11:45:57 -07:00

secret_sources

fix(update): avoid cryptography self-lock on Windows

2026-08-15 13:28:14 -07:00

transports

refactor(codex): route transports _pair_ids through shared fc_ canonicalization

2026-08-26 12:58:35 +05:30

verify

Integrate verify subsystem with the existing verification stack

2026-08-07 10:11:05 -07:00

__init__.py

…

account_usage.py

fix(auth): detect upstream Codex quota resets and lift stale pool cooldowns (#69494)

2026-07-22 10:58:22 -07:00

acp_openai_bridge.py

refactor(acp): share one OpenAI bridge between the ACP clients

2026-08-26 10:10:11 -07:00

agent_init.py

fix(acp): key the ACP runtime exclusions on the scheme, not on one vendor

2026-08-26 10:10:11 -07:00

agent_runtime_helpers.py

fix(fallback): surface provider transitions and primary recovery

2026-08-25 12:12:08 +05:30

anthropic_adapter.py

fix(anthropic): send an explicit thinking disable on the native Messages wire

2026-08-20 12:29:11 -05:00

async_utils.py

refactor(gateway): dedupe detached-task consumer + reconnect backoff policy

2026-07-18 20:01:55 +05:30

aux_accounting.py

feat(analytics): record auxiliary model usage per task in session accounting (#65537)

2026-07-16 04:23:12 -07:00

auxiliary_client.py

fix(agent): honor explicit free OpenRouter models

2026-08-27 04:37:36 -07:00

azure_identity_adapter.py

fix(secrets): scope-aware Azure credential presence reads in identity diagnostics

2026-08-02 10:02:33 -07:00

backend_identity.py

refactor(fallback): single owner for backend identity and failure-scoped skips

2026-07-26 23:41:54 -07:00

background_review.py

fix(background-review): skip the fork when the provider can't emit tool calls

2026-08-26 10:10:11 -07:00

battery.py

feat(status-bar): add /battery toggle for a color-coded battery read-out

2026-07-21 13:54:11 -05:00

bedrock_adapter.py

refactor(bedrock): make resolve_bedrock_runtime_region the single region chokepoint

2026-08-21 15:02:29 -07:00

billing_links.py

feat(billing): shared cross-surface out-of-credits signal

2026-07-22 18:08:59 -05:00

billing_usage.py

chore: remove unused imports and dead locals (ruff F401/F841 sweep)

2026-07-29 11:53:39 -07:00

billing_view.py

chore: remove unused imports and dead locals (ruff F401/F841 sweep)

2026-07-29 11:53:39 -07:00

bounded_response.py

fix(agents): bound streaming error-response body reads

2026-07-05 14:00:20 -07:00

browser_provider.py

feat(browser): integrate Browser Use CLI 3.0

2026-08-10 10:45:44 -07:00

browser_registry.py

fix(plugins): isolate ownership by profile

2026-08-12 19:13:32 -07:00

chat_completion_helpers.py

fix(fallback): surface provider transitions and primary recovery

2026-08-25 12:12:08 +05:30

codex_responses_adapter.py

refactor(codex): extract shared fc_->call_ canonicalization helper

2026-08-26 12:58:35 +05:30

codex_runtime.py

fix(memory): bind the checkpoint gate to every compaction authority

2026-08-25 03:55:55 -07:00

coding_context.py

fix: system prompt no longer references tools/skills the session can't use; hermes-agent skill is always kept

2026-08-24 20:25:10 -07:00

command_token_source.py

fix(auth): bound the no-TTL key_cmd token cache; docs for key_cmd

2026-08-15 03:16:21 -07:00

compaction_display.py

fix(clients): hide compaction carriers across surfaces

2026-08-21 21:53:11 +05:30

context_breakdown.py

feat(cli,gateway): unify /context into a visual context-usage breakdown

2026-07-26 18:06:21 -07:00

context_compressor.py

fix(compaction): exclude operational notifications from tail anchor and auto-focus (#92703)

2026-08-26 21:38:20 -07:00

context_engine.py

Revert "fix: rewire DCP context engine to current main architecture"

2026-08-08 23:09:45 +05:30

context_references.py

feat(plugins): allow plugins to register custom @-prefix context references

2026-08-12 18:41:59 -07:00

conversation_compression.py

refactor(compression): fold review follow-ups on #71488 salvage

2026-08-27 12:42:38 +05:30

conversation_loop.py

fix(acp): key the ACP runtime exclusions on the scheme, not on one vendor

2026-08-26 10:10:11 -07:00

copilot_acp_client.py

refactor(acp): share one OpenAI bridge between the ACP clients

2026-08-26 10:10:11 -07:00

credential_persistence.py

feat(auth): make xAI Grok OAuth device-code-only, drop loopback login

2026-07-02 13:17:41 -07:00

credential_pool.py

fix(auth): rotate credentials for named custom providers after 401/429

2026-08-23 20:01:18 -07:00

credential_sources.py

fix(auth): read .env as utf-8-sig in the dotenv-vs-shell detector

2026-08-08 12:32:23 -07:00

credits_tracker.py

refactor(credits): fold review findings for stealth free-tier fix

2026-08-22 04:20:48 +05:30

curator_backup.py

fix(curator): abort rollback when safety snapshot fails

2026-08-16 22:06:41 -07:00

curator.py

fix(curator): tell the background reviewer to read before it writes

2026-08-25 00:31:14 -07:00

deadline.py

fix(deadline): remove the dead second SuspectableBackend class shadowing the Phase 3a Protocol

2026-08-27 17:22:02 +05:30

delegation_context.py

fix(cron): don't let a cron job inherit a kanban worker's dispatcher identity

2026-08-06 03:16:26 +05:30

display.py

feat(delegation): live orchestration of running subagents via delegate_task action param

2026-08-13 09:34:36 -07:00

empty_response_guard.py

refactor(agent): move empty-response guard settings from env vars to config.yaml

2026-08-16 22:06:08 -07:00

error_classifier.py

fix(classifier): 429 quota walls route to billing across providers; reset signals stay rate-limited

2026-08-23 20:02:07 -07:00

error_surface.py

fix(desktop): error card honors the classifier's retry verdict + failing-session identity (review feedback)

2026-08-21 15:24:03 -07:00

errors.py

fix(moa): surface stale presets without retries

2026-07-17 13:49:12 -07:00

estop.py

fix: post-merge audit follow-ups for #81138/#81139/#81141/#81148

2026-08-08 05:21:09 -07:00

file_safety.py

fix(browser): real-profile snapshot is a first-class secret store + preserve channel identity

2026-08-26 19:25:33 -07:00

gemini_native_adapter.py

fix(gemini): embed images in Gemini 3.x functionResponse.parts for multimodal tool results

2026-08-27 21:07:21 +05:30

gemini_schema.py

fix(gemini): preserve typed enum constraints as strings

2026-07-26 20:59:16 -07:00

i18n.py

perf: use load_config_readonly() at read-only call sites in agent/

2026-07-29 15:28:15 -07:00

image_gen_provider.py

feat(media): opt-in upscale pass for image_generate and video_generate across FAL and Krea

2026-08-08 14:49:28 -07:00

image_gen_registry.py

fix(tools): dispatch image/video FAL strictly on the stored hermes tools selection

2026-08-19 16:10:01 -07:00

image_routing.py

fix(vision): forward the API key to the server-type probe and cache failed verdicts

2026-08-23 17:47:50 -07:00

insights.py

refactor(usage): simplify-pass follow-ups

2026-08-14 02:04:24 +05:30

interrupt_compat.py

fix(agent): separate cancellation diagnostics from tool output

2026-08-23 18:25:19 -07:00

iteration_budget.py

feat: raise default tool-calling iteration limit from 90 to 500

2026-07-26 12:54:45 -07:00

jiter_preload.py

…

kanban_stop.py

follow-up: integrate agent nudge + dispatcher retry docs and tests

2026-07-14 16:47:33 +05:30

learn_prompt.py

fix(learn): process large sources incrementally

2026-08-08 05:09:43 -07:00

learning_graph_render.py

chore: remove unused imports and dead locals (ruff F401/F841 sweep)

2026-07-29 11:53:39 -07:00

learning_graph.py

fix(learning_graph): guard non-dict metadata so /journey can't crash

2026-07-01 16:25:48 -05:00

learning_mutations.py

refactor(journey): route memory mutations through MemoryStore atomic I/O

2026-06-30 15:16:21 -05:00

lmstudio_reasoning.py

fix(lmstudio): clamp max/ultra reasoning effort to LM Studio's ceiling

2026-07-16 07:57:51 -07:00

manual_compression_feedback.py

fix(compression): /compress refusal no longer reports a successful rewrite

2026-08-19 22:41:05 -07:00

markdown_tables.py

…

memory_manager.py

refactor(memory): renumber checkpoint API — v1 is the implicit historical contract, v2 opts into fail-closed checkpoints

2026-08-25 03:55:55 -07:00

memory_provider.py

refactor(memory): renumber checkpoint API — v1 is the implicit historical contract, v2 opts into fail-closed checkpoints

2026-08-25 03:55:55 -07:00

message_content.py

fix(openviking): preserve structured sync attribution

2026-06-19 15:23:41 +08:00

message_metadata.py

fix: create message_metadata module and update chat_completion_helpers

2026-08-15 01:04:19 -07:00

message_sanitization.py

fix(agent): preserve tool results across ID variants

2026-08-23 18:24:43 -07:00

moa_loop.py

fix: restore stale-base revert hunks in conversation_loop.py and moa_loop.py

2026-08-13 23:10:16 +05:30

moa_trace.py

feat(langfuse): widen tracing to errors, sessions, subagents, and MoA fan-out

2026-08-13 23:10:16 +05:30

model_metadata.py

fix: follow-up for salvaged PRs #93250 + #96234

2026-08-27 03:24:20 -07:00

models_dev.py

refactor(models): simplify-pass follow-ups on the refresh path

2026-08-14 03:31:22 +05:30

moonshot_schema.py

fix(agent): inject empty required array on Moonshot object schemas

2026-07-20 11:05:52 -07:00

native_compaction.py

fix(memory): bind the checkpoint gate to every compaction authority

2026-08-25 03:55:55 -07:00

nous_rate_guard.py

fix: decode config and state files as UTF-8 on non-UTF-8 locales

2026-07-24 17:10:39 -07:00

onboarding.py

feat(surfaces): route busy-input corrections through active-turn redirect

2026-07-22 12:10:58 -05:00

oneshot.py

feat(agent): one-shot LLM helper + llm.oneshot gateway RPC (#51261)

2026-06-23 08:01:50 +00:00

outbound_webhooks.py

fix(hooks): flush outbound queue at interpreter exit

2026-08-02 15:01:11 -07:00

plugin_llm.py

fix(plugins): complete task-routed LLM integration

2026-08-12 16:25:20 -07:00

plugin_stream_hooks.py

feat(plugins): add streaming output observer hooks

2026-08-12 18:38:25 -07:00

portal_tags.py

feat(portal): ambient conversation context entangles aux/MoA/delegate calls

2026-07-16 01:13:43 -07:00

process_bootstrap.py

docs: note httpcore pin dependency in _enable_happy_eyeballs

2026-08-24 21:46:05 -07:00

prompt_builder.py

refactor(computer_use): diet schema + delete prompt block (~1.4K tok/call); remove max_elements, ladder moves to response verdicts

2026-08-26 04:50:13 -07:00

prompt_cache_boundary.py

refactor(cache): share the boundary-declaration helper and simplify the registry

2026-08-09 15:25:12 +05:30

prompt_cache_scope.py

refactor(cache): never-raising scope resolver shared by both call sites

2026-08-15 11:09:56 +05:30

prompt_caching.py

refactor(prompt_caching): shallow strip copy, exact-count guards, dedupe idempotency tests

2026-08-21 12:23:47 +05:30

provider_projection.py

feat(agent): fold an agent-as-provider's own tool work back into the turn

2026-08-26 10:10:11 -07:00

rate_limit_tracker.py

…

reactions.py

feat(agent): core affection reaction detector + reaction_callback

2026-07-10 05:41:59 -05:00

reasoning_effort.py

fix(zai): GLM-5.3 low/medium reasoning effort reaches the wire instead of clamping to high

2026-08-21 14:58:37 -07:00

reasoning_summaries.py

fix(reasoning): keep gpt-5.x summary parts as separate blocks on the chat wire

2026-08-06 22:02:37 -05:00

reasoning_timeouts.py

fix: follow-up for salvaged PRs #93250 + #96234

2026-08-27 03:24:20 -07:00

redact.py

fix: reuse redact_sensitive_text, fix leaky abstraction, fix test data

2026-08-13 23:10:16 +05:30

relay_llm.py

feat(relay): activate configured dynamic plugins

2026-08-19 08:52:01 -07:00

relay_runtime.py

fix(relay): complete native plugin cutover

2026-08-19 08:52:03 -07:00

relay_tools.py

fix(relay): bypass managed execution for nested calls inside managed callbacks

2026-08-08 23:46:17 -07:00

repetition_guard.py

fix(agent+discord): guard truncated-response continuation loops and cap Discord split delivery (#86581)

2026-08-16 01:55:51 -07:00

replay_cleanup.py

refactor: shared helpers for api_content sidecar pop/drop/extract

2026-07-19 08:25:35 +05:30

retry_utils.py

refactor: single shared Retry-After parser

2026-07-29 10:13:50 -07:00

review_engine.py

feat: every subagent's prompt embeds the workspace's project context files

2026-08-23 19:04:37 -07:00

runtime_cwd.py

fix(agent): scope install-tree guard to fallback-picked cwds, allow cli/tui in-tree dev

2026-07-16 04:32:23 -07:00

secret_scope.py

fix(gateway): classify relay routing stamps as process-global deployment config

2026-08-21 13:21:46 +10:00

session_activity.py

refactor(agent): pin session activity heartbeat cadence + harden best-effort write

2026-08-02 16:16:36 -07:00

shell_hooks.py

feat(hooks): pre_tool_call content transformation via modify directive

2026-08-15 23:03:52 -07:00

skill_bundles.py

feat(observability): add Relay skill metrics

2026-07-29 12:20:34 -07:00

skill_commands.py

fix(skills): publish and read the (map, platform) pair under one lock

2026-08-19 19:41:42 -07:00

skill_preprocessing.py

perf: use load_config_readonly() at read-only call sites in agent/

2026-07-29 15:28:15 -07:00

skill_utils.py

fix: system prompt no longer references tools/skills the session can't use; hermes-agent skill is always kept

2026-08-24 20:25:10 -07:00

ssl_guard.py

fix(ssl_guard): tolerate truststore SSLContext.get_ca_certs() NotImplementedError on Windows

2026-08-08 15:10:24 +05:30

ssl_verify.py

fix(agent): honor custom CA certs on aux client + harden TLS resolution

2026-07-02 04:51:56 +05:30

stream_diag.py

…

stream_single_writer.py

fix(streaming): make the single-writer fence best-effort so a missing guard can't crash a turn (#66448)

2026-07-17 18:31:09 +00:00

subagent_lifecycle.py

fix(agent): separate cancellation diagnostics from tool output

2026-08-23 18:25:19 -07:00

subdirectory_hints.py

Port from earendil-works/pi#7681: support AGENTS.override.md context override

2026-08-16 22:07:43 -07:00

subscription_view.py

feat(cli): plan catalog on Free + plan= deep link + top-up/auto-refill copy split (#68689)

2026-07-22 08:11:09 +05:30

system_prompt.py

refactor(computer_use): diet schema + delete prompt block (~1.4K tok/call); remove max_elements, ladder moves to response verdicts

2026-08-26 04:50:13 -07:00

terminal_env_provider.py

feat(terminal): pluggable terminal environment backends via plugin registry

2026-08-24 20:10:44 -07:00

terminal_env_registry.py

feat(terminal): pluggable terminal environment backends via plugin registry

2026-08-24 20:10:44 -07:00

think_scrubber.py

fix(agent): re-arm think scrubber boundary after stream flush

2026-07-16 01:07:29 +05:30

thinking_timeout_guidance.py

fix(agent): detect thinking-timeout for reasoning models and surface actionable guidance instead of misleading file-write advice

2026-06-25 19:00:48 -07:00

thread_scoped_output.py

fix(agent): stop thread output descriptor leaks

2026-08-15 00:33:01 -07:00

title_generator.py

Port from can1357/oh-my-pi#7306: reject answer-shaped auto-title output

2026-08-16 22:10:02 -07:00

tool_dispatch_helpers.py

fix(tool-search): bridge batch barrier, listing truncation, source indexing (salvage #92693, part 1)

2026-08-25 15:24:15 -07:00

tool_executor.py

Merge pull request #86412 from kshitijk4poor/salvage/83225-overflow-clamp

2026-08-27 14:26:38 +05:30

tool_guardrails.py

refactor(tools): remove expired BFL FLUX 3 promo core tools (migration v39); FLUX 3 stays via video_gen/FAL for subscribers (#94599)

2026-08-25 02:45:10 -07:00

tool_result_classification.py

fix(agent): preserve none vs unknown tool effects (#61783)

2026-07-11 05:41:58 -07:00

trace_upload.py

fix(state): close leaked SessionDB connections on exception paths (#83226)

2026-08-14 21:41:26 -07:00

trajectory.py

…

transcription_provider.py

feat(plugins): add pre_transcription hook and STT prompt threading

2026-08-12 19:01:30 -07:00

transcription_registry.py

fix(plugins): isolate ownership by profile

2026-08-12 19:13:32 -07:00

tts_provider.py

…

tts_registry.py

fix(plugins): isolate ownership by profile

2026-08-12 19:13:32 -07:00

turn_context.py

Merge branch 'pr-81234' into salv/81234-retry-carrier

2026-08-24 03:15:07 -07:00

turn_finalizer.py

fix(memory): bind the checkpoint gate to post-turn micro-compaction too

2026-08-25 03:55:55 -07:00

turn_retry_state.py

feat(compression): native OpenAI Responses server-side compaction for gpt-5.6

2026-08-08 11:24:45 -07:00

turn_summary.py

fix(cli): report unknown line deltas for content-free diffs instead of +0 -0

2026-07-26 17:47:51 -07:00

usage_pricing.py

fix(pricing): support Gemini context-tiered rates in pricing snapshot (#93469)

2026-08-24 03:23:07 -07:00

verification_evidence.py

fix(agent): reject masked verification results

2026-08-16 00:14:58 -07:00

verification_stop.py

fix: make verify_on_stop opt-in everywhere (default False, not auto) (#84383)

2026-08-12 01:15:25 -07:00

verify_hooks.py

feat(agent): add pre_verify hook and verify-on-stop coding guidance

2026-06-30 00:59:29 -05:00

vertex_adapter.py

fix(picker): harden keyless provider gate logging and credentials path validation

2026-08-22 23:25:43 -07:00

video_gen_provider.py

feat(media): opt-in upscale pass for image_generate and video_generate across FAL and Krea

2026-08-08 14:49:28 -07:00

video_gen_registry.py

fix(tools): dispatch image/video FAL strictly on the stored hermes tools selection

2026-08-19 16:10:01 -07:00

web_search_provider.py

feat: web search/extract now work keyless on fresh installs via Parallel + Exa free tiers

2026-08-19 15:15:27 -07:00

web_search_registry.py

feat: keyless web tier becomes a 5-vendor round-robin ring (adds Tavily, Firecrawl, Keenable)

2026-08-20 00:17:25 -07:00