Skip to content

Python: [BREAKING] Add Gemini Embedding 2 and vector task options - #8798

Merged
Eduard van Valkenburg (eavanvalkenburg) merged 4 commits into
microsoft:mainfrom
eavanvalkenburg:google-embeddings-client
Sep 29, 2026
Merged

Eduard van Valkenburg (eavanvalkenburg) merged 4 commits into
microsoft:mainfrom
eavanvalkenburg:google-embeddings-client

Conversation

@eavanvalkenburg

Copy link
Copy Markdown
Member

Motivation & Context

The Google portion of #4171 needs an embedding client for the current Gemini Embedding 2 service. Asymmetric retrieval requires different document and query instructions; generic vector operations previously supplied only dimensions, leaving no way to select those tasks without application-specific adapters.

Description & Review Guide

  • What are the major changes? Add text and multimodal GeminiEmbeddingClient / RawGeminiEmbeddingClient using google-genai 2.25+, defaulting to stable gemini-embedding-2. Map Google-supported text tasks per call, preserve unprefixed media and mixed-media content, and handle Developer API batching versus Enterprise's single-content requests. Forward mutually exclusive flat or by-field embedding options through vector upserts, and query options through search and generated tools; field dimensions remain authoritative. Add provider exports, SDK-backed tests, and text/image search Agent samples using create_vector_search_tool.
  • What is the impact of these changes? Gemini chat and embeddings use the GOOGLE_* settings namespace and support the current GOOGLE_GENAI_USE_ENTERPRISE mode. Breaking beta change: chat no longer reads GEMINI_API_KEY or GEMINI_MODEL or exports GeminiSettings; use GOOGLE_API_KEY and GOOGLE_MODEL. The new embedding client reads GOOGLE_EMBEDDING_MODEL. Vector operations without embedding options retain their previous behavior; conflicting dimensions fail before embedding.
  • What do you want reviewers to focus on? The per-operation options contract (broadcast versus by-field, dimension conflicts, and precomputed vectors), Gemini Embedding 2 task formatting and multimodal behavior, and shared Developer/Enterprise authentication. Python: accept nullable auto-generated keys in vector upsert tools #8631 also touches Core vector tool files for an unrelated nullable-key fix.

Related Issue

Part of #4171: this PR implements only Google AI / Vertex AI embeddings. NVIDIA is still outstanding, so merging this PR must not close the parent issue. No other open PR was found for the Google embedding task.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after a language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Integration-test gates currently skip the default embedding model and the new Enterprise chat mode.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
What changed in this PR

Adds Gemini Embedding 2 support and provider-specific vector embedding options while standardizing Gemini configuration on GOOGLE_*.

Changes:

  • Adds text/multimodal Gemini embedding clients and shared authentication.
  • Forwards per-operation embedding options through vector APIs and tools.
  • Adds tests, documentation, and retrieval samples.
File Description
python/​uv.lock Updates Gemini package and Google SDK versions.
python/​samples/​README.md Documents Google configuration variables.
python/​samples/​02-agents/​vector_stores/​README.md Explains vector embedding options.
python/​samples/​02-agents/​providers/​gemini/​README.md Documents new Gemini samples and settings.
python/​samples/​02-agents/​providers/​gemini/​gemini_with_google_search.py Updates configuration guidance.
python/​samples/​02-agents/​providers/​gemini/​gemini_with_google_maps.py Updates configuration guidance.
python/​samples/​02-agents/​providers/​gemini/​gemini_with_code_execution.py Updates configuration guidance.
python/​samples/​02-agents/​providers/​gemini/​gemini_search_agent.py Adds text retrieval sample.
python/​samples/​02-agents/​providers/​gemini/​gemini_image_search_agent.py Adds cross-modal image search sample.
python/​samples/​02-agents/​providers/​gemini/​gemini_embeddings.py Adds embedding task sample.
python/​samples/​02-agents/​providers/​gemini/​gemini_basic.py Updates configuration guidance.
python/​samples/​02-agents/​providers/​gemini/​gemini_advanced.py Updates configuration guidance.
python/​packages/​gemini/​tests/​test_gemini_client.py Tests shared chat authentication changes.
python/​packages/​gemini/​tests/​gemini/​test_gemini_embedding_client.py Tests embedding behavior and integration.
python/​packages/​gemini/​README.md Documents embeddings and migration.
python/​packages/​gemini/​pyproject.toml Bumps package and SDK versions.
python/​packages/​gemini/​AGENTS.md Records Gemini package conventions.
python/​packages/​gemini/​agent_framework_gemini/​_sdk_client.py Centralizes SDK authentication and endpoints.
python/​packages/​gemini/​agent_framework_gemini/​_embedding_client.py Implements Gemini embedding clients.
python/​packages/​gemini/​agent_framework_gemini/​_chat_client.py Adopts shared authentication and Google settings.
python/​packages/​gemini/​agent_framework_gemini/​__init__.py Exports embedding APIs.
python/​packages/​core/​tests/​core/​test_vectors.py Tests embedding-option forwarding.
python/​packages/​core/​tests/​core/​test_gemini_namespace.py Tests lazy Gemini exports.
python/​packages/​core/​AGENTS.md Documents vector option contracts.
python/​packages/​core/​agent_framework/​gemini/​__init__.pyi Adds embedding type exports.
python/​packages/​core/​agent_framework/​gemini/​__init__.py Adds lazy embedding exports.
python/​packages/​core/​agent_framework/​_vectors.py Adds embedding options across vector operations.
python/​.env.example Migrates Gemini environment names.
docs/​specs/​feature-usage-bit-registry.md Extends Gemini feature usage description.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread python/packages/gemini/tests/gemini/test_gemini_embedding_client.py Outdated
Comment thread python/packages/gemini/tests/test_gemini_client.py

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (2 commit(s)): b88e1b88d5f3, 1d5960d21f71
Model: gpt-5.6-sol-fast

Overview

The review found 1 verified inline finding(s).

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 medium) across 1 file. Details are attached to the affected lines below.

Affected areas: python/packages/core/agent_framework/gemini/__init__.py

Comment thread python/packages/core/agent_framework/gemini/__init__.py
Comment thread python/packages/gemini/agent_framework_gemini/_embedding_client.py Outdated

@jpalvarezl Jose Alvarez (jpalvarezl) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. The new embedding and search options are wired through consistently, and the settings migration is documented. No substantive concerns from my review.

Merged via the queue into microsoft:main with commit f8f15f2 Sep 29, 2026
46 checks passed
@eavanvalkenburg
Eduard van Valkenburg (eavanvalkenburg) deleted the google-embeddings-client branch September 29, 2026 16:05

This branch was successfully deployed

1 active deployment
github-app-auth — 2e0cb5ea Deployed Sep 29, 2026 by eavanvalkenburg via add_label #23924
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking change Usage: [PRs], Target: all PRs that introduce changes that are not backward compatible documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants