Skip to content

Add pipeline snippets for Firestore - #919

Open
morganchen12 wants to merge 32 commits into
mainfrom
mc/pipelines-snippets
Open

morganchen12 wants to merge 32 commits into
mainfrom
mc/pipelines-snippets

Conversation

@morganchen12

Copy link
Copy Markdown
Contributor

No description provided.


default_app = firebase_admin.initialize_app()
client = firestore.client(default_app, "your-new-enterprise-database")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also have an async version of everything.

Often, we re-write the samples for both sync and async, and show both options in the language selection pickers. I'm not sure how you want to handle that here.

Maybe a couple samples for pipeline_initialization_async and basic_read_async is enough to show it exists, since the API is almost identical. Or maybe samples for async aren't needed for preview

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to get to the async versions after Java and if I don't get to it by launch day it'll just be a follow-on.

.execute()
# [END vector_length]
for res in result:
print(res)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great if we could have a test file that exercises these functions. Even just running them without verifying outputs would be helpful to make sure no exceptions are thrown, but reading back results would be ideal

It looks like the other samples in this repo aren't tested though, so maybe that would be too much to set up

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will defer to another PR

Field.of("rating").average().as_("avg_rating"),
groups=[Field.of("genre")]
) \
.execute()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to get rid of all the \, IIRC think you could do something like this:

results = (
    client.pipeline()
    .collection("books")
    .aggregate(
        Field.of("rating").average().as_("avg_rating"),
        groups=[Field.of("genre")]
    ).execute()
)

(But test it first)

@daniel-sanche daniel-sanche left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding more imports for Vector, and others that may be missing through the file

But overall LGTM

@morganchen12

Copy link
Copy Markdown
Contributor Author

After adding the snippets for expressions and inputs, I'm getting these errors:

(.venv) morganchen-mac:firebase-admin-python morganchen$ ./lint.sh 
Running linter on snippets/firestore/firestore_pipelines.py
************* Module snippets.firestore.firestore_pipelines
snippets/firestore/firestore_pipelines.py:1770:20: E1101: Module 'firebase_admin.firestore' has no 'SERVER_TIMESTAMP' member (no-member)
snippets/firestore/firestore_pipelines.py:2009:41: E1101: Class 'Expression' has no 'array' member (no-member)
snippets/firestore/firestore_pipelines.py:2010:16: E1101: Class 'Expression' has no 'array' member (no-member)
snippets/firestore/firestore_pipelines.py:2288:12: E1101: Instance of 'Field' has no 'array_get' member (no-member)

For the errors:

  1. I'm not sure why I'm getting this error, it seems to be invoked correctly based on our docs
  2. I'm also not sure why I'm getting this error since Array is documented here
  3. See 2
  4. The array_get method seems to just be missing from the Python SDK. @daniel-sanche this is likely me not reading something somewhere, but just in case it was overlooked

@daniel-sanche

Copy link
Copy Markdown
  1. One of those is from google.cloud import firestore, and the other is from firebase_admin import firestore. I'm not super familiar with the firebase_admin wrapper, but I don't think those are fully equivalent

  2. In the end, I think Array was actually implemented as a stand-alone class:

from google.cloud.firestore_v1.pipeline_expressions import Array
Array(["bar", Field.of("baz")])

It looks like there's an out-dated docstring. I can fix that.

Let me know if you think it's important to have array exposed on Expression as well though. I removed it in an effort to keep the API surface manageable, instead of exposing multiple ways of expressing the same thing. But I can add it back if that's more consistent with other languages.

  1. array_get was a late addition, but it should be in the preview branch as of yesterday. Maybe try running again

morganchen12 and others added 13 commits November 12, 2025 14:00
* add fid arg and token deprecate warning to Message class and add unit tests

* add fids to MulticastMessage and token deprecate warning and add unit tests

* Fix lint formatting error

* Position `tokens` first to guarantee backward compatibility for legacy positional arguments and add a unit test for it

* Extract multicast-to-message-list conversion logic into a private helper function

* Add docstring for the helper function

* Add unit tests for the async function, change deprecate message and address other review comments

* Fix error messages and MulticastMessage constructor and add mix types of fids and tokens

* Add integration tests for fid as argument

* Update integration tests error code for invalid fid target
…redential (#954)

Added a universe_domain property override to MockGoogleComputeEngineCredential in tests/testutils.py to prevent unexpected network requests to the GCE metadata server during test runs with newer versions of google-auth.
Updates the requirements on [pyjwt](https://github.com/jpadilla/pyjwt) to permit the latest version.
- [Release notes](https://github.com/jpadilla/pyjwt/releases)
- [Changelog](https://github.com/jpadilla/pyjwt/blob/master/CHANGELOG.rst)
- [Commits](jpadilla/pyjwt@2.10.1...2.12.1)

---
updated-dependencies:
- dependency-name: pyjwt
  dependency-version: 2.12.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yvonne Pan <103622026+yvonnep165@users.noreply.github.com>
Updates the requirements on [google-cloud-firestore](https://github.com/googleapis/google-cloud-python) to permit the latest version.
- [Release notes](https://github.com/googleapis/google-cloud-python/releases)
- [Changelog](https://github.com/googleapis/google-cloud-python/blob/main/packages/google-cloud-documentai/CHANGELOG.md)
- [Commits](googleapis/google-cloud-python@google-cloud-firestore-v2.23.0...google-cloud-firestore-v2.27.0)

---
updated-dependencies:
- dependency-name: google-cloud-firestore
  dependency-version: 2.27.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yvonne Pan <103622026+yvonnep165@users.noreply.github.com>
Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@8.2.2...8.4.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-version: 8.4.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yvonne Pan <103622026+yvonnep165@users.noreply.github.com>
morganchen12 and others added 12 commits June 29, 2026 13:24
* add snippets for joins and explain

* fix snippet string
* add ml api deprecation warnings

* Verify deprecation warnings in unit tests

* add helper function to prevent duplicate warning flooding and unit tests for it
* add one time token verification support

* add unit tests

* fix lint error

* change to use _utils.handle_platform_error_from_requests

* change the function description

* change docstring to state limited-use tokens

* add validation for consume variable

* add check_boolean to _Validators and add unit tests to check non-boolean consume

* raise `exceptions.UnknownError` if body is not a dict

* move the replay verification to a separate helper method and add unit tests

* add test_verify_token_with_consume_false_makes_no_http_call unit test

* replace the broad exceptions.FirebaseError assertion with more specific error types

* Refactor token verification tests to use mock_jwt fixture

* Document ValueError when consume is not a boolean in verify_token docstring
* feat(fcm): Migrate topic management to FCM v1 API

* fix(fcm): Prevent headers mutation race condition and deduplicate topic error parsing

- Pass a copy of self._fcm_headers in topic management requests to prevent concurrent mutation race conditions in ThreadPoolExecutor.
- Extract common error parsing logic from _build_topic_subscription_result_from_requests_error and _build_topic_subscription_result_from_httpx_error into _build_topic_subscription_result.

* fix(fcm): Improve topic management performance and fix test lint issues

- Hoist URL-encoded topic computation out of the per-token request loops.
- Mount an HTTPAdapter with a connection pool size of 100 on the FCM client session.
- Fix line length and method signature override lint warnings in test_messaging.py.

* fix(fcm): Addressed review improvements for FCM v1 topic management

- Added type annotations to public topic management functions and internal methods
- Updated HTTP status code mapping (401->UNAUTHENTICATED, 503->UNAVAILABLE, 408/504->DEADLINE_EXCEEDED) and prioritized status codes over free-form message strings
- Updated topic regex with \Z to reject trailing newlines
- Passed header copies in send, send_each, and send_each_async to prevent concurrency mutations
- Added test coverage for status code mapping, async argument validation, prefixed topics, and async batch unsubscribe

* chore(fcm): Trigger CI workflows
* fix(fac): Use v1beta endpoint for verifyAppCheckToken

Updated _AppCheckService._VERIFY_URL_FORMAT to point to the v1beta endpoint (https://firebaseappcheck.googleapis.com/v1beta/projects/{project_id}:verifyAppCheckToken) for replay protection verification.

* chore(fac): Fix line length lint in test_app_check.py
…meters (#996)

* fix(auth): validate tenant and provider IDs and URL-encode query parameters

* fix(auth): simplify tenant and provider ID validation checks
@morganchen12
morganchen12 requested a review from a team October 30, 2026 23:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants