Skip to content

Commit 5a395ec

Browse files
tpellissier-msfttpellissierclaudeAbel MilashSaurabh Badenkal
authored
Add QueryBuilder with fluent API and composable filter expressions (#118)
## Summary Implements the QueryBuilder feature from the SDK redesign design doc (ADO PR 1504429): - **Fluent query builder** via `client.query.builder("table")` with 20 chainable methods including `select`, `filter_eq/ne/gt/ge/lt/le`, `filter_contains/startswith/endswith`, `filter_in`, `filter_between`, `filter_null/not_null`, `filter_raw`, `where`, `order_by`, `top`, `page_size`, `expand`, and `execute` - **Composable filter expression tree** (`models/filters.py`) with Python operator overloads (`&`, `|`, `~`) for AND, OR, NOT composition - **Value auto-formatting** for `str`, `int`, `float`, `bool`, `None`, `datetime`, `date`, `uuid.UUID` - 126 new unit tests (57 filters + 69 query builder), 309 total passing ### Usage examples ```python # Fluent builder for page in (client.query.builder("account") .select("name", "revenue") .filter_eq("statecode", 0) .filter_gt("revenue", 1000000) .order_by("revenue", descending=True) .top(100) .page_size(50) .execute()): for record in page: print(record["name"]) # Composable expression tree with where() from PowerPlatform.Dataverse.models.filters import eq, gt, filter_in for page in (client.query.builder("account") .where((eq("statecode", 0) | eq("statecode", 1)) & gt("revenue", 100000)) .execute()): for record in page: print(record["name"]) ``` ### Design decisions - **Regular class, not dataclass** — prevents leaking internal state as constructor params - **Unified `_filter_parts` list** — preserves call order when mixing `filter_*()` and `where()` - **`execute()` calls `build()` internally** — single source of truth for filter compilation - **No public `get()` on QueryOperations** — only `builder()` added; paginated queries remain on `records.get()` - **Parenthesized `filter_between`** — `(col ge low and col le high)` for correct precedence ### Files changed | File | Description | |------|-------------| | `src/.../models/filters.py` | **NEW** — Composable expression tree | | `src/.../models/query_builder.py` | **NEW** — Fluent QueryBuilder class | | `src/.../operations/query.py` | Add `builder()` to QueryOperations | | `src/.../models/__init__.py` | Updated docstring | | `tests/.../models/test_filters.py` | **NEW** — 57 filter tests | | `tests/.../models/test_query_builder.py` | **NEW** — 69 builder tests | | `tests/.../test_query_operations.py` | 6 new integration tests | ### Merge conflict note `operations/query.py` may conflict with PR #115 (typed return models) — resolution is straightforward since we only add a `builder()` method. ## Test plan - [x] `pytest tests/unit/models/test_filters.py` — 57 passed - [x] `pytest tests/unit/models/test_query_builder.py` — 69 passed - [x] `pytest tests/unit/test_query_operations.py` — 9 passed - [x] `pytest tests/` — 309 passed, 0 failed 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: tpellissier <tpellissier@microsoft.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Abel Milash <abelmilash@microsoft.com> Co-authored-by: Saurabh Badenkal <sbadenkal@microsoft.com> Co-authored-by: Saurabh Ravindra Badenkal <32964911+saurabhrb@users.noreply.github.com>
1 parent 9788cbb commit 5a395ec

17 files changed

Lines changed: 3388 additions & 39 deletions

README.md

Lines changed: 112 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ A Python client library for Microsoft Dataverse that provides a unified interfac
2525
- [Bulk operations](#bulk-operations)
2626
- [Upsert operations](#upsert-operations)
2727
- [DataFrame operations](#dataframe-operations)
28-
- [Query data](#query-data)
28+
- [Query data](#query-data) *(QueryBuilder, SQL, raw OData)*
2929
- [Table management](#table-management)
3030
- [Relationship management](#relationship-management)
3131
- [File operations](#file-operations)
@@ -37,7 +37,8 @@ A Python client library for Microsoft Dataverse that provides a unified interfac
3737

3838
- **🔄 CRUD Operations**: Create, read, update, and delete records with support for bulk operations and automatic retry
3939
- **⚡ True Bulk Operations**: Automatically uses Dataverse's native `CreateMultiple`, `UpdateMultiple`, `UpsertMultiple`, and `BulkDelete` Web API operations for maximum performance and transactional integrity
40-
- **📊 SQL Queries**: Execute read-only SQL queries via the Dataverse Web API `?sql=` parameter
40+
- **🔍 Fluent QueryBuilder**: Type-safe query construction with method chaining, composable filter expressions, and automatic OData generation
41+
- **📊 SQL Queries**: Execute read-only SQL queries via the Dataverse Web API `?sql=` parameter
4142
- **🏗️ Table Management**: Create, inspect, and delete custom tables and columns programmatically
4243
- **🔗 Relationship Management**: Create one-to-many and many-to-many relationships between tables with full metadata control
4344
- **🐼 DataFrame Support**: Pandas wrappers for all CRUD operations, returning DataFrames and Series
@@ -116,7 +117,7 @@ The SDK provides a simple, pythonic interface for Dataverse operations:
116117
|---------|-------------|
117118
| **DataverseClient** | Main entry point; provides `records`, `query`, `tables`, and `files` namespaces |
118119
| **Context Manager** | Use `with DataverseClient(...) as client:` for automatic cleanup and HTTP connection pooling |
119-
| **Namespaces** | Operations are organized into `client.records` (CRUD & OData queries), `client.query` (query & search), `client.tables` (metadata), and `client.files` (file uploads) |
120+
| **Namespaces** | Operations are organized into `client.records` (CRUD & OData queries), `client.query` (QueryBuilder & SQL), `client.tables` (metadata), and `client.files` (file uploads) |
120121
| **Records** | Dataverse records represented as Python dictionaries with column schema names |
121122
| **Schema names** | Use table schema names (`"account"`, `"new_MyTestTable"`) and column schema names (`"name"`, `"new_MyTestColumn"`). See: [Table definitions in Microsoft Dataverse](https://learn.microsoft.com/en-us/power-apps/developer/data-platform/entity-metadata) |
122123
| **Bulk Operations** | Efficient bulk processing for multiple records with automatic optimization |
@@ -272,42 +273,129 @@ client.dataframe.delete("account", new_accounts["accountid"])
272273

273274
### Query data
274275

276+
The **QueryBuilder** is the recommended way to query records. It provides a fluent, type-safe interface that generates correct OData queries automatically — no need to remember OData filter syntax.
277+
278+
```python
279+
# Fluent query builder (recommended)
280+
for record in (client.query.builder("account")
281+
.select("name", "revenue")
282+
.filter_eq("statecode", 0)
283+
.filter_gt("revenue", 1000000)
284+
.order_by("revenue", descending=True)
285+
.top(100)
286+
.page_size(50)
287+
.execute()):
288+
print(f"{record['name']}: {record['revenue']}")
289+
```
290+
291+
The QueryBuilder handles value formatting, column name casing, and OData syntax automatically. All filter methods are discoverable via IDE autocomplete:
292+
293+
```python
294+
# Get results as a pandas DataFrame (consolidates all pages)
295+
df = (client.query.builder("account")
296+
.select("name", "telephone1")
297+
.filter_eq("statecode", 0)
298+
.top(100)
299+
.to_dataframe())
300+
print(f"Got {len(df)} accounts")
301+
```
302+
303+
```python
304+
# Comparison filters
305+
query = (client.query.builder("contact")
306+
.filter_eq("statecode", 0) # statecode eq 0
307+
.filter_gt("revenue", 1000000) # revenue gt 1000000
308+
.filter_contains("name", "Corp") # contains(name, 'Corp')
309+
.filter_in("statecode", [0, 1]) # Microsoft.Dynamics.CRM.In(...)
310+
.filter_between("revenue", 100000, 500000) # (revenue ge 100000 and revenue le 500000)
311+
.filter_null("telephone1") # telephone1 eq null
312+
)
313+
```
314+
315+
For complex logic (OR, NOT, grouping), use the composable expression tree with `where()`:
316+
317+
```python
318+
from PowerPlatform.Dataverse.models.filters import eq, gt, filter_in, between
319+
320+
# OR conditions: (statecode = 0 OR statecode = 1) AND revenue > 100k
321+
for record in (client.query.builder("account")
322+
.select("name", "revenue")
323+
.where((eq("statecode", 0) | eq("statecode", 1))
324+
& gt("revenue", 100000))
325+
.execute()):
326+
print(record["name"])
327+
328+
# NOT, between, and in operators
329+
for record in (client.query.builder("account")
330+
.where(~eq("statecode", 2)) # NOT inactive
331+
.where(between("revenue", 100000, 500000)) # revenue in range
332+
.execute()):
333+
print(record["name"])
334+
```
335+
336+
**Formatted values and annotations** -- request localized labels, currency symbols, and display names:
337+
338+
```python
339+
# Get formatted values (choice labels, currency, lookup names)
340+
for record in (client.query.builder("account")
341+
.select("name", "statecode", "revenue")
342+
.include_formatted_values()
343+
.execute()):
344+
status = record["statecode@OData.Community.Display.V1.FormattedValue"]
345+
print(f"{record['name']}: {status}")
346+
```
347+
348+
**Nested expand with options** -- expand navigation properties with `$select`, `$filter`, `$orderby`, and `$top`:
349+
350+
```python
351+
from PowerPlatform.Dataverse.models.query_builder import ExpandOption
352+
353+
# Expand related tasks with filtering and sorting
354+
for record in (client.query.builder("account")
355+
.select("name")
356+
.expand(ExpandOption("Account_Tasks")
357+
.select("subject", "createdon")
358+
.filter("contains(subject,'Task')")
359+
.order_by("createdon", descending=True)
360+
.top(5))
361+
.execute()):
362+
print(record["name"], record.get("Account_Tasks"))
363+
```
364+
365+
**Record count** -- include `$count=true` in the request:
366+
367+
```python
368+
# Request count alongside results
369+
results = (client.query.builder("account")
370+
.filter_eq("statecode", 0)
371+
.count()
372+
.execute())
373+
```
374+
375+
**SQL queries** provide an alternative read-only query syntax:
376+
275377
```python
276-
# SQL query (read-only)
277378
results = client.query.sql(
278379
"SELECT TOP 10 accountid, name FROM account WHERE statecode = 0"
279380
)
280381
for record in results:
281382
print(record["name"])
383+
```
384+
385+
**Raw OData queries** are available via `records.get()` for cases where you need direct control over the OData filter string:
282386

283-
# OData query with paging
284-
# Note: filter and expand parameters are case sensitive
387+
```python
285388
for page in client.records.get(
286389
"account",
287-
select=["accountid", "name"], # select is case-insensitive (automatically lowercased)
288-
filter="statecode eq 0", # filter must use lowercase logical names (not transformed)
390+
select=["name"],
391+
filter="statecode eq 0", # Raw OData: column names must be lowercase
392+
expand=["primarycontactid"], # Navigation properties are case-sensitive
289393
top=100,
290394
):
291395
for record in page:
292396
print(record["name"])
293-
294-
# Query with navigation property expansion (case-sensitive!)
295-
for page in client.records.get(
296-
"account",
297-
select=["name"],
298-
expand=["primarycontactid"], # Navigation property names are case-sensitive
299-
filter="statecode eq 0", # Column names must be lowercase logical names
300-
):
301-
for account in page:
302-
contact = account.get("primarycontactid", {})
303-
print(f"{account['name']} - Contact: {contact.get('fullname', 'N/A')}")
304397
```
305398

306-
> **Important**: When using `filter` and `expand` parameters:
307-
> - **`filter`**: Column names must use exact lowercase logical names (e.g., `"statecode eq 0"`, not `"StateCode eq 0"`)
308-
> - **`expand`**: Navigation property names are case-sensitive and must match the exact server names
309-
> - **`select`** and **`orderby`**: Case-insensitive; automatically converted to lowercase
310-
311399
### Table management
312400

313401
```python

0 commit comments

Comments
 (0)