Skip to content

Commit 56c4f81

Browse files
author
Samson Gebre
committed
implement batch API:
Add unit tests for batch serialization, OData key formatting, SQL parsing, and batch operations - Implemented unit tests for internal batch multipart serialization and response parsing in `test_batch_serialization.py`. - Added tests for `_ODataClient._format_key` functionality in `test_format_key.py`. - Enhanced SQL parsing tests in `test_sql_parse.py` to cover URL encoding scenarios. - Created comprehensive tests for batch operations, including record and table operations, in `test_batch_operations.py`.
1 parent 4dad4bd commit 56c4f81

16 files changed

Lines changed: 3003 additions & 139 deletions

File tree

‎.claude/skills/dataverse-sdk-dev/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ This skill provides guidance for developers working on the PowerPlatform Dataver
1313

1414
### API Design
1515

16-
1. **Public methods in operation namespaces** - New public methods go in the appropriate namespace module under `src/PowerPlatform/Dataverse/operations/` (`records.py`, `query.py`, `tables.py`). The `client.py` file exposes these via namespace properties (`client.records`, `client.query`, `client.tables`). Public types and constants live in their own modules (e.g., `models/metadata.py`, `common/constants.py`)
16+
1. **Public methods in operation namespaces** - New public methods go in the appropriate namespace module under `src/PowerPlatform/Dataverse/operations/` (`records.py`, `query.py`, `tables.py`, `batch.py`). The `client.py` file exposes these via namespace properties (`client.records`, `client.query`, `client.tables`, `client.batch`). Public types and constants live in their own modules (e.g., `models/metadata.py`, `models/batch.py`, `common/constants.py`)
1717
2. **Every public method needs README example** - Public API methods must have examples in README.md
1818
3. **Reuse existing APIs** - Always check if an existing method can be used before making direct Web API calls
1919
4. **Update documentation** when adding features - Keep README and SKILL files (both copies) in sync

‎.claude/skills/dataverse-sdk-use/SKILL.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ Use the PowerPlatform Dataverse Client Python SDK to interact with Microsoft Dat
2121
- `client.records` -- CRUD and OData queries
2222
- `client.query` -- query and search operations
2323
- `client.tables` -- table metadata, columns, and relationships
24+
- `client.batch` -- batch multiple operations into a single HTTP request
2425

2526
### Bulk Operations
2627
The SDK supports Dataverse's native bulk operations: Pass lists to `create()`, `update()` for automatic bulk processing, for `delete()`, set `use_bulk_delete` when passing lists to use bulk operation
@@ -275,6 +276,50 @@ client.upload_file(
275276
)
276277
```
277278

279+
### Batch Operations
280+
281+
Use `client.batch` to send multiple operations in one HTTP request. All batch methods return `None`; results arrive via `BatchResult` after `execute()`.
282+
283+
```python
284+
# Build a batch request
285+
batch = client.batch.new()
286+
batch.records.create("account", {"name": "Contoso"})
287+
batch.records.update("account", account_id, {"telephone1": "555-0100"})
288+
batch.records.get("account", account_id, select=["name"])
289+
batch.query.sql("SELECT TOP 5 name FROM account")
290+
291+
result = batch.execute()
292+
for item in result.responses:
293+
if item.is_success:
294+
print(f"[OK] {item.status_code} entity_id={item.entity_id}")
295+
if item.body:
296+
# GET responses populate item.body with the parsed JSON record
297+
print(item.body.get("name"))
298+
else:
299+
print(f"[ERR] {item.status_code}: {item.error_message}")
300+
301+
# Transactional changeset (all succeed or roll back)
302+
with batch.changeset() as cs:
303+
ref = cs.records.create("contact", {"firstname": "Alice"})
304+
cs.records.update("account", account_id, {"primarycontactid@odata.bind": ref})
305+
306+
# Continue on error
307+
result = batch.execute(continue_on_error=True)
308+
print(f"Succeeded: {len(result.succeeded)}, Failed: {len(result.failed)}")
309+
```
310+
311+
**BatchResult properties:**
312+
- `result.responses` -- list of `BatchItemResponse` in submission order
313+
- `result.succeeded` -- responses with 2xx status codes
314+
- `result.failed` -- responses with non-2xx status codes
315+
- `result.has_errors` -- True if any response failed
316+
- `result.created_ids` -- GUIDs from successful create operations
317+
318+
**Batch limitations:**
319+
- Maximum 1000 operations per batch
320+
- Paginated `records.get()` (without `record_id`) is not supported in batch
321+
- `flush_cache()` is not supported in batch
322+
278323
## Error Handling
279324

280325
The SDK provides structured exceptions with detailed error information:

‎README.md‎

Lines changed: 65 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ A Python client library for Microsoft Dataverse that provides a unified interfac
2727
- [Table management](#table-management)
2828
- [Relationship management](#relationship-management)
2929
- [File operations](#file-operations)
30+
- [Batch operations](#batch-operations)
3031
- [Next steps](#next-steps)
3132
- [Troubleshooting](#troubleshooting)
3233
- [Contributing](#contributing)
@@ -39,6 +40,7 @@ A Python client library for Microsoft Dataverse that provides a unified interfac
3940
- **🏗️ Table Management**: Create, inspect, and delete custom tables and columns programmatically
4041
- **🔗 Relationship Management**: Create one-to-many and many-to-many relationships between tables with full metadata control
4142
- **📎 File Operations**: Upload files to Dataverse file columns with automatic chunking for large files
43+
- **📦 Batch Operations**: Send multiple CRUD, table metadata, and SQL query operations in a single HTTP request with optional transactional changesets
4244
- **🔐 Azure Identity**: Built-in authentication using Azure Identity credential providers with comprehensive support
4345
- **🛡️ Error Handling**: Structured exception hierarchy with detailed error context and retry guidance
4446

@@ -111,8 +113,8 @@ The SDK provides a simple, pythonic interface for Dataverse operations:
111113

112114
| Concept | Description |
113115
|---------|-------------|
114-
| **DataverseClient** | Main entry point; provides `records`, `query`, and `tables` namespaces |
115-
| **Namespaces** | Operations are organized into `client.records` (CRUD & OData queries), `client.query` (query & search), and `client.tables` (metadata) |
116+
| **DataverseClient** | Main entry point; provides `records`, `query`, `tables`, and `batch` namespaces |
117+
| **Namespaces** | Operations are organized into `client.records` (CRUD & OData queries), `client.query` (query & search), `client.tables` (metadata), and `client.batch` (batch requests) |
116118
| **Records** | Dataverse records represented as Python dictionaries with column schema names |
117119
| **Schema names** | Use table schema names (`"account"`, `"new_MyTestTable"`) and column schema names (`"name"`, `"new_MyTestColumn"`). See: [Table definitions in Microsoft Dataverse](https://learn.microsoft.com/en-us/power-apps/developer/data-platform/entity-metadata) |
118120
| **Bulk Operations** | Efficient bulk processing for multiple records with automatic optimization |
@@ -334,6 +336,66 @@ client.upload_file(
334336
)
335337
```
336338

339+
### Batch operations
340+
341+
Use `client.batch` to send multiple operations in one HTTP request. The batch namespace mirrors `client.records`, `client.tables`, and `client.query`.
342+
343+
```python
344+
# Build a batch request and add operations
345+
batch = client.batch.new()
346+
batch.records.create("account", {"name": "Contoso"})
347+
batch.records.create("account", [{"name": "Fabrikam"}, {"name": "Woodgrove"}])
348+
batch.records.update("account", account_id, {"telephone1": "555-0100"})
349+
batch.records.delete("account", old_id)
350+
batch.records.get("account", account_id, select=["name"])
351+
352+
result = batch.execute()
353+
for item in result.responses:
354+
if item.is_success:
355+
print(f"[OK] {item.status_code} entity_id={item.entity_id}")
356+
else:
357+
print(f"[ERR] {item.status_code}: {item.error_message}")
358+
```
359+
360+
**Transactional changeset** — all operations in a changeset succeed or roll back together:
361+
362+
```python
363+
batch = client.batch.new()
364+
with batch.changeset() as cs:
365+
lead_ref = cs.records.create("lead", {"firstname": "Ada"})
366+
contact_ref = cs.records.create("contact", {"firstname": "Ada"})
367+
cs.records.create("account", {
368+
"name": "Babbage & Co.",
369+
"originatingleadid@odata.bind": lead_ref,
370+
"primarycontactid@odata.bind": contact_ref,
371+
})
372+
result = batch.execute()
373+
print(f"Created {len(result.created_ids)} records atomically")
374+
```
375+
376+
**Table metadata and SQL queries in a batch:**
377+
378+
```python
379+
batch = client.batch.new()
380+
batch.tables.create("new_Product", {"new_Price": "decimal", "new_InStock": "bool"})
381+
batch.tables.add_columns("new_Product", {"new_Rating": "int"})
382+
batch.tables.get("new_Product")
383+
batch.query.sql("SELECT TOP 5 name FROM account")
384+
385+
result = batch.execute()
386+
```
387+
388+
**Continue on error** — attempt all operations even when one fails:
389+
390+
```python
391+
result = batch.execute(continue_on_error=True)
392+
print(f"Succeeded: {len(result.succeeded)}, Failed: {len(result.failed)}")
393+
for item in result.failed:
394+
print(f"[ERR] {item.status_code}: {item.error_message}")
395+
```
396+
397+
For a complete example see [examples/advanced/batch.py](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/batch.py).
398+
337399
## Next steps
338400

339401
### More sample code
@@ -348,6 +410,7 @@ Explore our comprehensive examples in the [`examples/`](https://github.com/micro
348410
- **[Complete Walkthrough](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/walkthrough.py)** - Full feature demonstration with production patterns
349411
- **[Relationship Management](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/relationships.py)** - Create and manage table relationships
350412
- **[File Upload](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/file_upload.py)** - Upload files to Dataverse file columns
413+
- **[Batch Operations](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/batch.py)** - Send multiple operations in a single request with changesets
351414

352415
📖 See the [examples README](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/README.md) for detailed guidance and learning progression.
353416

‎examples/advanced/batch.py‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# Copyright (c) Microsoft Corporation.
2+
# Licensed under the MIT license.
3+
4+
"""
5+
Batch operations example for the Dataverse Python SDK.
6+
7+
Demonstrates how to use client.batch to send multiple operations in a single
8+
HTTP request to the Dataverse Web API.
9+
10+
Requirements:
11+
pip install "PowerPlatform.Dataverse" azure-identity
12+
"""
13+
14+
from __future__ import annotations
15+
16+
# ---------------------------------------------------------------------------
17+
# Setup — replace with your environment URL and credential
18+
# ---------------------------------------------------------------------------
19+
20+
from azure.identity import InteractiveBrowserCredential
21+
from PowerPlatform.Dataverse.client import DataverseClient
22+
23+
credential = InteractiveBrowserCredential()
24+
client = DataverseClient("https://org.crm.dynamics.com", credential)
25+
26+
27+
# ---------------------------------------------------------------------------
28+
# Example 1: Record CRUD in a single batch
29+
# ---------------------------------------------------------------------------
30+
31+
print("\n[INFO] Example 1: Record CRUD in a single batch")
32+
33+
batch = client.batch.new()
34+
35+
# Create a single record
36+
batch.records.create("account", {"name": "Contoso Ltd", "telephone1": "555-0100"})
37+
38+
# Create multiple records via CreateMultiple (one batch item)
39+
batch.records.create(
40+
"contact",
41+
[
42+
{"firstname": "Alice", "lastname": "Smith"},
43+
{"firstname": "Bob", "lastname": "Jones"},
44+
],
45+
)
46+
47+
# Assume we have an existing account_id from a prior operation
48+
# batch.records.update("account", account_id, {"telephone1": "555-9999"})
49+
# batch.records.delete("account", old_id)
50+
51+
result = batch.execute()
52+
53+
print(f"[OK] Total: {len(result.responses)}, Succeeded: {len(result.succeeded)}, Failed: {len(result.failed)}")
54+
for guid in result.created_ids:
55+
print(f"[OK] Created: {guid}")
56+
for item in result.failed:
57+
print(f"[ERR] {item.status_code}: {item.error_message}")
58+
59+
60+
# ---------------------------------------------------------------------------
61+
# Example 2: Transactional changeset with content-ID chaining
62+
# ---------------------------------------------------------------------------
63+
64+
print("\n[INFO] Example 2: Transactional changeset")
65+
66+
batch = client.batch.new()
67+
68+
with batch.changeset() as cs:
69+
# Each create() returns a "$n" reference usable in subsequent operations
70+
lead_ref = cs.records.create(
71+
"lead",
72+
{"firstname": "Ada", "lastname": "Lovelace"},
73+
)
74+
contact_ref = cs.records.create("contact", {"firstname": "Ada"})
75+
76+
# Reference the newly created lead and contact in the account
77+
cs.records.create(
78+
"account",
79+
{
80+
"name": "Babbage & Co.",
81+
"originatingleadid@odata.bind": lead_ref,
82+
"primarycontactid@odata.bind": contact_ref,
83+
},
84+
)
85+
86+
# Update using a content-ID reference as the record_id
87+
cs.records.update("contact", contact_ref, {"lastname": "Lovelace"})
88+
89+
result = batch.execute()
90+
91+
if result.has_errors:
92+
print("[ERR] Changeset rolled back")
93+
for item in result.failed:
94+
print(f" {item.status_code}: {item.error_message}")
95+
else:
96+
print(f"[OK] {len(result.created_ids)} records created atomically")
97+
98+
99+
# ---------------------------------------------------------------------------
100+
# Example 3: Table metadata operations in a batch
101+
# ---------------------------------------------------------------------------
102+
103+
print("\n[INFO] Example 3: Table metadata operations")
104+
105+
batch = client.batch.new()
106+
107+
# Create a new custom table
108+
batch.tables.create(
109+
"new_Product",
110+
{"new_Price": "decimal", "new_InStock": "bool"},
111+
solution="MySolution",
112+
)
113+
114+
# Read table metadata
115+
batch.tables.get("new_Product")
116+
117+
# List all non-private tables
118+
batch.tables.list()
119+
120+
result = batch.execute()
121+
print(f"[OK] Table ops: {[(r.status_code, r.is_success) for r in result.responses]}")
122+
123+
124+
# ---------------------------------------------------------------------------
125+
# Example 4: SQL query in a batch
126+
# ---------------------------------------------------------------------------
127+
128+
print("\n[INFO] Example 4: SQL query in batch")
129+
130+
batch = client.batch.new()
131+
batch.query.sql("SELECT TOP 5 accountid, name FROM account ORDER BY name")
132+
133+
result = batch.execute()
134+
if result.responses and result.responses[0].is_success and result.responses[0].body:
135+
rows = result.responses[0].body.get("value", [])
136+
print(f"[OK] Retrieved {len(rows)} accounts")
137+
for row in rows:
138+
print(f" {row.get('name')}")
139+
140+
141+
# ---------------------------------------------------------------------------
142+
# Example 5: Mixed batch — changeset writes + standalone GETs
143+
# ---------------------------------------------------------------------------
144+
145+
print("\n[INFO] Example 5: Mixed batch")
146+
147+
# Assume account_id exists
148+
# batch = client.batch.new()
149+
#
150+
# with batch.changeset() as cs:
151+
# cs.records.update("account", account_id, {"statecode": 0})
152+
#
153+
# batch.records.get("account", account_id, select=["name", "statecode"])
154+
#
155+
# result = batch.execute()
156+
# update_response = result.responses[0]
157+
# account_data = result.responses[1]
158+
# if account_data.is_success and account_data.body:
159+
# print(f"Account: {account_data.body.get('name')}")
160+
161+
162+
# ---------------------------------------------------------------------------
163+
# Example 6: Continue on error
164+
# ---------------------------------------------------------------------------
165+
166+
print("\n[INFO] Example 6: Continue on error")
167+
168+
batch = client.batch.new()
169+
batch.records.get("account", "nonexistent-guid-1111-1111-111111111111")
170+
batch.query.sql("SELECT TOP 1 name FROM account")
171+
172+
result = batch.execute(continue_on_error=True)
173+
print(f"[OK] Succeeded: {len(result.succeeded)}, Failed: {len(result.failed)}")
174+
for item in result.failed:
175+
print(f"[ERR] {item.status_code}: {item.error_message}")

‎src/PowerPlatform/Dataverse/claude_skill/dataverse-sdk-dev/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ This skill provides guidance for developers working on the PowerPlatform Dataver
1313

1414
### API Design
1515

16-
1. **Public methods in operation namespaces** - New public methods go in the appropriate namespace module under `src/PowerPlatform/Dataverse/operations/` (`records.py`, `query.py`, `tables.py`). The `client.py` file exposes these via namespace properties (`client.records`, `client.query`, `client.tables`). Public types and constants live in their own modules (e.g., `models/metadata.py`, `common/constants.py`)
16+
1. **Public methods in operation namespaces** - New public methods go in the appropriate namespace module under `src/PowerPlatform/Dataverse/operations/` (`records.py`, `query.py`, `tables.py`, `batch.py`). The `client.py` file exposes these via namespace properties (`client.records`, `client.query`, `client.tables`, `client.batch`). Public types and constants live in their own modules (e.g., `models/metadata.py`, `models/batch.py`, `common/constants.py`)
1717
2. **Every public method needs README example** - Public API methods must have examples in README.md
1818
3. **Reuse existing APIs** - Always check if an existing method can be used before making direct Web API calls
1919
4. **Update documentation** when adding features - Keep README and SKILL files (both copies) in sync

0 commit comments

Comments
 (0)