Skip to content

Commit e772134

Browse files
User/tpellissier/use logicalname (#19)
* Refactor crud ops to use LogicalName as input * self PR review * PR comments * merge with optionset change * extra entity_set --------- Co-authored-by: Tim Pellissier <tpellissier@microsoft.com>
1 parent 0226a22 commit e772134

9 files changed

Lines changed: 354 additions & 259 deletions

File tree

README.md

Lines changed: 39 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,11 @@
33
A Python package allowing developers to connect to Dataverse environments for DDL / DML operations.
44

55
- Read (SQL) — Execute constrained read-only SQL via the Dataverse Web API `?sql=` parameter. Returns `list[dict]`.
6-
- OData CRUD — Unified methods `create(entity, record|records)`, `update(entity, id|ids, patch|patches)`, `delete(entity, id|ids)` plus `get` / `get_multiple`.
6+
- OData CRUD — Unified methods `create(logical_name, record|records)`, `update(logical_name, id|ids, patch|patches)`, `delete(logical_name, id|ids)` plus `get` / `get_multiple`.
77
- Bulk create — Pass a list of records to `create(...)` to invoke the bound `CreateMultiple` action; returns `list[str]` of GUIDs. If any payload omits `@odata.type` the SDK resolves and stamps it (cached).
88
- Bulk update — Provide a list of IDs with a single patch (broadcast) or a list of per‑record patches to `update(...)`; internally uses the bound `UpdateMultiple` action; returns nothing. Each record must include the primary key attribute when sent to UpdateMultiple.
99
- Retrieve multiple (paging) — Generator-based `get_multiple(...)` that yields pages, supports `$top` and Prefer: `odata.maxpagesize` (`page_size`).
10-
- Upload files — Call `upload_file(entity_set, ...)` and a upload method will be auto picked (user can also overwrite the upload mode). See https://learn.microsoft.com/en-us/power-apps/developer/data-platform/file-column-data?tabs=sdk#upload-files
10+
- Upload files — Call `upload_file(logical_name, ...)` and an upload method will be auto picked (you can override the mode). See https://learn.microsoft.com/en-us/power-apps/developer/data-platform/file-column-data?tabs=sdk#upload-files
1111
- Metadata helpers — Create/inspect/delete simple custom tables (EntityDefinitions + Attributes).
1212
- Pandas helpers — Convenience DataFrame oriented wrappers for quick prototyping/notebooks.
1313
- Auth — Azure Identity (`TokenCredential`) injection.
@@ -17,8 +17,8 @@ A Python package allowing developers to connect to Dataverse environments for DD
1717
- Simple `DataverseClient` facade for CRUD, SQL (read-only), and table metadata.
1818
- SQL-over-API: Constrained SQL (single SELECT with limited WHERE/TOP/ORDER BY) via native Web API `?sql=` parameter.
1919
- Table metadata ops: create simple custom tables (supports string/int/decimal/float/datetime/bool/optionset) and delete them.
20-
- Bulk create via `CreateMultiple` (collection-bound) by passing `list[dict]` to `create(entity_set, payloads)`; returns list of created IDs.
21-
- Bulk update via `UpdateMultiple` (invoked internally) by calling unified `update(entity_set, ids, patch|patches)`; returns nothing.
20+
- Bulk create via `CreateMultiple` (collection-bound) by passing `list[dict]` to `create(logical_name, payloads)`; returns list of created IDs.
21+
- Bulk update via `UpdateMultiple` (invoked internally) by calling unified `update(logical_name, ids, patch|patches)`; returns nothing.
2222
- Retrieve multiple with server-driven paging: `get_multiple(...)` yields lists (pages) following `@odata.nextLink`. Control total via `$top` and per-page via `page_size` (Prefer: `odata.maxpagesize`).
2323
- Upload files, using either a single request (supports file size up to 128 MB) or chunk upload under the hood
2424
- Optional pandas integration (`PandasODataClient`) for DataFrame based create / get / query.
@@ -32,23 +32,23 @@ Auth:
3232

3333
| Method | Signature (simplified) | Returns | Notes |
3434
|--------|------------------------|---------|-------|
35-
| `create` | `create(entity_set, record_dict)` | `list[str]` (len 1) | Single create; GUID from `OData-EntityId`. |
36-
| `create` | `create(entity_set, list[record_dict])` | `list[str]` | Uses `CreateMultiple`; stamps `@odata.type` if missing. |
37-
| `get` | `get(entity_set, id)` | `dict` | One record; supply GUID (with/without parentheses). |
38-
| `get_multiple` | `get_multiple(entity_set, ..., page_size=None)` | `Iterable[list[dict]]` | Pages yielded (non-empty only). |
39-
| `update` | `update(entity_set, id, patch)` | `None` | Single update; no representation returned. |
40-
| `update` | `update(entity_set, list[id], patch)` | `None` | Broadcast; same patch applied to all IDs. Calls UpdateMultiple web API internally. |
41-
| `update` | `update(entity_set, list[id], list[patch])` | `None` | 1:1 patches; lengths must match. Calls UpdateMultiple web API internally. |
42-
| `delete` | `delete(entity_set, id)` | `None` | Delete one record. |
43-
| `delete` | `delete(entity_set, list[id])` | `None` | Delete many (sequential). |
35+
| `create` | `create(logical_name, record_dict)` | `list[str]` (len 1) | Single create; GUID from `OData-EntityId`. |
36+
| `create` | `create(logical_name, list[record_dict])` | `list[str]` | Uses `CreateMultiple`; stamps `@odata.type` if missing. |
37+
| `get` | `get(logical_name, id)` | `dict` | One record; supply GUID (with/without parentheses). |
38+
| `get_multiple` | `get_multiple(logical_name, ..., page_size=None)` | `Iterable[list[dict]]` | Pages yielded (non-empty only). |
39+
| `update` | `update(logical_name, id, patch)` | `None` | Single update; no representation returned. |
40+
| `update` | `update(logical_name, list[id], patch)` | `None` | Broadcast; same patch applied to all IDs (UpdateMultiple). |
41+
| `update` | `update(logical_name, list[id], list[patch])` | `None` | 1:1 patches; lengths must match (UpdateMultiple). |
42+
| `delete` | `delete(logical_name, id)` | `None` | Delete one record. |
43+
| `delete` | `delete(logical_name, list[id])` | `None` | Delete many (sequential). |
4444
| `query_sql` | `query_sql(sql)` | `list[dict]` | Constrained read-only SELECT via `?sql=`. |
45-
| `create_table` | `create_table(name, schema)` | `dict` | Creates custom table + columns. |
46-
| `get_table_info` | `get_table_info(name)` | `dict | None` | Basic table metadata. |
45+
| `create_table` | `create_table(tablename, schema)` | `dict` | Creates custom table + columns. Friendly name (e.g. `SampleItem`) becomes schema `new_SampleItem`; explicit schema name (contains `_`) used as-is. |
46+
| `get_table_info` | `get_table_info(schema_name)` | `dict | None` | Basic table metadata by schema name (e.g. `new_SampleItem`). Friendly names not auto-converted. |
4747
| `list_tables` | `list_tables()` | `list[dict]` | Lists non-private tables. |
48-
| `delete_table` | `delete_table(name)` | `None` | Drops custom table. |
49-
| `PandasODataClient.create_df` | `create_df(entity_set, series)` | `str` | Returns GUID (wrapper). |
50-
| `PandasODataClient.update` | `update(entity_set, id, series)` | `None` | Ignores empty Series. |
51-
| `PandasODataClient.get_ids` | `get_ids(entity_set, ids, select=None)` | `DataFrame` | One row per ID (errors inline). |
48+
| `delete_table` | `delete_table(tablename)` | `None` | Drops custom table. Accepts friendly or schema name; friendly converted to `new_<PascalCase>`. |
49+
| `PandasODataClient.create_df` | `create_df(logical_name, series)` | `str` | Create one record (returns GUID). |
50+
| `PandasODataClient.update` | `update(logical_name, id, series)` | `None` | Returns None; ignored if Series empty. |
51+
| `PandasODataClient.get_ids` | `get_ids(logical_name, ids, select=None)` | `DataFrame` | One row per ID (errors inline). |
5252
| `PandasODataClient.query_sql_df` | `query_sql_df(sql)` | `DataFrame` | DataFrame for SQL results. |
5353

5454
Guidelines:
@@ -128,30 +128,30 @@ base_url = "https://yourorg.crm.dynamics.com"
128128
client = DataverseClient(base_url=base_url, credential=DefaultAzureCredential())
129129

130130
# Create (returns list[str] of new GUIDs)
131-
account_id = client.create("accounts", {"name": "Acme, Inc.", "telephone1": "555-0100"})[0]
131+
account_id = client.create("account", {"name": "Acme, Inc.", "telephone1": "555-0100"})[0]
132132

133133
# Read
134-
account = client.get("accounts", account_id)
134+
account = client.get("account", account_id)
135135

136136
# Update (returns None)
137-
client.update("accounts", account_id, {"telephone1": "555-0199"})
137+
client.update("account", account_id, {"telephone1": "555-0199"})
138138

139139
# Bulk update (broadcast) – apply same patch to several IDs
140-
ids = client.create("accounts", [
140+
ids = client.create("account", [
141141
{"name": "Contoso"},
142142
{"name": "Fabrikam"},
143143
])
144-
client.update("accounts", ids, {"telephone1": "555-0200"}) # broadcast patch
144+
client.update("account", ids, {"telephone1": "555-0200"}) # broadcast patch
145145

146146
# Bulk update (1:1) – list of patches matches list of IDs
147-
client.update("accounts", ids, [
147+
client.update("account", ids, [
148148
{"telephone1": "555-1200"},
149149
{"telephone1": "555-1300"},
150150
])
151151
print({"multi_update": "ok"})
152152

153153
# Delete
154-
client.delete("accounts", account_id)
154+
client.delete("account", account_id)
155155

156156
# SQL (read-only) via Web API `?sql=`
157157
rows = client.query_sql("SELECT TOP 3 accountid, name FROM account ORDER BY createdon DESC")
@@ -160,7 +160,7 @@ for r in rows:
160160

161161
## Bulk create (CreateMultiple)
162162

163-
Pass a list of payloads to `create(entity_set, payloads)` to invoke the collection-bound `Microsoft.Dynamics.CRM.CreateMultiple` action. The method returns `list[str]` of created record IDs.
163+
Pass a list of payloads to `create(logical_name, payloads)` to invoke the collection-bound `Microsoft.Dynamics.CRM.CreateMultiple` action. The method returns `list[str]` of created record IDs.
164164

165165
```python
166166
# Bulk create accounts (returns list of GUIDs)
@@ -169,7 +169,7 @@ payloads = [
169169
{"name": "Fabrikam"},
170170
{"name": "Northwind"},
171171
]
172-
ids = client.create("accounts", payloads)
172+
ids = client.create("account", payloads)
173173
assert isinstance(ids, list) and all(isinstance(x, str) for x in ids)
174174
print({"created_ids": ids})
175175
```
@@ -180,10 +180,10 @@ Use the unified `update` method for both single and bulk scenarios:
180180

181181
```python
182182
# Broadcast
183-
client.update("accounts", ids, {"telephone1": "555-0200"})
183+
client.update("account", ids, {"telephone1": "555-0200"})
184184

185185
# 1:1 patches (length must match)
186-
client.update("accounts", ids, [
186+
client.update("account", ids, [
187187
{"telephone1": "555-1200"},
188188
{"telephone1": "555-1300"},
189189
])
@@ -216,12 +216,11 @@ Notes:
216216

217217
## Retrieve multiple with paging
218218

219-
Use `get_multiple(entity_set, ...)` to stream results page-by-page. You can cap total results with `$top` and hint the per-page size with `page_size` (sets Prefer: `odata.maxpagesize`).
219+
Use `get_multiple(logical_name, ...)` to stream results page-by-page. You can cap total results with `$top` and hint the per-page size with `page_size` (sets Prefer: `odata.maxpagesize`).
220220

221221
```python
222-
# Iterate pages of accounts ordered by name, selecting a few columns
223222
pages = client.get_multiple(
224-
"accounts",
223+
"account",
225224
select=["accountid", "name", "createdon"],
226225
orderby=["name asc"],
227226
top=10, # stop after 10 total rows (optional)
@@ -235,8 +234,8 @@ for page in pages: # each page is a list[dict]
235234
print({"total_rows": total})
236235
```
237236

238-
Parameters (all optional except `entity_set`)
239-
- `entity_set`: str — Entity set (plural logical name), e.g., `"accounts"`.
237+
Parameters (all optional except `logical_name`)
238+
- `logical_name`: str — Logical (singular) name, e.g., `"account"`.
240239
- `select`: list[str] | None — Columns -> `$select` (comma joined).
241240
- `filter`: str | None — OData `$filter` expression (e.g., `contains(name,'Acme') and statecode eq 0`).
242241
- `orderby`: list[str] | None — Sort expressions -> `$orderby` (comma joined).
@@ -257,7 +256,7 @@ Example (all parameters + expected response)
257256

258257
```python
259258
pages = client.get_multiple(
260-
"accounts",
259+
"account",
261260
select=["accountid", "name", "createdon", "primarycontactid"],
262261
filter="contains(name,'Acme') and statecode eq 0",
263262
orderby=["name asc", "createdon desc"],
@@ -320,7 +319,6 @@ info = client.create_table(
320319
},
321320
)
322321

323-
entity_set = info["entity_set_name"] # e.g., "new_sampleitems"
324322
logical = info["entity_logical_name"] # e.g., "new_sampleitem"
325323

326324
# Create a record in the new table
@@ -329,11 +327,11 @@ prefix = "new"
329327
name_attr = f"{prefix}_name"
330328
id_attr = f"{logical}id"
331329

332-
rec = client.create(entity_set, {name_attr: "Sample A"})
330+
rec_id = client.create(logical, {name_attr: "Sample A"})[0]
333331

334332
# Clean up
335-
client.delete(entity_set, rec[id_attr]) # delete record
336-
client.delete_table("SampleItem") # delete the table
333+
client.delete(logical, rec_id) # delete record
334+
client.delete_table("SampleItem") # delete table (friendly name or explicit schema new_SampleItem)
337335
```
338336

339337
Notes:
@@ -346,7 +344,7 @@ Notes:
346344

347345
### Pandas helpers
348346

349-
See `examples/quickstart_pandas.py` for a DataFrame workflow via `PandasODataClient`.
347+
`PandasODataClient` is a thin wrapper around the low-level client. All methods accept logical (singular) names (e.g. `account`, `new_sampleitem`), not entity set (plural) names. See `examples/quickstart_pandas.py` for a DataFrame workflow.
350348

351349
VS Code Tasks
352350
- Install deps: `Install deps (pip)`
@@ -356,7 +354,6 @@ VS Code Tasks
356354
- No general-purpose OData batching, upsert, or association operations yet.
357355
- `DeleteMultiple` not yet exposed.
358356
- Minimal retry policy in library (network-error only); examples include additional backoff for transient Dataverse consistency.
359-
- Entity naming conventions in Dataverse: for bulk create the SDK resolves logical names from entity set metadata.
360357

361358
## Contributing
362359

examples/quickstart.py

Lines changed: 17 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -141,8 +141,7 @@ class Status(IntEnum):
141141
pass
142142
# Fail fast: all operations must use the custom table
143143
sys.exit(1)
144-
entity_set = table_info.get("entity_set_name")
145-
logical = table_info.get("entity_logical_name") or entity_set.rstrip("s")
144+
logical = table_info.get("entity_logical_name")
146145

147146
# Derive attribute logical name prefix from the entity logical name (segment before first underscore)
148147
attr_prefix = logical.split("_", 1)[0] if "_" in logical else logical
@@ -261,15 +260,15 @@ def _status_value_for_index(idx: int, use_french: bool):
261260

262261
try:
263262
# Single create returns list[str] (length 1)
264-
log_call(f"client.create('{entity_set}', single_payload)")
265-
single_ids = backoff_retry(lambda: client.create(entity_set, single_payload))
263+
log_call(f"client.create('{logical}', single_payload)")
264+
single_ids = backoff_retry(lambda: client.create(logical, single_payload))
266265
if not (isinstance(single_ids, list) and len(single_ids) == 1):
267266
raise RuntimeError("Unexpected single create return shape (expected one-element list)")
268267
record_ids.extend(single_ids)
269268

270269
# Multi create returns list[str]
271-
log_call(f"client.create('{entity_set}', multi_payloads)")
272-
multi_ids = backoff_retry(lambda: client.create(entity_set, multi_payloads))
270+
log_call(f"client.create('{logical}', multi_payloads)")
271+
multi_ids = backoff_retry(lambda: client.create(logical, multi_payloads))
273272
if isinstance(multi_ids, list):
274273
record_ids.extend([mid for mid in multi_ids if isinstance(mid, str)])
275274
else:
@@ -301,8 +300,8 @@ def _status_value_for_index(idx: int, use_french: bool):
301300
if record_ids:
302301
# Read only the first record and move on
303302
target = record_ids[0]
304-
log_call(f"client.get('{entity_set}', '{target}')")
305-
rec = backoff_retry(lambda: client.get(entity_set, target))
303+
log_call(f"client.get('{logical}', '{target}')")
304+
rec = backoff_retry(lambda: client.get(logical, target))
306305
print_line_summaries("Read record summary:", [{"id": target, **summary_from_record(rec)}])
307306
else:
308307
raise RuntimeError("No record created; skipping read.")
@@ -348,10 +347,10 @@ def _status_value_for_index(idx: int, use_french: bool):
348347
pause("Execute Update")
349348

350349
# Update only the chosen record and summarize
351-
log_call(f"client.update('{entity_set}', '{target_id}', update_data)")
350+
log_call(f"client.update('{logical}', '{target_id}', update_data)")
352351
# Perform update (returns None); follow-up read to verify
353-
backoff_retry(lambda: client.update(entity_set, target_id, update_data))
354-
verify_rec = backoff_retry(lambda: client.get(entity_set, target_id))
352+
backoff_retry(lambda: client.update(logical, target_id, update_data))
353+
verify_rec = backoff_retry(lambda: client.get(logical, target_id))
355354
for k, v in expected_checks.items():
356355
assert verify_rec.get(k) == v, f"Field {k} expected {v}, got {verify_rec.get(k)}"
357356
got = verify_rec.get(amount_key)
@@ -376,16 +375,16 @@ def _status_value_for_index(idx: int, use_french: bool):
376375
id_key: rid,
377376
count_key: 100 + idx, # new count values
378377
})
379-
log_call(f"client.update('{entity_set}', <{len(bulk_updates)} ids>, <patches>)")
378+
log_call(f"client.update('{logical}', <{len(bulk_updates)} ids>, <patches>)")
380379
# Unified update handles multiple via list of patches (returns None)
381-
backoff_retry(lambda: client.update(entity_set, subset, bulk_updates))
380+
backoff_retry(lambda: client.update(logical, subset, bulk_updates))
382381
print({"bulk_update_requested": len(bulk_updates), "bulk_update_completed": True})
383382
# Verify the updated count values by refetching the subset
384383
verification = []
385384
# Small delay to reduce risk of any brief replication delay
386385
time.sleep(1)
387386
for rid in subset:
388-
rec = backoff_retry(lambda rid=rid: client.get(entity_set, rid))
387+
rec = backoff_retry(lambda rid=rid: client.get(logical, rid))
389388
verification.append({
390389
"id": rid,
391390
"count": rec.get(count_key),
@@ -443,7 +442,7 @@ def run_paging_demo(label: str, *, top: Optional[int], page_size: Optional[int])
443442
_select = [id_key, code_key, amount_key, when_key, status_key]
444443
_orderby = [f"{code_key} asc"]
445444
for page in client.get_multiple(
446-
entity_set,
445+
logical,
447446
select=_select,
448447
filter=None,
449448
orderby=_orderby,
@@ -496,15 +495,15 @@ def run_paging_demo(label: str, *, top: Optional[int], page_size: Optional[int])
496495
try:
497496
if record_ids:
498497
max_workers = min(8, len(record_ids))
499-
log_call(f"concurrent delete {len(record_ids)} items from '{entity_set}' (workers={max_workers})")
498+
log_call(f"concurrent delete {len(record_ids)} items from '{logical}' (workers={max_workers})")
500499

501500
successes: list[str] = []
502501
failures: list[dict] = []
503502

504503
def _del_one(rid: str) -> tuple[str, bool, str | None]:
505504
try:
506-
log_call(f"client.delete('{entity_set}', '{rid}')")
507-
backoff_retry(lambda: client.delete(entity_set, rid))
505+
log_call(f"client.delete('{logical}', '{rid}')")
506+
backoff_retry(lambda: client.delete(logical, rid))
508507
return (rid, True, None)
509508
except Exception as ex:
510509
return (rid, False, str(ex))

0 commit comments

Comments
 (0)