Add client.dataframe namespace for pandas DataFrame CRUD operations (#98)

## Summary

Adds a `client.dataframe` namespace with pandas DataFrame/Series
wrappers for all CRUD operations, plus two advanced example scripts, and
a minor SDK enhancement for table metadata. Users can now query, create,
update, and delete Dataverse records using DataFrame-native inputs and
outputs -- no manual dict conversion required.

## Quick Example

```python
import pandas as pd
from azure.identity import InteractiveBrowserCredential
from PowerPlatform.Dataverse.client import DataverseClient

credential = InteractiveBrowserCredential()

with DataverseClient("https://yourorg.crm.dynamics.com", credential) as client:

    # Query records as a DataFrame (all pages consolidated automatically)
    df = client.dataframe.get("account", select=["name", "telephone1"], top=5)

    # Create records from a DataFrame (returns Series of GUIDs)
    new_records = pd.DataFrame([
        {"name": "Acme Corp", "telephone1": "555-9000"},
        {"name": "Globex Inc", "telephone1": "555-9001"},
    ])
    new_records["accountid"] = client.dataframe.create("account", new_records)

    # Update records (NaN/None skipped by default; use clear_nulls=True to clear fields)
    new_records["telephone1"] = ["555-1111", "555-2222"]
    client.dataframe.update("account", new_records[["accountid", "telephone1"]], id_column="accountid")

    # Delete records
    client.dataframe.delete("account", new_records["accountid"])
```

## Changes

### DataFrame CRUD (`client.dataframe` namespace)
| File | Description |
|------|-------------|
| `src/.../operations/dataframe.py` | `DataFrameOperations` class:
`get()`, `create()`, `update()`, `delete()` |
| `src/.../utils/_pandas.py` | `dataframe_to_records()` helper --
normalizes NumPy, datetime, NaN/None |
| `client.py` | Added `self.dataframe = DataFrameOperations(self)` |
| `pyproject.toml` | Added `pandas>=2.0.0` required dependency |
| `README.md` | DataFrame usage examples |
| `operations/__init__.py` | Cleanup (`__all__ = []`) |

### SDK Enhancement: TableInfo primary column metadata (fixes #148)
| File | Description |
|------|-------------|
| `src/.../data/_odata.py` | `_get_entity_by_table_schema_name()` and
`_get_table_info()` now select `PrimaryNameAttribute` and
`PrimaryIdAttribute` from EntityDefinitions |
| `src/.../models/table_info.py` | `TableInfo` includes
`primary_name_attribute` and `primary_id_attribute` fields |
| `tests/unit/models/test_table_info.py` | Tests for new fields in
`from_dict`, `from_api_response`, and legacy key access |

### Advanced Examples
| File | Description |
|------|-------------|
| `examples/advanced/dataframe_operations.py` | DataFrame CRUD
walkthrough |
| `examples/advanced/prodev_quick_start.py` | Pro-dev: 4-table system
with relationships, DataFrame CRUD, query/analyze. Uses
`result.primary_name_attribute` from `tables.create()` |
| `examples/advanced/datascience_risk_assessment.py` | Data science:
5-step risk pipeline with 3 LLM provider options (Azure AI Inference,
OpenAI, GitHub Copilot SDK), matplotlib charts |

### Test Files
| File | Tests |
|------|-------|
| `test_dataframe_operations.py` | 44 |
| `test_client_dataframe.py` | 26 |
| `test_pandas_helpers.py` | 33 |
| `test_table_info.py` | +1 (primary fields) |

## API Design

| Method | Input | Output | Underlying API |
|--------|-------|--------|----------------|
| `get(table, ...)` | OData params | `pd.DataFrame` | `records.get()` |
| `get(table, record_id=...)` | GUID | 1-row `pd.DataFrame` |
`records.get()` |
| `create(table, df)` | `pd.DataFrame` | `pd.Series` of GUIDs |
`CreateMultiple` |
| `update(table, df, id_column)` | `pd.DataFrame` | `None` |
`UpdateMultiple` |
| `delete(table, ids)` | `pd.Series` | `Optional[str]` | `BulkDelete` |

### Design Decisions
- **`clear_nulls`**: Default `False` skips NaN (field unchanged). `True`
sends null to clear.
- **Type normalization**: np.int64/float64/bool_/ndarray,
datetime/date/np.datetime64, pd.Timestamp -- all auto-converted.
- **ID validation**: Strip whitespace, report DataFrame index labels in
errors.
- **pandas required**: Core dependency by team decision.

## Test Results

```
396 passed, 8 warnings (pre-existing deprecation), 4 subtests passed
```

| Check | Result |
|-------|--------|
| Full test suite | 396 pass, 0 fail |
| mypy | 0 errors |
| black / isort | Clean |
| E2E prodev | PASS (4 tables, 3 relationships, 13 records, full CRUD
cycle) |
| E2E datascience | PASS (7 accounts, 3 cases, 8 opportunities, risk
scoring, charts) |
| PR review threads | 54/54 resolved |

## Issues Addressed
- Fixes #148: `tables.create()` now exposes `primary_name_attribute` via
Dataverse metadata
- Followup #147: `QueryBuilder.to_dataframe()` tracked for future work

---------

Co-authored-by: Saurabh Badenkal <sbadenkal@microsoft.com>

Author: zhaodongwang-msft

.claude/skills/dataverse-sdk-use/SKILL.md

@@ -30,6 +30,9 @@ The SDK supports Dataverse's native bulk operations: Pass lists to `create()`, `
 - Control page size with `page_size` parameter
 - Use `top` parameter to limit total records returned
 
+### DataFrame Support
+- DataFrame operations are accessed via the `client.dataframe` namespace: `client.dataframe.get()`, `client.dataframe.create()`, `client.dataframe.update()`, `client.dataframe.delete()`
+
 ## Common Operations
 
 ### Import
@@ -129,7 +132,7 @@ client.records.update("account", [id1, id2, id3], {"industry": "Technology"})
 ```
 
 #### Upsert Records
-Creates or updates records identified by alternate keys. Single item → PATCH; multiple items → `UpsertMultiple` bulk action.
+Creates or updates records identified by alternate keys. Single item -> PATCH; multiple items -> `UpsertMultiple` bulk action.
 > **Prerequisite**: The table must have an alternate key configured in Dataverse for the columns used in `alternate_key`. Without it, Dataverse will reject the request with a 400 error.
 ```python
 from PowerPlatform.Dataverse.models.upsert import UpsertItem
@@ -171,6 +174,42 @@ client.records.delete("account", account_id)
 client.records.delete("account", [id1, id2, id3], use_bulk_delete=True)
 ```
 
+### DataFrame Operations
+
+The SDK provides DataFrame wrappers for all CRUD operations via the `client.dataframe` namespace, using pandas DataFrames and Series as input/output.
+
+```python
+import pandas as pd
+
+# Query records -- returns a single DataFrame
+df = client.dataframe.get("account", filter="statecode eq 0", select=["name"])
+print(f"Got {len(df)} rows")
+
+# Limit results with top for large tables
+df = client.dataframe.get("account", select=["name"], top=100)
+
+# Fetch single record as one-row DataFrame
+df = client.dataframe.get("account", record_id=account_id, select=["name"])
+
+# Create records from a DataFrame (returns a Series of GUIDs)
+new_accounts = pd.DataFrame([
+    {"name": "Contoso", "telephone1": "555-0100"},
+    {"name": "Fabrikam", "telephone1": "555-0200"},
+])
+new_accounts["accountid"] = client.dataframe.create("account", new_accounts)
+
+# Update records from a DataFrame (id_column identifies the GUID column)
+new_accounts["telephone1"] = ["555-0199", "555-0299"]
+client.dataframe.update("account", new_accounts, id_column="accountid")
+
+# Clear a field by setting clear_nulls=True (by default, NaN/None fields are skipped)
+df = pd.DataFrame([{"accountid": "guid-1", "websiteurl": None}])
+client.dataframe.update("account", df, id_column="accountid", clear_nulls=True)
+
+# Delete records by passing a Series of GUIDs
+client.dataframe.delete("account", new_accounts["accountid"])
+```
+
 ### SQL Queries
 
 SQL queries are **read-only** and support limited SQL syntax. A single SELECT statement with optional WHERE, TOP (integer literal), ORDER BY (column names only), and a simple table alias after FROM is supported. But JOIN and subqueries may not be. Refer to the Dataverse documentation for the current feature set.

.gitignore

@@ -25,3 +25,4 @@ Thumbs.db
 
 # Claude local settings
 .claude/*.local.json
+.claude/*.local.md

README.md

@@ -24,6 +24,7 @@ A Python client library for Microsoft Dataverse that provides a unified interfac
   - [Basic CRUD operations](#basic-crud-operations)
   - [Bulk operations](#bulk-operations)
   - [Upsert operations](#upsert-operations)
+  - [DataFrame operations](#dataframe-operations)
   - [Query data](#query-data)
   - [Table management](#table-management)
   - [Relationship management](#relationship-management)
@@ -39,6 +40,7 @@ A Python client library for Microsoft Dataverse that provides a unified interfac
 - **📊 SQL Queries**: Execute read-only SQL queries via the Dataverse Web API `?sql=` parameter  
 - **🏗️ Table Management**: Create, inspect, and delete custom tables and columns programmatically
 - **🔗 Relationship Management**: Create one-to-many and many-to-many relationships between tables with full metadata control
+- **🐼 DataFrame Support**: Pandas wrappers for all CRUD operations, returning DataFrames and Series
 - **📎 File Operations**: Upload files to Dataverse file columns with automatic chunking for large files
 - **🔐 Azure Identity**: Built-in authentication using Azure Identity credential providers with comprehensive support
 - **🛡️ Error Handling**: Structured exception hierarchy with detailed error context and retry guidance
@@ -232,6 +234,42 @@ client.records.upsert("account", [
 ])
 ```
 
+### DataFrame operations
+
+The SDK provides pandas wrappers for all CRUD operations via the `client.dataframe` namespace, using DataFrames and Series for input and output.
+
+```python
+import pandas as pd
+
+# Query records as a single DataFrame
+df = client.dataframe.get("account", filter="statecode eq 0", select=["name", "telephone1"])
+print(f"Found {len(df)} accounts")
+
+# Limit results with top for large tables
+df = client.dataframe.get("account", select=["name"], top=100)
+
+# Fetch a single record as a one-row DataFrame
+df = client.dataframe.get("account", record_id=account_id, select=["name"])
+
+# Create records from a DataFrame (returns a Series of GUIDs)
+new_accounts = pd.DataFrame([
+    {"name": "Contoso", "telephone1": "555-0100"},
+    {"name": "Fabrikam", "telephone1": "555-0200"},
+])
+new_accounts["accountid"] = client.dataframe.create("account", new_accounts)
+
+# Update records from a DataFrame (id_column identifies the GUID column)
+new_accounts["telephone1"] = ["555-0199", "555-0299"]
+client.dataframe.update("account", new_accounts, id_column="accountid")
+
+# Clear a field by setting clear_nulls=True (by default, NaN/None fields are skipped)
+df = pd.DataFrame([{"accountid": new_accounts["accountid"].iloc[0], "websiteurl": None}])
+client.dataframe.update("account", df, id_column="accountid", clear_nulls=True)
+
+# Delete records by passing a Series of GUIDs
+client.dataframe.delete("account", new_accounts["accountid"])
+```
+
 ### Query data
 
 ```python

examples/advanced/dataframe_operations.py

@@ -0,0 +1,174 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT license.
+
+"""
+PowerPlatform Dataverse Client - DataFrame Operations Walkthrough
+
+This example demonstrates how to use the pandas DataFrame extension methods
+for CRUD operations with Microsoft Dataverse.
+
+Prerequisites:
+    pip install PowerPlatform-Dataverse-Client
+    pip install azure-identity
+"""
+
+import sys
+import uuid
+
+import pandas as pd
+from azure.identity import InteractiveBrowserCredential
+
+from PowerPlatform.Dataverse.client import DataverseClient
+
+
+def main():
+    # -- Setup & Authentication ------------------------------------
+    base_url = input("Enter Dataverse org URL (e.g. https://yourorg.crm.dynamics.com): ").strip()
+    if not base_url:
+        print("[ERR] No URL entered; exiting.")
+        sys.exit(1)
+    base_url = base_url.rstrip("/")
+
+    print("[INFO] Authenticating via browser...")
+    credential = InteractiveBrowserCredential()
+
+    with DataverseClient(base_url, credential) as client:
+        _run_walkthrough(client)
+
+
+def _run_walkthrough(client):
+    table = input("Enter table schema name to use [default: account]: ").strip() or "account"
+    print(f"[INFO] Using table: {table}")
+
+    # Unique tag to isolate test records from existing data
+    tag = uuid.uuid4().hex[:8]
+    test_filter = f"contains(name,'{tag}')"
+    print(f"[INFO] Using tag '{tag}' to identify test records")
+
+    select_cols = ["name", "telephone1", "websiteurl", "lastonholdtime"]
+
+    # -- 1. Create records from a DataFrame ------------------------
+    print("\n" + "-" * 60)
+    print("1. Create records from a DataFrame")
+    print("-" * 60)
+
+    new_accounts = pd.DataFrame(
+        [
+            {
+                "name": f"Contoso_{tag}",
+                "telephone1": "555-0100",
+                "websiteurl": "https://contoso.com",
+                "lastonholdtime": pd.Timestamp("2024-06-15 10:30:00"),
+            },
+            {"name": f"Fabrikam_{tag}", "telephone1": "555-0200", "websiteurl": None, "lastonholdtime": None},
+            {
+                "name": f"Northwind_{tag}",
+                "telephone1": None,
+                "websiteurl": "https://northwind.com",
+                "lastonholdtime": pd.Timestamp("2024-12-01 08:00:00"),
+            },
+        ]
+    )
+    print(f"  Input DataFrame:\n{new_accounts.to_string(index=False)}\n")
+
+    # create_dataframe returns a Series of GUIDs aligned with the input rows
+    new_accounts["accountid"] = client.dataframe.create(table, new_accounts)
+    print(f"[OK] Created {len(new_accounts)} records")
+    print(f"  IDs: {new_accounts['accountid'].tolist()}")
+
+    # -- 2. Query records as a DataFrame -------------------------
+    print("\n" + "-" * 60)
+    print("2. Query records as a DataFrame")
+    print("-" * 60)
+
+    df_all = client.dataframe.get(table, select=select_cols, filter=test_filter)
+    print(f"[OK] Got {len(df_all)} records in one DataFrame")
+    print(f"  Columns: {list(df_all.columns)}")
+    print(f"{df_all.to_string(index=False)}")
+
+    # -- 3. Limit results with top ------------------------------
+    print("\n" + "-" * 60)
+    print("3. Limit results with top")
+    print("-" * 60)
+
+    df_top2 = client.dataframe.get(table, select=select_cols, filter=test_filter, top=2)
+    print(f"[OK] Got {len(df_top2)} records with top=2")
+    print(f"{df_top2.to_string(index=False)}")
+
+    # -- 4. Fetch a single record by ID ----------------------------
+    print("\n" + "-" * 60)
+    print("4. Fetch a single record by ID")
+    print("-" * 60)
+
+    first_id = new_accounts["accountid"].iloc[0]
+    print(f"  Fetching record {first_id}...")
+    single = client.dataframe.get(table, record_id=first_id, select=select_cols)
+    print(f"[OK] Single record DataFrame:\n{single.to_string(index=False)}")
+
+    # -- 5. Update records from a DataFrame ------------------------
+    print("\n" + "-" * 60)
+    print("5. Update records with different values per row")
+    print("-" * 60)
+
+    new_accounts["telephone1"] = ["555-1100", "555-1200", "555-1300"]
+    print(f"  New telephone numbers: {new_accounts['telephone1'].tolist()}")
+    client.dataframe.update(table, new_accounts[["accountid", "telephone1"]], id_column="accountid")
+    print("[OK] Updated 3 records")
+
+    # Verify the updates
+    verified = client.dataframe.get(table, select=select_cols, filter=test_filter)
+    print(f"  Verified:\n{verified.to_string(index=False)}")
+
+    # -- 6. Broadcast update (same value to all records) -----------
+    print("\n" + "-" * 60)
+    print("6. Broadcast update (same value to all records)")
+    print("-" * 60)
+
+    broadcast_df = new_accounts[["accountid"]].copy()
+    broadcast_df["websiteurl"] = "https://updated.example.com"
+    print(f"  Setting websiteurl to 'https://updated.example.com' for all {len(broadcast_df)} records")
+    client.dataframe.update(table, broadcast_df, id_column="accountid")
+    print("[OK] Broadcast update complete")
+
+    # Verify all records have the same websiteurl
+    verified = client.dataframe.get(table, select=select_cols, filter=test_filter)
+    print(f"  Verified:\n{verified.to_string(index=False)}")
+
+    # Default: NaN/None fields are skipped (not overridden on server)
+    print("\n  Updating with NaN values (default: clear_nulls=False, fields should stay unchanged)...")
+    sparse_df = pd.DataFrame(
+        [
+            {"accountid": new_accounts["accountid"].iloc[0], "telephone1": "555-9999", "websiteurl": None},
+        ]
+    )
+    client.dataframe.update(table, sparse_df, id_column="accountid")
+    verified = client.dataframe.get(table, select=select_cols, filter=test_filter)
+    print(f"  Verified (Contoso telephone1 updated, websiteurl unchanged):\n{verified.to_string(index=False)}")
+
+    # Opt-in: clear_nulls=True sends None as null to clear the field
+    print("\n  Clearing websiteurl for Contoso with clear_nulls=True...")
+    clear_df = pd.DataFrame([{"accountid": new_accounts["accountid"].iloc[0], "websiteurl": None}])
+    client.dataframe.update(table, clear_df, id_column="accountid", clear_nulls=True)
+    verified = client.dataframe.get(table, select=select_cols, filter=test_filter)
+    print(f"  Verified (Contoso websiteurl should be empty):\n{verified.to_string(index=False)}")
+
+    # -- 7. Delete records by passing a Series of GUIDs ------------
+    print("\n" + "-" * 60)
+    print("7. Delete records by passing a Series of GUIDs")
+    print("-" * 60)
+
+    print(f"  Deleting {len(new_accounts)} records...")
+    client.dataframe.delete(table, new_accounts["accountid"], use_bulk_delete=False)
+    print(f"[OK] Deleted {len(new_accounts)} records")
+
+    # Verify deletions - filter for our tagged records should return 0
+    remaining = client.dataframe.get(table, select=select_cols, filter=test_filter)
+    print(f"  Verified: {len(remaining)} test records remaining (expected 0)")
+
+    print("\n" + "=" * 60)
+    print("[OK] DataFrame operations walkthrough complete!")
+    print("=" * 60)
+
+
+if __name__ == "__main__":
+    main()