Prefix and logicalname fixes; readme/docstring updates

Author: tpellissier

README.md

@@ -4,7 +4,7 @@ A minimal Python SDK to use Microsoft Dataverse as a database for Azure AI Found
 
 - Read (SQL) — Execute read-only T‑SQL via the McpExecuteSqlQuery Custom API. Returns `list[dict]`.
 - OData CRUD — Thin wrappers over Dataverse Web API (create/get/update/delete).
-- Bulk create — Pass a list of records to `create(...)` to invoke the bound `CreateMultiple` action; returns `list[str]` of GUIDs.
+- Bulk create — Pass a list of records to `create(...)` to invoke the bound `CreateMultiple` action; returns `list[str]` of GUIDs. If `@odata.type` is absent the SDK resolves the logical name from metadata (cached).
 - Retrieve multiple (paging) — Generator-based `get_multiple(...)` that yields pages, supports `$top` and Prefer: `odata.maxpagesize` (`page_size`).
 - Metadata helpers — Create/inspect/delete simple custom tables (EntityDefinitions + Attributes).
 - Pandas helpers — Convenience DataFrame oriented wrappers for quick prototyping/notebooks.
@@ -127,6 +127,9 @@ print({"created_ids": ids})
 Notes:
 - The bulk create response typically includes IDs only; the SDK returns the list of GUID strings.
 - Single-record `create` still returns the full entity representation.
+- `@odata.type` handling: If any payload in the list omits `@odata.type`, the SDK performs a one-time metadata query (`EntityDefinitions?$filter=EntitySetName eq '<entity_set>'`) to resolve the logical name, caches it, and stamps each missing item with `Microsoft.Dynamics.CRM.<logical>`. If **all** payloads already include `@odata.type`, no metadata call is made.
+- The metadata lookup is per entity set and reused across subsequent multi-create calls in the same client instance (in-memory cache only).
+- You can explicitly set `@odata.type` yourself (e.g., for polymorphic scenarios); the SDK will not override it.
 
 ## Retrieve multiple with paging
 
@@ -149,16 +152,20 @@ for page in pages:         # each page is a list[dict]
 print({"total_rows": total})
 ```
 
-Parameters
-- `entity`: str — Entity set name (plural logical name), e.g., `"accounts"`.
-- `select`: list[str] | None — Columns to include; joined into `$select`.
-- `filter`: str | None — OData `$filter` expression (e.g., `"contains(name,'Acme') and statecode eq 0"`).
-- `orderby`: list[str] | None — Sort expressions (e.g., `["name asc", "createdon desc"]`).
-- `top`: int | None — Cap on total rows across all pages (sent as `$top` on first request).
-- `expand`: list[str] | None — Navigation expansions as raw OData strings. Example:
-	- `"primarycontactid($select=fullname,emailaddress1)"`
-	- `"parentaccountid($select=name)"`
-- `page_size`: int | None — Per-page hint via Prefer: `odata.maxpagesize=N`.
+Parameters (all optional except `entity_set`)
+- `entity_set`: str — Entity set (plural logical name), e.g., `"accounts"`.
+- `select`: list[str] | None — Columns -> `$select` (comma joined).
+- `filter`: str | None — OData `$filter` expression (e.g., `contains(name,'Acme') and statecode eq 0`).
+- `orderby`: list[str] | None — Sort expressions -> `$orderby` (comma joined).
+- `top`: int | None — Global cap via `$top` (applied on first request; service enforces across pages).
+- `expand`: list[str] | None — Navigation expansions -> `$expand`; pass raw clauses (e.g., `primarycontactid($select=fullname,emailaddress1)`).
+- `page_size`: int | None — Per-page hint using Prefer: `odata.maxpagesize=<N>` (not guaranteed; last page may be smaller).
+
+Return value & semantics
+- Returns a generator yielding non-empty pages (`list[dict]`). Empty pages are skipped.
+- Each yielded list corresponds to a `value` page from the Web API.
+- Iteration stops when no `@odata.nextLink` remains (or when `$top` satisfied server-side).
+- The generator does not materialize all results; pages are fetched lazily.
 
 Example (all parameters + expected response)
 
@@ -193,9 +200,11 @@ for page in pages:  # page is list[dict]
 ```
 
 Semantics:
-- `$top`: caps the total number of rows returned across all pages.
-- `page_size`: per-page size hint via Prefer: `odata.maxpagesize`; the service may return fewer/more.
-- The generator follows `@odata.nextLink` until exhausted or `$top` is satisfied.
+- `$select`, `$filter`, `$orderby`, `$expand`, `$top` map directly to corresponding OData query options on the first request.
+- `$top` caps total rows; the service may partition those rows across multiple pages.
+- `page_size` (Prefer: `odata.maxpagesize`) is a hint; the server decides actual page boundaries.
+- The generator follows `@odata.nextLink` until exhausted (service already accounts for `$top`).
+- Only non-empty pages are yielded; if the first response has no `value`, iteration ends immediately.
 ```
 
 ### Custom table (metadata) example
@@ -248,7 +257,7 @@ VS Code Tasks
 - No general-purpose OData batching, upsert, or association operations yet.
 - `DeleteMultiple`/`UpdateMultiple` are not exposed; quickstart may demonstrate faster deletes using client-side concurrency only.
 - Minimal retry policy in library (network-error only); examples include additional backoff for transient Dataverse consistency.
-- Entity naming conventions in Dataverse (schema/logical/entity set plural & publisher prefix) using the SDK is currently not well-defined
+- Entity naming conventions in Dataverse (schema/logical/entity set plural & publisher prefix) are only partially abstracted; for multi-create the SDK resolves logical names from entity set metadata.
 
 ## Contributing
 

examples/quickstart.py

@@ -21,7 +21,7 @@
 	sys.exit(1)
 
 base_url = entered.rstrip('/')
-delete_choice = input("Delete the SampleItem table at end? (Y/n): ").strip() or "y"
+delete_choice = input("Delete the new_SampleItem table at end? (Y/n): ").strip() or "y"
 delete_table_at_end = (str(delete_choice).lower() in ("y", "yes", "true", "1"))
 # Ask once whether to pause between steps during this run
 pause_choice = input("Pause between test steps? (y/N): ").strip() or "n"
@@ -68,11 +68,12 @@ def backoff_retry(op, *, delays=(0, 2, 5, 10, 20), retry_http_statuses=(400, 403
 table_info = None
 created_this_run = False
 
-# First check for existing table
-log_call("client.get_table_info('SampleItem')")
-existing = client.get_table_info("SampleItem")
-if existing:
-	table_info = existing
+# Check for existing table using list_tables
+log_call("client.list_tables()")
+tables = client.list_tables()
+existing_table = next((t for t in tables if t.get("SchemaName") == "new_SampleItem"), None)
+if existing_table:
+	table_info = client.get_table_info("new_SampleItem")
 	created_this_run = False
 	print({
 		"table": table_info.get("entity_schema"),
@@ -85,9 +86,9 @@ def backoff_retry(op, *, delays=(0, 2, 5, 10, 20), retry_http_statuses=(400, 403
 else:
 	# Create it since it doesn't exist
 	try:
-		log_call("client.create_table('SampleItem', schema={code,count,amount,when,active})")
+		log_call("client.create_table('new_SampleItem', schema={code,count,amount,when,active})")
 		table_info = client.create_table(
-			"SampleItem",
+			"new_SampleItem",
 			{
 				"code": "string",
 				"count": "int",
@@ -415,11 +416,11 @@ def _del_one(rid: str) -> tuple[str, bool, str | None]:
 print("Cleanup (Metadata):")
 if delete_table_at_end:
 	try:
-		log_call("client.get_table_info('SampleItem')")
-		info = client.get_table_info("SampleItem")
+		log_call("client.get_table_info('new_SampleItem')")
+		info = client.get_table_info("new_SampleItem")
 		if info:
-			log_call("client.delete_table('SampleItem')")
-			client.delete_table("SampleItem")
+			log_call("client.delete_table('new_SampleItem')")
+			client.delete_table("new_SampleItem")
 			print({"table_deleted": True})
 		else:
 			print({"table_deleted": False, "reason": "not found"})

src/dataverse_sdk/client.py

@@ -227,6 +227,16 @@ def delete_table(self, tablename: str) -> None:
         """
         self._get_odata().delete_table(tablename)
 
+    def list_tables(self) -> list[str]:
+        """List all custom tables in the Dataverse environment.
+
+        Returns
+        -------
+        list[str]
+            A list of table names.
+        """
+        return self._get_odata().list_tables()
+
 
 __all__ = ["DataverseClient"]
 

src/dataverse_sdk/odata.py

@@ -22,6 +22,8 @@ def __init__(self, auth, base_url: str, config=None) -> None:
             backoff=self.config.http_backoff,
             timeout=self.config.http_timeout,
         )
+        # Cache: entity set name -> logical name (resolved via metadata lookup)
+        self._entityset_logical_cache = {}
 
     def _headers(self) -> Dict[str, str]:
         """Build standard OData headers with bearer auth."""
@@ -42,13 +44,28 @@ def _request(self, method: str, url: str, **kwargs):
     def create(self, entity_set: str, data: Union[Dict[str, Any], List[Dict[str, Any]]]) -> Union[Dict[str, Any], List[str]]:
         """Create one or many records.
 
-        Behaviour:
-        - Single (dict): POST /{entity_set} with Prefer: return=representation and return the created record (dict).
-        - Multiple (list[dict]): POST bound action /{entity_set}/Microsoft.Dynamics.CRM.CreateMultiple
-          and return a list[str] of created record GUIDs (server only returns Ids for this action).
-
-        @odata.type is auto-inferred (Microsoft.Dynamics.CRM.<logical>) for each item when doing multi-create
-        if not already supplied.
+        Parameters
+        ----------
+        entity_set : str
+            Entity set (plural logical name), e.g. "accounts".
+        data : dict | list[dict]
+            Single entity payload or list of payloads for batch create.
+
+        Behaviour
+        ---------
+        - Single (dict): POST /{entity_set} with Prefer: return=representation. Returns created record (dict).
+        - Multiple (list[dict]): POST /{entity_set}/Microsoft.Dynamics.CRM.CreateMultiple. Returns list[str] of created GUIDs.
+
+        Multi-create logical name resolution
+        ------------------------------------
+        - If any payload omits ``@odata.type`` the client performs a metadata lookup (once per entity set, cached)
+          to resolve the logical name and stamps ``Microsoft.Dynamics.CRM.<logical>`` into missing payloads.
+        - If all payloads already include ``@odata.type`` no lookup or modification occurs.
+
+        Returns
+        -------
+        dict | list[str]
+            Created entity (single) or list of created IDs (multi).
         """
         if isinstance(data, dict):
             return self._create_single(entity_set, data)
@@ -71,17 +88,44 @@ def _create_single(self, entity_set: str, record: Dict[str, Any]) -> Dict[str, A
         except ValueError:
             return {}
 
-    def _infer_logical(self, entity_set: str) -> str:
-        # Basic heuristic: drop trailing 's'. (Metadata lookup could be added later.)
-        return entity_set[:-1] if entity_set.endswith("s") else entity_set
+    def _logical_from_entity_set(self, entity_set: str) -> str:
+        """Resolve logical name from an entity set using metadata (cached)."""
+        es = (entity_set or "").strip()
+        if not es:
+            raise ValueError("entity_set is required")
+        cached = self._entityset_logical_cache.get(es)
+        if cached:
+            return cached
+        url = f"{self.api}/EntityDefinitions"
+        params = {
+            "$select": "LogicalName,EntitySetName",
+            "$filter": f"EntitySetName eq '{es}'",
+        }
+        r = self._request("get", url, headers=self._headers(), params=params)
+        r.raise_for_status()
+        try:
+            body = r.json()
+            items = body.get("value", []) if isinstance(body, dict) else []
+        except ValueError:
+            items = []
+        if not items:
+            raise RuntimeError(f"Unable to resolve logical name for entity set '{es}'. Provide @odata.type explicitly.")
+        logical = items[0].get("LogicalName")
+        if not logical:
+            raise RuntimeError(f"Metadata response missing LogicalName for entity set '{es}'.")
+        self._entityset_logical_cache[es] = logical
+        return logical
 
     def _create_multiple(self, entity_set: str, records: List[Dict[str, Any]]) -> List[str]:
         if not all(isinstance(r, dict) for r in records):
             raise TypeError("All items for multi-create must be dicts")
-        logical = self._infer_logical(entity_set)
+        need_logical = any("@odata.type" not in r for r in records)
+        logical: Optional[str] = None
+        if need_logical:
+            logical = self._logical_from_entity_set(entity_set)
         enriched: List[Dict[str, Any]] = []
         for r in records:
-            if "@odata.type" in r:
+            if "@odata.type" in r or not logical:
                 enriched.append(r)
             else:
                 nr = r.copy()
@@ -104,7 +148,7 @@ def _create_multiple(self, entity_set: str, records: List[Dict[str, Any]]) -> Li
         ids = body.get("Ids")
         if isinstance(ids, list):
             return [i for i in ids if isinstance(i, str)]
-        # Future-proof: some environments might eventually return value/list of entities.
+
         value = body.get("value")
         if isinstance(value, list):
             # Extract IDs if possible
@@ -128,6 +172,22 @@ def _format_key(self, key: str) -> str:
         return f"({k})"
 
     def update(self, entity_set: str, key: str, data: Dict[str, Any]) -> Dict[str, Any]:
+        """Update an existing record and return the updated representation.
+
+        Parameters
+        ----------
+        entity_set : str
+            Entity set name (plural logical name).
+        key : str
+            Record GUID (with or without parentheses) or alternate key.
+        data : dict
+            Partial entity payload.
+
+        Returns
+        -------
+        dict
+            Updated record representation.
+        """
         url = f"{self.api}/{entity_set}{self._format_key(key)}"
         headers = self._headers().copy()
         headers["If-Match"] = "*"
@@ -137,13 +197,25 @@ def update(self, entity_set: str, key: str, data: Dict[str, Any]) -> Dict[str, A
         return r.json()
 
     def delete(self, entity_set: str, key: str) -> None:
+        """Delete a record by GUID or alternate key."""
         url = f"{self.api}/{entity_set}{self._format_key(key)}"
         headers = self._headers().copy()
         headers["If-Match"] = "*"
         r = self._request("delete", url, headers=headers)
         r.raise_for_status()
 
     def get(self, entity_set: str, key: str, select: Optional[str] = None) -> Dict[str, Any]:
+        """Retrieve a single record.
+
+        Parameters
+        ----------
+        entity_set : str
+            Entity set name.
+        key : str
+            Record GUID (with or without parentheses) or alternate key syntax.
+        select : str | None
+            Comma separated columns for $select.
+        """
         params = {}
         if select:
             params["$select"] = select
@@ -171,18 +243,20 @@ def get_multiple(
         select : list[str] | None
             Columns to select; joined with commas into $select.
         filter : str | None
-            OData $filter expression as a string.
+            ``$filter`` expression.
         orderby : list[str] | None
             Order expressions; joined with commas into $orderby.
         top : int | None
-            Max number of records across all pages. Passed as $top on the first request; the server will paginate via nextLink as needed.
+            Global cap via ``$top`` (applied on first request; server enforces across pages).
         expand : list[str] | None
-            Navigation properties to expand; joined with commas into $expand.
+            Navigation expansions -> ``$expand``; raw clauses accepted.
+        page_size : int | None
+            Hint for per-page size using Prefer: ``odata.maxpagesize``.
 
         Yields
         ------
         list[dict]
-            A page of records from the Web API (the "value" array for each page).
+            A non-empty page of entities (service ``value`` array). Empty pages are skipped.
         """
 
         # Build headers once; include odata.maxpagesize to force smaller pages for demos/testing
@@ -234,6 +308,23 @@ def _do_request(url: str, *, params: Optional[Dict[str, Any]] = None) -> Dict[st
 
     # --------------------------- SQL Custom API -------------------------
     def query_sql(self, tsql: str) -> list[dict[str, Any]]:
+        """Execute a read-only T-SQL query via the configured Custom API.
+
+        Parameters
+        ----------
+        tsql : str
+            SELECT-style Dataverse-supported T-SQL (read-only).
+
+        Returns
+        -------
+        list[dict]
+            Rows materialised as list of dictionaries (empty list if no rows).
+
+        Raises
+        ------
+        RuntimeError
+            If the Custom API response is missing the expected ``queryresult`` property or type is unexpected.
+        """
         payload = {"querytext": tsql}
         headers = self._headers()
         api_name = self.config.sql_api_name
@@ -392,20 +483,38 @@ def _attribute_payload(self, schema_name: str, dtype: str, *, is_primary_name: b
         return None
 
     def get_table_info(self, tablename: str) -> Optional[Dict[str, Any]]:
-        # Accept tablename as a display/logical root; infer a default schema using 'new_' if not provided.
-        # If caller passes a full SchemaName, use it as-is.
-        schema_name = tablename if "_" in tablename else f"new_{self._to_pascal(tablename)}"
-        entity_schema = schema_name
-        ent = self._get_entity_by_schema(entity_schema)
+        """Return basic metadata for a custom table if it exists.
+
+        Parameters
+        ----------
+        tablename : str
+            Friendly name or full schema name (with publisher prefix and underscore).
+
+        Returns
+        -------
+        dict | None
+            Metadata summary or ``None`` if not found.
+        """
+        ent = self._get_entity_by_schema(tablename)
         if not ent:
             return None
         return {
-            "entity_schema": ent.get("SchemaName") or entity_schema,
+            "entity_schema": ent.get("SchemaName") or tablename,
             "entity_logical_name": ent.get("LogicalName"),
             "entity_set_name": ent.get("EntitySetName"),
             "metadata_id": ent.get("MetadataId"),
             "columns_created": [],
         }
+    
+    def list_tables(self) -> List[Dict[str, Any]]:
+        """List all tables in the Dataverse, excluding private tables (IsPrivate=true)."""
+        url = f"{self.api}/EntityDefinitions"
+        params = {
+            "$filter": "IsPrivate eq false"
+        }
+        r = self._request("get", url, headers=self._headers(), params=params)
+        r.raise_for_status()
+        return r.json().get("value", [])
 
     def delete_table(self, tablename: str) -> None:
         schema_name = tablename if "_" in tablename else f"new_{self._to_pascal(tablename)}"