Add opt-in HTTP diagnostics logging with automatic header redaction

[Copilot]

Adds local file logging of HTTP request/response traffic to the SDK. Disabled by default (zero overhead); enabled by passing a LogConfig to DataverseConfig.

New modules

• core/log_config.py — LogConfig frozen dataclass: configurable log folder, file prefix (timestamp auto-appended), body capture limit, redacted headers, log level, and RotatingFileHandler rotation settings
• core/_http_logger.py — _HttpLogger: structured log formatting, case-insensitive header redaction (Authorization, Proxy-Authorization, OCP-Apim-Subscription-Key, X-MS-Authorization-Auxiliary), body truncation with byte limit, dedicated named logger (propagate=False to avoid side effects)

Wiring

• config.py — DataverseConfig gains log_config: Optional[LogConfig] = None (TYPE_CHECKING guard to avoid circular imports)
• _http.py — _HttpClient.__init__ accepts optional logger; _request() logs outbound request once (before retry loop), logs response with elapsed ms, logs transport errors on RequestException
• _odata.py — instantiates _HttpLogger from config.log_config and passes it through to _HttpClient

Usage

from PowerPlatform.Dataverse.client import DataverseClient
from PowerPlatform.Dataverse.core.config import DataverseConfig
from PowerPlatform.Dataverse.core.log_config import LogConfig

config = DataverseConfig(
    log_config=LogConfig(
        log_folder="./my_logs",
        log_file_prefix="crm_debug",  # → crm_debug_20260310_143022.log
        max_body_bytes=8192,
    )
)

with DataverseClient("https://myorg.crm.dynamics.com", credential, config=config) as client:
    client.records.create("account", {"name": "Contoso"})
    # All HTTP traffic logged; Authorization header → [REDACTED]

Sample log output:

[2026-03-10T14:30:22+0000] DEBUG >>> REQUEST  POST https://myorg.crm.dynamics.com/api/data/v9.2/accounts
    Headers: {'Authorization': '[REDACTED]', 'Content-Type': 'application/json', ...}
    Body:    {"name": "Contoso"}

[2026-03-10T14:30:22+0000] DEBUG <<< RESPONSE 201 POST https://myorg.crm.dynamics.com/api/data/v9.2/accounts (342.1ms)
    Headers: {'OData-EntityId': '...', ...}

Original prompt

Summary

Add opt-in local file logging of HTTP request/response diagnostics to the Dataverse Python SDK. When enabled via a LogConfig passed through DataverseConfig, every HTTP request and response is logged to timestamped .log files with automatic redaction of sensitive headers (e.g., Authorization bearer tokens).

Design

The implementation consists of 2 new files and 3 modified files, with zero new external dependencies (uses only stdlib logging, os, datetime).

New Files

1. src/PowerPlatform/Dataverse/core/log_config.py — LogConfig dataclass

A frozen dataclass for opt-in logging configuration:

from __future__ import annotations

import os
from dataclasses import dataclass, field
from typing import FrozenSet, Optional

# Headers whose values must never appear in log files
_DEFAULT_REDACTED_HEADERS: FrozenSet[str] = frozenset(
    {
        "authorization",
        "proxy-authorization",
        "x-ms-authorization-auxiliary",
        "ocp-apim-subscription-key",
    }
)


@dataclass(frozen=True)
class LogConfig:
    """
    Configuration for local HTTP diagnostics logging.

    When provided to DataverseClient via DataverseConfig, every HTTP request
    and response is logged to timestamped .log files in the specified folder.
    Sensitive headers (e.g. Authorization) are automatically redacted.

    :param log_folder: Directory path for log files. Created automatically if missing.
        Default: "./dataverse_logs"
    :param log_file_prefix: Filename prefix. Timestamp is appended automatically.
        Default: "dataverse"  →  dataverse_20260310_143022.log
    :param max_body_bytes: Maximum bytes of request/response body to capture.
        0 disables body logging. Default: 4096.
    :param redacted_headers: Header names (case-insensitive) whose values are
        replaced with "[REDACTED]" in logs. Defaults include
        Authorization, Proxy-Authorization, etc.
    :param log_level: Python logging level name. Default: "DEBUG".
    :param max_file_bytes: Max size per log file before rotation (bytes).
        Default: 10_485_760 (10 MB).
    :param backup_count: Number of rotated backup files to keep. Default: 5.
    """

    log_folder: str = "./dataverse_logs"
    log_file_prefix: str = "dataverse"
    max_body_bytes: int = 4096
    redacted_headers: FrozenSet[str] = _DEFAULT_REDACTED_HEADERS
    log_level: str = "DEBUG"
    max_file_bytes: int = 10_485_760  # 10 MB
    backup_count: int = 5

2. src/PowerPlatform/Dataverse/core/_http_logger.py — _HttpLogger internal helper

Handles structured log formatting, header redaction, body truncation, and file rotation:

from __future__ import annotations

import logging
import os
import uuid
from datetime import datetime, timezone
from logging.handlers import RotatingFileHandler
from typing import Any, Dict, Optional

from .log_config import LogConfig


class _HttpLogger:
    """Structured HTTP diagnostic logger with automatic header redaction."""

    def __init__(self, config: LogConfig) -> None:
        self._config = config
        self._redacted = {h.lower() for h in config.redacted_headers}

        # Ensure folder exists
        os.makedirs(config.log_folder, exist_ok=True)

        # Build timestamped filename
        ts = datetime.now(timezone.utc).strftime("%Y%m%d_%H%M%S")
        filename = f"{config.log_file_prefix}_{ts}.log"
        filepath = os.path.join(config.log_folder, filename)

        # Create a dedicated named logger (not root) to avoid side effects
        logger_name = f"PowerPlatform.Dataverse.http.{uuid.uuid4().hex[:8]}"
        self._logger = logging.getLogger(logger_name)
        self._logger.setLevel(getattr(logging, config.log_level.upper(), logging.DEBUG))
        self._logger.propagate = False  # don't bubble to root

        handler = RotatingFileHandler(
            filepath,
            maxBytes=config.max_file_bytes,
            backupCount=config.backup_count,
            encoding="utf-8",
        )
        formatter = logging.Formatter(
            "[%(asctime)s] %(levelname)s %(message)s",
            datefmt="%Y-%m-%dT%H:%M:%S%z",
        )
        handler.setFormatter(formatter)
        self._logger.addHandler(handler)

    def log_request(
        self,
        method: str,
        url: str,
        headers: Optional[Dict[str, str]] = None,
        body: Any = None,
    ) -> None:
        safe_headers = self._redact_headers(headers or {})
        body_text = self._truncate_body(body)
        lines = [
            f">>> REQUEST  {method.upper()} {url}",
            f"    Headers: {safe_headers}",
        ]
        if body_text:
            lines.append(f"    Body:    {body_text}")
        self._logger.debug("\n".join(lines))

    def log_response(
        self,
        method: str,
        url: str,
        status_code: int,
        headers: Optional[Dict[str, str]] = None,
        body: Any = None,
        elapsed_ms: Optional[float] ...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs.

[Copilot]

Pull request overview

Adds an opt-in, local file HTTP diagnostics logger to the Dataverse SDK, wired from DataverseConfig into the OData HTTP client so request/response traffic can be captured with header redaction and body truncation.

Changes:

• Introduces LogConfig and internal _HttpLogger to format and persist redacted HTTP request/response diagnostics with file rotation.
• Wires DataverseConfig.log_config through _ODataClient into _HttpClient for request/response/error logging.
• Adds unit tests validating defaults, redaction, truncation behavior, and basic integration with _HttpClient.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tests/unit/data/test_enum_optionset_payload.py	Updates test stub config to include the new log_config attribute expected by _ODataClient.
tests/unit/core/test_http_logger.py	Adds unit/integration tests for LogConfig, _HttpLogger, and _HttpClient logger plumbing.
src/PowerPlatform/Dataverse/data/_odata.py	Creates _HttpLogger from config.log_config and passes it into _HttpClient.
src/PowerPlatform/Dataverse/core/log_config.py	Defines frozen LogConfig dataclass with defaults and redacted header set.
src/PowerPlatform/Dataverse/core/config.py	Adds log_config optional field to DataverseConfig and from_env().
src/PowerPlatform/Dataverse/core/_http_logger.py	Implements structured diagnostics logging with redaction and truncation.
src/PowerPlatform/Dataverse/core/_http.py	Adds optional logger support; logs request (once), response (with elapsed), and transport errors.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.