[andreatnvidia]

What

Adds an experimental initial Gym integration: gym-resource and gym-harness columns call operator-managed services inside the Data Designer DAG, retain outputs and attempt evidence, and feed downstream processing. Task export and rollout ingestion remain available.

An optional persistent Pi service is a separate example with its own runtime and lifecycle.

Why

Execution evidence and completed harness answers can participate in generation, judging, filtering, and export. Manual service startup is the initial workflow.

Usage

from data_designer_gym.config import GymResourceColumnConfig

builder.add_column(GymResourceColumnConfig(
    name="execution",
    endpoint="http://127.0.0.1:8765",
    route="/execute",  # Route supplied by the example service.
    row_id_column="row_id",
    request_columns={"database": "database", "sql": "sql"},
))

The package README provides source installation, pinned runtime combinations, manual startup, a no-inference resource-to-downstream-column run, export, and cleanup. Harness columns return terminal assistant text plus a linked evidence column.

How and limits

• The operator owns service/model installation, readiness, placement, and shutdown. Configured row-local initialization and cleanup preserve cookies/session headers; the plugin does not retry requests.
• Concurrency limits apply per generator in each worker. Sharing a model endpoint does not share DD/Gym clients, model admission, retries, or usage accounting.
• Call/cleanup status and optional Gym episode outcomes remain separate. Cleanup/storage failures clear harness answers; cancellation retains attempt evidence and propagates. Acceptance stays in the workflow.
• The Pi example preserves completed turns and workspaces across an agent-service restart. Its local file tools do not establish sandbox isolation or interrupted-tool recovery.
• A supported UserSim session client/probe, managed service startup, sandbox workflows, and Slurm orchestration are not supplied. Episode-envelope tests do not establish a live Gym-hosted UserSim integration.

Validation

• 106 plugin tests passed on DD 0.9.4 and 0.9.5, and against the installed wheel on DD 0.9.5 (Python 3.12.12).
• The README resource recipe passed against public Gym 58b3240e04f1fa2061328a5d9a8ec7581262f8eb on Python 3.14.4: downstream output and export, retained evidence, zero active sessions, and service shutdown verified.
• 22 Pi adapter tests and the real-Pi/DD restart smoke passed with Pi 1.2.0 and Node 25.9.0: two rows, four turns, native history continuity and separate workspaces, with zero active Pi processes after each stage. The model fixture is deterministic and local.
• Ruff, installed plugin validation, distribution checks, generated docs/metadata checks, and strict documentation build passed. No hosted inference, GPU allocation, or cluster run was used for this preparation.

Proposed next steps

Discussion milestones, not approved delivery commitments:

1. Supported UserSim session client/probe and cleanup contract, with public multi-call success/failure/cancellation coverage and zero leaked sessions.
2. Shared DD/Gym model admission and accounting, including rate-limit and cancellation checks across both callers.
3. Worker-scoped startup/readiness/cleanup and static Slurm resource planning, including startup failure, service crash, and retry.
4. One pinned sandbox/coding-harness workflow that resumes a conversation and workspace after a service restart, verifies a follow-up file change and row isolation, and cleans up owned resources.