Skip to content

DOC GUI: Document dataset workflows and cover cross-flow journeys #2761

Open

Description

@romanlutz

Describe the issue linked to the documentation

The completed dataset explorer needs a concise user guide and contributor-friendly end-to-end examples. Users must understand the difference between loading a dataset, importing one example into an editable chat draft, and configuring a scenario from multiple selected examples.

Work item 17 of 17 in #2744. Prerequisites: the completed memory/chat, provider-loading, and custom-scenario milestones (items 1-16). Keep not ready yet until promoted. Earlier issues must ship their own focused tests; this item does not postpone that responsibility.

Suggest a potential alternative/fix

Add a short guide in the existing GUI documentation and cross-flow coverage using the established test harness.

Document:

  • Browsing memory-backed dataset cards, count semantics, missing metadata, filters, prompt details, and multimodal previews.
  • The three single-example import destinations and the guarantee that importing only fills an editable draft. Explain draft replacement and return-to-chat behavior.
  • Explicit provider loading, supported options, backend credential configuration, external gated-access prerequisites, error/retry behavior, and shared-memory scope. Do not include secrets or imply that a token automatically grants dataset access.
  • Multi-select to Configure scenario, exact selected-example semantics, prompt-sending/no-additional-converter defaults, configured scorer presets, missing-objective requirements, confirmation, and normal scenario progress/history.
  • Unsupported formats/seed shapes and relevant limitations, including the distinction between a non-refusal scorer and a task-achievement scorer.

Cross-flow coverage:

  • Browse a local dataset, inspect text plus media, import into each supported draft destination, and verify that no target call occurs before explicit Send.
  • Leave chat with a nonempty draft, visit Datasets, and return without losing text, attachments, or context.
  • Exercise provider loading success and access/failure guidance with deterministic local/fake providers.
  • Select examples across pages, configure a scenario, and verify preview/estimate/launch agree on exact selections and defaults. Exercise an alternate compatible technique/converter/scorer configuration.
  • Include keyboard navigation and representative narrow-viewport/error states.

Use the real backend for Playwright end-to-end flows with controlled local fixture providers/targets/scorers. Component tests may mock API services according to repository conventions; do not replace the real stack with API mocks in end-to-end tests. Tests and documentation examples must not require privileged Hugging Face access, real secrets, or paid model calls.

Acceptance criteria:

  • The guide matches the shipped labels, routes, defaults, and limitations.
  • A contributor can run the documented fixture-based workflows without gated datasets or credentials.
  • Cross-flow tests cover chat and scenario handoffs and detect accidental sends, lost drafts, and changed selections.
  • Existing GUI documentation is updated rather than duplicated in an unrelated guide.

Start with doc\gui and frontend\e2e plus the feature's local fixture/setup helpers. Follow applicable documentation, frontend, Python, and test instructions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    GUIUmbrella label for all feedback submitted via the Co-PyRIT GUIdocumentationImprovements or additions to documentationnot ready yetThis issue needs more definition or is blocked by a pending change.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions