Skip to content

examples: Separate extraction from manual generation - #377

Open
fingolfin wants to merge 1 commit into
masterfrom
claude/autodoc-issue-376-a4ac71
Open

examples: Separate extraction from manual generation#377
fingolfin wants to merge 1 commit into
masterfrom
claude/autodoc-issue-376-a4ac71

Conversation

@fingolfin

Copy link
Copy Markdown
Member

Makes it possible to run a package's manual examples from its test
suite without building the manual, and without storing the generated
.tst files in the repository.

Extraction without a manual build

New AutoDocExtractExamples, meant for a package's tst/testall.g:

LoadPackage( "mypkg" );
dirs := DirectoriesPackageLibrary( "mypkg", "tst" );
Add( dirs, AutoDocExtractExamples( "mypkg" ) );
TestDirectory( dirs, rec( exitGAP := true ) );

It reads the package's own makedoc.g, so the settings describing the
manual — source files, scaffolding, extract_examples options — are not
duplicated. The mode is requested through a GAP global option, the same
mechanism already used for nopdf and documented as
Read( "makedoc.g" : nopdf ), so makedoc.g itself is untouched. For a
makedoc.g ending in QUIT, the AUTODOC_EXTRACT_ONLY environment
variable does the same for a separate process.

The documentation directory is staged into a temporary directory and
generated into the copy. This is needed because ComposedDocument
resolves relative <#Include SYSTEM ...> against a single path, and
handwritten doc files commonly include generated ones by relative name
(<#Include SYSTEM "_entities.xml">). As a side effect nothing is
written into the package, so this also works when the package directory
is read-only, as with a distribution or system-wide install.

Real source locations

Extracted tests used to point at the intermediate XML file:

-# doc/_Chapter_Overview.xml:61-73
+# gap/CanonicalPcPres.gd:53-65

The parser already tracked file and line, but only for error messages.
Example nodes now record their position, generated XML carries it in an
XML comment which the GAPDoc backends ignore, and extraction rewrites
GAPDoc's origin list from those markers before parsing. Working on the
origin list rather than on the extraction results also covers chunks,
which are expanded at their insertion point.

This should also make the changeSources-style rewriting asked for in
#375 tractable, since the examples can now be traced back to the file
that has to be edited.

Not covered

Fenced markdown examples sitting directly in section text get no marker
and keep the previous behaviour: the parser records no source positions
for that content at all. The @BeginExample family works everywhere,
and fenced blocks work inside @Description text and chunks.

Testing

make check passes. Beyond the package's own suite, extraction was
checked against real packages: for congruence and circle — both
using scaffold.MainPage := false with a handwritten main page that
includes _entities.xml — the extracted files are byte-identical to
the ones committed in those packages today.

Fixes #376

AI disclosure

Prepared with Claude Code (Claude Opus 5), which investigated the
issue, proposed the approach and wrote the implementation and tests
under review.

🤖 Generated with Claude Code

Add AutoDocExtractExamples, which collects a package's manual
examples by running its makedoc.g with only the parts of the build
that extraction needs. Everything is written below a temporary
directory, so the extracted tests need not be stored in the
repository, and they can be produced from a read-only package
directory such as a system-wide install.

The settings describing a manual stay in makedoc.g alone: the mode
is requested through a global option, the mechanism already used
for nopdf, or through the AUTODOC_EXTRACT_ONLY environment
variable for a makedoc.g which ends in QUIT and therefore has to
run as a separate process.

Generated XML now records where each example was written, as an
XML comment which the GAPDoc output backends ignore. Extraction
rewrites GAPDoc's origin list from those markers, so a failing
example points at the .autodoc file or AutoDoc comment which has
to be edited, rather than at the XML generated from it.

Fenced markdown examples sitting directly in section text are not
covered, as the parser records no source positions there.

Fixes #376

AI disclosure: prepared with Claude Code (Claude Opus 5), which
investigated the issue, proposed the approach and wrote the
implementation under review.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@fingolfin
fingolfin force-pushed the claude/autodoc-issue-376-a4ac71 branch from 4346d9e to 181ac4e Compare August 28, 2026 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make it easier separate generating the manual from extracting example, remove need to store extracted .tst files

1 participant