Research: evaluate user-written Swift as input to a codegen CLI (Tuist-style manifest execution)

[leogdion]

Background

Tuist lets users describe an Xcode project by writing a Project.swift manifest in plain Swift, then evaluates that manifest at CLI invocation time — without the user having to compile and run a full app themselves. SwiftPM does the same thing with Package.swift.

We should investigate whether SyntaxKit could expose an analogous flow: the user writes a Swift file that uses the SyntaxKit DSL, and a CLI compiles + executes it to emit generated Swift source. This would let people drive code generation declaratively from a checked-in .swift file without setting up an executable target per generator.

Goals of this research

1. Understand exactly how Tuist (and, as a comparison point, SwiftPM) compiles and executes a user manifest.
2. Identify which pieces of that pipeline are reusable for a SyntaxKit-driven codegen CLI.
3. Surface the risks and open questions before we commit to a design.

Phase 1 — How Tuist does it

Source: tuist/tuist, primarily the ProjectDescription target and the manifest-loading subsystem.

Questions to answer (with file/line citations into the Tuist repo):

• Compilation pipeline. What exact swiftc invocation compiles Project.swift? Flags, framework search paths, linker args, output type (dylib vs. executable)?
• Execution model. Does Tuist (a) load a compiled dylib and call a known symbol, (b) run a compiled executable that dumps a description on stdout, or (c) something else? (SwiftPM uses (b) — confirm whether Tuist matches.)
• Bridging format. What's the wire format between the manifest process and the host tool? ProjectDescription types are Codable — is JSON the boundary?
• Helpers / shared code. How does ProjectDescriptionHelpers/ get compiled and made importable from the manifest?
• Caching. Does Tuist hash the manifest + ProjectDescription version and skip recompilation? Where is the cache?
• Failure modes. What happens when the manifest crashes, has a syntax error, or hangs?

Phase 2 — Map the pattern onto SyntaxKit

With Phase 1 in hand, sketch the equivalent for a SyntaxKit CLI:

• Manifest shape. What does the user write? A top-level expression returning a Group of CodeBlocks? An @main struct?
• What crosses the process boundary. SyntaxKit's output is generated Swift source, not a graph of Codable types. Two viable options:
  • Manifest executable prints generated Swift to stdout directly (simplest).
  • Manifest exports a Codable IR; host renders via SwiftSyntax (matches Tuist more closely, but duplicates rendering).
• Linking SyntaxKit into the manifest. The manifest needs import SyntaxKit. Compare (a) shipping a precompiled SyntaxKit binary next to the CLI and passing -F/-I/-L, vs. (b) generating a throwaway SwiftPM package per run.
• Helpers directory. Mirror Tuist's ProjectDescriptionHelpers so users can factor out reusable codegen.

Phase 3 — Open risks to probe early

• SwiftSyntax link cost. SwiftSyntax is large; compiling a manifest that transitively links it could be slow. Measure on a hello-world manifest before committing.
• macOS-only vs. Linux. Tuist is mac-focused. The SyntaxKit CLI should at least not paint itself into a macOS corner.
• Toolchain resolution. How does Tuist locate swiftc (xcrun --find vs. explicit toolchain paths)?

Deliverables

1. Phase 1 write-up: how Tuist's manifest pipeline works, with citations.
2. Design sketch: compile command, execution model, IR boundary (or lack of one), helpers layout, caching strategy.
3. Smallest possible proof-of-concept steps (enumerated, not yet implemented), e.g. "compile a manifest that imports SyntaxKit and prints a Hello.swift to stdout."

Out of scope (for this issue)

• Implementation of the CLI itself — this issue is research only.
• Choosing a CLI name or product surface.

[leogdion]

Phase 1 + Phase 2 deliverables drafted on branch research/swift-manifest-codegen:

• Phase 1 — How Tuist evaluates manifests: Docs/research/tuist-manifest-pipeline.md. File/line citations into tuist/tuist@main and swiftlang/swift-package-manager@main for the manifest pipeline, helpers compiler, dual cache, sandbox, and SwiftPM comparison.
• Phase 2 — Design sketch + POC steps: Docs/research/codegen-cli-design.md. Manifest shape (top-level Output(...) self-publishing), wire format (JSON envelope of { path, contents } records — no AST IR, no SyntaxKit on the host), distribution shape (bundled dylibs adjacent to the CLI), and a 7-step POC ladder where step 1 retires the largest risk (cold-start cost of swift Codegen.swift with SyntaxKit linked).

Key design departure from Tuist worth flagging up front: the manifest already renders Swift source via CodeBlock.generateCode(), so the host can carry the finished string across the process boundary instead of a Codable AST. The host never needs to link SyntaxKit. This is what keeps the CLI small and the design simple.

Headline open question: cold-start cost of swift <manifest> with SyntaxKit/SwiftSyntax linked. Step 1 of the POC ladder is designed to answer it before we invest in steps 2–7.

[leogdion]

POC step 1 done — approach is viable. Commit 23869f0. Full writeup: Docs/research/poc-step1-results.md.

Headline numbers (Apple Silicon, debug dylib):

	real	user	sys
cold	0.72s	0.77s	0.29s
warm	0.11s	0.07s	0.02s

The pure-DSL Input.swift spliced into a Group { … } wrapper compiles and runs end-to-end under xcrun swift with the bundled dylib. Cold-start is well inside the "feels instant" budget — the biggest design risk is retired.

Two surprises worth flagging:

1. -Xcc -I -Xcc <_SwiftSyntaxCShims/include> is required. Without it, the script compile fails with missing required module '_SwiftSyntaxCShims'. SyntaxKit transitively imports SwiftSyntax which has a C-shims target; the bundled-binary release layout has to ship that include directory alongside the dylib. Design doc updated.

2. if-in-Group is broken in SyntaxKit today. CodeBlockBuilderResult declares buildEither/buildOptional but a wrapped input containing if true { Struct(…) } inside Group { … } crashes the Swift type checker (error: failed to produce diagnostic for expression). No test in Tests/SyntaxKitTests/Unit/ exercises this pattern. This is independent of the CLI design — worth filing as its own SyntaxKit issue so users can write conditional codegen down the line. Non-blocking for v1.

Next step on the POC ladder is step 2: wrap up the wrap-and-spawn flow as a CLI subcommand (parse imports with SwiftSyntax, write the wrapped file to temp, spawn, capture stdout, rewrite stderr line numbers). Lower risk than step 1 — just engineering — but worth doing before steps 3-7.

[leogdion]

Three updates pushed.

(1) Filed the if-in-Group bug as its own issue: #155. Independent of this CLI work.

(2) Release-config dylib size (49de3ea): 25 MB debug → 18 MB release → 9.3 MB stripped. Comfortably shippable. Warm-run timings identical between debug and release. Design doc updated.

(3) POC step 2 done (e8c7a86) — new executable target skitrun (POC name): Sources/skitrun/Main.swift ≈180 lines. Full writeup: Docs/research/poc-step2-results.md.

Quickstart:

./Docs/research/poc-step1.sh                              # stage /tmp/syntaxkit-poc/lib/
swift build --product skitrun
.build/<triple>/debug/skitrun /tmp/syntaxkit-poc/Input.swift

Verified flows:

• Default → stdout
• -o <file> → atomic file write
• Hoisted import Foundation resolves UUID/Date in the rendered struct
• A deliberate NonexistentType in InputError.swift:4 produces InputError.swift:4:37: error: cannot find 'NonexistentType' in scope — #sourceLocation is doing all the diagnostic-mapping work, no manual stderr line arithmetic.

Notable surface: skitrun does not depend on SyntaxKit. It uses SwiftSyntax/SwiftParser only — the host has no need to link SyntaxKit because the manifest already renders its own source.

Next on the ladder is step 3 (folder mode — walk InputDir/**/*.swift, mirror into OutputDir/, with concurrency-limited fan-out and a _-prefix skip rule). Modest engineering, low risk.

[leogdion]

POC step 3 done — folder mode works (dc2720c). Writeup: Docs/research/poc-step3-results.md.

skitrun InputDir/ -o OutDir/

Walks **/*.swift, skips _-prefixed files, fans out over withTaskGroup capped at activeProcessorCount, mirrors paths into OutDir/. Failures don't abort the batch — successes are always written, exit code reflects overall pass/fail, stderr ends with skitrun: N/M succeeded.

Quick timing: 3 parallel files in 1.41s wall, vs. 0.72s baseline single cold-start. Parallelism helps but per-spawn compile cost isn't deduplicated — for bigger batches we'd eventually want one long-lived swift process driving all inputs. Out of scope for v1.

Next is step 4: bundled-binary release. Today the CLI needs Docs/research/poc-step1.sh to have run first (it expects /tmp/syntaxkit-poc/lib/); step 4 makes the CLI self-contained — the lib/ directory rides along with the binary across swift build, swift run, and brew install. Real systems-integration work and probably the right moment to graduate skitrun out of POC-name purgatory.