SQLite regression tracker

LLVM's own C++ sources are tracked in LLVM.md, starting with the complete Demangle library. See the LLVM tracker for its pinned source, scope and reproduction commands.

Vectorized input is tracked separately in VECTORS.md, using small SQLite-derived operation cases. See the vector tracker for its scope and reproduction commands.

A cached corpus of SQLite as generic MLIR, one module per function and per non-private global. The O3 corpus uses -O3 -fno-vectorize -fno-slp-vectorize. Every selected chunk must pass strict parsing, structural verification, printing and reparsing in VeIR. These are IR regression checks, not SQLite execution tests. The chunks are committed, so scoring needs no LLVM/MLIR toolchain -- only a built VeIR checkout.

./update-sqlite.py            # regenerate the corpus if source or toolchain moved
./update-sqlite.py --check    # is the corpus stale? (exit 1 if so)
./update-sqlite.py --verify   # validate cached inputs without LLVM tools
./make-leadership.py --require-supported  # write LEADERSHIP.md and fail on regressions

Layout

sqlite3.c                        the pinned amalgamation (3.53.3, checksummed)
chunks/<corpus>/<board>/*.mlir   one self-contained module per symbol
manifest.json                    what produced the corpus, and its digests
LEADERSHIP.md                    the standings, regenerated by make-leadership.py
.cache/                          bitcode; regenerable, not committed

<corpus> is O0 (clang -O0 then sroa, matching vcc's pipeline) or O3 (clang -O3 with both vectorizers disabled). <board> is functions or globals: llvm-extract -func keeps only declarations of the globals a function touches, so an initializer is only ever visible on a board of its own. A file is named for its IR symbol, which is the same string on every host, so chunks/O0/functions/absFunc.mlir names the same item as a row in veir's Test/sqlite3/O0/functions-baseline.txt.

The global board follows llvm-nm and excludes private constants such as string literals. The current Linux corpus contains 1,573 function chunks and 185 global chunks at O3; another 1,388 private globals are outside that board. Chunk coverage does not establish whole-module compilation or execution correctness.

The leaderboard

make-leadership.py builds veir-opt once in the neighbouring VeIR checkout, then puts every O3 chunk and its printed output through the built binary and sorts each into supported / parsed (accepted only with --allow-unregistered-dialect) / failed / timed out.

--require-supported exits 1 for any failure, timeout, or chunk accepted only with unregistered constructs. It requires both complete boards and cannot be combined with a sample limit. Reports and --json-out diagnostics are written before returning a regression failure. Without that flag, the scorer can still be used to investigate partial support and its first blocking operations.

Only O3 is scored. O0 remains available for comparison.

Keeping it current

The leadership workflow (.github/workflows/leadership.yml) polls veir's main every five minutes. If it has moved since the commit named in the Provenance table, the workflow builds veir-opt at the new commit, rescores and commits LEADERSHIP.md as github-actions[bot], with the change in the standings in the subject and the blockers that cleared or appeared in the body. A regression fails CI and uploads the failing report and JSON diagnostics; LEADERSHIP.md retains the last passing run. Pull requests and pushes to main also run the check. Pull requests never publish reports. The workflow can be run by hand from the Actions tab, with force to rescore an unchanged VeIR.

The tracker workflows can score concurrently. Publication reads the latest main and compares reports in VeIR commit order: an older or divergent VeIR revision cannot replace a newer report, and duplicate runs keep the first published result. A manual rescore can still refresh an unchanged report. If any tracked file other than the four generated reports changed during scoring, publication skips those results so the next run can score the updated inputs. Completed reports and JSON diagnostics remain in the workflow artifacts.

publish-reports.py creates a commit on the latest main and makes a normal push. If another writer wins the race, it refreshes main and repeats the input, revision and baseline checks, including the SQLite commit message, for up to five push attempts. Persistent push errors fail CI. No workflow lock is needed. SQLite's requirement that every O3 chunk pass is checked during scoring; it is independent of the report used to describe changes in the publication commit.

CI validates the pinned source, compiler pipelines, complete chunk counts and content hashes before scoring. Deleted or modified chunks, unrecorded compiler flags and extraction failures cannot silently reduce the regression corpus.

The corpus is a function of its toolchain

Chunks are not portable between toolchains. The compiler decides what gets inlined, how generated symbols are numbered, and what target metadata rides along; the MLIR tools decide how all of that is spelled. manifest.json therefore records tool versions, the target triple and both compiler pipelines. update-sqlite.py detects changed flags, versions and corpus contents. Source or toolchain changes require regenerating both corpora together; --corpus O3 can refresh just O3 when the source and toolchain match.

The corrected corpus was generated with Debian Clang/LLVM 19.1.7 and MLIR 24.0.0git on Linux x86-64. The MLIR tool source and build environment are documented in llvm/TOOLCHAIN.md. To reproduce it:

python3 update-sqlite.py --force --llvm-bindir /usr/lib/llvm-19/bin \
  --mlir-bindir /path/to/mlir/bin --jobs 4
python3 update-sqlite.py --verify
python3 make-leadership.py --veir /path/to/veir --require-supported --jobs 4

So: a corpus is comparable with another corpus built the same way, and with nothing else. Compare digests in manifest.json before comparing results.