To find the next test to work on, run:
go run ./cmd/next-testThis tool finds all tests with todo: true in their metadata and returns the one with the shortest query.sql file.
- Run
go run ./cmd/next-testto find the next test to implement - Check the test's
query.sqlto understand what SQL needs parsing - Check the test's
ast.jsonto understand the expected output format - Implement the necessary AST types in
ast/ - Add parser logic in
parser/parser.go - Add JSON marshaling functions in
parser/parser.go - Run
go test ./parser/... -check-todo -vto auto-enable passing todo tests - Run
go test ./parser/...to verify all enabled tests pass
After implementing parser changes, run:
go test ./parser/... -check-todo -vThis runs todo tests and automatically updates metadata.json for any tests that now pass (removes the todo: true flag). Look for "ENABLED:" in the output to see which tests were updated.
Each test in parser/testdata/ contains:
metadata.json-{}for enabled tests,{"todo": true}for pending tests, or{"invalid_syntax": true}for tests with invalid SQLquery.sql- T-SQL to parseast.json- Expected AST output
- NEVER modify
ast.jsonfiles - These are golden files containing the expected output. If tests fail due to JSON mismatches, fix the Go code to match the expected output, not the other way around.
The TsqlAstParser/ directory contains a C# tool that generates ast.json files using Microsoft's official T-SQL parser (ScriptDom).
-
Install .NET 8.0 SDK:
curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin --channel 8.0 --install-dir ~/.dotnet
-
Download the NuGet package (if
packages/directory is empty):mkdir -p packages curl -L -o packages/microsoft.sqlserver.transactsql.scriptdom.170.128.0.nupkg \ "https://api.nuget.org/v3-flatcontainer/microsoft.sqlserver.transactsql.scriptdom/170.128.0/microsoft.sqlserver.transactsql.scriptdom.170.128.0.nupkg" -
Build the tool:
~/.dotnet/dotnet build TsqlAstParser -c Release
Generate ast.json for a single test:
~/.dotnet/dotnet run --project TsqlAstParser -c Release -- parser/testdata/TestName/query.sql parser/testdata/TestName/ast.jsonGenerate ast.json for all tests missing it:
for dir in parser/testdata/*/; do
if [ -f "$dir/query.sql" ] && [ ! -f "$dir/ast.json" ]; then
~/.dotnet/dotnet run --project TsqlAstParser -c Release -- "$dir/query.sql" "$dir/ast.json"
fi
doneTsqlAstParser uses TSql160Parser (SQL Server 2022) and cannot parse:
- SQL Server 170+ features (VECTOR indexes, AI functions, JSON enhancements)
- Fabric DW-specific syntax (CLONE TABLE, CLUSTER BY)
- Deprecated syntax removed in newer versions
- Intentionally invalid SQL (error test cases)
Tests for unsupported syntax will not have ast.json files generated.