From 0d6082ab327e50b49b5ee5de9eac694820617b73 Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 17:11:10 +0200 Subject: [PATCH 01/13] docs: complete the documentation site and slim the README Build the docs site out into a Getting Started page, an eleven page guide, and a four page reference, modeled on the two tier onboarding and uniform page template used by hikari-lightbulb. The guide covers configuration, drivers, model types, writing queries, enums, type overrides, converters, working with JSON, docstrings, SQLite type conversion, and naming. The reference documents every configuration option and config object schema, the SQL to Python type mappings for both engines, and per driver feature support. Generated code examples are taken verbatim from the committed test fixtures. Slim the README down to the example config plus a feature list that links into the site, since those sections now live there. --- README.md | 204 +++------------ docs/content/docs/_index.md | 15 +- docs/content/docs/converters.md | 125 --------- docs/content/docs/getting-started.md | 12 +- docs/content/docs/guide/_index.md | 48 ++++ docs/content/docs/guide/configuration.md | 95 +++++++ docs/content/docs/guide/converters.md | 95 +++++++ docs/content/docs/guide/docstrings.md | 137 ++++++++++ docs/content/docs/guide/drivers.md | 67 +++++ docs/content/docs/guide/enums.md | 68 +++++ docs/content/docs/guide/model-types.md | 95 +++++++ docs/content/docs/guide/naming.md | 97 +++++++ .../docs/guide/sqlite-type-conversion.md | 87 +++++++ docs/content/docs/guide/type-overrides.md | 90 +++++++ docs/content/docs/guide/working-with-json.md | 241 ++++++++++++++++++ docs/content/docs/guide/writing-queries.md | 133 ++++++++++ docs/content/docs/reference/_index.md | 16 ++ .../docs/reference/configuration-options.md | 78 ++++++ .../content/docs/reference/feature-support.md | 44 ++++ docs/content/docs/reference/type-mappings.md | 74 ++++++ 20 files changed, 1510 insertions(+), 311 deletions(-) delete mode 100644 docs/content/docs/converters.md create mode 100644 docs/content/docs/guide/_index.md create mode 100644 docs/content/docs/guide/configuration.md create mode 100644 docs/content/docs/guide/converters.md create mode 100644 docs/content/docs/guide/docstrings.md create mode 100644 docs/content/docs/guide/drivers.md create mode 100644 docs/content/docs/guide/enums.md create mode 100644 docs/content/docs/guide/model-types.md create mode 100644 docs/content/docs/guide/naming.md create mode 100644 docs/content/docs/guide/sqlite-type-conversion.md create mode 100644 docs/content/docs/guide/type-overrides.md create mode 100644 docs/content/docs/guide/working-with-json.md create mode 100644 docs/content/docs/guide/writing-queries.md create mode 100644 docs/content/docs/reference/_index.md create mode 100644 docs/content/docs/reference/configuration-options.md create mode 100644 docs/content/docs/reference/feature-support.md create mode 100644 docs/content/docs/reference/type-mappings.md diff --git a/README.md b/README.md index 9864d8b8..62fb77f9 100644 --- a/README.md +++ b/README.md @@ -12,10 +12,15 @@ A WASM plugin for SQLC allowing the generation of Python code. The generated code requires **Python 3.12 or newer** (it uses PEP 695 type aliases and generics, and `enum.StrEnum`). -**Documentation: https://rayakame.github.io/sqlc-gen-better-python/** -(The docs site is new; some feature references still live in this README below -while they are ported over.) +## Documentation +**https://rayakame.github.io/sqlc-gen-better-python/** + +- [Getting Started](https://rayakame.github.io/sqlc-gen-better-python/docs/getting-started/) - install the plugin and generate your first models. +- [Guide](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/) - configuration, drivers, model types, writing queries, and every feature, each with real generated output. +- [Reference](https://rayakame.github.io/sqlc-gen-better-python/docs/reference/) - all configuration options, SQL-to-Python type mappings, and per-driver feature support. + +Questions or feedback? Join the [Discord](https://discord.gg/hikari). > [!NOTE] > Every Release before `v1.0.0`, including this one is an beta release. @@ -25,7 +30,6 @@ while they are ported over.) > Since `v0.5.0` this includes full support for PostgreSQL enums and a fourth model type, `pydantic`. > Feel free to lmk any wanted features and I'm going to do my best on implementing them with the time I have rn. - ## Example Config ```yaml @@ -51,171 +55,33 @@ sql: ``` -More options at the [`sqlc` config reference](https://docs.sqlc.dev/en/stable/reference/config.html) - -## Configuration Options - -| Name | Type | Required | Description | -|----------------------------------|----------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `package` | string | yes | The name of the package where the generated files will be located | -| `emit_init_file` | bool | yes | If set to to `false` there will be no `__init__.py` file created in the package that you specified. Only set this to false if you know that you already have a `__init__.py` file otherwise the generated code wont work. | -| `sql_driver` | string | yes | The name of the sql driver you want to use. Valid options are listed [here](#feature-support) | -| `model_type` | string | no | The model type you want to use. This can be one of `dataclass`, `msgspec`, `attrs` or [`pydantic`](#pydantic-models). Defaults to `dataclass` | -| `initialisms` | list[string] | no | An array of [initialisms](https://google.github.io/styleguide/go/decisions.html#initialisms) to upper-case. For example, `app_id` becomes `AppID`. Defaults to `["id"]`. | -| `emit_exact_table_names` | bool | no | If `true`, model names will mirror table names. Otherwise sqlc attempts to singularize plural table names. | -| `emit_classes` | bool | no | If `true`, every query function will be put into a class called `Querier`. Otherwise every function will be a standalone function. | -| `inflection_exclude_table_names` | list[string] | no | An array of table names that should not be turned singular. Only applies if `emit_exact_table_names` is `false`. | -| `omit_unused_models` | bool | no | If set to `true` and there are models/tables that are not used in any query, they wont be turned into models. | -| `omit_typechecking_block` | bool | no | If set to `true`, will not wrap all non-builtin types behind a `typing.TYPE_CHECKING` block. Defaults to `false` | -| `docstrings` | string | no | If set, there will be docstrings generated in the selected format. This can be one of `google`, `numpy`, `pep257` and `none`. `none` will not generate any docstrings. | -| `docstrings_emit_sql` | bool | no | If set to `false` the SQL code for each query wont be included in the docstrings. This defaults to `true` but is not used when `docstrings` is not set or set to `none` | -| `query_parameter_limit` | integer | no | If set to a non-negative value, queries with more parameters than the limit get them bundled into a single `params: Params` argument instead of expanded arguments. If unset or negative, parameters are never bundled (`:copyfrom` always uses a Params class). | -| `omit_kwargs_limit` | integer | no | This can be used to set a limit where any query with less or equal amounts of parameters will not require kwargs for the parameters. This defaults to `0` which makes every query require kwargs for their parameters. | -| `speedups` | bool | no | If set to `true` the plugin will use other librarys for type conversion. Needs extra dependecys to be installed. This option currently only affects `sqlite3` & `aiosqlite` and uses the library `ciso8601` | -| `overrides` | list[Override] | no | A list of [type overrides](#type-overrides). | -| `converters` | list[Converter]| no | A list of [converters](#converters): named pairs of your own functions that serialize and deserialize a column value, referenced by an override. | -| `debug` | bool | no | If set to `true`, there will be debug logs generated into a `log.txt` file when executing `sqlc generate`. Defaults to `false` | - -### Type Overrides - -Similar to `sqlc-gen-go` this plugin supports overriding types with your own. You can either override the type of every -column that has a specific sql type, or you can overwrite the type of specific columns. - -```yaml -# filename: sqlc.yaml -# ... -options: - # ... - overrides: - - db_type: text - py_type: - import: collections - package: UserString - type: UserString - - column: table_name.text_column - py_type: - import: collections - type: collections.UserString - -``` - -### Converters - -An override replaces a column's type, but conversion is done by calling that type -(`Preferences(value)`), which does not work for things like JSON. A converter names -two of your own functions instead, used whenever the column is read or written: - -```yaml -# filename: sqlc.yaml -# ... -options: - # ... - converters: - - name: prefs - py_type: - import: myapp.models - package: Preferences - type: Preferences - to_db: myapp.converters.encode_preferences - from_db: myapp.converters.decode_preferences - overrides: - - db_type: jsonb - converter: prefs - - column: users.preferences - converter: prefs -``` - -```python -# generated -preferences=myapp.converters.decode_preferences(row[1]) # read -await conn.execute(CREATE_USER, id_, myapp.converters.encode_preferences(preferences)) # write -``` - -A converter is referenced by an override, so it applies to whichever columns that -override matches, and `converter` replaces `py_type` on the override itself. - -Both directions are required. `to_db` and `from_db` are dotted paths; their module is -imported and the function called fully qualified, so the names can never collide with -generated code. - -`to_db` must return the type the column would have had without the override (`jsonb` is -a `str`, `bytea` a `memoryview`), and `from_db` receives that same type. For a column -whose SQL type the plugin does not recognise there is no such type, so `to_db` must -return one the driver accepts. - -Your functions never see `None`: nullable columns are guarded, so a NULL stays `None` -without calling the converter. List columns convert element-wise. - -### Enums - -PostgreSQL enum types generate an `enums.py` module containing `enum.StrEnum` classes: - -```python -class Mood(enum.StrEnum): - SAD = "sad" - OK = "ok" - HAPPY = "happy" -``` - -Enum columns are typed with these classes in models and query functions, and values read -from the database are coerced into them. Enums in non-default schemas get schema-qualified -class names (e.g. `CustomMood` for `custom.mood`), so same-named enums never collide. - -### Identifier sanitization - -SQL identifiers that are not valid Python names are sanitized: invalid characters become -underscores, digit-leading columns/parameters get a `column_` prefix, table names that -would produce a digit-leading, keyword or empty class name get a `Model` prefix, digit- -or underscore-leading enum values get a `VALUE_` prefix, and colliding results are -deduplicated with numeric suffixes. Field names also prefix leading underscores, since -attrs and pydantic treat such fields as private. - -One caveat: sanitization checks characters with Go's Unicode tables, which differ from -Python's identifier rules in exotic cases (e.g. characters that may not START an -identifier in Python, or two names that Python normalizes to the same identifier via -NFKC). Such schemas are not detected; stick to ASCII identifiers if in doubt. - -### Pydantic models - -With `model_type: pydantic`, models are generated as `pydantic.BaseModel` subclasses -(requires `pydantic >= 2.9`). Two things to be aware of: - -- Every generated class sets `model_config = pydantic.ConfigDict(arbitrary_types_allowed=True)`. - This is required for field types pydantic has no core schema for (e.g. `memoryview` for - `bytea`/`blob` columns and custom override types); those fields are still validated with - an `isinstance` check, and all other fields get full pydantic validation on construction. -- Unlike the other model types, pydantic resolves field annotations at runtime, so the - generated files import the referenced modules at runtime instead of inside - `if typing.TYPE_CHECKING:` blocks. If you lint generated code with ruff, set - `lint.flake8-type-checking.runtime-evaluated-base-classes = ["pydantic.BaseModel"]`. - -### SQLite type conversion - -For the `sqlite3` and `aiosqlite` drivers, the generated code registers adapters and -converters (for `date`, `datetime`/`timestamp`, `decimal`, `bool` and `blob` columns) via -`register_adapter`/`register_converter`. For these to work, you must open your connection -with declared-type parsing enabled: - -```python -conn = sqlite3.connect(dsn, detect_types=sqlite3.PARSE_DECLTYPES) -``` - -## Feature Support - -Every [sqlc macro](https://docs.sqlc.dev/en/latest/reference/macros.html) is supported. -The supported [query commands](https://docs.sqlc.dev/en/latest/reference/query-annotations.html) depend on the SQL -driver you are using, supported commands are listed below. -> Every `:batch*` command is not supported by this plugin and probably will never be. - -> Prepared Queries are not planned for the near future, but will be implemented sooner or later - -| | `:exec` | `:execresult` | `:execrows` | `:execlastid` | `:many` | `:one` | `:copyfrom` | -|-----------|---------|---------------|-------------|---------------|---------|--------|-------------| -| aiosqlite | yes | yes | yes | yes | yes | yes | no | -| sqlite3 | yes | yes | yes | yes | yes | yes | no | -| asyncpg | yes | yes | yes | no | yes | yes | yes | -| psycopg2 | no | no | no | no | no | no | no | -| mysql | no | no | no | no | no | no | no | +More options at the [`sqlc` config reference](https://docs.sqlc.dev/en/stable/reference/config.html), +and the full plugin option list in the +[configuration reference](https://rayakame.github.io/sqlc-gen-better-python/docs/reference/configuration-options/). + +## Features + +- **Four model types** - `dataclass`, `attrs`, `msgspec`, or `pydantic` + ([docs](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/model-types/)). +- **Three drivers** - `asyncpg` for PostgreSQL, `aiosqlite` and `sqlite3` for SQLite + ([docs](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/drivers/)). +- **Typed query functions** - one module per query file, one function per query + ([docs](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/writing-queries/)). +- **PostgreSQL enums** as `enum.StrEnum` classes + ([docs](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/enums/)). +- **Type overrides and converters** - swap a column's Python type, or plug in your + own encode/decode functions + ([overrides](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/type-overrides/), + [converters](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/converters/)). +- **Typed JSON columns** via msgspec structs + ([docs](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/working-with-json/)). +- **Optional docstrings** in `google`, `numpy`, or `pep257` convention + ([docs](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/docstrings/)). +- Generated code passes **pyright strict** and **ruff**. + +Every [sqlc macro](https://docs.sqlc.dev/en/latest/reference/macros.html) is +supported. Which query commands are available depends on the driver - see the +[feature support matrix](https://rayakame.github.io/sqlc-gen-better-python/docs/reference/feature-support/). ## Development diff --git a/docs/content/docs/_index.md b/docs/content/docs/_index.md index 5b28102d..2f86baea 100644 --- a/docs/content/docs/_index.md +++ b/docs/content/docs/_index.md @@ -8,18 +8,11 @@ sidebar: --- `sqlc-gen-better-python` is a [sqlc](https://sqlc.dev) plugin that turns your SQL -schema and queries into modern, fully typed Python database code: models plus -async query functions, ready to run. - -Start with **[Getting Started](getting-started)** for installation and a first -generate, then explore the individual features. +schema and queries into modern, fully typed Python database code: models, typed +query functions, and enums. {{< cards >}} {{< card link="getting-started" title="Getting Started" subtitle="Install the plugin and generate your first models." >}} - {{< card link="converters" title="Converters" subtitle="Serialize and deserialize columns with your own functions." >}} + {{< card link="guide" title="Guide" subtitle="Learn each feature with real schema, queries, and generated output." >}} + {{< card link="reference" title="Reference" subtitle="Configuration options, type mappings, and driver support." >}} {{< /cards >}} - -{{< callout type="info" >}} - These docs are new and growing. More feature pages (model types, drivers, - enums, type overrides, docstrings) are being ported from the README. -{{< /callout >}} diff --git a/docs/content/docs/converters.md b/docs/content/docs/converters.md deleted file mode 100644 index a4b53516..00000000 --- a/docs/content/docs/converters.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -title: Converters -weight: 3 -prev: getting-started ---- - -A [type override](https://github.com/rayakame/sqlc-gen-better-python#type-overrides) -replaces a column's Python type, but it converts by *calling that type* -(`Preferences(value)`), which cannot express JSON or model (de)serialization. A -**converter** names two of your own functions instead, used whenever the column -is read from or written to the database. - -## Defining a converter - -Declare the converter under `converters`, then reference it from an override: - -```yaml -options: - # ... - converters: - - name: prefs - py_type: - import: myapp.models - package: Preferences - type: Preferences - to_db: myapp.converters.encode_preferences - from_db: myapp.converters.decode_preferences - overrides: - - db_type: jsonb # every jsonb column - converter: prefs - - column: users.preferences # or one specific column - converter: prefs -``` - -The generated code calls your functions on both sides: - -```python -# read -preferences = myapp.converters.decode_preferences(row[1]) -# write -await conn.execute(CREATE_USER, id_, myapp.converters.encode_preferences(preferences)) -``` - -## Rules - -{{< callout type="info" >}} - A converter is referenced *by an override*, so it applies to whichever columns - that override matches. `converter` replaces `py_type` on the override. -{{< /callout >}} - -- **Both directions are required.** `to_db` and `from_db` are dotted paths to - module-level functions; the module is imported and the function called fully - qualified, so the names can never collide with generated code. -- **Wire type.** `to_db` must return the type the column would have had *without* - the override (`jsonb` is a `str`, `bytea` a `memoryview`), and `from_db` - receives that same type. For an unrecognised SQL type there is no such type, so - `to_db` must return one the driver accepts. -- **Your functions never see `None`.** Nullable columns are guarded, so a SQL - `NULL` stays `None` without calling the converter. -- **List columns convert element-wise.** - -## Example: msgspec structs to and from JSON - -Because `msgspec.json.decode` needs a `type=` argument and returns `bytes`, you -wrap it in two small functions that match the wire type of a `jsonb` column -(`str`): - -{{< tabs >}} - - {{< tab name="converters.py" >}} - -```python -import msgspec - -from myapp.models import User - - -def encode_user(value: User) -> str: - # jsonb's wire type is str, so decode msgspec's bytes back to str. - return msgspec.json.encode(value).decode() - - -def decode_user(value: str) -> User: - return msgspec.json.decode(value, type=User) -``` - - {{< /tab >}} - - {{< tab name="models.py" >}} - -```python -import msgspec - - -class User(msgspec.Struct): - name: str - groups: set[str] - email: str | None = None -``` - - {{< /tab >}} - - {{< tab name="sqlc.yaml" >}} - -```yaml -converters: - - name: user - py_type: - import: myapp.models - package: User - type: User - to_db: myapp.converters.encode_user - from_db: myapp.converters.decode_user -overrides: - - db_type: jsonb - converter: user -``` - - {{< /tab >}} - -{{< /tabs >}} - -A `get_user` query then returns a fully typed `User` (set field and all), and a -`create_user` query accepts one directly - the plugin inserts the -`decode_user` / `encode_user` calls for you. diff --git a/docs/content/docs/getting-started.md b/docs/content/docs/getting-started.md index c33c46ff..6bfb8972 100644 --- a/docs/content/docs/getting-started.md +++ b/docs/content/docs/getting-started.md @@ -1,8 +1,8 @@ --- title: Getting Started -weight: 2 +weight: 1 prev: /docs -next: converters +next: /docs/guide --- ## Prerequisites @@ -56,7 +56,7 @@ per query file, and (when your schema has enums) an `enums.py`. ## Next steps -- Full option reference and driver support: see the - [project README](https://github.com/rayakame/sqlc-gen-better-python#configuration-options) - while the remaining pages are ported here. -- Plug in your own serialization with [Converters](converters). +- Work through the [Guide](/docs/guide) - configuration, drivers, model types, + writing queries, and every feature, each with real generated output. +- Look up specifics in the [Reference](/docs/reference): the full option list, + SQL-to-Python type mappings, and per-driver support. diff --git a/docs/content/docs/guide/_index.md b/docs/content/docs/guide/_index.md new file mode 100644 index 00000000..098d88f3 --- /dev/null +++ b/docs/content/docs/guide/_index.md @@ -0,0 +1,48 @@ +--- +title: Guide +weight: 2 +prev: /docs/getting-started +next: /docs/guide/configuration +sidebar: + open: true +--- + +This guide walks through `sqlc-gen-better-python` one feature at a time. Each +page states what a feature does, the configuration it needs, and shows real +schema, queries, and the exact Python the plugin generates for them. + +## How the pieces fit + +You never call this plugin directly. You write a SQL schema and queries, and +`sqlc` runs the plugin for you: + +1. You write **schema** (`CREATE TABLE`, `CREATE TYPE`) and **queries** + (annotated `SELECT`/`INSERT`/...). +2. `sqlc` parses them into a typed **catalog** and hands it to the plugin. +3. The plugin walks the catalog and emits **Python**: a `models.py` (one class + per table), one query module per query `.sql` file (typed functions), and an + `enums.py` when your schema has enums. + +So everything in this guide is driven by two inputs - your SQL and your +`sqlc.yaml` options - and observed through one output: the generated Python. + +{{< callout type="info" >}} + New here? Start with [Getting Started](/docs/getting-started) for install and + a first `sqlc generate`, then come back to this guide. +{{< /callout >}} + +## How to read this guide + +Start with the essentials, then dip into feature pages as you need them: + +{{< cards >}} + {{< card link="configuration" title="Configuration" subtitle="The sqlc.yaml plugin block and the core options." >}} + {{< card link="drivers" title="Drivers" subtitle="asyncpg, aiosqlite, and sqlite3 - and which query commands each supports." >}} + {{< card link="model-types" title="Model types" subtitle="dataclass, attrs, msgspec, or pydantic models." >}} + {{< card link="writing-queries" title="Writing queries" subtitle="How query annotations become typed Python functions." >}} +{{< /cards >}} + +The remaining pages - enums, type overrides, converters, working with JSON, +docstrings, SQLite type conversion, and naming - cover specific features and can +be read in any order. The [Reference](/docs/reference) section holds the +canonical option, type-mapping, and driver-support tables. diff --git a/docs/content/docs/guide/configuration.md b/docs/content/docs/guide/configuration.md new file mode 100644 index 00000000..14215305 --- /dev/null +++ b/docs/content/docs/guide/configuration.md @@ -0,0 +1,95 @@ +--- +title: Configuration +weight: 10 +prev: /docs/guide +next: /docs/guide/drivers +--- + +Everything the plugin does is driven by your `sqlc.yaml`. This page covers how +the plugin is wired in and how its options are structured. For the exhaustive +option list, see the [configuration options reference](/docs/reference/configuration-options). + +## Anatomy of `sqlc.yaml` + +Two parts matter: a `plugins` entry that declares the WASM plugin, and a +`codegen` entry inside your `sql` block that runs it with a set of `options`. + +```yaml +# filename: sqlc.yaml +version: "2" +plugins: + - name: python + wasm: + url: https://github.com/rayakame/sqlc-gen-better-python/releases/download/v0.5.1/sqlc-gen-better-python.wasm + sha256: c7cc470df7625ae3232c2b042060b948180ae784ce3d81c32e8a2c040fe04fa7 +sql: + - engine: "postgresql" + queries: "query.sql" + schema: "schema.sql" + codegen: + - out: "app/db" + plugin: python + options: + package: "db" + emit_init_file: true + sql_driver: "asyncpg" + model_type: "msgspec" +``` + +- **`plugins`** declares the plugin once, pinned to a release `url` and its + `sha256`. `sqlc` downloads and runs the WASM binary. +- **`sql`** points `sqlc` at your `engine`, `queries`, and `schema`. +- **`codegen`** runs the plugin: `out` is the output directory, `plugin` refers + to the name declared above, and `options` configures this plugin. + +{{< callout type="warning" >}} + The `sha256` must match the release you pin - `sqlc` refuses to run a plugin + whose hash does not match. Each release lists its hash. +{{< /callout >}} + +## The options block + +`options` is where all plugin configuration lives. Three options are required: + +| Option | What it does | +|---|---| +| `package` | The name of the generated package. | +| `sql_driver` | `asyncpg`, `aiosqlite`, or `sqlite3` - must match the `engine`. See [Drivers](/docs/guide/drivers). | +| `emit_init_file` | Whether to emit `__init__.py`. Must be set explicitly. | + +Everything else is optional and has a sensible default. The most common ones to +reach for next are [`model_type`](/docs/guide/model-types) and +[`docstrings`](/docs/guide/docstrings). The full list, with types and defaults, +is in the [reference](/docs/reference/configuration-options). + +## Multiple outputs + +A single `sql` block can have several `codegen` entries, each writing to its own +`out`. This is handy to generate more than one flavor from the same schema and +queries - for example a `msgspec` package and a `dataclass` package: + +```yaml + codegen: + - out: "app/db_msgspec" + plugin: python + options: + package: "db_msgspec" + emit_init_file: true + sql_driver: "asyncpg" + model_type: "msgspec" + - out: "app/db_dataclass" + plugin: python + options: + package: "db_dataclass" + emit_init_file: true + sql_driver: "asyncpg" + model_type: "dataclass" +``` + +## Common pitfalls + +- **Driver/engine mismatch.** `sql_driver: asyncpg` requires `engine: "postgresql"`; + `aiosqlite`/`sqlite3` require `engine: "sqlite"`. A mismatch is an error. +- **Forgetting `emit_init_file`.** It has no default and generation fails if it + is omitted. Set it to `true` unless the package already has an `__init__.py`. +- **A stale `sha256`.** When you bump the plugin version, update the hash too. diff --git a/docs/content/docs/guide/converters.md b/docs/content/docs/guide/converters.md new file mode 100644 index 00000000..95a2fb3a --- /dev/null +++ b/docs/content/docs/guide/converters.md @@ -0,0 +1,95 @@ +--- +title: Converters +weight: 70 +prev: /docs/guide/type-overrides +next: /docs/guide/working-with-json +--- + +A [type override](/docs/guide/type-overrides) replaces a column's Python type, +but it converts by *calling that type* (`Preferences(value)`), which cannot +express JSON or model (de)serialization. A **converter** names two of your own +functions instead, used whenever the column is read from or written to the +database. + +## Specification + +A converter is declared under `converters` and referenced *by an override*: + +| Field | Description | +|---|---| +| `name` | Unique name, referenced by an override's `converter`. | +| `py_type` | The Python type the converter produces and accepts (`import`/`package`/`type`). | +| `to_db` | Dotted path to a function turning the Python value into the column's wire type. | +| `from_db` | Dotted path to a function turning the wire value into the Python type. | + +```yaml +options: + converters: + - name: prefs + py_type: + import: myapp.models + package: Preferences + type: Preferences + to_db: myapp.converters.encode_preferences + from_db: myapp.converters.decode_preferences + overrides: + - db_type: jsonb # every jsonb column + converter: prefs + - column: users.preferences # or one specific column + converter: prefs +``` + +Because the converter is referenced by an override, it applies to whichever +columns that override matches, and `converter` replaces `py_type` on the +override. + +## Generated code + +The column is typed with your `py_type`, and the plugin inserts calls to your +functions on both sides: + +```python +# read - from_db on each column +return models.User(id_=row[0], preferences=myapp.converters.decode_preferences(row[1])) + +# write - to_db on each parameter +await conn.execute(CREATE_USER, id_, myapp.converters.encode_preferences(preferences)) +``` + +Nullable columns are guarded, so your functions never receive `None`: + +```python +maybe_prefs=myapp.converters.decode_preferences(row[2]) if row[2] is not None else None +``` + +and list columns convert element-wise: + +```python +[myapp.converters.encode_label(v) for v in labels] +``` + +## Rules + +- **Both directions are required.** `to_db` and `from_db` are dotted paths to + module-level functions; the module is imported and the function called fully + qualified, so the names can never collide with generated code. +- **Wire type.** `to_db` must return the type the column would have had *without* + the override (`jsonb` is a `str`, `bytea` a `memoryview` - see + [type mappings](/docs/reference/type-mappings)), and `from_db` receives that + same type. For an unrecognised SQL type there is no such type, so `to_db` must + return one the driver accepts. +- **Your functions never see `None`.** A SQL `NULL` stays `None` without calling + the converter. +- **List columns convert element-wise.** + +{{< callout type="warning" >}} + To reach an `ANY($1::type[])` array parameter, match the override with + `db_type` rather than `column` - sqlc does not link column overrides to those + parameters. +{{< /callout >}} + +## Next + +Converters are most often reached for with JSON columns. See +[Working with JSON](/docs/guide/working-with-json) for a complete walkthrough +using `msgspec` structs. diff --git a/docs/content/docs/guide/docstrings.md b/docs/content/docs/guide/docstrings.md new file mode 100644 index 00000000..d683a281 --- /dev/null +++ b/docs/content/docs/guide/docstrings.md @@ -0,0 +1,137 @@ +--- +title: Docstrings +weight: 80 +prev: /docs/guide/working-with-json +next: /docs/guide/sqlite-type-conversion +--- + +The plugin can document everything it generates - models, enums, and query +functions - in the docstring convention of your choice. + +```yaml +options: + docstrings: "google" # none | google | numpy | pep257 + docstrings_emit_sql: true # embed each query's SQL (default true) +``` + +The default is `none`, which emits no docstrings at all. + +## The three conventions + +The same `:one` query, generated under each convention: + +{{< tabs >}} + + {{< tab name="google" >}} +````python +async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFieldNaming | None: + """Fetch one from the db using the SQL query with `name: GetFieldNaming :one`. + + ```sql + SELECT id, outputs + FROM test_field_namings + WHERE id = $1 LIMIT 1 + ``` + + Args: + conn: + Connection object of type `ConnectionLike` used to execute the query. + id_: int. + + Returns: + Result of type `models.TestFieldNaming` fetched from the db. Will be `None` if not found. + """ +```` + {{< /tab >}} + + {{< tab name="numpy" >}} +````python +async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFieldNaming | None: + """Fetch one from the db using the SQL query with `name: GetFieldNaming :one`. + + ```sql + SELECT id, outputs + FROM test_field_namings + WHERE id = $1 LIMIT 1 + ``` + + Parameters + ---------- + conn : ConnectionLike + Connection object of type `ConnectionLike` used to execute the query. + id_ : int + + Returns + ------- + models.TestFieldNaming + Result fetched from the db. Will be `None` if not found. + + """ +```` + {{< /tab >}} + + {{< tab name="pep257" >}} +````python +async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFieldNaming | None: + """Fetch one from the db using the SQL query with `name: GetFieldNaming :one`. + + ```sql + SELECT id, outputs + FROM test_field_namings + WHERE id = $1 LIMIT 1 + ``` + + Arguments: + conn -- Connection object of type `ConnectionLike` used to execute the query. + id_ -- int. + + Returns: + models.TestFieldNaming -- Result fetched from the db. Will be `None` if not found. + """ +```` + {{< /tab >}} + +{{< /tabs >}} + +## Embedded SQL + +By default each query function's docstring includes the exact SQL it runs, in a +fenced `sql` block - handy when reading generated code or hovering a call in your +editor. Set `docstrings_emit_sql: false` to leave it out. + +{{< callout type="info" >}} + `:copyfrom` functions never embed SQL, since they do not execute a statement - + they stream rows via the driver's bulk copy API. +{{< /callout >}} + +## Models and enums + +Models and enums are documented too. Under `numpy`, the generated model carries an +`Attributes` section: + +```python +@attrs.define() +class TestFieldNaming: + """Model representing TestFieldNaming. + + Attributes + ---------- + id_ : int + outputs : str + + """ + + id_: int + outputs: str +``` + +## Linting generated code + +If you run a docstring linter (ruff's `pydocstyle` rules, for example) over the +generated package, set its convention to match the one you generated, or every +file will report style violations. In `ruff.toml`: + +```toml +[lint.pydocstyle] +convention = "google" +``` diff --git a/docs/content/docs/guide/drivers.md b/docs/content/docs/guide/drivers.md new file mode 100644 index 00000000..e1ae9916 --- /dev/null +++ b/docs/content/docs/guide/drivers.md @@ -0,0 +1,67 @@ +--- +title: Drivers +weight: 20 +prev: /docs/guide/configuration +next: /docs/guide/model-types +--- + +The `sql_driver` option picks which database library the generated code targets. +It must match your `engine`. Three drivers are supported: + +| Driver | Engine | Style | +|---|---|---| +| `asyncpg` | `postgresql` | async | +| `aiosqlite` | `sqlite` | async | +| `sqlite3` | `sqlite` | sync | + +Every generated query function takes the connection as its first argument, so you +open and manage the connection yourself and pass it in. + +## asyncpg (PostgreSQL) + +```python +import asyncpg + +from app.db import queries + +conn = await asyncpg.connect("postgresql://user:pass@localhost/db") +user = await queries.get_field_naming(conn, id_=1) +``` + +asyncpg is the only driver that supports `:copyfrom` (bulk insert via +`copy_records_to_table`). + +## aiosqlite (async SQLite) + +```python +import aiosqlite + +from app.db import queries + +async with aiosqlite.connect("app.db", detect_types=aiosqlite.PARSE_DECLTYPES) as conn: + user = await queries.get_field_naming(conn, id_=1) +``` + +## sqlite3 (sync SQLite) + +```python +import sqlite3 + +from app.db import queries + +conn = sqlite3.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) +user = queries.get_field_naming(conn, id_=1) +``` + +{{< callout type="warning" >}} + Both SQLite drivers require `detect_types=PARSE_DECLTYPES` on the connection. + The generated code registers adapters and converters (for dates, decimals, + booleans, and blobs) that only run when declared-type parsing is enabled. See + [SQLite type conversion](/docs/guide/sqlite-type-conversion). +{{< /callout >}} + +## Command support + +Not every [query command](/docs/guide/writing-queries) works on every driver - +for example `:copyfrom` is asyncpg-only and `:execlastid` is SQLite-only. The +full matrix is in the [feature support reference](/docs/reference/feature-support). diff --git a/docs/content/docs/guide/enums.md b/docs/content/docs/guide/enums.md new file mode 100644 index 00000000..26ed966d --- /dev/null +++ b/docs/content/docs/guide/enums.md @@ -0,0 +1,68 @@ +--- +title: Enums +weight: 50 +prev: /docs/guide/writing-queries +next: /docs/guide/type-overrides +--- + +PostgreSQL enum types become `enum.StrEnum` classes in a generated `enums.py` +module. Columns of that type are annotated with the class, and values read from +the database are coerced into it. + +## Example + +```sql +CREATE TYPE test_mood AS ENUM ('sad', 'ok', 'happy', '24h', '_hidden'); +``` + +generates: + +```python +class TestMood(enum.StrEnum): + SAD = "sad" + OK = "ok" + HAPPY = "happy" + VALUE_24H = "24h" + VALUE__HIDDEN = "_hidden" +``` + +Notice the member names are uppercased, and values that are not valid Python +identifiers are sanitized: the digit-leading `24h` becomes `VALUE_24H` and the +underscore-leading `_hidden` becomes `VALUE__HIDDEN`. The string *values* are +untouched, so round-tripping to the database is exact. See +[Naming and identifiers](/docs/guide/naming) for the full sanitization rules. + +## Using enums + +A column of an enum type is annotated with the generated class, and nullable +columns get `| None`: + +```sql +CREATE TABLE test_enum_types +( + id int PRIMARY KEY NOT NULL, + mood test_mood NOT NULL, + maybe_mood test_mood +); +``` + +```python +class TestEnumType(msgspec.Struct): + id_: int + mood: enums.TestMood + maybe_mood: enums.TestMood | None +``` + +Query functions coerce database values into these classes automatically, so you +get real `TestMood` members back, not bare strings. + +## Enums in other schemas + +Enums in a non-default schema get schema-qualified class names so same-named +enums never collide - for example `custom.mood` becomes `CustomMood`, distinct +from a `public.mood` that would become `Mood`. + +{{< callout type="info" >}} + Enum classes are a PostgreSQL feature - SQLite has no native enum type, so this + applies to the `asyncpg` driver. +{{< /callout >}} diff --git a/docs/content/docs/guide/model-types.md b/docs/content/docs/guide/model-types.md new file mode 100644 index 00000000..ca5539ae --- /dev/null +++ b/docs/content/docs/guide/model-types.md @@ -0,0 +1,95 @@ +--- +title: Model types +weight: 30 +prev: /docs/guide/drivers +next: /docs/guide/writing-queries +--- + +The `model_type` option controls what kind of class the plugin generates for each +table. All four produce the same fields with the same annotations - they differ +only in the class machinery and dependencies. + +```yaml +options: + model_type: "msgspec" # dataclass | attrs | msgspec | pydantic +``` + +## The four forms + +Given this table: + +```sql +CREATE TABLE test_field_namings +( + id bigint PRIMARY KEY NOT NULL, + outputs jsonb NOT NULL +); +``` + +the plugin generates (shown with the default `docstrings: none`): + +{{< tabs >}} + + {{< tab name="dataclass" >}} +```python +@dataclasses.dataclass() +class TestFieldNaming: + id_: int + outputs: str +``` +Standard library, no dependencies. The safe default. + {{< /tab >}} + + {{< tab name="attrs" >}} +```python +@attrs.define() +class TestFieldNaming: + id_: int + outputs: str +``` +Requires `attrs`. Slotted classes with concise definitions. + {{< /tab >}} + + {{< tab name="msgspec" >}} +```python +class TestFieldNaming(msgspec.Struct): + id_: int + outputs: str +``` +Requires `msgspec`. Very fast serialization/validation; pairs well with +[working with JSON](/docs/guide/working-with-json). + {{< /tab >}} + + {{< tab name="pydantic" >}} +```python +class TestFieldNaming(pydantic.BaseModel): + model_config = pydantic.ConfigDict(arbitrary_types_allowed=True) + + id_: int + outputs: str +``` +Requires `pydantic >= 2.9`. Full runtime validation on construction. + {{< /tab >}} + +{{< /tabs >}} + +Which to pick: `dataclass` if you want zero dependencies, `msgspec` for speed and +JSON, `pydantic` for runtime validation, `attrs` for its ergonomics. + +## Pydantic specifics + +`model_type: pydantic` differs from the others in two ways worth knowing: + +- **`arbitrary_types_allowed=True`** is set on every model. It is required for + field types pydantic has no core schema for (`memoryview` for `bytea`/`blob`, + and custom [override](/docs/guide/type-overrides) types). Those fields are still + validated with an `isinstance` check; all other fields get full validation. +- **Runtime annotations.** Unlike the other model types, pydantic resolves field + annotations at runtime, so generated files import referenced modules at module + level instead of inside `if typing.TYPE_CHECKING:`. + +{{< callout type="info" >}} + If you lint generated pydantic code with ruff, set + `lint.flake8-type-checking.runtime-evaluated-base-classes = ["pydantic.BaseModel"]` + so it does not try to move those runtime imports into a `TYPE_CHECKING` block. +{{< /callout >}} diff --git a/docs/content/docs/guide/naming.md b/docs/content/docs/guide/naming.md new file mode 100644 index 00000000..b2d579cb --- /dev/null +++ b/docs/content/docs/guide/naming.md @@ -0,0 +1,97 @@ +--- +title: Naming and identifiers +weight: 100 +prev: /docs/guide/sqlite-type-conversion +--- + +SQL identifiers are not always valid Python names, and SQL naming conventions are +not Python's. This page covers how table, column, and enum names are turned into +Python names, and how invalid ones are sanitized. + +## Table names to class names + +Table names are converted to `CamelCase` and singularized, so a table of many +rows yields a class describing one row: + +| Table | Model | +|---|---| +| `test_field_namings` | `TestFieldNaming` | +| `test_invalid_identifiers` | `TestInvalidIdentifier` | + +Two options change this: + +- **`emit_exact_table_names: true`** - skip singularization; model names mirror + table names exactly. +- **`inflection_exclude_table_names`** - a list of tables to leave + un-singularized while everything else is still singularized. Entries match both + bare and schema-qualified names. + +## Initialisms + +Segments listed in `initialisms` are fully upper-cased when building class names, +so you get `AppID` rather than `AppId`. It defaults to `["id"]`: + +```yaml +options: + initialisms: ["id", "url", "api"] +``` + +## Column names to field names + +Column names stay `snake_case`. Names that collide with Python keywords or +builtins get a trailing underscore - which is why an `id` column becomes `id_`: + +```python +class TestFieldNaming(msgspec.Struct): + id_: int + outputs: str +``` + +## Sanitization + +Identifiers that are not valid Python names are rewritten. Invalid characters +become underscores, and prefixes are added where a name would otherwise be +illegal: + +```sql +CREATE TABLE IF NOT EXISTS test_invalid_identifiers +( + id bigint PRIMARY KEY NOT NULL, + "3p%" text, + "new notes" text NOT NULL, + "%pct" text +); +``` + +```python +class TestInvalidIdentifier(msgspec.Struct): + id_: int + column_3p_: str | None + new_notes: str + column__pct: str | None +``` + +The rules at work: + +| Situation | Rule | Example | +|---|---|---| +| Invalid characters | Replaced with `_` | `new notes` -> `new_notes` | +| Column starts with a digit | Prefixed with `column_` | `3p%` -> `column_3p_` | +| Field starts with `_` | Prefixed with `column_` | `%pct` -> `column__pct` | +| Table yields a digit-leading, keyword, or empty class name | Prefixed with `Model` | `3rd_party_stats` -> `Model3RdPartyStat` | +| Enum value starts with a digit or `_` | Prefixed with `VALUE_` | `24h` -> `VALUE_24H` | +| Two results collide | Numeric suffix | `outputs`, `outputs` -> `outputs`, `outputs_2` | + +A leading underscore is prefixed rather than kept because `attrs` and `pydantic` +treat underscore-leading fields as private. + +The collision rule also covers queries that select the same column name twice - +for example a self-join selecting `a.outputs, b.outputs` produces a row class with +`outputs` and `outputs_2`. + +{{< callout type="warning" >}} + Sanitization checks characters with Go's Unicode tables, which differ from + Python's identifier rules in exotic cases - characters that may not *start* an + identifier in Python, or two names Python normalizes to the same identifier via + NFKC. Such schemas are not detected. Stick to ASCII identifiers if in doubt. +{{< /callout >}} diff --git a/docs/content/docs/guide/sqlite-type-conversion.md b/docs/content/docs/guide/sqlite-type-conversion.md new file mode 100644 index 00000000..611edc68 --- /dev/null +++ b/docs/content/docs/guide/sqlite-type-conversion.md @@ -0,0 +1,87 @@ +--- +title: SQLite type conversion +weight: 90 +prev: /docs/guide/docstrings +next: /docs/guide/naming +--- + +SQLite only stores a handful of native types, so dates, decimals, booleans and +blobs have to be translated in both directions. For the `sqlite3` and `aiosqlite` +drivers the generated code registers **adapters** (Python value -> SQLite) and +**converters** (SQLite value -> Python) for you. + +## Enable declared-type parsing + +Converters only run when the connection parses declared column types, so you must +open the connection with `PARSE_DECLTYPES`: + +{{< tabs >}} + + {{< tab name="sqlite3" >}} +```python +conn = sqlite3.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) +``` + {{< /tab >}} + + {{< tab name="aiosqlite" >}} +```python +async with aiosqlite.connect("app.db", detect_types=aiosqlite.PARSE_DECLTYPES) as conn: + ... +``` + {{< /tab >}} + +{{< /tabs >}} + +{{< callout type="warning" >}} + Without `detect_types=PARSE_DECLTYPES` the converters never fire and you get raw + strings and bytes back instead of `datetime`, `Decimal`, `bool`, or `memoryview` + values. This is the most common SQLite setup mistake. +{{< /callout >}} + +## What gets converted + +| Declared SQL type | Python type | Sent to SQLite as | Read back with | +|---|---|---|---| +| `date` | `datetime.date` | `str` | `datetime.date.fromisoformat` | +| `datetime`, `timestamp` | `datetime.datetime` | `str` | `datetime.datetime.fromisoformat` | +| `decimal`, `decimal(p,s)` | `decimal.Decimal` | `str` | `decimal.Decimal` | +| `bool`, `boolean` | `bool` | `int` | `bool(int(...))` | +| `blob` | `memoryview` | `bytes` | `memoryview` | + +The converter is keyed on the column's **declared type name**, which is why the +declared types in your schema matter for SQLite. + +## Only what each module needs + +Registration is emitted per generated module, and only for what that module +actually uses: + +- A type used as a **parameter** gets its **adapter** registered. +- A type used in a **returned column** gets its **converter** registered. + +{{< callout type="info" >}} + `register_converter` is process-global in Python's `sqlite3`. Registering only + what a module needs keeps modules from stepping on each other's conversions. +{{< /callout >}} + +## Interaction with overrides + +A column with a [type override](/docs/guide/type-overrides) or +[converter](/docs/guide/converters) is converted inline by the generated code, so +**no** SQLite converter is registered for it - registering one would convert the +value twice. Overridden values used as *parameters* still need the adapter, so +that half is still registered. + +## Faster date parsing + +`speedups: true` swaps the standard-library date parsing for +[`ciso8601`](https://github.com/closeio/ciso8601): + +```yaml +options: + speedups: true +``` + +This affects the `date` and `datetime`/`timestamp` converters only - `decimal`, +`bool` and `blob` are unchanged. It adds `ciso8601` as a runtime dependency of the +generated code, so only enable it if you install that package. diff --git a/docs/content/docs/guide/type-overrides.md b/docs/content/docs/guide/type-overrides.md new file mode 100644 index 00000000..29d6da27 --- /dev/null +++ b/docs/content/docs/guide/type-overrides.md @@ -0,0 +1,90 @@ +--- +title: Type overrides +weight: 60 +prev: /docs/guide/enums +next: /docs/guide/converters +--- + +An override replaces the Python type the plugin would pick for a column with one +of your own. You can target a specific column or every column of a given SQL +type. + +## Specification + +Each entry in `overrides` matches columns in one of two ways (exactly one is +required): + +- **`column`** - a specific column, as `[schema.]table.column` (wildcards + allowed). +- **`db_type`** - every column of a SQL type, e.g. `text`. + +and supplies a `py_type` describing the replacement type and how to import it +(`import`, `package`, `type`). Full schema in the +[reference](/docs/reference/configuration-options#override). + +## Example + +Override one `text` column with `collections.UserString`: + +```yaml +options: + overrides: + - column: test_type_override.text_test + py_type: + import: collections + package: UserString + type: UserString +``` + +The model field now uses your type: + +```python +class TestTypeOverride(msgspec.Struct): + id_: int + text_test: UserString | None +``` + +and query functions convert on the way in and out: + +```python +# read: construct your type from the column value +return models.TestTypeOverride(id_=row[0], text_test=UserString(row[1]) if row[1] is not None else None) + +# write: convert back to the column's default type (str for text) +await conn.execute(INSERT_TYPE_OVERRIDE, id_, str(text_test) if text_test is not None else None) +``` + +## How conversion works + +An override converts by **calling the type**. On read the plugin builds the value +with `YourType(column_value)`; on write it calls the column's default type +(`str(...)` here) to turn it back. Nullable columns are guarded, so a SQL `NULL` +stays `None` and the type is never called with `None`. + +This is why an override works for anything constructible from its wire value +(`UserString(str)`, `PurePosixPath(str)`, ...) but *not* for things like JSON, +where `MyModel(json_string)` is meaningless. For those, use a +[converter](/docs/guide/converters), which names your own encode/decode functions +instead. + +## Matching by SQL type + +Use `db_type` to override every column of a type at once: + +```yaml +overrides: + - db_type: text + py_type: + import: collections + package: UserString + type: UserString +``` + +When both a `column` and a `db_type` override could match a column, the `column` +override wins. + +{{< callout type="warning" >}} + Column overrides do not attach to `ANY($1::type[])` array parameters - sqlc does + not link a column override to those. Only `db_type` overrides (and converters + matched by `db_type`) reach them. +{{< /callout >}} diff --git a/docs/content/docs/guide/working-with-json.md b/docs/content/docs/guide/working-with-json.md new file mode 100644 index 00000000..4749fbc3 --- /dev/null +++ b/docs/content/docs/guide/working-with-json.md @@ -0,0 +1,241 @@ +--- +title: Working with JSON +weight: 75 +prev: /docs/guide/converters +next: /docs/guide/docstrings +--- + +A `json`/`jsonb` column maps to `str` by default - correct, but untyped: you get +a string and parse it yourself everywhere. With a [converter](/docs/guide/converters) +and a `msgspec.Struct` you get a fully typed, validated Python object instead, +and the plugin inserts the encode/decode calls for you. + +## Why JSON needs a converter + +A [type override](/docs/guide/type-overrides) converts by *calling the type*, so +it would generate `Preferences(row[2])` - handing a JSON string to a struct +constructor, which is meaningless. JSON needs a real parse step, which is exactly +what a converter's `from_db`/`to_db` pair provides. + +## The wire type contract + +This is the one rule to internalize: + +{{< callout type="info" >}} + A `jsonb` (or `json`) column's wire type is **`str`**. So `to_db` must *return* + `str`, and `from_db` *receives* `str`. +{{< /callout >}} + +That matters for `msgspec`, because `msgspec.json.encode()` returns **`bytes`** - +you must `.decode()` it. `msgspec.json.decode()` happily accepts either. + +## A complete example + +A `users` table with a nested preferences document: + +```sql +CREATE TABLE users +( + id bigint PRIMARY KEY NOT NULL, + name text NOT NULL, + preferences jsonb NOT NULL +); +``` + +```sql +-- name: GetUser :one +SELECT * FROM users WHERE id = $1; + +-- name: CreateUser :exec +INSERT INTO users (id, name, preferences) VALUES ($1, $2, $3); +``` + +{{< tabs >}} + + {{< tab name="models.py" >}} +```python +import msgspec + + +class Theme(msgspec.Struct): + name: str + dark_mode: bool = False + + +class Preferences(msgspec.Struct): + theme: Theme + languages: list[str] = msgspec.field(default_factory=list) + email_opt_in: bool | None = None +``` +Structs nest freely - `Preferences` holds a `Theme`, plus a list and an optional +field. `msgspec` handles all of it. + {{< /tab >}} + + {{< tab name="converters.py" >}} +```python +import msgspec + +from myapp.models import Preferences + + +def encode_preferences(value: Preferences) -> str: + # jsonb's wire type is str, and msgspec encodes to bytes. + return msgspec.json.encode(value).decode() + + +def decode_preferences(value: str) -> Preferences: + return msgspec.json.decode(value, type=Preferences) +``` +The `type=` argument is what makes decoding *typed* - msgspec builds real +`Preferences` objects and validates the payload while parsing. + {{< /tab >}} + + {{< tab name="sqlc.yaml" >}} +```yaml +options: + model_type: "msgspec" + converters: + - name: preferences + py_type: + import: myapp.models + package: Preferences + type: Preferences + to_db: myapp.converters.encode_preferences + from_db: myapp.converters.decode_preferences + overrides: + - column: users.preferences + converter: preferences +``` + {{< /tab >}} + +{{< /tabs >}} + +### What gets generated + +The model field is typed as your struct, and both query functions call your +converter: + +```python +class User(msgspec.Struct): + id_: int + name: str + preferences: Preferences + + +async def get_user(conn: ConnectionLike, *, id_: int) -> models.User | None: + row = await conn.fetchrow(GET_USER, id_) + if row is None: + return None + return models.User(id_=row[0], name=row[1], preferences=myapp.converters.decode_preferences(row[2])) + + +async def create_user(conn: ConnectionLike, *, id_: int, name: str, preferences: Preferences) -> None: + await conn.execute(CREATE_USER, id_, name, myapp.converters.encode_preferences(preferences)) +``` + +### Using it + +```python +await create_user( + conn, + id_=1, + name="ada", + preferences=Preferences(theme=Theme(name="solarized", dark_mode=True), languages=["en"]), +) + +user = await get_user(conn, id_=1) +user.preferences.theme.dark_mode # True - typed all the way down +``` + +No `json.loads`, no `dict["theme"]["dark_mode"]`, and pyright checks every access. + +## Several JSON columns with different shapes + +Match **per column**, one converter per struct: + +```yaml +options: + converters: + - name: preferences + py_type: { import: myapp.models, package: Preferences, type: Preferences } + to_db: myapp.converters.encode_preferences + from_db: myapp.converters.decode_preferences + - name: audit_meta + py_type: { import: myapp.models, package: AuditMeta, type: AuditMeta } + to_db: myapp.converters.encode_audit_meta + from_db: myapp.converters.decode_audit_meta + overrides: + - column: users.preferences + converter: preferences + - column: audit_log.meta + converter: audit_meta +``` + +{{< callout type="warning" >}} + Reaching for `db_type: jsonb` applies **one** struct to *every* `jsonb` column + in your schema. That is only what you want when all of them genuinely share a + shape - otherwise match by `column`. +{{< /callout >}} + +## Nullable and array JSON columns + +- **Nullable** (`preferences jsonb` without `NOT NULL`): the field becomes + `Preferences | None` and the plugin guards the call, so `decode_preferences` + never receives `None`. +- **Arrays** (`jsonb[]`): the field becomes + `collections.abc.Sequence[Preferences]` and conversion happens element-wise - + your functions still deal with a single value at a time. To also cover an + `ANY($1::jsonb[])` parameter, match with `db_type: jsonb`. + +## Validation + +`msgspec.json.decode(..., type=Preferences)` validates while parsing and raises +`msgspec.ValidationError` when a stored document does not match the struct (a +missing required field, a wrong type). That is usually what you want - it surfaces +bad data at the boundary. If you would rather tolerate legacy rows, give fields +defaults or widen them to `| None`. + +## Other libraries + +The converter contract is just "return the wire type, accept the wire type", so +any library works: + +{{< tabs >}} + + {{< tab name="stdlib json" >}} +```python +import dataclasses +import json + +from myapp.models import Preferences + + +def encode_preferences(value: Preferences) -> str: + return json.dumps(dataclasses.asdict(value)) + + +def decode_preferences(value: str) -> Preferences: + return Preferences(**json.loads(value)) +``` +No dependencies, but no validation and no nested-struct rebuilding. + {{< /tab >}} + + {{< tab name="pydantic" >}} +```python +from myapp.models import Preferences + + +def encode_preferences(value: Preferences) -> str: + return value.model_dump_json() + + +def decode_preferences(value: str) -> Preferences: + return Preferences.model_validate_json(value) +``` +Full validation, and `model_validate_json` parses straight from the string. + {{< /tab >}} + +{{< /tabs >}} + +The same approach works for the SQLite drivers: a `json` column is `str` there +too, so the identical converter applies. diff --git a/docs/content/docs/guide/writing-queries.md b/docs/content/docs/guide/writing-queries.md new file mode 100644 index 00000000..261179d4 --- /dev/null +++ b/docs/content/docs/guide/writing-queries.md @@ -0,0 +1,133 @@ +--- +title: Writing queries +weight: 40 +prev: /docs/guide/model-types +next: /docs/guide/enums +--- + +Each query `.sql` file becomes one Python module, and each annotated query in it +becomes one typed function. This page shows how the `-- name:` annotation maps to +a function and what each query command generates. + +## Anatomy of a query + +A query is a `-- name: :` comment followed by SQL: + +```sql +-- name: GetFieldNaming :one +SELECT id, outputs +FROM test_field_namings +WHERE id = $1 LIMIT 1; +``` + +From that the plugin generates: + +```python +GET_FIELD_NAMING: typing.Final[str] = """-- name: GetFieldNaming :one +SELECT id, outputs +FROM test_field_namings +WHERE id = $1 LIMIT 1 +""" + + +async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFieldNaming | None: + row = await conn.fetchrow(GET_FIELD_NAMING, id_) + if row is None: + return None + return models.TestFieldNaming(id_=row[0], outputs=row[1]) +``` + +- The SQL is stored once as a module-level constant. +- `GetFieldNaming` becomes the function `get_field_naming`. +- The connection is the first positional argument (`conn`); its type + `ConnectionLike` is a generated alias for the driver's connection (or pool). +- Query parameters (`$1`) become keyword-only arguments (`id_`). +- The `:one` command sets the return type and body. + +## Query commands + +### `:one` + +Returns a single model instance or `None`. Shown above. + +When a query's columns do not match one table exactly (a join, a partial select, +an aggregate), the plugin generates a dedicated `Row` class instead of +reusing a table model: + +```python +class GetJoinedFieldNamingsRow(msgspec.Struct): + outputs: str + outputs_2: str + + +async def get_joined_field_namings(conn: ConnectionLike, *, id_: int) -> GetJoinedFieldNamingsRow | None: + row = await conn.fetchrow(GET_JOINED_FIELD_NAMINGS, id_) + if row is None: + return None + return GetJoinedFieldNamingsRow(outputs=row[0], outputs_2=row[1]) +``` + +### `:many` + +Returns a `QueryResults[T]` - a helper that supports both async iteration and +one-shot fetching, so you do not pay for materializing the whole result set +unless you want it: + +```python +def get_many_test_timestamp_postgres_type(conn: ConnectionLike, *, id_: int) -> QueryResults[datetime.datetime]: + return QueryResults(conn, GET_MANY_TEST_TIMESTAMP_POSTGRES_TYPE, operator.itemgetter(0), id_) +``` + +Here the query selects one column, so `T` is a scalar (`datetime.datetime`). A +`:many` that selects full rows returns `QueryResults[models.Foo]` (or a +`Row`). + +### `:exec` + +Runs the statement and returns `None`: + +```python +async def set_field_naming_outputs(conn: ConnectionLike, *, id_: int, outputs: str, outputs_2: str) -> None: + await conn.execute(SET_FIELD_NAMING_OUTPUTS, id_, outputs, outputs_2) +``` + +### `:execresult`, `:execrows`, `:execlastid` + +Variants of `:exec` that return something about the write: + +- **`:execrows`** - the number of affected rows (`int`). +- **`:execlastid`** - the id of the last inserted row (`int`); SQLite drivers only. +- **`:execresult`** - the driver's raw result/status object. + +See the [feature support matrix](/docs/reference/feature-support) for which +driver supports which. + +### `:copyfrom` + +asyncpg only. Bulk-inserts rows via `copy_records_to_table`, taking a sequence of +generated `Params` objects and returning the affected row count: + +```python +async def test_copy_from(conn: ConnectionLike, *, params: collections.abc.Sequence[TestCopyFromParams]) -> int: + records = [(param.id_, param.float_test, param.int_test) for param in params] + r = await conn.copy_records_to_table("test_copy_from", columns=["id", "float_test", "int_test"], records=records) + return int(n) if (p := r.split()) and (n := p[-1]).isdigit() else 0 +``` + +## Parameters + +By default every parameter is **keyword-only** (note the `*,` in the signatures +above), which keeps call sites readable and order-independent. Two options tune +this: + +- **`omit_kwargs_limit`** - queries with this many parameters or fewer allow + positional arguments. Defaults to `0` (always keyword-only). +- **`query_parameter_limit`** - when a query has more parameters than the limit, + they are bundled into a single generated `Params` object instead of + expanded into the signature. `:copyfrom` always uses a params class. + +## Grouping into a class + +With `emit_classes: true`, the standalone functions become methods on a single +`Querier` class instead. The bodies are identical; only the call style changes +(`Querier(conn).get_field_naming(id_=1)`). diff --git a/docs/content/docs/reference/_index.md b/docs/content/docs/reference/_index.md new file mode 100644 index 00000000..9c91167a --- /dev/null +++ b/docs/content/docs/reference/_index.md @@ -0,0 +1,16 @@ +--- +title: Reference +weight: 3 +prev: /docs/guide +sidebar: + open: true +--- + +Canonical lookup material. The [Guide](/docs/guide) explains when and why to use +a feature; the Reference is the exhaustive *what*. + +{{< cards >}} + {{< card link="configuration-options" title="Configuration options" subtitle="Every plugin option, its type, default, and meaning." >}} + {{< card link="type-mappings" title="Type mappings" subtitle="How SQL types map to Python types, per engine." >}} + {{< card link="feature-support" title="Feature support" subtitle="Supported macros and per-driver query commands." >}} +{{< /cards >}} diff --git a/docs/content/docs/reference/configuration-options.md b/docs/content/docs/reference/configuration-options.md new file mode 100644 index 00000000..6b0ffb60 --- /dev/null +++ b/docs/content/docs/reference/configuration-options.md @@ -0,0 +1,78 @@ +--- +title: Configuration options +weight: 10 +prev: /docs/reference +next: /docs/reference/type-mappings +--- + +Every option the plugin accepts, set under `options:` in the plugin's `codegen` +block. See [Configuration](/docs/guide/configuration) for where this block lives +in `sqlc.yaml`. + +## Options + +`package`, `sql_driver`, and `emit_init_file` are required; everything else is +optional. + +| Option | Type | Default | Description | +|---|---|---|---| +| `package` | string | *required* | Name of the generated package. | +| `sql_driver` | string | *required* | One of `asyncpg`, `aiosqlite`, `sqlite3`. Must match the engine (`asyncpg` -> `postgresql`; the other two -> `sqlite`). | +| `emit_init_file` | bool | *required* | Whether to emit an `__init__.py` in the package. Must be set explicitly. Set `false` only if the package already has one. | +| `model_type` | string | `dataclass` | One of `dataclass`, `attrs`, `msgspec`, `pydantic`. See [Model types](/docs/guide/model-types). | +| `initialisms` | list[string] | `["id"]` | Identifier segments to upper-case, e.g. `app_id` -> `AppID`. | +| `emit_exact_table_names` | bool | `false` | If `true`, model names mirror table names; otherwise plural table names are singularized. | +| `emit_classes` | bool | `false` | If `true`, query functions become methods on a `Querier` class. See [Writing queries](/docs/guide/writing-queries). | +| `inflection_exclude_table_names` | list[string] | `[]` | Table names to leave un-singularized (only used when `emit_exact_table_names` is `false`). | +| `omit_unused_models` | bool | `false` | If `true`, tables not referenced by any query are not turned into models. | +| `omit_typechecking_block` | bool | `false` | If `true`, non-builtin types are imported at module level instead of inside a `typing.TYPE_CHECKING` block. | +| `docstrings` | string | `none` | One of `none`, `google`, `numpy`, `pep257`. See [Docstrings](/docs/guide/docstrings). | +| `docstrings_emit_sql` | bool | `true` | Include each query's SQL in its docstring. Ignored when `docstrings` is `none`. | +| `query_parameter_limit` | int | *unset* | When set to a non-negative value, queries with more parameters than the limit bundle them into a single `params` argument. Unset or negative never bundles (except `:copyfrom`, which always uses a params class). | +| `omit_kwargs_limit` | int | `0` | Queries with this many parameters or fewer do not require keyword arguments. `0` makes every parameter keyword-only. Must not be negative. | +| `speedups` | bool | `false` | Use faster third-party libraries for type conversion. Currently affects `sqlite3`/`aiosqlite` only (uses `ciso8601`). See [SQLite type conversion](/docs/guide/sqlite-type-conversion). | +| `overrides` | list[Override] | `[]` | Replace the Python type of matching columns. See [Type overrides](/docs/guide/type-overrides). | +| `converters` | list[Converter] | `[]` | Named encode/decode function pairs referenced by an override. See [Converters](/docs/guide/converters). | +| `indent_char` | string | `" "` (space) | Character used for one indent step. | +| `chars_per_indent_level` | int | `4` | Number of `indent_char`s per indent level. | +| `debug` | bool | `false` | Emit a `log.txt` debug log during `sqlc generate`. | + +## Object schemas + +Some options take structured objects rather than scalars. + +### `py_type` + +Describes a Python type and how to import it. Used by an `override` and by a +`converter`. + +| Field | Type | Description | +|---|---|---| +| `type` | string | *required* - the type expression used in annotations, e.g. `UserString` or `collections.UserString`. | +| `import` | string | Module to import, e.g. `collections`. | +| `package` | string | Name to import from `import` (`from import `). If empty, emits `import `. | + +### `override` + +An entry in `overrides`. Matches columns either by SQL type or by column name; +specifying both, or neither, is an error. + +| Field | Type | Description | +|---|---|---| +| `db_type` | string | Match a SQL type exactly (case-insensitive), e.g. `text`, `pg_catalog.int4`. Mutually exclusive with `column`. | +| `column` | string | Match `[catalog.][schema.]table.column`; wildcards are supported. Mutually exclusive with `db_type`. | +| `py_type` | py_type | Replacement type. Required unless `converter` is set. | +| `converter` | string | Name of a `converters` entry, which supplies the type and its encode/decode functions instead of `py_type`. | + +When both a `column` and a `db_type` override could match, the column match wins. + +### `converter` + +An entry in `converters`. See [Converters](/docs/guide/converters). + +| Field | Type | Description | +|---|---|---| +| `name` | string | *required* - unique name, referenced by an override's `converter`. | +| `py_type` | py_type | *required* - the Python type the converter produces and accepts. | +| `to_db` | string | *required* - dotted path to a function that turns the Python value into the column's wire type. | +| `from_db` | string | *required* - dotted path to a function that turns the wire value into the Python type. | diff --git a/docs/content/docs/reference/feature-support.md b/docs/content/docs/reference/feature-support.md new file mode 100644 index 00000000..877424d2 --- /dev/null +++ b/docs/content/docs/reference/feature-support.md @@ -0,0 +1,44 @@ +--- +title: Feature support +weight: 30 +prev: /docs/reference/type-mappings +--- + +Which sqlc features and query commands the plugin supports. + +## Macros + +Every [sqlc macro](https://docs.sqlc.dev/en/latest/reference/macros.html) is +supported (`sqlc.arg`, `sqlc.narg`, `sqlc.embed`, `sqlc.slice`). + +## Query commands + +The supported [query annotations](https://docs.sqlc.dev/en/latest/reference/query-annotations.html) +depend on the driver: + +| Command | aiosqlite | sqlite3 | asyncpg | +|---|---|---|---| +| `:one` | yes | yes | yes | +| `:many` | yes | yes | yes | +| `:exec` | yes | yes | yes | +| `:execresult` | yes | yes | yes | +| `:execrows` | yes | yes | yes | +| `:execlastid` | yes | yes | no | +| `:copyfrom` | no | no | yes | + +See [Writing queries](/docs/guide/writing-queries) for what each command +generates. + +{{< callout type="info" >}} + `:execlastid` relies on a last-inserted-row id, which asyncpg/PostgreSQL do not + provide; use a `RETURNING` clause with `:one` instead. `:copyfrom` maps to + asyncpg's bulk `copy_records_to_table`, which the SQLite drivers have no + equivalent for. +{{< /callout >}} + +## Not supported + +- **`:batch*` commands** (`:batchexec`, `:batchmany`, `:batchone`) are not + supported and likely never will be. +- **Prepared queries** are not planned for the near future. +- **`psycopg2` and `mysql`** drivers are not currently supported. diff --git a/docs/content/docs/reference/type-mappings.md b/docs/content/docs/reference/type-mappings.md new file mode 100644 index 00000000..9908b9e3 --- /dev/null +++ b/docs/content/docs/reference/type-mappings.md @@ -0,0 +1,74 @@ +--- +title: Type mappings +weight: 20 +prev: /docs/reference/configuration-options +next: /docs/reference/feature-support +--- + +How the plugin maps SQL types to Python types, per engine. These are the +built-in defaults; a [type override](/docs/guide/type-overrides) or +[converter](/docs/guide/converters) replaces the type for a specific column or +SQL type. + +## How a column's type is built + +The tables below give the **base** Python type for a SQL type. Two modifiers are +applied on top, based on the column: + +- **Nullable** (the column is not `NOT NULL`): ` | None` is appended, e.g. + `str | None`. +- **Array / list** (a Postgres array column, or a `sqlc.slice` parameter): the + base type is wrapped as `collections.abc.Sequence[T]`. + +Both can combine: a nullable `bytea[]` becomes +`collections.abc.Sequence[memoryview] | None`. + +## PostgreSQL + +| SQL type | Python type | +|---|---| +| `smallint`, `integer`, `bigint` (and `int2`/`int4`/`int8`, `smallserial`/`serial`/`bigserial`, `pg_catalog.*` forms) | `int` | +| `real`, `double precision` (`float4`/`float8`) | `float` | +| `numeric` | `decimal.Decimal` | +| `money` | `str` | +| `boolean` (`bool`) | `bool` | +| `json`, `jsonb` | `str` | +| `bytea` | `memoryview` | +| `date` | `datetime.date` | +| `time`, `timetz` | `datetime.time` | +| `timestamp`, `timestamptz` | `datetime.datetime` | +| `interval` | `datetime.timedelta` | +| `text`, `varchar`, `char`/`bpchar`, `citext` | `str` | +| `uuid` | `uuid.UUID` | +| `inet`, `cidr`, `macaddr`, `macaddr8` | `str` | +| `ltree`, `lquery`, `ltxtquery` | `str` | +| a user-defined `ENUM` type | the generated [enum class](/docs/guide/enums), e.g. `enums.Mood` | +| anything else | `typing.Any` | + +{{< callout type="info" >}} + A SQL type the plugin does not recognize falls back to `typing.Any`. If that + happens for a type you use often, add a [type override](/docs/guide/type-overrides) + or [converter](/docs/guide/converters) so it gets a real Python type. +{{< /callout >}} + +## SQLite + +SQLite type names are matched case-insensitively, by exact name first and then by +prefix (so `varchar(255)` matches `varchar`, and `decimal(10,5)` matches +`decimal`). + +| SQL type | Python type | +|---|---| +| `int`, `integer`, `tinyint`, `smallint`, `mediumint`, `bigint`, `int2`, `int8`, `bigserial` | `int` | +| `real`, `double`, `double precision`, `float`, `numeric` | `float` | +| `decimal`, `decimal(p,s)` | `decimal.Decimal` | +| `bool`, `boolean` | `bool` | +| `date` | `datetime.date` | +| `datetime`, `timestamp` | `datetime.datetime` | +| `text`, `clob`, `json`, `character`, `varchar`, `varyingcharacter`, `nchar`, `nativecharacter`, `nvarchar` | `str` | +| `blob` | `memoryview` | +| anything else | `typing.Any` | + +For the two SQLite drivers, several of these types also need runtime adapters and +converters (registered in the generated code) to round-trip correctly - see +[SQLite type conversion](/docs/guide/sqlite-type-conversion). From 845db5fb9f591b7b8c00f9ef657a37946abe6b4f Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 17:31:32 +0200 Subject: [PATCH 02/13] docs: fix sqlite connection examples and tighten guide wording aiosqlite does not re-export PARSE_DECLTYPES, so the connection examples raised AttributeError. Use sqlite3.PARSE_DECLTYPES with aiosqlite.connect, matching how the test suite opens those connections, and import sqlite3 in both examples. Scope the PARSE_DECLTYPES requirement to schemas that actually generate type conversions, and correct the register_converter note: registration is process-global, so limiting it per module avoids unused conversions but does not isolate modules from each other. Make the alternative JSON examples self consistent by defining the model with the same library the converter uses, and rebuild nested objects in the stdlib decoder. State that query_parameter_limit only bundles when set to a non-negative value, and fix two wording and formatting nits. --- docs/content/docs/guide/converters.md | 2 +- docs/content/docs/guide/drivers.md | 11 +++++---- .../docs/guide/sqlite-type-conversion.md | 13 ++++++++--- docs/content/docs/guide/working-with-json.md | 23 +++++++++++++++---- docs/content/docs/guide/writing-queries.md | 8 ++++--- docs/content/docs/reference/type-mappings.md | 2 +- 6 files changed, 42 insertions(+), 17 deletions(-) diff --git a/docs/content/docs/guide/converters.md b/docs/content/docs/guide/converters.md index 95a2fb3a..376e875e 100644 --- a/docs/content/docs/guide/converters.md +++ b/docs/content/docs/guide/converters.md @@ -77,7 +77,7 @@ and list columns convert element-wise: the override (`jsonb` is a `str`, `bytea` a `memoryview` - see [type mappings](/docs/reference/type-mappings)), and `from_db` receives that same type. For an unrecognised SQL type there is no such type, so `to_db` must - return one the driver accepts. + return a type the driver accepts. - **Your functions never see `None`.** A SQL `NULL` stays `None` without calling the converter. - **List columns convert element-wise.** diff --git a/docs/content/docs/guide/drivers.md b/docs/content/docs/guide/drivers.md index e1ae9916..714e50b3 100644 --- a/docs/content/docs/guide/drivers.md +++ b/docs/content/docs/guide/drivers.md @@ -34,11 +34,13 @@ asyncpg is the only driver that supports `:copyfrom` (bulk insert via ## aiosqlite (async SQLite) ```python +import sqlite3 + import aiosqlite from app.db import queries -async with aiosqlite.connect("app.db", detect_types=aiosqlite.PARSE_DECLTYPES) as conn: +async with aiosqlite.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) as conn: user = await queries.get_field_naming(conn, id_=1) ``` @@ -54,9 +56,10 @@ user = queries.get_field_naming(conn, id_=1) ``` {{< callout type="warning" >}} - Both SQLite drivers require `detect_types=PARSE_DECLTYPES` on the connection. - The generated code registers adapters and converters (for dates, decimals, - booleans, and blobs) that only run when declared-type parsing is enabled. See + Whenever the generated code registers type conversions - which it does for + `date`, `datetime`/`timestamp`, `decimal`, `bool`, and `blob` columns - both + SQLite drivers need `detect_types=sqlite3.PARSE_DECLTYPES` on the connection, + because converters only run when declared-type parsing is enabled. See [SQLite type conversion](/docs/guide/sqlite-type-conversion). {{< /callout >}} diff --git a/docs/content/docs/guide/sqlite-type-conversion.md b/docs/content/docs/guide/sqlite-type-conversion.md index 611edc68..5c9c0eb3 100644 --- a/docs/content/docs/guide/sqlite-type-conversion.md +++ b/docs/content/docs/guide/sqlite-type-conversion.md @@ -25,7 +25,11 @@ conn = sqlite3.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) {{< tab name="aiosqlite" >}} ```python -async with aiosqlite.connect("app.db", detect_types=aiosqlite.PARSE_DECLTYPES) as conn: +import sqlite3 + +import aiosqlite + +async with aiosqlite.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) as conn: ... ``` {{< /tab >}} @@ -60,8 +64,11 @@ actually uses: - A type used in a **returned column** gets its **converter** registered. {{< callout type="info" >}} - `register_converter` is process-global in Python's `sqlite3`. Registering only - what a module needs keeps modules from stepping on each other's conversions. + `register_converter` is process-global in Python's `sqlite3`, so a registration + made by one generated module applies to every connection in the process. + Registering only what a module needs avoids installing conversions it never + uses - it does not isolate modules from one another, and two modules + registering the same declared type will share that conversion. {{< /callout >}} ## Interaction with overrides diff --git a/docs/content/docs/guide/working-with-json.md b/docs/content/docs/guide/working-with-json.md index 4749fbc3..8dd469f2 100644 --- a/docs/content/docs/guide/working-with-json.md +++ b/docs/content/docs/guide/working-with-json.md @@ -198,16 +198,20 @@ defaults or widen them to `| None`. ## Other libraries The converter contract is just "return the wire type, accept the wire type", so -any library works: +any library works - as long as your model and your converter use the same one. +The examples below assume `Theme`/`Preferences` are defined with that library +rather than as `msgspec.Struct`s. {{< tabs >}} {{< tab name="stdlib json" >}} +With `Theme` and `Preferences` defined as `dataclasses.dataclass`: + ```python import dataclasses import json -from myapp.models import Preferences +from myapp.models import Preferences, Theme def encode_preferences(value: Preferences) -> str: @@ -215,12 +219,20 @@ def encode_preferences(value: Preferences) -> str: def decode_preferences(value: str) -> Preferences: - return Preferences(**json.loads(value)) + raw = json.loads(value) + return Preferences( + theme=Theme(**raw["theme"]), + languages=raw.get("languages", []), + email_opt_in=raw.get("email_opt_in"), + ) ``` -No dependencies, but no validation and no nested-struct rebuilding. +No dependencies, but no validation - and nested objects must be rebuilt by hand, +since `json.loads` leaves `theme` as a plain `dict`. {{< /tab >}} {{< tab name="pydantic" >}} +With `Theme` and `Preferences` defined as `pydantic.BaseModel`: + ```python from myapp.models import Preferences @@ -232,7 +244,8 @@ def encode_preferences(value: Preferences) -> str: def decode_preferences(value: str) -> Preferences: return Preferences.model_validate_json(value) ``` -Full validation, and `model_validate_json` parses straight from the string. +Full validation, nested models rebuilt for you, and `model_validate_json` parses +straight from the string. {{< /tab >}} {{< /tabs >}} diff --git a/docs/content/docs/guide/writing-queries.md b/docs/content/docs/guide/writing-queries.md index 261179d4..6d579900 100644 --- a/docs/content/docs/guide/writing-queries.md +++ b/docs/content/docs/guide/writing-queries.md @@ -122,9 +122,11 @@ this: - **`omit_kwargs_limit`** - queries with this many parameters or fewer allow positional arguments. Defaults to `0` (always keyword-only). -- **`query_parameter_limit`** - when a query has more parameters than the limit, - they are bundled into a single generated `Params` object instead of - expanded into the signature. `:copyfrom` always uses a params class. +- **`query_parameter_limit`** - when set to a non-negative value, queries with + more parameters than the limit bundle them into a single generated + `Params` object instead of expanding them into the signature. Left unset + or negative, parameters are never bundled. `:copyfrom` always uses a params + class regardless. ## Grouping into a class diff --git a/docs/content/docs/reference/type-mappings.md b/docs/content/docs/reference/type-mappings.md index 9908b9e3..187f05e7 100644 --- a/docs/content/docs/reference/type-mappings.md +++ b/docs/content/docs/reference/type-mappings.md @@ -15,7 +15,7 @@ SQL type. The tables below give the **base** Python type for a SQL type. Two modifiers are applied on top, based on the column: -- **Nullable** (the column is not `NOT NULL`): ` | None` is appended, e.g. +- **Nullable** (the column is not `NOT NULL`): `| None` is appended, e.g. `str | None`. - **Array / list** (a Postgres array column, or a `sqlc.slice` parameter): the base type is wrapped as `collections.abc.Sequence[T]`. From 14eaca9202ee56b75ec7c040ab7dbcf9083c3159 Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 17:39:17 +0200 Subject: [PATCH 03/13] docs: scope the PARSE_DECLTYPES note to converters detect_types governs converters, which decode returned columns, not the adapters that encode parameters. Scope the drivers callout to queries that return date, datetime/timestamp, decimal, bool, or blob columns, and say outright that parameter adapters work without the setting. Separate the tab commentary from the preceding code fence with a blank line throughout the JSON and model type pages, and drop the roadmap link from the README. --- README.md | 2 -- docs/content/docs/guide/drivers.md | 9 +++++---- docs/content/docs/guide/model-types.md | 4 ++++ docs/content/docs/guide/working-with-json.md | 4 ++++ 4 files changed, 13 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 62fb77f9..17b7b540 100644 --- a/README.md +++ b/README.md @@ -85,8 +85,6 @@ supported. Which query commands are available depends on the driver - see the ## Development -A roadmap of what is planned & worked on can be found [here](https://github.com/users/rayakame/projects/1/). - Contributions are very welcome, for more information and help please read the [contribution guidelines](https://github.com/rayakame/sqlc-gen-better-python/blob/main/CONTRIBUTING.md). diff --git a/docs/content/docs/guide/drivers.md b/docs/content/docs/guide/drivers.md index 714e50b3..5522d7eb 100644 --- a/docs/content/docs/guide/drivers.md +++ b/docs/content/docs/guide/drivers.md @@ -56,10 +56,11 @@ user = queries.get_field_naming(conn, id_=1) ``` {{< callout type="warning" >}} - Whenever the generated code registers type conversions - which it does for - `date`, `datetime`/`timestamp`, `decimal`, `bool`, and `blob` columns - both - SQLite drivers need `detect_types=sqlite3.PARSE_DECLTYPES` on the connection, - because converters only run when declared-type parsing is enabled. See + When a query *returns* a `date`, `datetime`/`timestamp`, `decimal`, `bool`, or + `blob` column, the generated code registers a converter for it - and converters + only run if the connection was opened with + `detect_types=sqlite3.PARSE_DECLTYPES`. Adapters, which send those types as + parameters, work without it. See [SQLite type conversion](/docs/guide/sqlite-type-conversion). {{< /callout >}} diff --git a/docs/content/docs/guide/model-types.md b/docs/content/docs/guide/model-types.md index ca5539ae..40795276 100644 --- a/docs/content/docs/guide/model-types.md +++ b/docs/content/docs/guide/model-types.md @@ -37,6 +37,7 @@ class TestFieldNaming: id_: int outputs: str ``` + Standard library, no dependencies. The safe default. {{< /tab >}} @@ -47,6 +48,7 @@ class TestFieldNaming: id_: int outputs: str ``` + Requires `attrs`. Slotted classes with concise definitions. {{< /tab >}} @@ -56,6 +58,7 @@ class TestFieldNaming(msgspec.Struct): id_: int outputs: str ``` + Requires `msgspec`. Very fast serialization/validation; pairs well with [working with JSON](/docs/guide/working-with-json). {{< /tab >}} @@ -68,6 +71,7 @@ class TestFieldNaming(pydantic.BaseModel): id_: int outputs: str ``` + Requires `pydantic >= 2.9`. Full runtime validation on construction. {{< /tab >}} diff --git a/docs/content/docs/guide/working-with-json.md b/docs/content/docs/guide/working-with-json.md index 8dd469f2..d7cb510f 100644 --- a/docs/content/docs/guide/working-with-json.md +++ b/docs/content/docs/guide/working-with-json.md @@ -67,6 +67,7 @@ class Preferences(msgspec.Struct): languages: list[str] = msgspec.field(default_factory=list) email_opt_in: bool | None = None ``` + Structs nest freely - `Preferences` holds a `Theme`, plus a list and an optional field. `msgspec` handles all of it. {{< /tab >}} @@ -86,6 +87,7 @@ def encode_preferences(value: Preferences) -> str: def decode_preferences(value: str) -> Preferences: return msgspec.json.decode(value, type=Preferences) ``` + The `type=` argument is what makes decoding *typed* - msgspec builds real `Preferences` objects and validates the payload while parsing. {{< /tab >}} @@ -226,6 +228,7 @@ def decode_preferences(value: str) -> Preferences: email_opt_in=raw.get("email_opt_in"), ) ``` + No dependencies, but no validation - and nested objects must be rebuilt by hand, since `json.loads` leaves `theme` as a plain `dict`. {{< /tab >}} @@ -244,6 +247,7 @@ def encode_preferences(value: Preferences) -> str: def decode_preferences(value: str) -> Preferences: return Preferences.model_validate_json(value) ``` + Full validation, nested models rebuilt for you, and `model_validate_json` parses straight from the string. {{< /tab >}} From d929f698f32e8e294e6851d58ac585a770db322c Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 17:41:14 +0200 Subject: [PATCH 04/13] docs: add changelog fragment for the documentation site --- .changes/unreleased/Added-20260721-140000.yaml | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 .changes/unreleased/Added-20260721-140000.yaml diff --git a/.changes/unreleased/Added-20260721-140000.yaml b/.changes/unreleased/Added-20260721-140000.yaml new file mode 100644 index 00000000..28d91fd4 --- /dev/null +++ b/.changes/unreleased/Added-20260721-140000.yaml @@ -0,0 +1,6 @@ +kind: Added +body: 'Documentation site at https://rayakame.github.io/sqlc-gen-better-python/, with a getting started page, a guide covering configuration, drivers, model types, writing queries, enums, type overrides, converters, working with JSON, docstrings, SQLite type conversion and naming, and a reference for every configuration option, the SQL to Python type mappings of both engines and per-driver feature support. Every generated code example is taken from the committed test fixtures. The README now links to the site instead of documenting each feature inline.' +time: 2026-07-21T14:00:00.0000000Z +custom: + Author: Rayakame + PR: "208" From 643d1135a9504e344d4fe674cb21bb0361b3bde5 Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 17:52:10 +0200 Subject: [PATCH 05/13] docs: turn getting started into a full quickstart Carry the page from install and configuration through to a first working query: a schema, an annotated query file, the generated model and query functions, and the call sites that use them. Every driver specific block is a synced tab group, so picking asyncpg, aiosqlite or sqlite3 once switches the whole page. That includes how a :many result is consumed, which differs between the async drivers, where the result is awaited or iterated, and sqlite3, where it is called. --- docs/content/docs/getting-started.md | 352 ++++++++++++++++++++++++++- 1 file changed, 341 insertions(+), 11 deletions(-) diff --git a/docs/content/docs/getting-started.md b/docs/content/docs/getting-started.md index 6bfb8972..4b5a4f8f 100644 --- a/docs/content/docs/getting-started.md +++ b/docs/content/docs/getting-started.md @@ -3,19 +3,51 @@ title: Getting Started weight: 1 prev: /docs next: /docs/guide +tabs: + sync: true --- +From an empty project to your first typed queries. Pick your driver in the tabs +below and the whole page follows your choice. + ## Prerequisites You need [`sqlc`](https://docs.sqlc.dev/en/latest/overview/install.html) on your -`PATH`. The generated code targets **Python 3.12 or newer** (it uses PEP 695 type -aliases and generics, and `enum.StrEnum`). +`PATH` and **Python 3.12 or newer** (the generated code uses PEP 695 type aliases +and generics, and `enum.StrEnum`). + +Then install the database driver you want to use: + +{{< tabs >}} + + {{< tab name="asyncpg" >}} +```bash +pip install asyncpg +``` + {{< /tab >}} + + {{< tab name="aiosqlite" >}} +```bash +pip install aiosqlite +``` + {{< /tab >}} + + {{< tab name="sqlite3" >}} +```text +Nothing to install - sqlite3 is in the standard library. +``` + {{< /tab >}} + +{{< /tabs >}} -## Configure the plugin +## 1. Configure the plugin -The plugin is a WASM binary that `sqlc generate` downloads and runs. Point your -`sqlc.yaml` at a release and select a driver and model type: +The plugin is a WASM binary that `sqlc generate` downloads and runs. Create a +`sqlc.yaml`: +{{< tabs >}} + + {{< tab name="asyncpg" >}} ```yaml # filename: sqlc.yaml version: "2" @@ -35,15 +67,153 @@ sql: package: "db" emit_init_file: true sql_driver: "asyncpg" - model_type: "msgspec" + model_type: "dataclass" +``` + {{< /tab >}} + + {{< tab name="aiosqlite" >}} +```yaml +# filename: sqlc.yaml +version: "2" +plugins: + - name: python + wasm: + url: https://github.com/rayakame/sqlc-gen-better-python/releases/download/v0.5.1/sqlc-gen-better-python.wasm + sha256: c7cc470df7625ae3232c2b042060b948180ae784ce3d81c32e8a2c040fe04fa7 +sql: + - engine: "sqlite" + queries: "query.sql" + schema: "schema.sql" + codegen: + - out: "app/db" + plugin: python + options: + package: "db" + emit_init_file: true + sql_driver: "aiosqlite" + model_type: "dataclass" ``` + {{< /tab >}} + + {{< tab name="sqlite3" >}} +```yaml +# filename: sqlc.yaml +version: "2" +plugins: + - name: python + wasm: + url: https://github.com/rayakame/sqlc-gen-better-python/releases/download/v0.5.1/sqlc-gen-better-python.wasm + sha256: c7cc470df7625ae3232c2b042060b948180ae784ce3d81c32e8a2c040fe04fa7 +sql: + - engine: "sqlite" + queries: "query.sql" + schema: "schema.sql" + codegen: + - out: "app/db" + plugin: python + options: + package: "db" + emit_init_file: true + sql_driver: "sqlite3" + model_type: "dataclass" +``` + {{< /tab >}} + +{{< /tabs >}} {{< callout type="warning" >}} - Always pin the `sha256` of the release you use - `sqlc` refuses to run a - plugin whose hash does not match. Each release lists its hash. + Always pin the `sha256` of the release you use - `sqlc` refuses to run a plugin + whose hash does not match. Each release lists its hash. {{< /callout >}} -## Generate +`model_type: "dataclass"` is the default and needs no extra dependency. See +[Model types](/docs/guide/model-types) for `attrs`, `msgspec`, and `pydantic`. + +## 2. Describe your schema + +{{< tabs >}} + + {{< tab name="asyncpg" >}} +```sql +-- filename: schema.sql +CREATE TABLE users +( + id bigint PRIMARY KEY NOT NULL, + name text NOT NULL +); +``` + {{< /tab >}} + + {{< tab name="aiosqlite" >}} +```sql +-- filename: schema.sql +CREATE TABLE users +( + id INTEGER PRIMARY KEY NOT NULL, + name TEXT NOT NULL +); +``` + {{< /tab >}} + + {{< tab name="sqlite3" >}} +```sql +-- filename: schema.sql +CREATE TABLE users +( + id INTEGER PRIMARY KEY NOT NULL, + name TEXT NOT NULL +); +``` + {{< /tab >}} + +{{< /tabs >}} + +## 3. Write your queries + +Each query is a `-- name: :` comment followed by SQL. The name +becomes the Python function, and the command decides what it returns - `:one` +returns a single row or `None`, `:many` returns all matching rows. + +{{< tabs >}} + + {{< tab name="asyncpg" >}} +```sql +-- filename: query.sql +-- name: GetUser :one +SELECT * FROM users WHERE id = $1; + +-- name: ListUsers :many +SELECT * FROM users ORDER BY name; +``` + {{< /tab >}} + + {{< tab name="aiosqlite" >}} +```sql +-- filename: query.sql +-- name: GetUser :one +SELECT * FROM users WHERE id = ?; + +-- name: ListUsers :many +SELECT * FROM users ORDER BY name; +``` + {{< /tab >}} + + {{< tab name="sqlite3" >}} +```sql +-- filename: query.sql +-- name: GetUser :one +SELECT * FROM users WHERE id = ?; + +-- name: ListUsers :many +SELECT * FROM users ORDER BY name; +``` + {{< /tab >}} + +{{< /tabs >}} + +PostgreSQL uses `$1` placeholders, SQLite uses `?`. Everything else is the same. + +## 4. Generate Run `sqlc` from the directory containing your `sqlc.yaml`: @@ -51,8 +221,168 @@ Run `sqlc` from the directory containing your `sqlc.yaml`: sqlc generate ``` -This writes a Python package to `out` containing `models.py`, one query module -per query file, and (when your schema has enums) an `enums.py`. +This writes a Python package to `out` (`app/db` above) containing `models.py`, +one query module per query file (`query.py`), and an `__init__.py`. + +## 5. What you got + +A model per table, and a typed function per query: + +{{< tabs >}} + + {{< tab name="asyncpg" >}} +```python +# models.py +@dataclasses.dataclass() +class User: + id_: int + name: str + + +# query.py +async def get_user(conn: ConnectionLike, *, id_: int) -> models.User | None: + row = await conn.fetchrow(GET_USER, id_) + if row is None: + return None + return models.User(id_=row[0], name=row[1]) + + +def list_users(conn: ConnectionLike) -> QueryResults[models.User]: + ... +``` + {{< /tab >}} + + {{< tab name="aiosqlite" >}} +```python +# models.py +@dataclasses.dataclass() +class User: + id_: int + name: str + + +# query.py +async def get_user(conn: aiosqlite.Connection, *, id_: int) -> models.User | None: + row = await (await conn.execute(GET_USER, (id_,))).fetchone() + if row is None: + return None + return models.User(id_=row[0], name=row[1]) + + +def list_users(conn: aiosqlite.Connection) -> QueryResults[models.User]: + ... +``` + {{< /tab >}} + + {{< tab name="sqlite3" >}} +```python +# models.py +@dataclasses.dataclass() +class User: + id_: int + name: str + + +# query.py +def get_user(conn: sqlite3.Connection, *, id_: int) -> models.User | None: + row = conn.execute(GET_USER, (id_,)).fetchone() + if row is None: + return None + return models.User(id_=row[0], name=row[1]) + + +def list_users(conn: sqlite3.Connection) -> QueryResults[models.User]: + ... +``` + {{< /tab >}} + +{{< /tabs >}} + +Two things to notice: + +- The `id` column became `id_`, because `id` is a Python builtin. See + [Naming and identifiers](/docs/guide/naming). +- Parameters are keyword-only (`*,`), so you call `get_user(conn, id_=1)`. + +## 6. Use it + +`:one` gives you a model or `None`. `:many` returns a `QueryResults`, which you +can consume all at once or stream row by row: + +{{< tabs >}} + + {{< tab name="asyncpg" >}} +```python +import asyncpg + +from app.db import query + +conn = await asyncpg.connect("postgresql://user:pass@localhost/mydb") + +user = await query.get_user(conn, id_=1) +if user is not None: + print(user.name) + +# every row at once +users = await query.list_users(conn) + +# or stream them +async for user in query.list_users(conn): + print(user.name) +``` + {{< /tab >}} + + {{< tab name="aiosqlite" >}} +```python +import sqlite3 + +import aiosqlite + +from app.db import query + +async with aiosqlite.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) as conn: + user = await query.get_user(conn, id_=1) + if user is not None: + print(user.name) + + # every row at once + users = await query.list_users(conn) + + # or stream them + async for user in query.list_users(conn): + print(user.name) +``` + {{< /tab >}} + + {{< tab name="sqlite3" >}} +```python +import sqlite3 + +from app.db import query + +conn = sqlite3.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) + +user = query.get_user(conn, id_=1) +if user is not None: + print(user.name) + +# every row at once - call the result +users = query.list_users(conn)() + +# or iterate +for user in query.list_users(conn): + print(user.name) +``` + {{< /tab >}} + +{{< /tabs >}} + +{{< callout type="info" >}} + The `detect_types=sqlite3.PARSE_DECLTYPES` above is not needed for this schema, + but it is what makes generated converters work once your tables use dates, + decimals, booleans, or blobs. See + [SQLite type conversion](/docs/guide/sqlite-type-conversion). +{{< /callout >}} ## Next steps From 24f328188ec8ed9fce17295a9b092ce9f8d64df1 Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 18:09:33 +0200 Subject: [PATCH 06/13] docs: flesh out the docs landing page and link the feature cards Give the documentation landing page something to land on: icons on the section cards, a short SQL to Python showcase, a grid linking every feature page, a support summary table, and cards for issues, Discord and the changelog. Point the home page feature cards at the guide pages they describe, now that those pages exist. Card links are relative, because the card shortcode does not resolve a leading slash against the site base path the way markdown links do. --- docs/content/_index.md | 6 ++++ docs/content/docs/_index.md | 64 ++++++++++++++++++++++++++++++++++--- 2 files changed, 66 insertions(+), 4 deletions(-) diff --git a/docs/content/_index.md b/docs/content/_index.md index 93e20017..092d8397 100644 --- a/docs/content/_index.md +++ b/docs/content/_index.md @@ -30,26 +30,32 @@ layout: hextra-home {{< hextra/feature-grid >}} {{< hextra/feature-card title="Four model types" + link="docs/guide/model-types" subtitle="Generate dataclass, attrs, msgspec, or pydantic models - pick per codegen block." >}} {{< hextra/feature-card title="Three drivers" + link="docs/guide/drivers" subtitle="asyncpg for PostgreSQL, plus aiosqlite and sqlite3 for SQLite." >}} {{< hextra/feature-card title="Strictly typed output" + link="docs/guide/writing-queries" subtitle="Generated code passes pyright strict and ruff, targeting Python 3.12+." >}} {{< hextra/feature-card title="Enums" + link="docs/guide/enums" subtitle="PostgreSQL enums become enum.StrEnum classes, wired through models and queries." >}} {{< hextra/feature-card title="Type overrides & converters" + link="docs/guide/type-overrides" subtitle="Swap a column's Python type, or plug in your own encode/decode functions." >}} {{< hextra/feature-card title="Docstrings" + link="docs/guide/docstrings" subtitle="Optional google, numpy, or pep257 docstrings on every generated function." >}} {{< /hextra/feature-grid >}} diff --git a/docs/content/docs/_index.md b/docs/content/docs/_index.md index 2f86baea..873a8a67 100644 --- a/docs/content/docs/_index.md +++ b/docs/content/docs/_index.md @@ -9,10 +9,66 @@ sidebar: `sqlc-gen-better-python` is a [sqlc](https://sqlc.dev) plugin that turns your SQL schema and queries into modern, fully typed Python database code: models, typed -query functions, and enums. +query functions, and enums. You keep writing SQL; the Python stays in sync with +it. {{< cards >}} - {{< card link="getting-started" title="Getting Started" subtitle="Install the plugin and generate your first models." >}} - {{< card link="guide" title="Guide" subtitle="Learn each feature with real schema, queries, and generated output." >}} - {{< card link="reference" title="Reference" subtitle="Configuration options, type mappings, and driver support." >}} + {{< card link="getting-started" title="Getting Started" icon="play" subtitle="Install the plugin and go from a schema to your first typed queries." >}} + {{< card link="guide" title="Guide" icon="book-open" subtitle="Every feature, explained with real schema, queries, and generated output." >}} + {{< card link="reference" title="Reference" icon="clipboard-list" subtitle="Configuration options, type mappings, and per-driver support." >}} +{{< /cards >}} + +## What it looks like + +You write a query: + +```sql +-- name: GetUser :one +SELECT * FROM users WHERE id = $1; +``` + +and get a typed function, with a model built from your schema: + +```python +async def get_user(conn: ConnectionLike, *, id_: int) -> models.User | None: + row = await conn.fetchrow(GET_USER, id_) + if row is None: + return None + return models.User(id_=row[0], name=row[1]) +``` + +No ORM, no hand-written row unpacking, and pyright checks every field access. + +## Explore by feature + +{{< cards >}} + {{< card link="guide/drivers" title="Drivers" icon="database" subtitle="asyncpg, aiosqlite, and sqlite3." >}} + {{< card link="guide/model-types" title="Model types" icon="cube" subtitle="dataclass, attrs, msgspec, or pydantic." >}} + {{< card link="guide/writing-queries" title="Writing queries" icon="code" subtitle="How query annotations become typed functions." >}} + {{< card link="guide/enums" title="Enums" icon="collection" subtitle="PostgreSQL enums as enum.StrEnum classes." >}} + {{< card link="guide/type-overrides" title="Type overrides" icon="adjustments" subtitle="Swap the Python type of a column." >}} + {{< card link="guide/converters" title="Converters" icon="puzzle" subtitle="Your own encode/decode functions for a column." >}} + {{< card link="guide/working-with-json" title="Working with JSON" icon="document-text" subtitle="Typed JSON columns with msgspec structs." >}} + {{< card link="guide/docstrings" title="Docstrings" icon="document" subtitle="google, numpy, or pep257 docstrings." >}} +{{< /cards >}} + +## At a glance + +| | | +|---|---| +| **Python** | 3.12 or newer | +| **Engines** | PostgreSQL, SQLite | +| **Drivers** | `asyncpg`, `aiosqlite`, `sqlite3` | +| **Model types** | `dataclass`, `attrs`, `msgspec`, `pydantic` | +| **Docstrings** | `google`, `numpy`, `pep257`, or none | +| **Checked with** | pyright (strict) and ruff | + +Full details in the [feature support reference](/docs/reference/feature-support). + +## Getting help + +{{< cards >}} + {{< card link="https://github.com/rayakame/sqlc-gen-better-python/issues" title="Issues" icon="github" subtitle="Report a bug or request a feature." >}} + {{< card link="https://discord.gg/hikari" title="Discord" icon="chat" subtitle="Ask a question or share what you built." >}} + {{< card link="https://github.com/rayakame/sqlc-gen-better-python/blob/main/CHANGELOG.md" title="Changelog" icon="clipboard-check" subtitle="What changed in each release." >}} {{< /cards >}} From 11c29945a6d7cf6242d61309e74730eb7a3798ce Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 18:16:33 +0200 Subject: [PATCH 07/13] docs: build out the root landing page The home page stopped at the feature grid and left the rest of the screen empty. Follow it with the two things a visitor actually wants: what the plugin produces, shown as an annotated query next to the typed function and model it generates, and how to start, shown as the sqlc.yaml block, the generate command, and a link into the getting started guide. Drop the same showcase from the documentation landing page, so it lives in one place and that page stays a navigation hub. --- docs/content/_index.md | 63 +++++++++++++++++++++++++++++++++++++ docs/content/docs/_index.md | 21 ------------- 2 files changed, 63 insertions(+), 21 deletions(-) diff --git a/docs/content/_index.md b/docs/content/_index.md index 092d8397..0037a5ff 100644 --- a/docs/content/_index.md +++ b/docs/content/_index.md @@ -59,3 +59,66 @@ layout: hextra-home subtitle="Optional google, numpy, or pep257 docstrings on every generated function." >}} {{< /hextra/feature-grid >}} + +
+ +{{< hextra/hero-section >}} + From SQL to Python +{{< /hextra/hero-section >}} + +You write a query, annotated with the name and shape you want: + +```sql +-- name: GetUser :one +SELECT * FROM users WHERE id = $1; +``` + +and get a typed function back, with a model built from your schema: + +```python +async def get_user(conn: ConnectionLike, *, id_: int) -> models.User | None: + row = await conn.fetchrow(GET_USER, id_) + if row is None: + return None + return models.User(id_=row[0], name=row[1]) +``` + +No ORM, no hand-written row unpacking, and pyright checks every field access. + +
+ +{{< hextra/hero-section >}} + Set up in three steps +{{< /hextra/hero-section >}} + +Point `sqlc` at the plugin, pick a driver and a model type, and generate: + +```yaml +# sqlc.yaml +plugins: + - name: python + wasm: + url: https://github.com/rayakame/sqlc-gen-better-python/releases/download/v0.5.1/sqlc-gen-better-python.wasm + sha256: c7cc470df7625ae3232c2b042060b948180ae784ce3d81c32e8a2c040fe04fa7 +sql: + - engine: "postgresql" + queries: "query.sql" + schema: "schema.sql" + codegen: + - out: "app/db" + plugin: python + options: + package: "db" + emit_init_file: true + sql_driver: "asyncpg" +``` + +```bash +sqlc generate +``` + +
+ +{{< hextra/hero-button text="Read the Getting Started guide" link="docs/getting-started" >}} + +
diff --git a/docs/content/docs/_index.md b/docs/content/docs/_index.md index 873a8a67..40737bda 100644 --- a/docs/content/docs/_index.md +++ b/docs/content/docs/_index.md @@ -18,27 +18,6 @@ it. {{< card link="reference" title="Reference" icon="clipboard-list" subtitle="Configuration options, type mappings, and per-driver support." >}} {{< /cards >}} -## What it looks like - -You write a query: - -```sql --- name: GetUser :one -SELECT * FROM users WHERE id = $1; -``` - -and get a typed function, with a model built from your schema: - -```python -async def get_user(conn: ConnectionLike, *, id_: int) -> models.User | None: - row = await conn.fetchrow(GET_USER, id_) - if row is None: - return None - return models.User(id_=row[0], name=row[1]) -``` - -No ORM, no hand-written row unpacking, and pyright checks every field access. - ## Explore by feature {{< cards >}} From ea53d7a79626d41da08e1eede1385a4576e5584d Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 18:25:09 +0200 Subject: [PATCH 08/13] docs: fill out the root page and top align card text Follow the feature grid with the things a visitor still has to go looking for: the same table rendered as all four model types, the correctness story behind the generated code, and the setup snippet, ending on a link into the getting started guide. Cards bottom align their text so it can sit under an image, which leaves a gap above short cards once a taller sibling stretches the grid row. Top align text only cards and leave cards with an image alone. --- docs/assets/css/custom.css | 11 +++++++ docs/content/_index.md | 63 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 74 insertions(+) create mode 100644 docs/assets/css/custom.css diff --git a/docs/assets/css/custom.css b/docs/assets/css/custom.css new file mode 100644 index 00000000..90ed1171 --- /dev/null +++ b/docs/assets/css/custom.css @@ -0,0 +1,11 @@ +/* Hextra bottom-aligns card content (margin-top: auto) so that text sits under + a card image. Text-only cards in a stretched grid row then get a gap above + the title when a taller sibling sets the row height, so top-align those and + keep the original behaviour for cards that do have an image. */ +.hextra-card > div { + margin-top: 0; +} + +.hextra-card-image ~ div { + margin-top: auto; +} diff --git a/docs/content/_index.md b/docs/content/_index.md index 0037a5ff..5b7b6f46 100644 --- a/docs/content/_index.md +++ b/docs/content/_index.md @@ -87,6 +87,69 @@ No ORM, no hand-written row unpacking, and pyright checks every field access.
+{{< hextra/hero-section >}} + One schema, four model types +{{< /hextra/hero-section >}} + +Set `model_type` and the same table generates whichever flavour your project +already uses - the fields and annotations are identical: + +{{< tabs >}} + + {{< tab name="dataclass" >}} +```python +@dataclasses.dataclass() +class User: + id_: int + name: str +``` + {{< /tab >}} + + {{< tab name="attrs" >}} +```python +@attrs.define() +class User: + id_: int + name: str +``` + {{< /tab >}} + + {{< tab name="msgspec" >}} +```python +class User(msgspec.Struct): + id_: int + name: str +``` + {{< /tab >}} + + {{< tab name="pydantic" >}} +```python +class User(pydantic.BaseModel): + model_config = pydantic.ConfigDict(arbitrary_types_allowed=True) + + id_: int + name: str +``` + {{< /tab >}} + +{{< /tabs >}} + +
+ +{{< hextra/hero-section >}} + Built to be trusted +{{< /hextra/hero-section >}} + +Generated code is held to the same standard as hand-written code: + +- **Type-checked.** Every generated file passes pyright in strict mode and ruff. +- **Tested against real databases.** The suite runs the generated code against + live PostgreSQL and SQLite, across every driver and model type. +- **Deterministic.** Output is byte-identical between runs, and CI fails if a + change would silently alter what you get. + +
+ {{< hextra/hero-section >}} Set up in three steps {{< /hextra/hero-section >}} From 2f62100664513bf26cc5c0e4800a75ffb75ce95a Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 18:29:28 +0200 Subject: [PATCH 09/13] docs: set the site footer copyright to the license holder The footer showed the theme's default placeholder. Override the copyright i18n string at site level, matching the LICENSE holder so the two cannot drift. --- docs/i18n/en.yaml | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 docs/i18n/en.yaml diff --git a/docs/i18n/en.yaml b/docs/i18n/en.yaml new file mode 100644 index 00000000..5258e9fc --- /dev/null +++ b/docs/i18n/en.yaml @@ -0,0 +1,4 @@ +# Overrides theme defaults. Hugo merges these over the theme's own i18n strings. + +# Footer copyright. Kept in step with the LICENSE holder. +copyright: "Copyright (c) 2025-present Rayakame" From 33ecb12f44fdf66091820ea15529c395a38be900 Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 18:35:48 +0200 Subject: [PATCH 10/13] docs: make the example snippets valid Python and sqlc config The async examples used await, async for and async with at module scope, which is a SyntaxError, so anyone copying them got nothing. Run them from an async def main under asyncio.run, in the getting started page and in the drivers, sqlite type conversion and JSON pages that had the same problem. Guard the JSON example against a missing row before reaching into the returned model, since the query returns an optional. Add the required version key to the sqlc.yaml snippet on the home page, which was not a valid v2 configuration without it. Separate fenced blocks from the surrounding tab shortcodes with blank lines, matching how the converter page already wrote them. --- docs/content/_index.md | 9 ++ docs/content/docs/getting-started.md | 85 +++++++++++++++---- docs/content/docs/guide/docstrings.md | 6 ++ docs/content/docs/guide/drivers.md | 21 ++++- docs/content/docs/guide/model-types.md | 4 + .../docs/guide/sqlite-type-conversion.md | 10 ++- docs/content/docs/guide/working-with-json.md | 24 ++++-- 7 files changed, 126 insertions(+), 33 deletions(-) diff --git a/docs/content/_index.md b/docs/content/_index.md index 5b7b6f46..c4c5af7e 100644 --- a/docs/content/_index.md +++ b/docs/content/_index.md @@ -97,32 +97,39 @@ already uses - the fields and annotations are identical: {{< tabs >}} {{< tab name="dataclass" >}} + ```python @dataclasses.dataclass() class User: id_: int name: str ``` + {{< /tab >}} {{< tab name="attrs" >}} + ```python @attrs.define() class User: id_: int name: str ``` + {{< /tab >}} {{< tab name="msgspec" >}} + ```python class User(msgspec.Struct): id_: int name: str ``` + {{< /tab >}} {{< tab name="pydantic" >}} + ```python class User(pydantic.BaseModel): model_config = pydantic.ConfigDict(arbitrary_types_allowed=True) @@ -130,6 +137,7 @@ class User(pydantic.BaseModel): id_: int name: str ``` + {{< /tab >}} {{< /tabs >}} @@ -158,6 +166,7 @@ Point `sqlc` at the plugin, pick a driver and a model type, and generate: ```yaml # sqlc.yaml +version: "2" plugins: - name: python wasm: diff --git a/docs/content/docs/getting-started.md b/docs/content/docs/getting-started.md index 4b5a4f8f..19efddbe 100644 --- a/docs/content/docs/getting-started.md +++ b/docs/content/docs/getting-started.md @@ -21,21 +21,27 @@ Then install the database driver you want to use: {{< tabs >}} {{< tab name="asyncpg" >}} + ```bash pip install asyncpg ``` + {{< /tab >}} {{< tab name="aiosqlite" >}} + ```bash pip install aiosqlite ``` + {{< /tab >}} {{< tab name="sqlite3" >}} + ```text Nothing to install - sqlite3 is in the standard library. ``` + {{< /tab >}} {{< /tabs >}} @@ -48,6 +54,7 @@ The plugin is a WASM binary that `sqlc generate` downloads and runs. Create a {{< tabs >}} {{< tab name="asyncpg" >}} + ```yaml # filename: sqlc.yaml version: "2" @@ -69,9 +76,11 @@ sql: sql_driver: "asyncpg" model_type: "dataclass" ``` + {{< /tab >}} {{< tab name="aiosqlite" >}} + ```yaml # filename: sqlc.yaml version: "2" @@ -93,9 +102,11 @@ sql: sql_driver: "aiosqlite" model_type: "dataclass" ``` + {{< /tab >}} {{< tab name="sqlite3" >}} + ```yaml # filename: sqlc.yaml version: "2" @@ -117,6 +128,7 @@ sql: sql_driver: "sqlite3" model_type: "dataclass" ``` + {{< /tab >}} {{< /tabs >}} @@ -134,6 +146,7 @@ sql: {{< tabs >}} {{< tab name="asyncpg" >}} + ```sql -- filename: schema.sql CREATE TABLE users @@ -142,9 +155,11 @@ CREATE TABLE users name text NOT NULL ); ``` + {{< /tab >}} {{< tab name="aiosqlite" >}} + ```sql -- filename: schema.sql CREATE TABLE users @@ -153,9 +168,11 @@ CREATE TABLE users name TEXT NOT NULL ); ``` + {{< /tab >}} {{< tab name="sqlite3" >}} + ```sql -- filename: schema.sql CREATE TABLE users @@ -164,6 +181,7 @@ CREATE TABLE users name TEXT NOT NULL ); ``` + {{< /tab >}} {{< /tabs >}} @@ -177,6 +195,7 @@ returns a single row or `None`, `:many` returns all matching rows. {{< tabs >}} {{< tab name="asyncpg" >}} + ```sql -- filename: query.sql -- name: GetUser :one @@ -185,9 +204,11 @@ SELECT * FROM users WHERE id = $1; -- name: ListUsers :many SELECT * FROM users ORDER BY name; ``` + {{< /tab >}} {{< tab name="aiosqlite" >}} + ```sql -- filename: query.sql -- name: GetUser :one @@ -196,9 +217,11 @@ SELECT * FROM users WHERE id = ?; -- name: ListUsers :many SELECT * FROM users ORDER BY name; ``` + {{< /tab >}} {{< tab name="sqlite3" >}} + ```sql -- filename: query.sql -- name: GetUser :one @@ -207,6 +230,7 @@ SELECT * FROM users WHERE id = ?; -- name: ListUsers :many SELECT * FROM users ORDER BY name; ``` + {{< /tab >}} {{< /tabs >}} @@ -231,6 +255,7 @@ A model per table, and a typed function per query: {{< tabs >}} {{< tab name="asyncpg" >}} + ```python # models.py @dataclasses.dataclass() @@ -250,9 +275,11 @@ async def get_user(conn: ConnectionLike, *, id_: int) -> models.User | None: def list_users(conn: ConnectionLike) -> QueryResults[models.User]: ... ``` + {{< /tab >}} {{< tab name="aiosqlite" >}} + ```python # models.py @dataclasses.dataclass() @@ -272,9 +299,11 @@ async def get_user(conn: aiosqlite.Connection, *, id_: int) -> models.User | Non def list_users(conn: aiosqlite.Connection) -> QueryResults[models.User]: ... ``` + {{< /tab >}} {{< tab name="sqlite3" >}} + ```python # models.py @dataclasses.dataclass() @@ -294,6 +323,7 @@ def get_user(conn: sqlite3.Connection, *, id_: int) -> models.User | None: def list_users(conn: sqlite3.Connection) -> QueryResults[models.User]: ... ``` + {{< /tab >}} {{< /tabs >}} @@ -312,49 +342,67 @@ can consume all at once or stream row by row: {{< tabs >}} {{< tab name="asyncpg" >}} + ```python +import asyncio + import asyncpg from app.db import query -conn = await asyncpg.connect("postgresql://user:pass@localhost/mydb") -user = await query.get_user(conn, id_=1) -if user is not None: - print(user.name) +async def main() -> None: + conn = await asyncpg.connect("postgresql://user:pass@localhost/mydb") -# every row at once -users = await query.list_users(conn) + user = await query.get_user(conn, id_=1) + if user is not None: + print(user.name) -# or stream them -async for user in query.list_users(conn): - print(user.name) + # every row at once + users = await query.list_users(conn) + + # or stream them + async for user in query.list_users(conn): + print(user.name) + + +asyncio.run(main()) ``` + {{< /tab >}} {{< tab name="aiosqlite" >}} + ```python +import asyncio import sqlite3 import aiosqlite from app.db import query -async with aiosqlite.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) as conn: - user = await query.get_user(conn, id_=1) - if user is not None: - print(user.name) - # every row at once - users = await query.list_users(conn) +async def main() -> None: + async with aiosqlite.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) as conn: + user = await query.get_user(conn, id_=1) + if user is not None: + print(user.name) - # or stream them - async for user in query.list_users(conn): - print(user.name) + # every row at once + users = await query.list_users(conn) + + # or stream them + async for user in query.list_users(conn): + print(user.name) + + +asyncio.run(main()) ``` + {{< /tab >}} {{< tab name="sqlite3" >}} + ```python import sqlite3 @@ -373,6 +421,7 @@ users = query.list_users(conn)() for user in query.list_users(conn): print(user.name) ``` + {{< /tab >}} {{< /tabs >}} diff --git a/docs/content/docs/guide/docstrings.md b/docs/content/docs/guide/docstrings.md index d683a281..21fff0c1 100644 --- a/docs/content/docs/guide/docstrings.md +++ b/docs/content/docs/guide/docstrings.md @@ -23,6 +23,7 @@ The same `:one` query, generated under each convention: {{< tabs >}} {{< tab name="google" >}} + ````python async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFieldNaming | None: """Fetch one from the db using the SQL query with `name: GetFieldNaming :one`. @@ -42,9 +43,11 @@ async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFiel Result of type `models.TestFieldNaming` fetched from the db. Will be `None` if not found. """ ```` + {{< /tab >}} {{< tab name="numpy" >}} + ````python async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFieldNaming | None: """Fetch one from the db using the SQL query with `name: GetFieldNaming :one`. @@ -68,9 +71,11 @@ async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFiel """ ```` + {{< /tab >}} {{< tab name="pep257" >}} + ````python async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFieldNaming | None: """Fetch one from the db using the SQL query with `name: GetFieldNaming :one`. @@ -89,6 +94,7 @@ async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFiel models.TestFieldNaming -- Result fetched from the db. Will be `None` if not found. """ ```` + {{< /tab >}} {{< /tabs >}} diff --git a/docs/content/docs/guide/drivers.md b/docs/content/docs/guide/drivers.md index 5522d7eb..1e931258 100644 --- a/docs/content/docs/guide/drivers.md +++ b/docs/content/docs/guide/drivers.md @@ -20,12 +20,19 @@ open and manage the connection yourself and pass it in. ## asyncpg (PostgreSQL) ```python +import asyncio + import asyncpg from app.db import queries -conn = await asyncpg.connect("postgresql://user:pass@localhost/db") -user = await queries.get_field_naming(conn, id_=1) + +async def main() -> None: + conn = await asyncpg.connect("postgresql://user:pass@localhost/db") + user = await queries.get_field_naming(conn, id_=1) + + +asyncio.run(main()) ``` asyncpg is the only driver that supports `:copyfrom` (bulk insert via @@ -34,14 +41,20 @@ asyncpg is the only driver that supports `:copyfrom` (bulk insert via ## aiosqlite (async SQLite) ```python +import asyncio import sqlite3 import aiosqlite from app.db import queries -async with aiosqlite.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) as conn: - user = await queries.get_field_naming(conn, id_=1) + +async def main() -> None: + async with aiosqlite.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) as conn: + user = await queries.get_field_naming(conn, id_=1) + + +asyncio.run(main()) ``` ## sqlite3 (sync SQLite) diff --git a/docs/content/docs/guide/model-types.md b/docs/content/docs/guide/model-types.md index 40795276..0d5b14a1 100644 --- a/docs/content/docs/guide/model-types.md +++ b/docs/content/docs/guide/model-types.md @@ -31,6 +31,7 @@ the plugin generates (shown with the default `docstrings: none`): {{< tabs >}} {{< tab name="dataclass" >}} + ```python @dataclasses.dataclass() class TestFieldNaming: @@ -42,6 +43,7 @@ Standard library, no dependencies. The safe default. {{< /tab >}} {{< tab name="attrs" >}} + ```python @attrs.define() class TestFieldNaming: @@ -53,6 +55,7 @@ Requires `attrs`. Slotted classes with concise definitions. {{< /tab >}} {{< tab name="msgspec" >}} + ```python class TestFieldNaming(msgspec.Struct): id_: int @@ -64,6 +67,7 @@ Requires `msgspec`. Very fast serialization/validation; pairs well with {{< /tab >}} {{< tab name="pydantic" >}} + ```python class TestFieldNaming(pydantic.BaseModel): model_config = pydantic.ConfigDict(arbitrary_types_allowed=True) diff --git a/docs/content/docs/guide/sqlite-type-conversion.md b/docs/content/docs/guide/sqlite-type-conversion.md index 5c9c0eb3..99532ad5 100644 --- a/docs/content/docs/guide/sqlite-type-conversion.md +++ b/docs/content/docs/guide/sqlite-type-conversion.md @@ -18,20 +18,26 @@ open the connection with `PARSE_DECLTYPES`: {{< tabs >}} {{< tab name="sqlite3" >}} + ```python conn = sqlite3.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) ``` + {{< /tab >}} {{< tab name="aiosqlite" >}} + ```python import sqlite3 import aiosqlite -async with aiosqlite.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) as conn: - ... + +async def main() -> None: + async with aiosqlite.connect("app.db", detect_types=sqlite3.PARSE_DECLTYPES) as conn: + ... ``` + {{< /tab >}} {{< /tabs >}} diff --git a/docs/content/docs/guide/working-with-json.md b/docs/content/docs/guide/working-with-json.md index d7cb510f..f02f621f 100644 --- a/docs/content/docs/guide/working-with-json.md +++ b/docs/content/docs/guide/working-with-json.md @@ -53,6 +53,7 @@ INSERT INTO users (id, name, preferences) VALUES ($1, $2, $3); {{< tabs >}} {{< tab name="models.py" >}} + ```python import msgspec @@ -73,6 +74,7 @@ field. `msgspec` handles all of it. {{< /tab >}} {{< tab name="converters.py" >}} + ```python import msgspec @@ -93,6 +95,7 @@ The `type=` argument is what makes decoding *typed* - msgspec builds real {{< /tab >}} {{< tab name="sqlc.yaml" >}} + ```yaml options: model_type: "msgspec" @@ -108,6 +111,7 @@ options: - column: users.preferences converter: preferences ``` + {{< /tab >}} {{< /tabs >}} @@ -138,15 +142,17 @@ async def create_user(conn: ConnectionLike, *, id_: int, name: str, preferences: ### Using it ```python -await create_user( - conn, - id_=1, - name="ada", - preferences=Preferences(theme=Theme(name="solarized", dark_mode=True), languages=["en"]), -) - -user = await get_user(conn, id_=1) -user.preferences.theme.dark_mode # True - typed all the way down +async def main() -> None: + await create_user( + conn, + id_=1, + name="ada", + preferences=Preferences(theme=Theme(name="solarized", dark_mode=True), languages=["en"]), + ) + + user = await get_user(conn, id_=1) + if user is not None: + print(user.preferences.theme.dark_mode) # True - typed all the way down ``` No `json.loads`, no `dict["theme"]["dark_mode"]`, and pyright checks every access. From bcae564c5abe53497ae36624fbe54c388b3f4180 Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 18:56:53 +0200 Subject: [PATCH 11/13] docs: correct the query command and emit_classes descriptions Checking the guide against the generated fixtures turned up several claims that were simply wrong. There is no Querier class. Each query file gets its own class named after the file, taking the connection in the constructor and exposing it again as a conn property, so document that shape instead. The exec family was described loosely: execlastid is int | None and refers to the last affected row, execresult is a status string on asyncpg but a cursor on the sqlite drivers, and execrows reports 0 or -1 depending on the driver when nothing is affected. A one query can also return a generated row class or a bare scalar rather than a model, and a many function is not a coroutine even on the async drivers. Stop claiming every sqlc macro works. sqlc.slice only reaches the type layer: the SLICE placeholder is never expanded, so the query fails once the driver is handed a list. Point people at ANY instead. Redirect the old converters URL, which moved under the guide. --- README.md | 5 +- docs/content/docs/guide/converters.md | 2 + docs/content/docs/guide/writing-queries.md | 51 ++++++++++++++++--- .../docs/reference/configuration-options.md | 2 +- .../content/docs/reference/feature-support.md | 12 ++++- 5 files changed, 60 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 17b7b540..9e2e8c1b 100644 --- a/README.md +++ b/README.md @@ -79,8 +79,9 @@ and the full plugin option list in the ([docs](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/docstrings/)). - Generated code passes **pyright strict** and **ruff**. -Every [sqlc macro](https://docs.sqlc.dev/en/latest/reference/macros.html) is -supported. Which query commands are available depends on the driver - see the +The [sqlc macros](https://docs.sqlc.dev/en/latest/reference/macros.html) +`sqlc.arg`, `sqlc.narg` and `sqlc.embed` are supported (`sqlc.slice` is not). +Which query commands are available depends on the driver - see the [feature support matrix](https://rayakame.github.io/sqlc-gen-better-python/docs/reference/feature-support/). ## Development diff --git a/docs/content/docs/guide/converters.md b/docs/content/docs/guide/converters.md index 376e875e..7c39803c 100644 --- a/docs/content/docs/guide/converters.md +++ b/docs/content/docs/guide/converters.md @@ -3,6 +3,8 @@ title: Converters weight: 70 prev: /docs/guide/type-overrides next: /docs/guide/working-with-json +aliases: + - /docs/converters/ --- A [type override](/docs/guide/type-overrides) replaces a column's Python type, diff --git a/docs/content/docs/guide/writing-queries.md b/docs/content/docs/guide/writing-queries.md index 6d579900..043ab5b6 100644 --- a/docs/content/docs/guide/writing-queries.md +++ b/docs/content/docs/guide/writing-queries.md @@ -48,7 +48,9 @@ async def get_field_naming(conn: ConnectionLike, *, id_: int) -> models.TestFiel ### `:one` -Returns a single model instance or `None`. Shown above. +Returns the row or `None`. The row is a `models.*` class when the query's columns +match a table, a generated `Row` class when they do not, or a bare scalar +when the query selects a single column. Shown above. When a query's columns do not match one table exactly (a join, a partial select, an aggregate), the plugin generates a dedicated `Row` class instead of @@ -82,6 +84,12 @@ Here the query selects one column, so `T` is a scalar (`datetime.datetime`). A `:many` that selects full rows returns `QueryResults[models.Foo]` (or a `Row`). +{{< callout type="info" >}} + Note that a `:many` function is **not** a coroutine, even on the async drivers - + it returns the `QueryResults` helper synchronously. You await (or iterate) the + helper, not the call. +{{< /callout >}} + ### `:exec` Runs the statement and returns `None`: @@ -95,9 +103,15 @@ async def set_field_naming_outputs(conn: ConnectionLike, *, id_: int, outputs: s Variants of `:exec` that return something about the write: -- **`:execrows`** - the number of affected rows (`int`). -- **`:execlastid`** - the id of the last inserted row (`int`); SQLite drivers only. -- **`:execresult`** - the driver's raw result/status object. +- **`:execrows`** - the number of affected rows (`int`). For statements that + affect no rows, such as `CREATE TABLE`, asyncpg reports `0` and the SQLite + drivers report `-1`. +- **`:execlastid`** - the cursor's `lastrowid`, typed `int | None` - it is `None` + when no row was affected. SQLite drivers only, and note it is the last + *affected* row, not strictly the last inserted one. +- **`:execresult`** - the driver's raw result, which differs per driver: a `str` + status tag on asyncpg, and a `sqlite3.Cursor` / `aiosqlite.Cursor` on the + SQLite drivers. See the [feature support matrix](/docs/reference/feature-support) for which driver supports which. @@ -130,6 +144,29 @@ this: ## Grouping into a class -With `emit_classes: true`, the standalone functions become methods on a single -`Querier` class instead. The bodies are identical; only the call style changes -(`Querier(conn).get_field_naming(id_=1)`). +With `emit_classes: true`, the standalone functions of each query file become +methods on a class named after that file - `queries_field_namings.sql` yields +`QueriesFieldNamings` - so you get one class per query module rather than one +class overall. + +The connection is passed once to the constructor and is also exposed as a +read-only `conn` property. The bodies are otherwise unchanged, except that `conn` +becomes `self._conn`: + +```python +class QueriesFieldNamings: + __slots__ = ("_conn",) + + def __init__(self, conn: ConnectionLike) -> None: + self._conn = conn + + @property + def conn(self) -> ConnectionLike: + return self._conn + + async def get_field_naming(self, *, id_: int) -> models.TestFieldNaming | None: + row = await self._conn.fetchrow(GET_FIELD_NAMING, id_) + ... +``` + +so the call becomes `QueriesFieldNamings(conn).get_field_naming(id_=1)`. diff --git a/docs/content/docs/reference/configuration-options.md b/docs/content/docs/reference/configuration-options.md index 6b0ffb60..f3543a0d 100644 --- a/docs/content/docs/reference/configuration-options.md +++ b/docs/content/docs/reference/configuration-options.md @@ -22,7 +22,7 @@ optional. | `model_type` | string | `dataclass` | One of `dataclass`, `attrs`, `msgspec`, `pydantic`. See [Model types](/docs/guide/model-types). | | `initialisms` | list[string] | `["id"]` | Identifier segments to upper-case, e.g. `app_id` -> `AppID`. | | `emit_exact_table_names` | bool | `false` | If `true`, model names mirror table names; otherwise plural table names are singularized. | -| `emit_classes` | bool | `false` | If `true`, query functions become methods on a `Querier` class. See [Writing queries](/docs/guide/writing-queries). | +| `emit_classes` | bool | `false` | If `true`, each query file's functions become methods on a class named after that file (`queries_field_namings.sql` -> `QueriesFieldNamings`). See [Writing queries](/docs/guide/writing-queries). | | `inflection_exclude_table_names` | list[string] | `[]` | Table names to leave un-singularized (only used when `emit_exact_table_names` is `false`). | | `omit_unused_models` | bool | `false` | If `true`, tables not referenced by any query are not turned into models. | | `omit_typechecking_block` | bool | `false` | If `true`, non-builtin types are imported at module level instead of inside a `typing.TYPE_CHECKING` block. | diff --git a/docs/content/docs/reference/feature-support.md b/docs/content/docs/reference/feature-support.md index 877424d2..11b475b0 100644 --- a/docs/content/docs/reference/feature-support.md +++ b/docs/content/docs/reference/feature-support.md @@ -8,8 +8,16 @@ Which sqlc features and query commands the plugin supports. ## Macros -Every [sqlc macro](https://docs.sqlc.dev/en/latest/reference/macros.html) is -supported (`sqlc.arg`, `sqlc.narg`, `sqlc.embed`, `sqlc.slice`). +Of the [sqlc macros](https://docs.sqlc.dev/en/latest/reference/macros.html), +`sqlc.arg`, `sqlc.narg`, and `sqlc.embed` are supported. + +{{< callout type="warning" >}} + **`sqlc.slice` is not supported.** The parameter is typed as a + `collections.abc.Sequence[...]`, but the `/*SLICE:name*/` placeholder that sqlc + emits is never expanded, so the query fails at runtime when the driver is handed + a list. On PostgreSQL, use `= ANY($1::type[])` instead - that is fully + supported, including with [converters](/docs/guide/converters). +{{< /callout >}} ## Query commands From 0366b978d907036c6ce0b68541f4d2ac713aba13 Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 19:24:19 +0200 Subject: [PATCH 12/13] docs: keep sqlc.slice listed as supported sqlc.slice is a supported macro whose placeholder expansion is being fixed before the next release, so list all four macros as supported again rather than calling it unsupported. Leave a short note about the current release behaviour pointing at the tracking issue, to be removed once the fix lands. --- README.md | 5 ++--- .../content/docs/reference/feature-support.md | 19 ++++++++++--------- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 9e2e8c1b..17b7b540 100644 --- a/README.md +++ b/README.md @@ -79,9 +79,8 @@ and the full plugin option list in the ([docs](https://rayakame.github.io/sqlc-gen-better-python/docs/guide/docstrings/)). - Generated code passes **pyright strict** and **ruff**. -The [sqlc macros](https://docs.sqlc.dev/en/latest/reference/macros.html) -`sqlc.arg`, `sqlc.narg` and `sqlc.embed` are supported (`sqlc.slice` is not). -Which query commands are available depends on the driver - see the +Every [sqlc macro](https://docs.sqlc.dev/en/latest/reference/macros.html) is +supported. Which query commands are available depends on the driver - see the [feature support matrix](https://rayakame.github.io/sqlc-gen-better-python/docs/reference/feature-support/). ## Development diff --git a/docs/content/docs/reference/feature-support.md b/docs/content/docs/reference/feature-support.md index 11b475b0..bc020d22 100644 --- a/docs/content/docs/reference/feature-support.md +++ b/docs/content/docs/reference/feature-support.md @@ -8,15 +8,16 @@ Which sqlc features and query commands the plugin supports. ## Macros -Of the [sqlc macros](https://docs.sqlc.dev/en/latest/reference/macros.html), -`sqlc.arg`, `sqlc.narg`, and `sqlc.embed` are supported. - -{{< callout type="warning" >}} - **`sqlc.slice` is not supported.** The parameter is typed as a - `collections.abc.Sequence[...]`, but the `/*SLICE:name*/` placeholder that sqlc - emits is never expanded, so the query fails at runtime when the driver is handed - a list. On PostgreSQL, use `= ANY($1::type[])` instead - that is fully - supported, including with [converters](/docs/guide/converters). +Every [sqlc macro](https://docs.sqlc.dev/en/latest/reference/macros.html) is +supported (`sqlc.arg`, `sqlc.narg`, `sqlc.embed`, `sqlc.slice`). + +{{< callout type="info" >}} + Known issue in `v0.5.1`: a query using `sqlc.slice` is generated with sqlc's + `/*SLICE:name*/` placeholder left in place, so it fails at runtime. A fix is + landing before the next release - see + [issue #210](https://github.com/rayakame/sqlc-gen-better-python/issues/210). + On PostgreSQL, `= ANY($1::type[])` is the idiomatic form anyway and is + unaffected. {{< /callout >}} ## Query commands From f5a3acc87cc862381fcec476f6d404e1ae82b8aa Mon Sep 17 00:00:00 2001 From: Rayakame Date: Tue, 21 Jul 2026 19:51:05 +0200 Subject: [PATCH 13/13] docs: await the query method in the emit_classes example The generated method is a coroutine on the async drivers, so calling it without await hands back an unexecuted coroutine rather than the row. Note that sqlite3 methods are plain functions and need no await. --- docs/content/docs/guide/writing-queries.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/content/docs/guide/writing-queries.md b/docs/content/docs/guide/writing-queries.md index 043ab5b6..7f4dd034 100644 --- a/docs/content/docs/guide/writing-queries.md +++ b/docs/content/docs/guide/writing-queries.md @@ -169,4 +169,7 @@ class QueriesFieldNamings: ... ``` -so the call becomes `QueriesFieldNamings(conn).get_field_naming(id_=1)`. +so the call becomes +`user = await QueriesFieldNamings(conn).get_field_naming(id_=1)` - without the +`await` on the synchronous `sqlite3` driver, where the method is a plain +function.