Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/cardinality-helpers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@maple-dev/effect-orm": minor
---

`/database` adds `runOne`, `runOption` and `runNonEmpty`: `run` with a check on how many rows came back. `runOne` wants exactly one row, `runOption` returns an `Option` and wants at most one, and `runNonEmpty` returns a `NonEmptyReadonlyArray` and wants at least one. A wrong count fails with `ResultCardinalityError` (`expected`, `rows`) in the error channel. The query runs as written, with no `LIMIT` added, so a duplicate row is found, not hidden.
27 changes: 27 additions & 0 deletions docs/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ Calling `withdraw(1, 30)` outside `transfer` does not compile: `requireTransacti
| `layerSqlClient(options)` | A `Database` layer over the `SqlClient` in context |
| `Database` | The service |
| `run(query, params?)` | Compile a query for the database's dialect, run it, decode its rows |
| `runOne`, `runOption`, `runNonEmpty` | `run`, with a check on how many rows came back ([Cardinality](#cardinality)) |
| `sql\`...\`` | A statement with every `${value}` bound; `sql.identifier(name)` for a table or column name |
| `query(statement, schema?)` | Run a statement and return its rows, decoded when a schema is given |
| `execute(statement)` | Run a statement and discard its rows |
Expand Down Expand Up @@ -133,6 +134,32 @@ so an object parsed from request JSON is bound as a value, never spliced:
Db.sql`SELECT * FROM t WHERE id IN (${Db.sql.join(ids)})${archived ? Db.sql` AND archived` : Db.sql.empty}`
```

### Cardinality

`run` returns every row. When a query must return a known number of rows, use the helper
that says so. It takes the same arguments as `run`:

| Helper | Returns | Fails with `ResultCardinalityError` when |
| --- | --- | --- |
| `runOne(query, params?)` | The row | Zero rows, or more than one |
| `runOption(query, params?)` | `Option.some(row)`, or `Option.none()` for zero rows | More than one row |
| `runNonEmpty(query, params?)` | Every row, as a `NonEmptyReadonlyArray` | Zero rows |

```ts
const user = yield* Db.runOne(
PG.from(Users)
.select("id", "email")
.where(($) => [$.id.eq(PG.param.string("id"))]),
{ id },
) // { id: string; email: string }, or ResultCardinalityError
```

`ResultCardinalityError` is in the error channel, not a defect: the count depends on the data.
`expected` is `"one"`, `"atMostOne"` or `"nonEmpty"`, and `rows` is how many came back. The
query runs as you wrote it, with no `LIMIT` added, so a duplicate row is found, not hidden.
Add a `limit` yourself when you only want the first row; `runOption` then never fails on a
duplicate.

For SQL inside a builder query rather than a whole statement, use
[`CH.sql`](./extending.md#chsql--sql-templates-inside-a-query).

Expand Down
4 changes: 2 additions & 2 deletions docs/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -512,8 +512,8 @@ placeholders, double-quoted identifiers.
| `ClickHouseStatementFromString`, `splitTerminalClauses`, `maskLiteralsAndComments` | `/sql` |
| Schema tooling: `isSchemaObject`, `makeSnapshot`, `renderSchema`, `diffSchemas`, `diffPgSchemas`, `fromDrizzleSnapshot`, snapshot and entity schemas (`PgEnumEntity`, `PgUniqueEntity`, …), `Hint`; for SQLite, `sqliteEntitiesOf`, `renderSqliteSchema`, `renderSqliteCreateTable`, `renderSqliteCreateIndex`, `diffSqliteSchemas`, `fromDrizzleSqliteSnapshot`, `SqliteMigrationOp`, `renderSqliteOp`, `renderSqliteMigrationSql`, `labelOfSqlite`, `rebuildName`, the `SqliteSnapshot` schema and the `SqliteTableEntity`, `SqliteColumnEntity`, `SqliteIndexEntity`, `SqliteUniqueEntity` schemas | `/schema` |
| `run`, `status`, `verify`, `baseline`, `MigrationDriver` and the migrate errors | `/migrate` |
| `Database`, `run`, `sql`, `query`, `execute`, `transaction`, `requireTransaction`, `afterCommit`, `retryContention`, `Transaction` | `/database` |
| `DatabaseError`, `TransactionCommitFailed`, `TransactionRollbackFailed` and the other transaction errors | `/database` |
| `Database`, `run`, `runOne`, `runOption`, `runNonEmpty`, `sql`, `query`, `execute`, `transaction`, `requireTransaction`, `afterCommit`, `retryContention`, `Transaction` | `/database` |
| `DatabaseError`, `ResultCardinalityError`, `TransactionCommitFailed`, `TransactionRollbackFailed` and the other transaction errors | `/database` |
| `withQuerySummary`, `QuerySummary` — the `db.query.summary` of the statements an effect runs, for spans | `/database` |
| `ClickhouseHttpClient`, `layer`, `make`, `ClickhouseHttpClientConfig` — a ClickHouse `SqlClient` over `fetch`, for Workers | `/clickhouse-http` |
| `defineConfig`, `generate`, `check`, `pull`, `loadSchema`, `readMigrations`, `analyze`, `KitError` | `/kit` |
Expand Down
2 changes: 1 addition & 1 deletion docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ alongside `compiled.rowSchemaSource`, `compiled.untypedColumns`, and `compiled.r
| Timestamps shifted by several hours | The codec assumes zone-less text is UTC, but the server/column emitted another timezone. Normalize SQL output or use a matching custom codec. |
| Microseconds/nanoseconds disappeared | Parsed `DateTime.Utc` has millisecond precision. Preserve the text with `CH.dateTime64String`. |
| An average/percentile is NULL | Empty/non-finite aggregate results can serialize as JSON null. Keep the nullable type or explicitly define a fallback. |
| “No rows” crashes a point lookup | Use `decodeFirstRow` and handle `Option.none`; an empty result is not a decode failure. |
| “No rows” crashes a point lookup | Use `Db.runOption` (or `decodeFirstRow`) and handle `Option.none`; an empty result is not a decode failure. |

The built-in numeric codecs are wire decoders, not full range validators. `CH.uint64` does not
make JavaScript numbers lossless or enforce UInt64 bounds. `CH.untyped` validates nothing for its
Expand Down
4 changes: 4 additions & 0 deletions src/database.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ export {
requireTransaction,
retryContention,
run,
runNonEmpty,
runOne,
runOption,
transaction,
type DatabaseApi,
type RowOf,
Expand All @@ -31,6 +34,7 @@ export {
} from "./database/database"
export {
DatabaseError,
ResultCardinalityError,
TenantScopeViolation,
TransactionClosed,
TransactionCommitFailed,
Expand Down
15 changes: 14 additions & 1 deletion src/database/database.test-d.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Type-level tests: rows from `run`, `requireTransaction`, `afterCommit`, and what `transaction` removes.

import { Context, Effect, Schema } from "effect"
import { type Array as Arr, Context, Effect, Option, Schema } from "effect"
import { expectTypeOf } from "expect-type"
import * as CH from "../ch/index"
import * as PG from "../postgres"
Expand All @@ -24,6 +24,19 @@ expectTypeOf(Db.run(PG.compileUnsafe(CH.from(Items).select("id", "note"), {}))).
Effect.Effect<ReadonlyArray<Row>, RunError, Db.Database>
>()

// The cardinality helpers keep the row type and add `ResultCardinalityError`.
const point = CH.from(Items)
.select("id", "note")
.where(($) => [$.id.eq(CH.param.int("id"))])
type CardinalityError = RunError | Db.ResultCardinalityError
expectTypeOf(Db.runOne(point, { id: 1 })).toEqualTypeOf<Effect.Effect<Row, CardinalityError, Db.Database>>()
expectTypeOf(Db.runOption(point, { id: 1 })).toEqualTypeOf<Effect.Effect<Option.Option<Row>, CardinalityError, Db.Database>>()
expectTypeOf(Db.runNonEmpty(point, { id: 1 })).toEqualTypeOf<
Effect.Effect<Arr.NonEmptyReadonlyArray<Row>, CardinalityError, Db.Database>
>()
// @ts-expect-error the param is still required
Db.runOne(point)

// `query` is untyped without a schema and typed with one.
expectTypeOf(Db.query(Db.sql`SELECT 1`)).toEqualTypeOf<
Effect.Effect<ReadonlyArray<Record<string, unknown>>, Db.DatabaseError, Db.Database>
Expand Down
67 changes: 66 additions & 1 deletion src/database/database.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

import { PgliteClient } from "@effect/sql-pglite"
import { assert, describe, expect, it, layer } from "@effect/vitest"
import { Cause, DateTime, Deferred, Effect, Exit, Fiber, Layer, Ref, Schedule, Schema } from "effect"
import { Cause, DateTime, Deferred, Effect, Exit, Fiber, Layer, Option, Ref, Schedule, Schema } from "effect"
import * as SqlClient from "effect/sql/SqlClient"
import * as CH from "../ch/index"
import * as PG from "../postgres"
Expand Down Expand Up @@ -115,6 +115,71 @@ layer(Live, { excludeTestServices: true })("Database on PGlite", (it) => {
}),
)

describe("cardinality", () => {
const notes = (table: string) => {
const Items = CH.table(table, { id: PG.int4, note: PG.text })
return CH.from(Items)
.select("id")
.where(($) => [$.note.eq(CH.param.string("note"))])
.orderBy(["id", "asc"])
}
const seeded = Effect.gen(function* () {
const table = yield* freshTable
yield* Effect.all([insert(table, 1, "one"), insert(table, 2, "two"), insert(table, 3, "two")])
return notes(table)
})

it.effect("runOne returns the only row", () =>
Effect.gen(function* () {
const query = yield* seeded
expect(yield* Db.runOne(query, { note: "one" })).toEqual({ id: 1 })
}),
)

it.effect("runOne fails with ResultCardinalityError on zero rows and on more than one", () =>
Effect.gen(function* () {
const query = yield* seeded
const none = yield* Effect.flip(Db.runOne(query, { note: "none" }))
assert(none instanceof Db.ResultCardinalityError)
expect({ expected: none.expected, rows: none.rows }).toEqual({ expected: "one", rows: 0 })
const many = yield* Effect.flip(Db.runOne(query, { note: "two" }))
assert(many instanceof Db.ResultCardinalityError)
expect({ expected: many.expected, rows: many.rows }).toEqual({ expected: "one", rows: 2 })
}),
)

it.effect("runOption returns None on zero rows, Some on one, and fails on more than one", () =>
Effect.gen(function* () {
const query = yield* seeded
expect(yield* Db.runOption(query, { note: "none" })).toEqual(Option.none())
expect(yield* Db.runOption(query, { note: "one" })).toEqual(Option.some({ id: 1 }))
const many = yield* Effect.flip(Db.runOption(query, { note: "two" }))
assert(many instanceof Db.ResultCardinalityError)
expect({ expected: many.expected, rows: many.rows }).toEqual({ expected: "atMostOne", rows: 2 })
}),
)

it.effect("runNonEmpty returns every row and fails on zero rows", () =>
Effect.gen(function* () {
const query = yield* seeded
expect(yield* Db.runNonEmpty(query, { note: "two" })).toEqual([{ id: 2 }, { id: 3 }])
const none = yield* Effect.flip(Db.runNonEmpty(query, { note: "none" }))
assert(none instanceof Db.ResultCardinalityError)
expect({ expected: none.expected, rows: none.rows }).toEqual({ expected: "nonEmpty", rows: 0 })
}),
)

it.effect("sends the query unchanged: no LIMIT is added", () =>
Effect.gen(function* () {
const query = yield* seeded
statements.length = 0
yield* Effect.flip(Db.runOne(query, { note: "two" }))
expect(statements).toHaveLength(1)
expect(statements[0]).not.toMatch(/LIMIT/i)
}),
)
})

it.effect("commits on success", () =>
Effect.gen(function* () {
const table = yield* freshTable
Expand Down
52 changes: 51 additions & 1 deletion src/database/database.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
// statements outliving their transaction, and contention retry. See
// `design/transactions.md`.

import { Cause, Context, Effect, ErrorReporter, Exit, Layer, Option, Schedule, Schema } from "effect"
import { Array as Arr, Cause, Context, Effect, ErrorReporter, Exit, Layer, Option, Schedule, Schema } from "effect"
import { dual } from "effect/Function"
import * as SqlClient from "effect/sql/SqlClient"
import { isSqlError } from "effect/sql/SqlError"
Expand All @@ -21,6 +21,7 @@ import type { CHUnionQuery, QueryParams } from "../ch/union"
import type { ParamsSatisfied } from "../ch/expr"
import {
DatabaseError,
ResultCardinalityError,
TransactionClosed,
TransactionCommitFailed,
TransactionOptionsRejected,
Expand Down Expand Up @@ -498,6 +499,55 @@ export const run = <Q extends Runnable, const Given extends object = {}>(
): Effect.Effect<ReadonlyArray<RowOf<Q>>, DatabaseError | QueryBuilderError | CompiledQueryDecodeError, Database> =>
Effect.flatMap(Effect.service(Database), (db) => db.run(query, params))

/** What `run` can fail with. */
type RunError = DatabaseError | QueryBuilderError | CompiledQueryDecodeError

const cardinalityError = (expected: ResultCardinalityError["expected"], rows: number) =>
new ResultCardinalityError({
message: `expected ${expected === "one" ? "exactly one row" : expected === "atMostOne" ? "at most one row" : "at least one row"}, got ${rows}`,
expected,
rows,
})

/**
* `run`, for a query that must return exactly one row. Zero rows or more than
* one fail with `ResultCardinalityError`. No `LIMIT` is added, so a duplicate
* is found, not hidden.
*/
export const runOne = <Q extends Runnable, const Given extends object = {}>(
query: Q & RunCheck<Q> & ParamsSatisfied<QueryParams<Q>, Given>,
params?: Given,
): Effect.Effect<RowOf<Q>, RunError | ResultCardinalityError, Database> =>
Effect.flatMap(run(query, params), (rows) =>
Arr.isReadonlyArrayNonEmpty(rows) && rows.length === 1 ? Effect.succeed(rows[0]) : Effect.fail(cardinalityError("one", rows.length)),
)

/**
* `run`, for a query that returns at most one row: `Option.none()` for zero
* rows, `Option.some(row)` for one. More than one fails with
* `ResultCardinalityError`. No `LIMIT` is added.
*/
export const runOption = <Q extends Runnable, const Given extends object = {}>(
query: Q & RunCheck<Q> & ParamsSatisfied<QueryParams<Q>, Given>,
params?: Given,
): Effect.Effect<Option.Option<RowOf<Q>>, RunError | ResultCardinalityError, Database> =>
Effect.flatMap(run(query, params), (rows) =>
rows.length > 1 ? Effect.fail(cardinalityError("atMostOne", rows.length)) : Effect.succeed(Arr.head(rows)),
)

/**
* `run`, for a query that must return at least one row. Zero rows fail with
* `ResultCardinalityError`; otherwise every row comes back, typed as a
* non-empty array.
*/
export const runNonEmpty = <Q extends Runnable, const Given extends object = {}>(
query: Q & RunCheck<Q> & ParamsSatisfied<QueryParams<Q>, Given>,
params?: Given,
): Effect.Effect<Arr.NonEmptyReadonlyArray<RowOf<Q>>, RunError | ResultCardinalityError, Database> =>
Effect.flatMap(run(query, params), (rows) =>
Arr.isReadonlyArrayNonEmpty(rows) ? Effect.succeed(rows) : Effect.fail(cardinalityError("nonEmpty", 0)),
)

/** `query` on the `Database` in context. */
export const query: {
(statement: StatementInput): Effect.Effect<ReadonlyArray<Record<string, unknown>>, DatabaseError, Database>
Expand Down
15 changes: 15 additions & 0 deletions src/database/errors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,21 @@ export class TenantScopeViolation extends Schema.TaggedError<TenantScopeViolatio
{ message: Schema.String, sql: Schema.String },
) {}

/**
* A query returned a number of rows its cardinality helper does not allow:
* `runOne` wants exactly one, `runOption` at most one, `runNonEmpty` at least
* one. `rows` is how many came back. It is an error, not a defect, because the
* count depends on the data. The query ran as written, with no `LIMIT` added.
*/
export class ResultCardinalityError extends Schema.TaggedError<ResultCardinalityError>()(
"@maple-dev/effect-orm/ResultCardinalityError",
{
message: Schema.String,
expected: Schema.Literals(["one", "atMostOne", "nonEmpty"]),
rows: Schema.Number,
},
) {}

export type TransactionError =
| TransactionUnsupported
| TransactionOptionsRejected
Expand Down
Loading