Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .tabularium
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"id": "libsql",
"name": "libSQL",
"version": "1.0.2",
"version": "1.0.3",
"kind": "driver",
"description": "libSQL / Turso driver. Connects to local libSQL/SQLite files (embedded libSQL fork: ALTER COLUMN and foreign-key add/drop work locally) and to remote Turso or sqld servers over the Hrana HTTP protocol.",
"engine": "libsql",
Expand Down
42 changes: 42 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# AGENTS.md

Tabularis driver plugin for libSQL/Turso: a standalone binary speaking Tabularis' JSON-RPC 2.0 over stdin/stdout (one request line in, one response line out). Two backends: local files through the bundled **libSQL fork** of SQLite (`libsql` crate, `core` feature, no system lib) and remote Turso/sqld over Hrana HTTP `/v2/pipeline` with blocking `ureq` + rustls. No async runtime: `futures_executor::block_on` bridges libsql's async API. A C compiler is required to build the bundled SQLite.

## Commands

- `just test` — `cargo test`; unit tests only, inline `#[cfg(test)]`, no integration harness. Single test: `cargo test <name>`.
- `just lint` / `just fmt` — clippy with `-D warnings` / rustfmt. CI runs build + `cargo test --locked --all-targets` + RustSec audit, but not clippy or fmt, so run both locally.
- `just repl` — `cargo run --bin test_plugin`: REPL over the real dispatch. Bare method names (e.g. `get_databases`) are auto-wrapped; full JSON-RPC objects pass through; `exit` quits.
- `just dev-install` — builds debug, copies binary + `.tabularium` to the OS plugin dir: Linux `~/.local/share/tabularis/plugins/drivers/libsql`, macOS `~/Library/Application Support/tabularis/plugins/drivers/libsql`, Windows `%APPDATA%\tabularis\plugins\drivers\libsql`. Tabularis resolves kind-first (`plugins/drivers/<id>` before flat `plugins/<id>`), so the `drivers` segment is mandatory — a flat copy is shadowed by an existing installed bundle.
- Manual smoke: `echo '{"jsonrpc":"2.0","method":"get_tables","params":{"params":{"database":":memory:"}},"id":1}' | ./target/debug/libsql-plugin`
- Windows shell is PowerShell; `just` recipes use `windows-shell`, and each recipe line runs as a separate `powershell.exe`. Variable-setting recipes must be one `;`-joined line (see `dev-install`).

## Architecture

- Logic lives in the library (`src/lib.rs`); `src/main.rs` is thin (read line → `rpc::handle_line` → write line). `test_plugin` drives the same dispatch. New RPC method = handler in `src/handlers/{metadata,query,crud,ddl}.rs` plus an arm in `rpc.rs::handle_line`; handlers return `Result<Value, PluginError>` via `handlers::respond`.
- Connection params live in the nested `params.params` block; use `handlers::connect` / `models::inner_params`. A fresh `Client` is built per call; there is no shared state.
- Preview builders `get_create_table_sql`, `get_add_column_sql`, `get_create_index_sql` are pure (no connection params). `get_create_foreign_key_sql` always receives `params.params` (the host's RpcDriver passes them); `get_alter_column_sql` uses params only when present and otherwise stays pure. Do not call `connect()` unconditionally in builders. Methods that touch a database (`execute_query`, CRUD, `drop_index`, `drop_foreign_key`, schema-introspecting metadata) do receive params.
- `execute_query` must match the host's strict `QueryResult`: `affected_rows`, `truncated`, and `pagination` are required. Requests send `limit` (code also accepts `page_size`); the response `pagination` is `{page, page_size, total_rows: null, has_more}`. `has_more` comes from a LIMIT+1 fetch; there is no `total_count` field.
- `client.rs::resolve_backend`: `connection_uri` is authoritative over decomposed fields. A URL in `database`/`host` (or any non-empty `host`) is remote; otherwise `database` is a local path. `libsql://`/`turso://`/`wss://` → `https://`, `ws://` → `http://`; localhost or `ssl_mode=disable` uses `http://`; `file:` paths and `~/` expand. Auth token from the URL query (`authToken`, etc.) wins over `password`.
- sqld namespaces go in the `x-namespace` header, not the URL path: sqld 0.24.x ignores `/dev/<ns>/...` and resolves from Host. `hrana.rs::namespace_from_url_path` maps `.../dev/example` → `example`; plain Turso hosts send no header. When `host` is bare, a decomposed `database` containing `/` is kept as a URL path prefix; `.tabularium` declares `http`/`https` so those URIs reach the plugin.
- Errors must become JSON-RPC error responses, never panics; one bad request must not kill the process.

## Conventions / gotchas

- Quote identifiers with `utils::identifiers::quote` (ANSI `"name"`); bind CRUD values, never interpolate them. Booleans are stored `0`/`1`; BLOBs are returned base64-encoded.
- `get_databases` is static `["main"]`; `get_schemas` and `get_routines` return empty (libSQL has no schemas or stored routines).
- `update_record` / `delete_record` take `pk_map` (preferred, supports composite keys); legacy `pk_col`/`pk_val` is also accepted. Keep both.
- Both backends run the libSQL fork, so `ALTER TABLE ... ALTER COLUMN` (retype, FK add/drop) works locally and remotely. The ALTER/FK builders emit the libSQL form unconditionally; renames use vanilla `RENAME COLUMN`.
- FK add emits `ALTER COLUMN col TO col <type> REFERENCES ...`, reproducing type/NOT NULL/DEFAULT from `PRAGMA table_info`. Constraint names are cosmetic; names are re-derived from `PRAGMA foreign_key_list`. Composite FK drops error; FK enforcement needs `PRAGMA foreign_keys=ON` per connection.
- `.tabularium` capabilities must stay in sync with actual support (`alter_column`, `create_foreign_keys` are true).
- `get_schema_snapshot` returns the host's `Vec<TableSchema>` shape: a top-level array of `{name, columns, foreign_keys}` (empty schema → `[]`). The host deserializes it directly into the ER diagram, so an envelope or name-keyed map fails the whole load.
- Tests that need one database across connections must use a temp file: local `:memory:` databases are per-connection.

## Release

- Tag `v*` triggers `.github/workflows/release.yml`: 5 targets (Linux x64/arm64 — arm64 via `cross` — macOS x64/arm64, Windows x64), zipping the binary, `.tabularium`, and assets. `.tabularium` also ships as a standalone asset because the registry resolves the manifest from release assets, not the git ref.
- `Cargo.lock` is committed; CI and release builds use `--locked`. Keep the `.tabularium` version in sync with `Cargo.toml`.

## Demo

- `demo/tabularis_demo.db` and its `.db-wal` are committed; regenerate with `demo/make_demo_db.py` + `demo/seed.sql`. `demo/turso_demo.py` targets remote Turso.
18 changes: 17 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [1.0.3] - 2026-10-26

### Fixed

- The ER diagram could never load on a libSQL connection: `get_schema_snapshot`
returned a `{tables, columns, foreign_keys}` envelope, while the host
deserializes the result straight into `Vec<TableSchema>` and rejected it with
`invalid type: map, expected a sequence`. The handler now returns one
`{name, columns, foreign_keys}` entry per table, and an empty schema is `[]`.
- `just dev-install` and `uninstall` pointed at the flat `plugins/libsql`
bundle, which the host shadows once a kind-scoped `plugins/drivers/libsql`
bundle exists. Both now target the kind-first path on Linux, macOS and
Windows.

## [1.0.2] - 2026-10-25

### Fixed
Expand Down Expand Up @@ -117,6 +131,8 @@ Thanks to @debba, @NewtTheWolf and @jonaspm — see the
[v1.0.0 release notes](https://github.com/TabularisDB/tabularis-libsql-plugin/releases/tag/v1.0.0)
for the full list.

[Unreleased]: https://github.com/TabularisDB/tabularis-libsql-plugin/compare/v1.0.1...HEAD
[Unreleased]: https://github.com/TabularisDB/tabularis-libsql-plugin/compare/v1.0.3...HEAD
[1.0.3]: https://github.com/TabularisDB/tabularis-libsql-plugin/releases/tag/v1.0.3
[1.0.2]: https://github.com/TabularisDB/tabularis-libsql-plugin/releases/tag/v1.0.2
[1.0.1]: https://github.com/TabularisDB/tabularis-libsql-plugin/releases/tag/v1.0.1
[1.0.0]: https://github.com/TabularisDB/tabularis-libsql-plugin/releases/tag/v1.0.0
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "libsql-plugin"
version = "1.0.2"
version = "1.0.3"
edition = "2021"
description = "Tabularis driver plugin for libSQL / Turso (local files and remote Hrana-over-HTTP)"
license = "Apache-2.0"
Expand Down
86 changes: 65 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
# Tabularis libSQL / Turso driver

[![CI](https://github.com/TabularisDB/tabularis-libsql-plugin/actions/workflows/ci.yml/badge.svg)](https://github.com/TabularisDB/tabularis-libsql-plugin/actions/workflows/ci.yml)

> ⚠️ **Work in progress** — this plugin is under active development. APIs,
> behavior and feature coverage may change, and things may break. Use at your
> own risk.

A [Tabularis](https://github.com/TabularisDB/tabularis) database driver plugin
for **libSQL**. It connects to:

- **Local libSQL / SQLite files** on disk (via bundled SQLite — nothing to
install), and
- **Local libSQL / SQLite files** on disk (via the bundled libSQL fork — nothing
to install), and
- **Remote Turso / sqld servers** over the **Hrana HTTP** protocol.

The plugin is a standalone executable that speaks Tabularis' JSON-RPC protocol
Expand All @@ -23,14 +25,24 @@ The driver picks a backend from the connection form automatically:
| You enter | Backend |
|-----------|---------|
| A file path in **Database** (e.g. `/data/app.db`, `~/notes.db`, `:memory:`) | Local SQLite file |
| A `file:` URI in **Database** / connection string (e.g. `file:///data/app.db`) | Local SQLite file |
| A URL in **Database** or **Host** (e.g. `libsql://my-db.turso.io`, `http://localhost:8080/dev/example/`) | Remote Hrana HTTP |
| A bare host in **Host** (e.g. `db.turso.io`) | Remote Hrana HTTP (`https://`) |

For Turso, put the **auth token** in the **Password** field, or append it to the
URL as `?authToken=...`. `libsql://`, `wss://` and `turso://` URLs are
A connection URI (`connection_uri`) is authoritative over the decomposed
`host`/`port`/`database` fields. `libsql://`, `wss://` and `turso://` URLs are
automatically rewritten to `https://` (and `ws://` to `http://`) for the Hrana
HTTP endpoint. `http://` and `https://` URLs are used as-is. A bare
`localhost:8080` host uses `http://` automatically.
HTTP endpoint; `http://` and `https://` URLs are used as-is.

For a decomposed `host`, the scheme is chosen from `ssl_mode`:

- `ssl_mode=disable` forces `http://` for any host.
- A loopback host (`localhost`, `127.0.0.1`, `[::1]`) without
`ssl_mode=require` also uses `http://`.
- Everything else uses `https://`.

For Turso, put the **auth token** in the **Password** field, or append it to the
URL as `?authToken=...`.

Self-hosted sqld namespaces: the namespace is derived from the URL path
(`http://localhost:8080/dev/example/` → `example`) and sent as the `x-namespace`
Expand All @@ -50,13 +62,15 @@ http://localhost:8080/dev/example/

| Area | Status |
|------|--------|
| `test_connection`, `ping` | ✅ |
| `initialize`, `ping`, `test_connection` | ✅ |
| Connection-string import (`connection_uri` + `turso`/`http`/`https`/`ws`/`wss`/`file` schemes) | ✅ |
| Tables, columns, indexes, foreign keys | ✅ |
| Triggers: list + definition | ✅ |
| Views: list, definition, columns, create/alter/drop | ✅ |
| Query execution with server-side LIMIT/OFFSET pagination + total count | ✅ |
| Query execution with LIMIT/OFFSET pagination for wrappable `SELECT`s (`has_more` from a one-row-ahead fetch; `total_rows` is always `null`) | ✅ |
| `EXPLAIN QUERY PLAN` | ✅ |
| Insert / update / delete rows (bound parameters) | ✅ |
| Schema snapshot + batch columns/FKs (ER diagram) | ✅ |
| Schema snapshot + batch columns/FKs — powers the ER diagram | ✅ |
| `CREATE TABLE` SQL, add column, create/drop index | ✅ |
| Schemas, stored routines | ❌ (not a SQLite concept — Turso has no stored procedures, and its multi-database model is separate databases, not schemas) |
| Rename column | ✅ everywhere (vanilla `RENAME COLUMN`) |
Expand All @@ -67,6 +81,10 @@ http://localhost:8080/dev/example/
Identifiers are quoted ANSI-style (`"name"`). Booleans are stored as `0`/`1` and
BLOBs are returned base64-encoded.

`get_schema_snapshot` returns the host's `TableSchema` array shape — one
`{name, columns, foreign_keys}` entry per table — which is what Tabularis
consumes for the ER diagram. An empty schema returns `[]`.

### Schema changes via ALTER COLUMN

Both local files and remote Turso / sqld servers run the **libSQL fork** of
Expand All @@ -93,17 +111,27 @@ silently doing nothing.

## Build & test

Requires a Rust toolchain (and a C compiler for the bundled SQLite).
Requires a Rust toolchain (`rust-toolchain.toml` pins the stable channel with
`rustfmt` and `clippy`) and a C compiler for the bundled SQLite. `just` is
convenient but optional — every recipe is a thin wrapper over cargo.

```bash
just test # cargo test — unit tests for SQL builders, parsing, RPC
just build # debug build
just release # optimized release build
just lint # clippy -D warnings
just dev-install # build + copy binary + manifest into the Tabularis plugins dir
just fmt # rustfmt
just dev-install # build + copy binary + manifest into plugins/drivers/libsql
just repl # local JSON-RPC REPL over stdio
```

CI runs the same suite with the lockfile pinned:

```bash
cargo build --locked
cargo test --locked --all-targets
```

Or directly with cargo:

```bash
Expand All @@ -120,29 +148,45 @@ echo '{"jsonrpc":"2.0","method":"get_tables","params":{"params":{"database":"/tm

## Installing

`just dev-install` copies `libsql-plugin` and `.tabularium` into the Tabularis
plugins folder (the location Tabularis actually scans — derived from
`ProjectDirs::from("com", "debba", "tabularis")`):
### Release archive

Download `libsql-plugin-<platform>.zip` from the
[releases page](https://github.com/TabularisDB/tabularis-libsql-plugin/releases)
and unpack it so the driver folder contains the executable, `.tabularium` and
`assets/`:

- **Linux:** `~/.local/share/tabularis/plugins/drivers/libsql/`
- **macOS:** `~/Library/Application Support/tabularis/plugins/drivers/libsql/`
- **Windows:** `%APPDATA%\tabularis\plugins\drivers\libsql\`

The root is the Tabularis config directory
(`ProjectDirs::from("com", "debba", "tabularis")`). Tabularis resolves
kind-scoped bundles (`plugins/drivers/<id>`) before flat `plugins/<id>` bundles,
so keep the `drivers` segment — a flat copy is shadowed by an installed bundle.
Each release also publishes `.tabularium` as a standalone asset.

### From source

- **Linux:** `~/.local/share/tabularis/plugins/libsql/`
- **macOS:** `~/Library/Application Support/tabularis/plugins/libsql/`
- **Windows:** `%APPDATA%\tabularis\plugins\libsql\`
`just dev-install` builds a debug binary and copies it plus `.tabularium` into
the same `plugins/drivers/libsql` folder.

Restart Tabularis (or toggle the plugin in Settings) and **libSQL** appears in
the Database Type list.
Either way, restart Tabularis (or toggle the plugin in Settings) and **libSQL**
appears in the Database Type list.

## Architecture

```
src/
├── main.rs # stdio JSON-RPC loop
├── main.rs # stdio JSON-RPC loop (one request line in, one response line out)
├── lib.rs # shared dispatch, used by main.rs and the REPL
├── rpc.rs # method routing + response helpers
├── client.rs # backend resolution (local vs remote) + unified query/execute
├── hrana.rs # Hrana-over-HTTP pipeline client (remote)
├── models.rs # ConnectionParams
├── error.rs # PluginError
├── handlers/ # metadata / query / crud / ddl
└── utils/ # identifiers, pagination, SQL classification, value conversion
├── utils/ # identifiers, pagination, SQL classification, value conversion
└── bin/test_plugin.rs # `just repl` driver
```

Local files use the embedded **libSQL fork of SQLite** (bundled — nothing to
Expand Down
24 changes: 12 additions & 12 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,33 +31,33 @@ fmt:

[linux]
dev-install: build
mkdir -p ~/.local/share/tabularis/plugins/libsql
cp target/debug/libsql-plugin ~/.local/share/tabularis/plugins/libsql/
cp .tabularium ~/.local/share/tabularis/plugins/libsql/
@echo "Installed to ~/.local/share/tabularis/plugins/libsql"
mkdir -p ~/.local/share/tabularis/plugins/drivers/libsql
cp target/debug/libsql-plugin ~/.local/share/tabularis/plugins/drivers/libsql/
cp .tabularium ~/.local/share/tabularis/plugins/drivers/libsql/
@echo "Installed to ~/.local/share/tabularis/plugins/drivers/libsql"
@echo "Restart Tabularis (or toggle the plugin in Settings) to pick up changes."

[macos]
dev-install: build
mkdir -p "$HOME/Library/Application Support/tabularis/plugins/libsql"
cp target/debug/libsql-plugin "$HOME/Library/Application Support/tabularis/plugins/libsql/"
cp .tabularium "$HOME/Library/Application Support/tabularis/plugins/libsql/"
@echo "Installed to ~/Library/Application Support/tabularis/plugins/libsql"
mkdir -p "$HOME/Library/Application Support/tabularis/plugins/drivers/libsql"
cp target/debug/libsql-plugin "$HOME/Library/Application Support/tabularis/plugins/drivers/libsql/"
cp .tabularium "$HOME/Library/Application Support/tabularis/plugins/drivers/libsql/"
@echo "Installed to ~/Library/Application Support/tabularis/plugins/drivers/libsql"
@echo "Restart Tabularis (or toggle the plugin in Settings) to pick up changes."

[windows]
dev-install: build
$dest = Join-Path $env:APPDATA "tabularis\plugins\libsql"; New-Item -ItemType Directory -Force -Path $dest | Out-Null; Copy-Item "target\debug\libsql-plugin.exe" $dest; Copy-Item ".tabularium" $dest; Write-Host "Installed to $dest"; Write-Host "Restart Tabularis (or toggle the plugin in Settings) to pick up changes."
$dest = Join-Path $env:APPDATA "tabularis\plugins\drivers\libsql"; New-Item -ItemType Directory -Force -Path $dest | Out-Null; Copy-Item "target\debug\libsql-plugin.exe" $dest; Copy-Item ".tabularium" $dest; Write-Host "Installed to $dest"; Write-Host "Restart Tabularis (or toggle the plugin in Settings) to pick up changes."

[linux]
uninstall:
rm -rf ~/.local/share/tabularis/plugins/libsql
rm -rf ~/.local/share/tabularis/plugins/drivers/libsql

[macos]
uninstall:
rm -rf "$HOME/Library/Application Support/tabularis/plugins/libsql"
rm -rf "$HOME/Library/Application Support/tabularis/plugins/drivers/libsql"

[windows]
uninstall:
$dest = Join-Path $env:APPDATA "tabularis\plugins\libsql"
$dest = Join-Path $env:APPDATA "tabularis\plugins\drivers\libsql"
if (Test-Path $dest) { Remove-Item -Recurse -Force $dest }
Loading
Loading