Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions skills/hotdata/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ hotdata databases detach <database> [--database <id>] # or t

# Preferred: load by catalog alias (server declares the table/schema if missing).
# Loads csv, json, or parquet — format read from the extension.
hotdata databases load --catalog <alias> --table <table> [--schema public] (--file <path> | --url <url> | --upload-id <id> | --result-id <id>) [--mode replace|append|delete|update|upsert] [--append] [--format csv|json|parquet] [--key <col>]... [--retry-timeout <secs>] [--workspace-id <workspace_id>]
hotdata databases load (--catalog <alias> | --database <id>) --table <table> [--schema public] (--file <path> | --url <url> | --upload-id <id> | --result-id <id>) [--mode replace|append|delete|update|upsert] [--append] [--format csv|json|parquet] [--key <col>]... [--retry-timeout <secs>] [--workspace-id <workspace_id>]

# Also available via tables subcommand
hotdata databases tables list [--database <id>] [--schema <name>] [--workspace-id <workspace_id>] [--output table|json|yaml]
Expand All @@ -136,8 +136,9 @@ hotdata databases tables remove <table> [--database <id>] [--schema public] [--w
- `use` — saves the database as the current database (accepts an id, catalog alias, or name; prefer the id). Subsequent `query`, `databases tables`, and `databases context` commands use it automatically. This is the **only** command that changes the current database.
- `unset` — clears the current database from config.
- `show <id>` / `<id>` — inspect one database (returns id, catalog, name, expires_at, attached databases; a fork also shows its `forked_from` record).
- `remove` — removes the instant database; clears the active-database config if it matched.
- `remove` — removes the instant database immediately, with no confirmation prompt (safe to script; there is no `--yes` because none is needed); clears the active-database config if it matched.
- `load` (top-level shorthand) — loads a file into `--catalog.--schema.--table`. Accepts `--file`, `--url`, `--upload-id`, or `--result-id` (load a saved query result by id — from `hotdata databases results` or a query's `[result-id: …]` footer — instead of a file; the result must belong to the target database). **Formats:** csv, json (`.json`/`.jsonl`/`.ndjson`), and parquet; the format comes from the file's extension, and `--format` overrides it (needed when the extension is absent or misleading). An unrecognised extension is not rejected — the server reads the bytes, and a file that plainly opens a json array is taken as json even without an extension. A json source in any shape (array, pretty-printed, one object per line) is reshaped locally to newline-delimited json before upload; an already-newline-delimited file is uploaded untouched. A table or schema that was never declared is declared by the server as part of the load, so no up-front `--table` is required.
- **After a fork, `--catalog` is ambiguous.** A fork answers to the same catalog alias as its source, so `databases load --catalog <alias>` (and `tables show <alias>.schema.table`) fail with `multiple databases have catalog …` once a fork exists. Pass the database id instead: `databases load --database <id> …` or `databases tables load <table> --database <id> …`. Prefer the id form in scripts.
- **Concurrent loads** — loads into one table run one at a time. While another load holds the table the server answers 409 `RESOURCE_LOCKED` with a `Retry-After`; the CLI waits and retries for up to `--retry-timeout` seconds (default 60). Pass `--retry-timeout 0` to fail immediately instead. Loads into different tables never block each other.
- **Load modes** (`--mode`, default `replace`) — `replace` supersedes the table's contents; `append` adds rows; `delete`, `update`, and `upsert` match existing rows **by key**. `--append` is the old shorthand for `--mode append` and still works, but the two cannot be combined. The keyed modes need a key: declare one with `databases tables add --key`, or name it per-load with `--key` (repeat for a composite key). `delete` uploads only the key columns; `update` replaces matching rows and ignores unmatched ones; `upsert` inserts the unmatched instead. Keyed modes are not available with `--result-id`.
- `tables list` — lists tables with `TABLE` (`<catalog>.<schema>.<table>`), `SYNCED`, `LAST_SYNC`. Uses current database when `--database` is omitted.
Expand Down Expand Up @@ -255,7 +256,7 @@ hotdata databases context push <name> [--database <id>] [--dry-run]

```
hotdata query "<sql>" [--workspace-id <workspace_id>] [--database <database>] [--dialect hotsql|duckdb|postgres|snowflake] [--output table|json|csv|id]
hotdata query status <query_run_id>
hotdata query [-d <database_id>] status <query_run_id> # -d before `status`, or set a current database first

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Drop the "-d before status" instruction (not blocking). -d is now global = true in src/cli.rs:56, so query status <id> -d <db> also parses. The PR description cites that exact form as the fix. Agents that read this line apply an ordering rule that does not exist.

Suggested change
hotdata query [-d <database_id>] status <query_run_id> # -d before `status`, or set a current database first
hotdata query status <query_run_id> [-d <database_id>] # -d is optional when a current database is set


# Same commands under the databases group (identical flags and exit codes)
hotdata databases query "<sql>" [-d <database>] [--output table|json|csv|id]
Expand All @@ -280,7 +281,7 @@ hotdata jobs list [--workspace-id <workspace_id>] [--job-type <type>] [--status
hotdata jobs <job_id> [--workspace-id <workspace_id>] [--output table|json|yaml]
```
- `list` shows only active jobs (`pending`, `running`) by default. Use `--all` to see all jobs.
- `--job-type`: `create_index`, `managed_load`, `database_fork`. (`data_refresh_table` / `data_refresh_connection` were retired server-side — the engine is push-only; nothing refreshes.)
- `--job-type`: `create_index`, `managed_load`, `database_fork`. (`database_fork` is accepted for forward compatibility; today a fork completes synchronously and records no job.) (`data_refresh_table` / `data_refresh_connection` were retired server-side — the engine is push-only; nothing refreshes.)
- `--status`: `pending`, `running`, `succeeded`, `partially_succeeded`, `failed`.
- Use `hotdata jobs <job_id>` to inspect a specific job's status, error, and result.

Expand Down
4 changes: 2 additions & 2 deletions src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ pub enum Commands {
workspace_id: Option<String>,

/// Run against a specific instant database, for this command only (defaults to the current database; switch with `databases use`)
#[arg(long, short = 'd')]
#[arg(long, short = 'd', global = true)]
database: Option<String>,

/// SQL dialect the query is written in — a non-`hotsql` dialect is
Expand All @@ -64,7 +64,7 @@ pub enum Commands {

/// Output format. `id` prints only the stored result's id, without
/// downloading rows (for commands that read a result by id)
#[arg(long = "output", short = 'o', default_value = "table", value_parser = ["table", "json", "csv", "id"])]
#[arg(long = "output", short = 'o', default_value = "table", value_parser = ["table", "json", "csv", "id"], global = true)]
output: String,

#[command(subcommand)]
Expand Down
10 changes: 6 additions & 4 deletions src/commands/connections.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,13 @@ pub fn try_resolve_connection_id(api: &Api, name_or_id: &str) -> Result<String,
}

// Fall back to instant databases: treat name_or_id as a catalog alias.
if let Ok(db) = crate::commands::databases::try_resolve_database(api, name_or_id) {
return Ok(db.default_connection_id);
// An ambiguous alias (a fork shares its source's catalog) is reported as
// such, not as "no catalog": the caller's fix is to pass the id.
match crate::commands::databases::try_resolve_database(api, name_or_id) {
Ok(db) => Ok(db.default_connection_id),
Err(e) if e.contains("multiple databases") => Err(e),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Add a unit test for this ambiguous-alias branch (not blocking). The branch depends on a substring of the message at src/commands/databases.rs:796. A rewording of that message silently restores the misleading no catalog with id error. No test covers the branch or the new databases load --database path. The test at src/commands/databases.rs:3958 checks only try_resolve_database, not this caller.

Err(_) => Err(format!("no catalog with id '{name_or_id}'")),
}

Err(format!("no catalog with id '{name_or_id}'"))
}

/// Resolve a connection name or ID to a connection ID, exiting on failure.
Expand Down
20 changes: 14 additions & 6 deletions src/commands/databases.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ pub enum DatabasesCommands {
name: Option<String>,

/// SQL catalog alias used in queries: SELECT … FROM <catalog>.schema.table.
/// Must be [a-z_][a-z0-9_]*, globally unique.
/// Must be [a-z_][a-z0-9_]*. Not required to be unique — a fork answers
/// to its source's alias — so select databases by id when it matters.
#[arg(long)]
catalog: Option<String>,

Expand Down Expand Up @@ -201,9 +202,16 @@ pub enum DatabasesCommands {
/// Load a csv, json, or parquet file — or a saved query result — into an
/// instant database table
Load {
/// SQL catalog alias of the target database (e.g. `--catalog airbnb`)
#[arg(long)]
catalog: String,
/// SQL catalog alias of the target database (e.g. `--catalog airbnb`).
/// A fork answers to the same alias as its source, so once a database
/// has been forked the alias is ambiguous: pass `--database <id>` instead.
#[arg(long, required_unless_present = "database")]
catalog: Option<String>,

/// Database id (or name) to load into. Takes precedence over `--catalog`
/// and is the unambiguous form when forks share a catalog alias.
#[arg(long, conflicts_with = "catalog")]
database: Option<String>,

/// Schema to load into (default: public)
#[arg(long, default_value = "public")]
Expand Down Expand Up @@ -291,7 +299,7 @@ pub enum DatabasesCommands {
sql: Option<String>,

/// Instant database to run against (this command only; defaults to the current database)
#[arg(long, short = 'd')]
#[arg(long, short = 'd', global = true)]
database: Option<String>,

/// SQL dialect the query is written in — a non-`hotsql` dialect is
Expand All @@ -302,7 +310,7 @@ pub enum DatabasesCommands {

/// Output format. `id` prints only the stored result's id, without
/// downloading rows (for commands that read a result by id)
#[arg(long = "output", short = 'o', default_value = "table", value_parser = ["table", "json", "csv", "id"])]
#[arg(long = "output", short = 'o', default_value = "table", value_parser = ["table", "json", "csv", "id"], global = true)]
output: String,

#[command(subcommand)]
Expand Down
3 changes: 2 additions & 1 deletion src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,7 @@ fn main() {
}
Some(DatabasesCommands::Load {
catalog,
database,
schema,
table,
file,
Expand All @@ -325,7 +326,7 @@ fn main() {
retry_timeout,
}) => databases::tables_load(
&workspace_id,
Some(catalog.as_str()),
database.as_deref().or(catalog.as_deref()),
&table,
Some(schema.as_str()),
file.as_deref(),
Expand Down
Loading