docs: document live view TTL, retention and Parquet conversion - #549
Open
puzpuzpuz wants to merge 2 commits into
Open
docs: document live view TTL, retention and Parquet conversion#549puzpuzpuz wants to merge 2 commits into
puzpuzpuz wants to merge 2 commits into
Conversation
Add an ALTER LIVE VIEW reference page for the statements that manage a live view's disk tier: SET TTL, DROP PARTITION, CONVERT PARTITION between native and Parquet, and the pre-existing RESUME WAL and SUSPEND WAL. The page states what a live view's derived nature costs an operator: the change is applied asynchronously by the refresh worker, rows removed from disk can still be served from the in-memory tier until it is rebuilt, a manual DROP PARTITION is undone by any recovery that recomputes output over the dropped period, and an out-of-order base commit rewrites every Parquet partition its correction range covers. It also lists the clauses a live view rejects, including FORCE DROP PARTITION, the permission each clause is authorized with, and how a replica applies a relayed statement to its own copy of the view. Document the CREATE LIVE VIEW TTL clause alongside FLUSH EVERY, IN MEMORY and PARTITION BY, with the partition-size granularity rule and the behaviour of a retention window during the initial seed. Give the live views concept page a retention and storage format section and drop its "no TTL on the view" limitation, which this work makes false. Cover live views on the TTL concept page next to materialized views. Correct the in-place Parquet conversion example, which named the option bloom_filter_fpp and left the value unquoted. ALTER TABLE ... CONVERT PARTITION accepts fpp with a quoted value; bloom_filter_fpp is the COPY ... TO spelling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PbJoRNP8MurxdLzmSuRYzY
The relay fences a received command on the view's own base progress so that a partition removal or conversion affects the same rows on both nodes. SET TTL carries no fence, because the rule applies to whatever the view holds when it commits, so calling every relayed statement fenced overstated it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PbJoRNP8MurxdLzmSuRYzY
|
🚀 Build success! Latest successful preview: https://preview-549--questdb-documentation.netlify.app/docs/ Commit SHA: a9d82c7
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documents the durable-tier DDL that questdb/questdb#7612 adds to live views.
New page:
ALTER LIVE VIEW. The statement had no reference page at all, so it covers all five clauses:SET TTL,DROP PARTITION,CONVERT PARTITIONbetween native and Parquet, and the pre-existingRESUME WALandSUSPEND WAL. Beyond the grammar it states what a live view being a derived object costs an operator:FLUSH EVERYis a cadence and not a deadline;DROP PARTITIONnorSET TTLmakes data unreadable at a known point in time;DROP PARTITIONis undone by any recovery that recomputes output over the dropped period, which is whySET TTLis the retention that survives recovery;WITH (bloom_filter_columns = ...)is not stored, and a live view has no per-columnPARQUET()metadata to fall back on, so bloom filters last only until the partition is next rewritten.It also lists the clauses a live view rejects (including
FORCE DROP PARTITION), the permission each clause is authorized with, and how a replica applies a relayed statement to its own copy of the view.CREATE LIVE VIEW: theTTLclause, its partition-size granularity rule, and what a retention window does during the initial seed.Concept pages: a retention and storage format section on live views, replacing the now-false "no TTL on the view" limitation, plus the replication behaviour of these statements; and a live views subsection on the TTL page next to materialized views.
Unrelated correction: the in-place Parquet conversion example named the option
bloom_filter_fppand left the value unquoted, so it fails as written.ALTER TABLE ... CONVERT PARTITIONparsesfppand needs a quoted value;bloom_filter_fppis theCOPY ... TOspelling.Open question
The
SET TTLsection documents Enterprise as it behaves today:EntSqlCompilerImpl.alterTableOrMatViewSetTtllets only a materialized view or a system table through, so a non-zeroALTER LIVE VIEW ... SET TTLis rejected withTTL is not supported on Enterprise tables; use a storage policy instead, while theCREATE LIVE VIEW ... TTLclause is accepted. Whether live views follow the materialized-view rule (TTL allowed) or the table rule (storage policy) is still open on the feature branch. That caution block needs a revisit before this merges.Dependencies
🤖 Generated with Claude Code
https://claude.ai/code/session_01PbJoRNP8MurxdLzmSuRYzY