Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/build_docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@
name: "Publish Docs"

on:
pull_request:
paths:
- 'apidoc/**'
- 'docs/**'
- 'include/**'
push:
branches:
- main
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/release_candidate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ on:
pull_request:
paths:
- ".github/workflows/release_candidate.yaml"
- ".github/workflows/build_docs.yaml"
- ".github/.rat-excludes"
- "CMakeLists.txt"
- "LICENSE"
Expand Down
12 changes: 9 additions & 3 deletions docs/source/_static/versions.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
[
{
Comment thread
lucasfang marked this conversation as resolved.
Resolved
"name": "0.4.0",
"version": "0.4.0",
"url": "https://paimon.apache.org/docs/cpp/"
"name": "main",
"version": "main",
"url": "https://paimon.apache.org/docs/cpp/main/"
},
{
"name": "0.3.0 (stable)",
"version": "",
"url": "https://paimon.apache.org/docs/cpp/",
"preferred": true
}
]
4 changes: 0 additions & 4 deletions docs/source/api/global_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,6 @@ Interface
:members:
:undoc-members:

.. doxygenclass:: paimon::FullTextScan::Plan
:members:
:undoc-members:

.. doxygenclass:: paimon::FullTextRead
:members:
:undoc-members:
Expand Down
29 changes: 27 additions & 2 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,13 +116,38 @@

version = "0.4.0"

# Documentation channel, set by the paimon-website build pipeline:
# "main" (default) for the main-branch docs published under main/, "stable"
# for the latest release published at docs/cpp/, or "X.Y" for an archived
# release series published under docs/cpp/X.Y/.
docs_channel = os.environ.get("PAIMON_DOCS_VERSION", "main")
switcher_version = "" if docs_channel == "stable" else docs_channel

# The release string feeds the version warning banner: main-branch builds
# carry a ".dev" suffix so the banner calls them an unstable development
# version, as the Apache Arrow documentation does. Release builds (stable
# and archived series) keep the plain version. The stable channel disables
# the banner because its empty switcher identifier is not a release number.
release = f"{version}.dev" if docs_channel == "main" else version

html_theme_options = {
"show_toc_level": 2,
"show_nav_level": 2,
"use_edit_page_button": True,
"header_links_before_dropdown": 4,
"navbar_align": "left",
"navbar_end": ["theme-switcher", "navbar-icon-links"],
"navbar_end": ["version-switcher", "theme-switcher", "navbar-icon-links"],
"switcher": {
# The main build is republished daily by the paimon-website pipeline,
# so its copy of the version list stays the freshest one on the site.
# PAIMON_DOCS_VERSIONS_URL allows local previews to point at their
# own copy instead of the live site.
"json_url": os.environ.get(
"PAIMON_DOCS_VERSIONS_URL",
"https://paimon.apache.org/docs/cpp/main/_static/versions.json",
),
"version_match": switcher_version,
},
"icon_links": [
{
"name": "GitHub",
Expand All @@ -133,7 +158,7 @@
"logo": {
"text": "Apache Paimon C++",
},
"show_version_warning_banner": True,
"show_version_warning_banner": docs_channel != "stable",
}

html_context = {
Expand Down
2 changes: 1 addition & 1 deletion docs/source/user_guide/catalog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -418,7 +418,7 @@ the commit endpoint of ``tbl$branch_dev``, not the one of ``tbl``. A body sent
to the bare table's URL publishes the branch's snapshot on the main branch.

Authenticating with credentials of your own
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Credentials that a catalog does not issue — from a token service of your own, or
from a store Paimon has no client for — reach a file system through a
``CredentialProvider`` that you consult yourself. Implement it together with a
Expand Down
2 changes: 1 addition & 1 deletion docs/source/user_guide/file_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ File Index
==========

Full-text predicates and root TopK
---------------------------------
----------------------------------

``PredicateBuilder::FullTextMatch`` creates a ``LeafPredicate`` for a full-text
condition without TopK truncation. It holds a shared ``FullTextMatch`` request,
Expand Down
11 changes: 11 additions & 0 deletions scripts/releasing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,17 @@ scripts/releasing/bump_version.py 0.2.0 0.3.0
scripts/releasing/bump_version.py --check 0.3.0
```

The target version becomes the stable documentation label (for example,
`0.3.0 (stable)`). If its major.minor series differs from the previous stable
series, the previous release is moved to an archived entry. Archived labels
also show the full version, while their identifiers and URLs retain the
major.minor series. Patch releases update the stable label without adding
an archive. The documentation version may track the latest stable release
while the CMake version advances; release preparation updates both to the
target version. Use this command for release
preparation; it also promotes documentation and is not a development-only
version bump.

### Signing key setup and security

Complete signing-key setup well before creating the first release candidate:
Expand Down
110 changes: 89 additions & 21 deletions scripts/releasing/bump_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,33 @@

VERSION_PATTERN = re.compile(r"^\d+\.\d+\.\d+$")

DOCS_URL = "https://paimon.apache.org/docs/cpp/"
MAIN_ENTRY_VERSION = "main"


def series(version: str) -> str:
"""The major.minor documentation series a version belongs to."""
return ".".join(version.split(".")[:2])


def stable_entry(version: str) -> Dict[str, object]:
"""Dropdown entry for the latest release published at the site root."""
return {
"name": f"{version} (stable)",
"version": "",
"url": DOCS_URL,
"preferred": True,
}


def series_entry(version: str) -> Dict[str, object]:
"""Dropdown entry for an archived release series under <series>/."""
return {
"name": version,
"version": series(version),
"url": f"{DOCS_URL}{series(version)}/",
}


class VersionError(RuntimeError):
"""Version metadata is missing or inconsistent."""
Expand Down Expand Up @@ -55,6 +82,55 @@ def load_versions(path: Path) -> List[Dict[str, object]]:
return value


def check_version_list(versions: List[Dict[str, object]]) -> None:
"""Validate the dev/stable/archived layout of versions.json."""
if len(versions) < 2:
raise VersionError(
"versions.json must contain a dev entry followed by a stable entry"
)
dev = versions[0]
if (
dev.get("version") != MAIN_ENTRY_VERSION
# Older Git refs retain the previous display label.
or dev.get("name") not in ("main", "main (development)")
or dev.get("url") != f"{DOCS_URL}main/"
):
raise VersionError(
"the first versions.json entry must be the main-branch docs: "
f"name 'main', version {MAIN_ENTRY_VERSION!r}, "
f"url {DOCS_URL}main/"
)
stable = versions[1]
name = stable.get("name")
match = (
re.fullmatch(r"(\d+\.\d+\.\d+) \(stable\)", name)
if isinstance(name, str)
else None
)
if (
match is None
or stable.get("version") != ""
or stable.get("url") != DOCS_URL
or stable.get("preferred") is not True
):
raise VersionError(
"the second versions.json entry must be the stable docs: name "
"'X.Y.Z (stable)', version '', url the site root, preferred true"
)
for item in versions[2:]:
item_name = item.get("name")
if (
not isinstance(item_name, str)
or VERSION_PATTERN.fullmatch(item_name) is None
or item.get("version") != series(item_name)
or item.get("url") != f"{DOCS_URL}{series(item_name)}/"
):
raise VersionError(
"archived versions.json entries must use name X.Y.Z, "
f"version X.Y and url {DOCS_URL}X.Y/, found {item!r}"
)


def checked_files(root: Path, expected: str) -> Dict[Path, str]:
cmake_path = root / "CMakeLists.txt"
docs_path = root / "docs/source/conf.py"
Expand All @@ -70,22 +146,14 @@ def checked_files(root: Path, expected: str) -> Dict[Path, str]:
docs_matches = re.findall(
r'^version = "(\d+\.\d+\.\d+)"$', docs, re.MULTILINE
)
json_matches = [
item
for item in versions
if item.get("name") == expected and item.get("version") == expected
]

if cmake_matches != [expected]:
raise VersionError(f"CMake version is {cmake_matches}, expected [{expected!r}]")
if docs_matches != [expected]:
raise VersionError(
f"documentation version is {docs_matches}, expected [{expected!r}]"
)
if len(json_matches) != 1:
check_version_list(versions)
stable_version = versions[1]["name"].split(" ")[0]
if docs_matches not in ([expected], [stable_version]):
raise VersionError(
"versions.json must contain exactly one entry whose name and "
f"version are both {expected}"
f"documentation version is {docs_matches}, expected project version "
f"{expected!r} or stable documentation version {stable_version!r}"
)
return {
cmake_path: cmake,
Expand All @@ -108,19 +176,19 @@ def updated_files(root: Path, current: str, new: str) -> Dict[Path, str]:
)
files[docs_path] = replace_once(
files[docs_path],
rf'^version = "{re.escape(current)}"$',
r'^version = "\d+\.\d+\.\d+"$',
f'version = "{new}"',
f"documentation version {current}",
)

versions = load_versions(versions_path)
matches = [item for item in versions if item.get("version") == current]
if len(matches) != 1:
raise VersionError(
f"versions.json contains {len(matches)} entries for {current}"
)
matches[0]["name"] = new
matches[0]["version"] = new
previous_version = versions[1]["name"].split(" ")[0]
if series(previous_version) != series(new):
# Release preparation promotes the target series, even when the
# project version already belongs to that series. Archive the
# previous stable series only when it changes.
versions.insert(2, series_entry(previous_version))
versions[1] = stable_entry(new)
files[versions_path] = json.dumps(versions, indent=4) + "\n"
return files

Expand Down
29 changes: 12 additions & 17 deletions scripts/releasing/create_source_release.sh
Original file line number Diff line number Diff line change
Expand Up @@ -146,20 +146,18 @@ for required_file in CMakeLists.txt LICENSE NOTICE; do
fail "${required_file} is missing from ${GIT_REF}"
done

CMAKE_VERSION=$(
git -C "${SOURCE_ROOT}" show "${GIT_REF}:CMakeLists.txt" |
awk '$1 == "VERSION" && $2 ~ /^[0-9]+\.[0-9]+\.[0-9]+$/ { print $2; exit }'
)
[[ "${CMAKE_VERSION}" == "${RELEASE_VERSION}" ]] ||
fail "CMake version ${CMAKE_VERSION:-<missing>} does not match ${RELEASE_VERSION}"

DOCS_VERSION=$(
git -C "${SOURCE_ROOT}" show "${GIT_REF}:docs/source/conf.py" |
sed -n 's/^version = "\([^"]*\)"$/\1/p' |
head -n 1
)
[[ "${DOCS_VERSION}" == "${RELEASE_VERSION}" ]] ||
fail "documentation version ${DOCS_VERSION:-<missing>} does not match ${RELEASE_VERSION}"
command -v python3 >/dev/null 2>&1 || fail "python3 is required"
TEMP_DIR=$(mktemp -d)
trap 'rm -rf "${TEMP_DIR}"' EXIT

# Check the immutable ref with the same metadata rules as archive verification.
mkdir -p "${TEMP_DIR}/metadata/docs/source/_static"
for metadata_file in CMakeLists.txt docs/source/conf.py docs/source/_static/versions.json; do
git -C "${SOURCE_ROOT}" show "${GIT_REF}:${metadata_file}" \
>"${TEMP_DIR}/metadata/${metadata_file}"
done
python3 "${SCRIPT_DIR}/bump_version.py" \
--root "${TEMP_DIR}/metadata" --check "${RELEASE_VERSION}"

GZIP_BIN=$(find_gnu_gzip)

Expand All @@ -174,9 +172,6 @@ for suffix in "" ".sha512" ".asc"; do
fail "refusing to overwrite ${OUTPUT_DIR}/${ARTIFACT_NAME}${suffix}"
done

TEMP_DIR=$(mktemp -d)
trap 'rm -rf "${TEMP_DIR}"' EXIT

echo "Creating ${ARTIFACT_NAME} from ${GIT_REF}..."
git -C "${SOURCE_ROOT}" -c tar.umask=0022 archive \
--format=tar \
Expand Down
Loading
Loading