Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CN/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,8 @@
*** xref:master/ecosystem_components/pgbackrest.adoc[pgBackRest]
*** xref:master/ecosystem_components/set_user.adoc[set_user]
*** xref:master/ecosystem_components/pg_jieba.adoc[pg_jieba]
*** xref:master/ecosystem_components/pg_track_settings.adoc[pg_track_settings]
*** xref:master/ecosystem_components/pg_track_settings.adoc[pg_track_settings]
*** xref:master/ecosystem_components/citus.adoc[Citus]
* 监控运维
** xref:master/getting-started/daily_monitoring.adoc[日常监控]
** xref:master/getting-started/daily_maintenance.adoc[日常维护]
Expand Down
173 changes: 173 additions & 0 deletions CN/modules/ROOT/pages/master/ecosystem_components/citus.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
:sectnums:
:sectnumlevels: 5

= Citus

== 概述

Citus 是 PostgreSQL 的开源分布式扩展(由 Microsoft 开源维护),将 PostgreSQL 转换为分布式数据库。它通过表分片(sharding)、分布式查询计划与并行执行,在多节点集群上实现水平扩展,同时保留标准 SQL 接口。

Citus 的核心能力:

* *分布式表*:按分布列将大表哈希分片到多个节点,透明支持 INSERT/UPDATE/DELETE/SELECT
* *引用表*:小维度表全节点复制,与分布式表 JOIN 零网络开销
* *分布式查询*:基于 Citus Adaptive 执行器的并行查询,支持分片裁剪(shard pruning)
* *在线扩容*:rebalance_table_shards 在集群间重新均衡分片

Citus 14.x 支持 PostgreSQL 16、17、18。IvorySQL 5.x 基于 PostgreSQL 18,兼容 Citus 14.2。

[TIP]
本文档源码安装环境为 Debian 12(bookworm)容器,IvorySQL 5.6 镜像,Citus 14.2.0。

== 安装

=== 前置条件

Citus 以 PostgreSQL 扩展形式编译,需要 IvorySQL 的开发头文件(pg_config)与编译工具链。

[literal]
----
# Debian / Ubuntu
sudo apt install gcc make bison flex pkg-config \
libcurl4-openssl-dev liblz4-dev libzstd-dev \
zlib1g-dev libssl-dev autoconf automake libtool
----

=== 源码安装

[literal]
----
git clone --depth 1 --branch v14.2.0 https://github.com/citusdata/citus.git
cd citus

# 确保 pg_config 位于 PATH(IvorySQL 默认安装路径示例)
export PATH=/usr/ivory-5/bin:$PATH

./configure
make -j$(nproc)
sudo make install
----

=== 验证安装

[literal]
----
ls /usr/ivory-5/lib/postgresql/citus.so
ls /usr/ivory-5/share/postgresql/extension/citus.control
----

== 配置

Citus 必须通过 `shared_preload_libraries` 预加载。

IvorySQL 默认已预加载自身组件(`gb18030_2022`、`liboracle_parser`、`ivorysql_ora`),配置时应**保留原有库并追加 citus**:

[literal]
----
# postgresql.conf
shared_preload_libraries = 'citus, gb18030_2022, liboracle_parser, ivorysql_ora'
----

修改后重启实例:

[literal]
----
pg_ctl restart -D $PGDATA
----

[TIP]
不建议使用 `ALTER SYSTEM SET shared_preload_libraries` 配置多库列表:实测该方式可能将整个列表写入为单一库名,导致启动失败(`could not access file`)。建议直接编辑 postgresql.conf。

== 使用

=== 创建扩展

[literal]
----
-- psql -U ivorysql -p 5432 -d postgres
CREATE EXTENSION citus;
SELECT citus_version();

-- citus_version
-- ----------------------------------------
-- Citus 14.2.0 on x86_64-pc-linux-gnu ...
----

=== 创建分布式表

[literal]
----
CREATE TABLE orders (
order_id bigint,
user_id bigint,
amount numeric(10,2),
created date DEFAULT current_date
);

-- 按 user_id 哈希分片
SELECT create_distributed_table('orders', 'user_id');

-- 查看分片情况(默认 32 个分片)
SELECT table_name, shard_count FROM citus_tables;

-- table_name | shard_count
-- ------------+-------------
-- orders | 32
----

=== 数据操作与分片裁剪

分布式表的 DML 与普通表语法完全一致,Citus 根据分布列自动路由:

[literal]
----
INSERT INTO orders VALUES (1, 100, 99.90, current_date);
INSERT INTO orders VALUES (2, 200, 59.50, current_date);
INSERT INTO orders VALUES (3, 100, 129.00, current_date);

-- 带分布列条件的查询会裁剪到单个分片
EXPLAIN SELECT * FROM orders WHERE user_id = 100;

-- QUERY PLAN
-- --------------------------------------------------------
-- Custom Scan (Citus Adaptive)
-- Task Count: 1
-- -> Task
-- Node: host=localhost port=5432 dbname=postgres
-- -> Seq Scan on orders_102009 orders
-- Filter: (user_id = 100)

-- 跨分片聚合自动并行汇总
SELECT count(*), sum(amount) FROM orders;
----

=== 分片再均衡

新增 worker 节点后,使用 rebalance 在集群间迁移分片:

[literal]
----
-- 添加节点(在所有 worker 上安装并预加载 citus 后)
SELECT citus_add_node('worker1', 5432);

-- 在线再均衡分片
SELECT rebalance_table_shards();

-- 查看集群节点
SELECT nodename, nodeport FROM pg_dist_node;
----

== Oracle 兼容模式说明

实测(Citus 14.2.0 + IvorySQL 5.6,双模式监听 5432/1521),Oracle 兼容模式下 Citus 存在以下限制:

. *扩展安装受限*:在 Oracle 模式连接(1521 端口)中执行 `CREATE EXTENSION citus` 会因 Oracle 解析器拒绝扩展脚本中的 PostgreSQL 专有语法(如 `CREATE SEQUENCE ... NO CYCLE`)而失败
. *Oracle 类型表无法分布式化*:对包含 `NUMBER`、`VARCHAR2` 等 Oracle 类型的表执行 `create_distributed_table` 时,DDL 将被下发到 worker 端重放,而 worker 端 Citus 仅能安装在 PostgreSQL 模式实例中,无法识别 Oracle 类型,导致失败

因此,当前版本建议在 **PostgreSQL 兼容模式(5432 端口)** 下使用 Citus 分布式能力;Oracle 模式下的表可继续使用 IvorySQL 的 Oracle 语法特性,但不建议纳入 Citus 分布式集群。

== 限制

* Citus 仅支持 PostgreSQL 兼容模式;Oracle 兼容模式下无法安装扩展
* Citus 版本须与后端 PostgreSQL 大版本匹配(Citus 14.x 对应 PostgreSQL 16/17/18)
* 所有 worker 节点均需安装相同版本的 Citus 与 IvorySQL,并正确配置 `shared_preload_libraries`
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ IvorySQL 作为一款兼容 Oracle 且基于 PostgreSQL 的高级开源数据库
| 30 | xref:master/ecosystem_components/set_user.adoc[set_user] | REL4_2_0 | PostgreSQL 安全审计扩展,可控角色切换,支持白名单、强制审计、拦截高危操作 | 可控角色切换、权限管理、审计日志
| 31 | xref:master/ecosystem_components/pg_jieba.adoc[pg_jieba] | master branch | 中文分词扩展,能够将连续的中文文本切分成有意义的词语序列,为中文全文搜索提供基础支持 | 全文搜索、自然语言处理
| 32 | xref:master/ecosystem_components/pg_track_settings.adoc[pg_track_settings] | 2.1.2 | PostgreSQL 配置跟踪扩展,记录配置变更历史 | 配置审计、变更追踪
| 33 | xref:master/ecosystem_components/citus.adoc[citus] | 14.2 | PostgreSQL 分布式扩展,通过表分片与分布式查询实现水平扩展 | 海量数据分布式存储、多节点并行查询、实时分析
|====

这些插件均经过 IvorySQL 团队的测试和适配,确保在 IvorySQL 环境下稳定运行。用户可以根据业务需求选择合适的插件,进一步提升数据库系统的能力和灵活性。
Expand Down
1 change: 1 addition & 0 deletions EN/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@
*** xref:master/ecosystem_components/set_user.adoc[set_user]
*** xref:master/ecosystem_components/pg_jieba.adoc[pg_jieba]
*** xref:master/ecosystem_components/pg_track_settings.adoc[pg_track_settings]
*** xref:master/ecosystem_components/citus.adoc[Citus]
* Monitor and O&M
** xref:master/getting-started/daily_monitoring.adoc[Monitoring]
** xref:master/getting-started/daily_maintenance.adoc[Maintenance]
Expand Down
173 changes: 173 additions & 0 deletions EN/modules/ROOT/pages/master/ecosystem_components/citus.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
:sectnums:
:sectnumlevels: 5

= Citus

== Overview

Citus is an open-source distributed extension for PostgreSQL (open-sourced and maintained by Microsoft) that turns PostgreSQL into a distributed database. Through table sharding, distributed query planning and parallel execution, it delivers horizontal scalability across a multi-node cluster while keeping the standard SQL interface.

Core capabilities of Citus:

* *Distributed tables*: large tables are hash-sharded across nodes by a distribution column, transparently supporting INSERT/UPDATE/DELETE/SELECT
* *Reference tables*: small dimension tables are replicated to every node, enabling zero-network-cost joins with distributed tables
* *Distributed queries*: parallel execution based on the Citus Adaptive executor, with shard pruning
* *Online scale-out*: rebalance_table_shards redistributes shards across the cluster

Citus 14.x supports PostgreSQL 16, 17 and 18. IvorySQL 5.x is based on PostgreSQL 18 and is compatible with Citus 14.2.

[TIP]
The installation in this document was verified on a Debian 12 (bookworm) container with the IvorySQL 5.6 image and Citus 14.2.0 built from source.

== Installation

=== Prerequisites

Citus is built as a PostgreSQL extension and requires the IvorySQL development headers (pg_config) and a build toolchain.

[literal]
----
# Debian / Ubuntu
sudo apt install gcc make bison flex pkg-config \
libcurl4-openssl-dev liblz4-dev libzstd-dev \
zlib1g-dev libssl-dev autoconf automake libtool
----

=== Building from source

[literal]
----
git clone --depth 1 --branch v14.2.0 https://github.com/citusdata/citus.git
cd citus

# Make sure pg_config is on PATH (example: default IvorySQL install prefix)
export PATH=/usr/ivory-5/bin:$PATH

./configure
make -j$(nproc)
sudo make install
----

=== Verify the installation

[literal]
----
ls /usr/ivory-5/lib/postgresql/citus.so
ls /usr/ivory-5/share/postgresql/extension/citus.control
----

== Configuration

Citus must be preloaded via `shared_preload_libraries`.

IvorySQL preloads its own components by default (`gb18030_2022`, `liboracle_parser`, `ivorysql_ora`). Keep the existing libraries and append citus:

[literal]
----
# postgresql.conf
shared_preload_libraries = 'citus, gb18030_2022, liboracle_parser, ivorysql_ora'
----

Restart the instance afterwards:

[literal]
----
pg_ctl restart -D $PGDATA
----

[TIP]
Avoid configuring the library list via `ALTER SYSTEM SET shared_preload_libraries`: in testing, this can write the whole list as a single library name and cause startup failure (`could not access file`). Edit postgresql.conf directly instead.

== Usage

=== Create the extension

[literal]
----
-- psql -U ivorysql -p 5432 -d postgres
CREATE EXTENSION citus;
SELECT citus_version();

-- citus_version
-- ----------------------------------------
-- Citus 14.2.0 on x86_64-pc-linux-gnu ...
----

=== Create distributed tables

[literal]
----
CREATE TABLE orders (
order_id bigint,
user_id bigint,
amount numeric(10,2),
created date DEFAULT current_date
);

-- Hash-shard by user_id
SELECT create_distributed_table('orders', 'user_id');

-- Inspect shards (32 shards by default)
SELECT table_name, shard_count FROM citus_tables;

-- table_name | shard_count
-- ------------+-------------
-- orders | 32
----

== Data operations and shard pruning

DML on distributed tables uses exactly the same syntax as regular tables; Citus routes statements automatically based on the distribution column:

[literal]
----
INSERT INTO orders VALUES (1, 100, 99.90, current_date);
INSERT INTO orders VALUES (2, 200, 59.50, current_date);
INSERT INTO orders VALUES (3, 100, 129.00, current_date);

-- Queries filtered on the distribution column are pruned to a single shard
EXPLAIN SELECT * FROM orders WHERE user_id = 100;

-- QUERY PLAN
-- --------------------------------------------------------
-- Custom Scan (Citus Adaptive)
-- Task Count: 1
-- -> Task
-- Node: host=localhost port=5432 dbname=postgres
-- -> Seq Scan on orders_102009 orders
-- Filter: (user_id = 100)

-- Cross-shard aggregation is gathered in parallel automatically
SELECT count(*), sum(amount) FROM orders;
----

=== Shard rebalancing

After adding worker nodes, rebalance shards across the cluster:

[literal]
----
-- Add a node (install and preload citus on every worker first)
SELECT citus_add_node('worker1', 5432);

-- Rebalance shards online
SELECT rebalance_table_shards();

-- List cluster nodes
SELECT nodename, nodeport FROM pg_dist_node;
----

== Oracle compatibility mode notes

Verified with Citus 14.2.0 + IvorySQL 5.6 (dual-mode listening on 5432/1521), Citus has the following limitations in Oracle compatibility mode:

. *Extension installation fails*: running `CREATE EXTENSION citus` on an Oracle-mode connection (port 1521) fails because the Oracle parser rejects PostgreSQL-specific syntax in the extension script (such as `CREATE SEQUENCE ... NO CYCLE`)
. *Oracle-typed tables cannot be distributed*: when running `create_distributed_table` on a table with Oracle types such as `NUMBER` or `VARCHAR2`, the DDL is replayed on the worker side, while Citus on the worker can only be installed in a PostgreSQL-mode instance and cannot recognize Oracle types, causing failure

Therefore, in the current version we recommend using Citus distributed capabilities in **PostgreSQL compatibility mode (port 5432)**. Tables in Oracle mode can keep using IvorySQL's Oracle syntax features, but should not be added to a Citus distributed cluster.

== Limitations

* Citus only supports the PostgreSQL compatibility mode; the extension cannot be installed in Oracle compatibility mode
* The Citus version must match the backend PostgreSQL major version (Citus 14.x corresponds to PostgreSQL 16/17/18)
* Every worker node must run the same version of Citus and IvorySQL with `shared_preload_libraries` properly configured
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ IvorySQL, as an advanced open-source database compatible with Oracle and based o
|*30*| xref:master/ecosystem_components/set_user.adoc[set_user] | REL4_2_0 | PostgreSQL security auditing extension with controlled role switching, supporting allowlists, enforced auditing, and blocking of high-risk operations | Controlled role switching, privilege management, audit logging
|*31*| xref:master/ecosystem_components/pg_jieba.adoc[pg_jieba] | master branch | Chinese word segmentation extension, which segments continuous Chinese text into meaningful word sequences, providing fundamental support for full-text search | Full-text search and natural language processing (NLP)
|*32*| xref:master/ecosystem_components/pg_track_settings.adoc[pg_track_settings] | 2.1.2 | PostgreSQL configuration tracking extension that records configuration change history | Configuration auditing, change tracking
|*33*| xref:master/ecosystem_components/citus.adoc[citus] | 14.2 | Distributed PostgreSQL extension delivering horizontal scale-out through table sharding and distributed queries | Large-scale distributed storage, parallel multi-node queries, real-time analytics
|====

These plugins have all been tested and adapted by the IvorySQL team to ensure stable operation in the IvorySQL environment. Users can select appropriate plugins based on business needs to further enhance the capabilities and flexibility of the database system.
Expand Down
Loading