Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions content/cn/docs/guides/hugegraph-docker-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,10 +196,13 @@ curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \
| `HG_STORE_GRPC_PORT` | 否 | `8500` | `grpc.port` |
| `HG_STORE_REST_PORT` | 否 | `8520` | `server.port` |
| `HG_STORE_DATA_PATH` | 否 | `/hugegraph-store/storage` | `app.data-path` |
| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | 否 | 无 | `rocksdb.total_memory_size`,单位字节 |
{.td-table--wide}

> **已弃用的别名**:`PD_ADDRESS` → `HG_STORE_PD_ADDRESS`、`GRPC_HOST` → `HG_STORE_GRPC_HOST`、`RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS`

RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时按容器内存限制自动计算;容器没有内存限制时,按 Docker 宿主机(Docker Desktop 虚拟机)的内存计算。上面的 Compose 文件都没有设置内存限制,因此每个 Store 都会按整台宿主机的内存计算预算;多个 Store 共用一台宿主机时,请为 Store 服务设置内存限制,或把该变量设为纯字节数。详见 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中的 `rocksdb.total_memory_size`。

### Server 变量

与 PD、Store 不同,Server 入口脚本没有必填变量:只有实际设置了的变量才会被写入配置文件。但分布式部署至少需要 `HG_SERVER_BACKEND` 和 `HG_SERVER_PD_PEERS`。
Expand Down
11 changes: 6 additions & 5 deletions content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ docker run -d \
| `HG_STORE_GRPC_PORT` | 否 | `8500` | `grpc.port` | gRPC 服务端口。 |
| `HG_STORE_REST_PORT` | 否 | `8520` | `server.port` | REST API 端口。 |
| `HG_STORE_DATA_PATH` | 否 | `/hugegraph-store/storage` | `app.data-path` | 数据存储路径。 |
| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | 否 | 无 | `rocksdb.total_memory_size` | RocksDB 内存预算,单位字节,只能是数字;其他值会使入口脚本以退出码 2 退出。未设置时使用 `application-pd.yml` 中的值或自动计算。 |
{.td-table--wide}

当前主线 entrypoint 将这些变量写入 `SPRING_APPLICATION_JSON`,覆盖镜像内 `conf/application.yml`;其余配置仍从该文件和 `application-pd.yml` 读取,然后以前台方式启动 Java。旧变量 `PD_ADDRESS`、`GRPC_HOST`、`RAFT_ADDRESS` 仍可用,但会输出弃用警告。
Expand All @@ -146,7 +147,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [

#### 4.1 application.yml

以下配置参考表核对的是 server 主线提交 `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4` 的发行目录值和 Java 默认注入值;1.7.0 发布包请使用其版本标签对应的配置文件,不要跨版本覆盖。
以下配置参考表核对的是 server 主线提交 `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4` 的发行目录值和 Java 默认注入值,其中 `rocksdb.total_memory_size`、`rocksdb.write_buffer_size` 和 `raft.disruptorBufferSize` 三行描述的是 apache/hugegraph#3355 的行为;1.7.0 发布包请使用其版本标签对应的配置文件,不要跨版本覆盖。

#### 4.2 application-pd.yml

Expand Down Expand Up @@ -178,7 +179,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [
| 配置项 | 当前主线目录值 | 代码默认值 | 说明 |
|--------|--------|------------|------|
| `raft.address` | `127.0.0.1:8510` | 必填 | 本节点的 Raft 服务地址,格式为 `host:port`,必须能被其他 Store 节点访问。这里不需要配置 peer 列表:分区 Raft 组的成员由 PD 下发。 |
| `raft.disruptorBufferSize` | `1024` | `0` | Raft 任务队列大小。设为 `0` 时按 `rocksdb.total_memory_size` 推导:将该内存量的 GB 数取最接近的 2 的幂,再乘以 32。 |
| `raft.disruptorBufferSize` | `1024` | `0` | Raft 任务队列大小。设为 `0` 时按 `rocksdb.total_memory_size` 推导:将该内存量的 GB 数(至少为 1)取最接近的 2 的幂,再乘以 32。 |
| `raft.max-log-file-size` | `600000000000` | `50000000000` | Raft 日志的最大字节数。 |
| `raft.snapshotInterval` | `1800` | `300` | 生成 Raft 快照的时间间隔,单位秒。 |
| `raft.snapshotLogIndexMargin` | 未设置 | `0` | 距上次快照的最小 applied index 差值,达到后才真正生成快照。设为 `0` 关闭该判断。 |
Expand Down Expand Up @@ -207,8 +208,8 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [

| 配置项 | 当前主线目录值 | 代码默认值 | 说明 |
|--------|--------|------------|------|
| `rocksdb.total_memory_size` | `32000000000` | `51539607552` | 本节点所有 RocksDB 实例共享的内存预算。缺失或为 `0` 时使用 JVM 最大堆内存。 |
| `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`。 |
| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 |
| `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 |
| `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 |
| `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 |
{.td-table--wide}
Expand Down Expand Up @@ -325,7 +326,7 @@ master 构建接受 `g1`、`G1`、`zgc` 和 `ZGC`。
- `-d true`(默认):以后台守护进程方式运行,脚本立即返回,并把 Java 进程号写入 `bin/pid`。
- `-d false`:以前台模式运行,脚本通过 `exec` 替换为 Java 进程,容器或进程管理器的进程即为 Java 本身。在 Docker 或进程管理器(systemd、supervisord)下运行时请使用此参数,以便在崩溃时自动检测并重启服务。

JVM 内存方面,如果没有自行设置 `JAVA_OPTIONS`:`-Xms512m`,`-Xmx` 取空闲内存的一半并限制在 512 MB 到 2048 MB 之间。脚本还会加上 `-XX:MetaspaceSize=256M`、OOM 时把堆转储写入 `logs/`,以及滚动的 GC 日志 `logs/gc.log`。生产节点通常需要大得多的堆,请显式指定,例如 `-j "-Xmx32g -Xms32g"`。
JVM 内存方面,如果没有自行设置 `JAVA_OPTIONS`:`-Xms512m`,`-Xmx` 取空闲内存的一半并限制在 512 MB 到 2048 MB 之间。脚本还会加上与该堆大小相同的 `-XX:MaxDirectMemorySize`、`-XX:MetaspaceSize=256M`、OOM 时把堆转储写入 `logs/`,以及滚动的 GC 日志 `logs/gc.log`。生产节点通常需要大得多的堆,请显式指定,并同时设置直接内存上限,例如 `-j "-Xmx32g -Xms32g -XX:MaxDirectMemorySize=2g"`。不设上限时,JVM 允许的直接内存与 32 GB 的堆相同,RocksDB 预算会把它计算在内,在内存不足约 66 GB 的主机上会降到 256 MiB 的下限。

当 `ulimit -n` 或 `ulimit -u` 低于 1024 时脚本会拒绝启动;在 x86_64 和 arm64 上,如果能成功下载并校验对应的动态库,脚本会预加载 jemalloc。

Expand Down
3 changes: 3 additions & 0 deletions content/en/docs/guides/hugegraph-docker-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,10 +196,13 @@ The following tables describe current master entrypoints and Compose files, not
| `HG_STORE_GRPC_PORT` | No | `8500` | `grpc.port` |
| `HG_STORE_REST_PORT` | No | `8520` | `server.port` |
| `HG_STORE_DATA_PATH` | No | `/hugegraph-store/storage` | `app.data-path` |
| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | No | None | `rocksdb.total_memory_size`, in bytes |
{.td-table--wide}

> **Deprecated aliases**: `PD_ADDRESS` → `HG_STORE_PD_ADDRESS`, `GRPC_HOST` → `HG_STORE_GRPC_HOST`, `RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS`.

RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or from the memory of the Docker host (the Docker Desktop VM) when the container has no limit. The Compose files above set no limit, so each Store sizes its budget from the whole host; set a memory limit on the Store services, or set this variable to a plain number of bytes, when several Stores share one host. See `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/).

### Server Variables

Unlike PD and Store, Server has no mandatory entrypoint variables: only set values are written to configuration. Distributed deployment nevertheless requires at least `HG_SERVER_BACKEND` and `HG_SERVER_PD_PEERS`.
Expand Down
11 changes: 6 additions & 5 deletions content/en/docs/quickstart/hugegraph/hugegraph-hstore.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ docker run -d \
| `HG_STORE_GRPC_PORT` | No | `8500` | `grpc.port` | gRPC service port. |
| `HG_STORE_REST_PORT` | No | `8520` | `server.port` | REST API port. |
| `HG_STORE_DATA_PATH` | No | `/hugegraph-store/storage` | `app.data-path` | Data path. |
| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | No | None | `rocksdb.total_memory_size` | RocksDB memory budget in bytes, digits only; other values stop the entrypoint with exit code 2. When unset, the value in `application-pd.yml` or auto-sizing applies. |
{.td-table--wide}

The master entrypoint writes these values into `SPRING_APPLICATION_JSON`, overriding image `conf/application.yml`. Other options still come from that file and `application-pd.yml`; Java runs in the foreground. Deprecated `PD_ADDRESS`, `GRPC_HOST`, and `RAFT_ADDRESS` remain supported with warnings.
Expand All @@ -147,7 +148,7 @@ Store reads two files under `conf/`. Release 1.7.0 sources: [application.yml](ht

#### 4.1 application.yml

The following tables were checked against distribution values and Java defaults at Server master commit `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4`. Use tagged configuration for release 1.7.0 without replacing it with another version's files.
The following tables were checked against distribution values and Java defaults at Server master commit `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4`, except the `rocksdb.total_memory_size`, `rocksdb.write_buffer_size` and `raft.disruptorBufferSize` rows, which describe apache/hugegraph#3355. Use tagged configuration for release 1.7.0 without replacing it with another version's files.

#### 4.2 application-pd.yml

Expand Down Expand Up @@ -180,7 +181,7 @@ This file contains only `rocksdb` and Actuator exposure settings.
| Key | Master distribution value | Code default | Meaning |
|-----|---------|--------------|---------|
| `raft.address` | `127.0.0.1:8510` | required | Raft service address of this node, `host:port`. Must be reachable from every other Store node. There is no peer list to configure: PD tells each node which peers belong to a partition's Raft group. |
| `raft.disruptorBufferSize` | `1024` | `0` | Raft task queue size. `0` derives it from `rocksdb.total_memory_size`, by rounding that size in GB to the nearest power of two and multiplying by 32. |
| `raft.disruptorBufferSize` | `1024` | `0` | Raft task queue size. `0` derives it from `rocksdb.total_memory_size`, by rounding that size in GB (at least 1) to the nearest power of two and multiplying by 32. |
| `raft.max-log-file-size` | `600000000000` | `50000000000` | Maximum byte size of Raft logs. |
| `raft.snapshotInterval` | `1800` | `300` | Seconds between Raft snapshots. |
| `raft.snapshotLogIndexMargin` | not set | `0` | Minimum applied-index distance since the last snapshot before a snapshot is actually written. `0` disables the distance check. |
Expand Down Expand Up @@ -209,8 +210,8 @@ This file contains only `rocksdb` and Actuator exposure settings.

| Key | Master distribution value | Code default | Meaning |
|-----|---------|--------------|---------|
| `rocksdb.total_memory_size` | `32000000000` | `51539607552` | Memory budget shared by all RocksDB instances on this node. When absent or `0`, the node uses the JVM max heap instead. |
| `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`. |
| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. |
Comment thread
byteayan marked this conversation as resolved.
| `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). |
| `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. |
| `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. |
{.td-table--wide}
Expand Down Expand Up @@ -330,7 +331,7 @@ Daemon mode:
- `-d true` (default): run as a background daemon. The script returns immediately and writes the Java pid to `bin/pid`.
- `-d false`: run in the foreground. The script `exec`s Java, so the container or supervisor process is Java itself. Use this under Docker or a process supervisor (systemd, supervisord) so crashes are detected and the service is restarted automatically.

JVM memory, unless you set `JAVA_OPTIONS` yourself: `-Xms512m`, and `-Xmx` set to half the free memory, clamped to the 512 MB to 2048 MB range. The script also adds `-XX:MetaspaceSize=256M`, a heap dump on out-of-memory into `logs/`, and a rolling GC log at `logs/gc.log`. Production nodes normally need a much larger heap, so pass one explicitly, for example `-j "-Xmx32g -Xms32g"`.
JVM memory, unless you set `JAVA_OPTIONS` yourself: `-Xms512m`, and `-Xmx` set to half the free memory, clamped to the 512 MB to 2048 MB range. The script also adds `-XX:MaxDirectMemorySize` equal to that heap, `-XX:MetaspaceSize=256M`, a heap dump on out-of-memory into `logs/`, and a rolling GC log at `logs/gc.log`. Production nodes normally need a much larger heap, so pass one explicitly together with a direct-memory cap, for example `-j "-Xmx32g -Xms32g -XX:MaxDirectMemorySize=2g"`. Without the cap the JVM allows as much direct memory as the 32 GB heap, the RocksDB budget counts it, and on a host with less than about 66 GB it falls to the 256 MiB floor.

The script refuses to start if `ulimit -n` or `ulimit -u` is below 1024, and it preloads jemalloc on x86_64 and arm64 when the shared object can be downloaded and verified.

Expand Down
Loading