Skip to content

docs(store): document the auto-sized RocksDB memory budget - #535

Open
byteayan wants to merge 8 commits into
apache:masterfrom
byteayan:doc/store-rocksdb-memory-budget
Open

byteayan wants to merge 8 commits into
apache:masterfrom
byteayan:doc/store-rocksdb-memory-budget

Conversation

@byteayan

Copy link
Copy Markdown
Contributor

Purpose of the PR

apache/hugegraph#3355 changes the Store's shipped rocksdb.total_memory_size to 0. With 0 or no value, the Store sizes the RocksDB budget as half of what is left after the JVM max heap, MaxDirectMemorySize and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. An unset MaxDirectMemorySize counts as the heap size, at most 1 GiB. Inside a container it starts from the cgroup memory limit, including a limit on a parent cgroup the Store can see. The Docker entrypoint also gains HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE. The site still documented the old 32000000000 value and said that 0 meant the JVM max heap.

Changes, in English and Chinese:

  • quickstart/hugegraph/hugegraph-hstore.md:
    • Rewrote the rocksdb.total_memory_size row: the new distribution value 0, the auto-sizing rule, the off-heap caches plus the separate 1 GiB Raft log cache, the 1 MiB minimum for an explicit value, and plain bytes only.
    • Added an upgrade note to that row: an application-pd.yml kept from an earlier release still says 32000000000 and has to be set to 0 to auto-size.
    • Added "at least 1" to the GB rounding in the raft.disruptorBufferSize row, which the code fix now guarantees.
    • Added the new variable to the master Docker env table.
  • guides/hugegraph-docker-cluster.md: added the variable to the Store table, with one line pointing to the configuration reference.

bash dist/validate-links.sh passes and git diff --check is clean. I did not build the site locally because Hugo is not installed on this machine.

The Store fix for apache/hugegraph#3254 ships rocksdb.total_memory_size
as 0 and sizes the budget from the memory left after the heap, direct
memory and a 1.5 GiB reserve, starting from the cgroup limit in a
container. The HStore page still listed the old 32000000000 value and
said 0 meant the JVM max heap.

Update the rocksdb.total_memory_size row with the new rule, note the
minimum of 1 GB in the raft.disruptorBufferSize derivation, and add
HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE to the Store env var tables in the
HStore quick start and the Docker cluster guide, in English and Chinese.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: The new rocksdb.total_memory_size text, the raft.disruptorBufferSize rounding note and the HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE rows match apache/hugegraph#3355 at 535c2d01b. One minor gap: the paired change also puts a 1 MiB floor on the derived rocksdb.write_buffer_size, and that row still gives the old formula. Evidence: read the full diff at a516891 (4 files, en and cn) and checked it against RocksdbMemoryBudget, AppConfig, RaftRocksdbOptions, RocksDBOptions, both application-pd.yml files and the Store docker-entrypoint.sh at the #3355 head. Note that #3355 is still open, so the 0 distribution value is correct only once it merges, as the PR description says.

Comment thread content/en/docs/quickstart/hugegraph/hugegraph-hstore.md Outdated
apache/hugegraph#3355 fills an absent or 0 rocksdb.write_buffer_size
with total_memory_size / 1000, but never below the 1 MiB that
RocksDBOptions accepts. Any budget under about 1 GB, including the
256 MiB auto-sizing floor, therefore gets 1048576. Say so in the EN and
CN rows.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The rocksdb.total_memory_size, write_buffer_size, raft.disruptorBufferSize and HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE text matches apache/hugegraph#3355 at 535c2d01b, and EN and CN agree. Two gaps remain: the Docker cluster guide says an unset variable sizes the budget from the container memory limit, but the Compose files it documents set no limit, so each Store sizes from the whole Docker host or VM; and the HStore page still says its tables were checked at master commit 2f827d6e, which predates the values this PR now lists. Evidence: read the full diff at 120c614 (4 files, EN and CN) and checked it against RocksdbMemoryBudget, AppConfig, RaftRocksdbOptions, RocksDBOptions, both application-pd.yml files, the Store docker-entrypoint.sh and start-hugegraph-store.sh at the #3355 head, the docker/*.yml Compose files (no mem_limit or deploy.resources), and application-pd.yml/AppConfig at 2f827d6e; links resolve to existing pages, git diff --check is clean and the site build is green. The earlier write_buffer_size floor comment is fixed in 120c614.

Comment thread content/en/docs/guides/hugegraph-docker-cluster.md Outdated
Comment thread content/en/docs/quickstart/hugegraph/hugegraph-hstore.md Outdated
The Docker guide said an unset HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE sizes
the budget from the container memory limit, but the Compose files it
documents set no limit, so each Store sizes from the whole Docker host.
Say so, and tell readers to set a limit or pin the variable when several
Stores share a host. The HStore row now says the cgroup limit applies
when one is set, otherwise the host's memory.

The HStore page also said its tables were checked at a master commit
from before apache/hugegraph#3355; note that the three RocksDB and
disruptor rows describe that PR until its merge commit replaces it.
apache/hugegraph#3355 now budgets the JVM's real direct-memory
allowance: an unset -XX:MaxDirectMemorySize counts as the heap, and
bin/start-hugegraph-store.sh sets the flag to the heap it picks unless
-j passes one. Drop the "at most 1 GiB" from the EN and CN rows.
apache/hugegraph#3355 now leaves -XX:MaxDirectMemorySize out when -j
sets -Xmx or the cap itself, so a heap-only -j gets the JVM default.
Update the EN and CN rows.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: The new rocksdb.total_memory_size, write_buffer_size, raft.disruptorBufferSize and HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE text matches apache/hugegraph#3355 and the EN and CN pages agree; one minor gap remains because #3355 changed the launcher's direct-memory rule after this head. Evidence: read the full diff at b9e2b34 (4 files, EN and CN) and checked it against RocksdbMemoryBudget, AppConfig, RaftRocksdbOptions, the Store RocksDBOptions, both application-pd.yml files, the Store docker-entrypoint.sh and start-hugegraph-store.sh at the #3355 head 9e6dc506, plus the docker/*.yml Compose files (no memory limits); the site build checks on this head pass.

Comment thread content/en/docs/quickstart/hugegraph/hugegraph-hstore.md Outdated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants