Skip to content

[Feature] Add an official Helm chart for distributed HStore deployment #3131

Description

@bitflicker64

Feature Description

Apache HugeGraph ships container images and Compose deployments, but it does
not currently provide an official Helm installation path for the distributed
PD + Store + Server topology on Kubernetes.

The main challenge is encoding the distributed startup and recovery contract,
not only rendering Kubernetes resources:

  • PD must reach Raft quorum before Store registers.
  • Server must not run concurrent init-store operations against the same
    distributed backend.
  • All Server replicas must use PD-backed graph metadata so they share one
    graph catalog.
  • Store addresses can change after rescheduling, so DNS and gRPC clients must
    recover rather than retain stale addresses or channels.
  • StatefulSet and PVC identities must remain stable when replica counts change.

Proposal

Add an application chart under helm/hugegraph that provides:

  • PD and Store StatefulSets with persistent storage and headless Services.
  • A Server Deployment with a client Service, optional HPA, PDB, and Ingress.
  • Always-on Server PD metadata mode (usePD=true plus chart-derived PD peers
    and Kubernetes service registration) for the distributed HStore topology.
  • Authentication on by default, with a kept chart-managed admin password
    Secret and a shared JWT auth.token_secret across Server replicas (BYO
    Secrets remain supported).
  • PD, Store, and Server scheduling, resource, security, and ServiceAccount
    controls.
  • PodDisruptionBudgets for the stateful components.
  • A helm test connection check.
  • values.schema.json validation so invalid combinations fail during render.
  • Default, single-node, and production-oriented values presets.
  • An optional Hubble UI component (off by default): single replica, pd (PD
    discovery with the cluster operations view) or direct wiring mode,
    optional Ingress and H2 persistence, wired to the deployed cluster. Current
    Hubble images authenticate their login against the cluster, so the chart
    requires server.auth when Hubble is enabled unless explicitly overridden.
  • server.advertiseUrl plus an exposable PD client Service so a standalone
    Hubble outside the cluster can use PD discovery; empty advertise URL keeps
    the in-cluster Server Service URL.
  • Helm lint, render, schema, compatibility, packaging, and kubeconform CI.
  • Installation, configuration, upgrade, troubleshooting, and limitations
    documentation.

The chart configures the existing HugeGraph images. It does not vendor or
duplicate server-side implementation.

The implementation is #3132.

Prerequisites

The first chart release is version 0.1.0. While its PR is a draft, the chart
tracks the latest HugeGraph images. Before stable publication, the image tags
and appVersion will be pinned to the next HugeGraph release. There is no
1.8.x release tag yet.

All six image-side prerequisites have merged:

PR Required behavior Merged as
#3128 One gRPC stub binding per channel b026a90a
#3129 Store readiness retries across all configured PD peers 8b2932c7
#3105 Bounded port preflight without requiring lsof 1716c774
#3119 Dedicated init_store.enabled gate for distributed Server startup (issue #3118) 39f4f856
#3126 Finite JVM DNS cache TTL, plus startup DNS-policy validation and JDK 24+ security-check handling de62d97f
#3130 Channel and stub refresh after a Store address change 431f6e6b

#3126 and #3130 form one recovery path: channel refresh cannot resolve a new
Store address while the JVM retains the old DNS result indefinitely.

#3119 was narrowed to the init_store.enabled option and its gate. The Docker
entrypoint and auth-bootstrap work that previously shared its branch is tracked
separately in #3133. The chart does not depend on #3133 landing first, but the
entrypoint contract it describes is the one this chart relies on for
PASSWORD and auth.admin_pa handling.

Validation

The implemented chart was tested on a four-node Kubernetes cluster using a
3 PD + 3 Store + 3 Server topology, against a composition built from these
exact prerequisite heads:

PR Head under test Status now
#3105 ee8f5559 merged as 1716c774
#3119 3e505a81 merged as 39f4f856
#3126 35e1a240 merged as de62d97f
#3128 0118e158 merged unchanged at this head
#3129 575a1122 merged at 61407213, one commit further
#3130 26218cb3 merged as 431f6e6b

The completed validation covers:

  • Static chart validation: PASS=239 FAIL=0 SKIP=0.
  • Edge and hardening validation: PASS=37 FAIL=0.
  • Fresh install with 9 Ready Pods and zero restarts.
  • Same-package Helm upgrade with cluster and dataset checks passing.
  • Server replacement: PASS=91 FAIL=0.
  • Store replacement: PASS=90 FAIL=0, with data preserved.
  • Deterministic PD REST failover: PASS=7 FAIL=0, followed by successful
    restoration.
  • Legacy --reuse-values rendering and client-side dry-run compatibility.

The auth bootstrap fix (the wrapper now writes auth.admin_pa from the auth
Secret) and the Hubble component were validated separately against a newer
composition; details are in #3132. Later chart commits enabled auth by default,
auto-generate kept Secrets, add server.advertiseUrl, and merge #3130 onto the
chart branch.

On 2026-08-17, #3132 head 84c56fc7 was validated on Kubernetes using the
published Docker Hub images hugegraph/{pd,store,server,hubble}:latest, with
authentication enabled and Hubble in pd mode:

  • A 3 PD + 3 Store + 3 Server + 1 Hubble install reached Ready with zero
    restarts and passed helm test.
  • Protected APIs rejected unauthenticated and wrong-password access; the chart
    admin Secret and shared JWT login succeeded on every Server replica.
  • REST and Hubble create/read/update/delete paths against HStore succeeded;
    Hubble discovered graph spaces through PD.

The full multi-node fault and upgrade lifecycle matrix rerun is still pending
before #3132 leaves draft.

Only #3128 merged at exactly the original matrix head under test, so that
matrix remains the multi-node evidence baseline rather than a statement about
today's master alone.

Resolved Server startup contracts

The chart now keeps init_store.enabled=false while always writing
usePD=true, chart-derived pd.peers, server.urls_to_pd, and
server.deploy_in_k8s=true before Server startup. This makes the graph catalog
PD-backed for every chart-managed Server replica, including the default install
with Hubble disabled.

Authentication is on by default. The wrapper writes auth.admin_pa from the
configured Secret, resolving the earlier bootstrap gap. auth.admin_pa
applies only when the admin is first created and does not rotate an existing
cluster password.

PD-backed graph metadata does not make local RocksDB data shared and does not
close the cross-replica CreateGraph readiness window tracked in #3137.

Scope and follow-ups

This contribution establishes the official Helm installation path. TLS,
backup and restore, an Operator, multi-cluster orchestration, automatic leader
transfer, ConfigMap-based component configuration, and a complete monitoring
stack remain separate follow-up work and are documented as limitations.

Entrypoint property handling for mounted and upgraded configs is tracked in
#3133 and is outside this chart's scope.

#3132 should remain a draft until the multi-node lifecycle matrix is rerun
against current master and image tags are pinned when the next HugeGraph
release exists. The #3130 merge is already on the chart branch (84c56fc7),
and Kubernetes auth + Hubble validation against that head is recorded in
#3132.

Related upstream issues

  • #3135 PD: periodic partition leader balancing, motivated by fault testing chart deployments.

  • #3136 Store: shard recovery metrics (sync progress, raft lag), motivated by fault testing chart deployments.

  • #3137 PD-orchestrated graph creation: new graphs are not consistently available across Server replicas; the chart documents the remaining cross-replica window as a limitation. Phase 1 (minimal patch) merged: #3138 (2026-08-12). Issue stays open for cluster-wide readiness (#3139) and PD-owned creation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ci-cdBuild or deployfeatureNew feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions