Repository navigation
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3330 +/- ##
=============================================
- Coverage 54.29% 35.00% -19.29%
- Complexity 2616 5503 +2887
=============================================
Files 770 691 -79
Lines 67952 60025 -7927
Branches 9213 8221 -992
=============================================
- Hits 36892 21012 -15880
- Misses 27482 36443 +8961
+ Partials 3578 2570 -1008 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: no. Summary: The preflight is wired into allocation, cold open, Raft start and snapshot activation as described, and I found no correctness problem in the version resolver, the snapshot validation or the delayed committedIndex publication. One small note: PartitionEngine.init reopens read-only a database that the factory already validated and holds open. Evidence: full diff 68fb045..2773193 read with HgStoreEngine, PartitionManager, StoreMetadata, SnapshotHandler, InnerKeyCreator, RocksDBFactory and RocksDBSession at this head, plus GraphIdManager at tag 1.7.0 for the 0xFFFE sentinel. No Maven tests were run for this review; pd_store / store, pd_store / hstore and cluster / cluster-test passed on this head and only codecov/project is red.
- inspect reserved-prefix rows before allocating graph IDs - preflight existing databases before partition Raft replay - validate staged snapshots before activation and index publication - reject unknown path access and cover native recovery boundaries
- validate retained DB versions before native pruning - reuse existing factory locks and cached sessions - cover the real metadata bootstrap entry point - preserve version data and document metadata limits
- reuse effective options for read-only checks - keep native causes and close failed-open handles - cover level-seven snapshots and corruption errors
2773193 to
70d03eb
Compare
Inspect the cached primary view before partition startup. Detect legacy rows written with WAL disabled before flush. Close the borrowed session without releasing its native owner. Preserve read-only path checks when the cache is empty.
Remove stdout diagnostics from bootstrap regressions. Retain existing failure messages and data-preservation assertions. Keep production preflight behavior unchanged.
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: yes. Score: 6/10 (10 - 2 for one Important = 8, capped at 6 because it is blocking). Summary: The preflight is wired into allocation, cold open, Raft start and snapshot activation, and the version resolver, snapshot validator and delayed committedIndex publication look correct, but a refusal at cold open aborts the whole Store bootstrap instead of the affected partition. Evidence: static trace of BusinessHandlerImpl.getSession -> PartitionMetaStoreWrapper.scan -> PartitionManager.loadPartitions -> HgStoreEngine.stateChanged at d9ff79f, plus the PR's GraphIdUpgradePreflightTest, which asserts loadPartition throws. CI has no required checks on master; dependency-review fails on spring-webmvc 5.3.27 in hg-pd-dist, which this PR does not touch.
Already raised: @bitflicker64 noted the extra read-only open in PartitionEngine.init; the current head reuses the cached session.
| dbSession = | ||
| factory.createGraphDB(partitionManager.getDbDataPath(partId, dbName), dbName, | ||
| version); | ||
| version, path -> GraphIdManager.checkLegacyGraphIdData(path, partId)); |
There was a problem hiding this comment.
Important: a refusal from this validator stops every partition on the Store, not only the affected one.
PartitionManager.loadPartitions (line 271) reaches this path through wrapper.scan with no catch, so the first affected partition aborts the loop and later partitions are never loaded. HgStoreEngine.stateChanged only catches InvalidShardException, so the exception escapes before restoreLocalPartitionEngine runs and no Raft group starts, while the Store stays registered Up. Raft replicas hold the same rows, so with 3 Stores and 3 replicas one affected partition takes every partition in the cluster offline. The PR's own tests assert that loadPartition throws.
Catch the refusal per partition in loadPartitions (log it and skip that partition), or fail the Store explicitly through requestExit, and document which one is intended. Add a two-partition test.
Purpose of the PR
Affected legacy graph rows must be rejected before missing-ID allocation, cold partition opening, Raft startup or staged snapshot activation. Earlier checks could run after metadata bootstrap had opened or pruned a partition DB, and default column-family options could reject a valid configured DB.
Source PR: hugegraph#301.
Main Changes
The persisted ID format stays unchanged. Store/Raft location metadata may already change before preflight. Refusal preserves the inspected DB and incoming snapshot evidence; it does not infer row ownership or perform automatic migration.
Verifying these changes
These component controls do not replace full SDK/Raft/cluster validation. New-head CI is pending; earlier results are not inherited. Both repositories' current-head CI, the complete GraphID compatibility design, deployment upgrade, old-data visibility and recovery acceptance remain open.
Does this PR potentially affect the following parts?
Documentation Status
Doc - TODODoc - DoneDoc - No NeedRepository guidance: docs/store-graphid-upgrade.md and the RocksDB README.
Paired website documentation: apache/hugegraph-doc#523 (organization source: hugegraph/hugegraph-doc#47). Repository documentation ships with the source change; historical website PR tracking is outside this task.
Conceptual overview
Legacy GraphID preflight at the matching source revision: refuse affected rows before protected startup/allocation/snapshot paths and preserve evidence. Not automatic migration, physical deletion, or complete deployment upgrade/rollback acceptance.