fix(tools): restore plain HDFS backups by imbajin · Pull Request #777 · apache/hugegraph-toolchain

imbajin · 2026-10-04T17:00:42Z

Purpose of the PR

HDFS restore previously attempted ZIP decoding for plain backup files. Read plain files directly and unzip only files with the .zip suffix, preserving uncompressed backups written by older Tools versions.

Original preparation and review: hugegraph#42.

Depends on #775. Merge the shared build preparation first. This PR targets master and currently includes prerequisite changes; after the prerequisite lands, rebase this same organization branch onto the updated upstream base and revalidate the module diff before merging.

Main Changes

Support plain and ZIP input streams and close invalid compressed streams before reporting failure.
Exercise backup/restore through Hadoop's local file URI, including UTF-8 data and compression suffix assertions.
Require explicit tools.test.disposable=true before destructive graph test setup; CI opts in only for its job-owned instance.
Complete independent cleanup actions without masking the primary failure.

Verifying these changes

Trivial rework / code cleanup without any test coverage.
Already covered by Hadoop stream tests and backup/restore regression tests.
Need tests and can be verified as follows:
- Prior functional validation used a dedicated released Server 1.7 instance.
- The latest stream and disposable-instance guard tests passed on native Java 17 without a backend.
- The current organization CI job passed both plain and ZIP CLI backup/restore round trips through Hadoop's local file URI on its disposable server.

Does this PR potentially affect the following parts?

Nope
Dependencies (build/test tooling only; runtime payload is unchanged by this preparation)
Modify configurations (CI/test configuration)
The public API (compatible behavior; signatures unchanged)
Other affects: tests and validation tooling

Documentation Status

Doc - TODO
Doc - Done
Doc - No Need

- manage compiler, surefire and coverage plugin versions - keep Java 8 output and use compatible Mockito 4 - resolve optional server repositories, refs and pins - verify checkout and package servers without install - share authenticated startup and filter AppleDouble files

- connect repository and fetch refs to all server installers - reuse the shared installer for Tools and Spark validation - preserve Loader source caching with verified checkouts - isolate fixture builds and retain separate HTTPS RPC ports

- preserve Mockito 2 interaction assertions with the equivalent API - align test-only Byte Buddy dependencies with Mockito 4 - retain the Spring Boot 2 and Java 11 shared baseline

- run shared server selection and installer tests in one CI job - document Maven 3.6.3 as the minimum source build version - report missing or ambiguous server archives before exiting

- trigger push checks for client installer changes - trigger pull request checks for the same source path - cover the installer exercised by the existing contracts

- encode legacy graph configuration as properties - keep task and target tests aligned with server contracts - inherit shared compiler and coverage plugin versions - preserve Java 8 APIs in compatibility tests

- read plain backup streams without ZIP decoding - close invalid compressed streams before reporting errors - cover ZIP and plain backup restore round trips - allow a dedicated test server URL

- pass the configured URL to auth backup commands - pass the same URL to auth restore commands - keep command and client requests on one test fixture

- disable persisted checkout credentials in the contract job - preserve read-only source checkout permissions - retain the existing local Git fixture contracts

- Give each HTTPS test its own client and clean it up only when opened. - Clear the client even when closing fails so teardown cannot reuse it. - Prepare the pinned official trust store once per class and verify downloads. - Compile and probe negative builder lifecycle on JDK 11 and 17 without server writes.

- report missing or ambiguous Loader server archives - stop before shared server startup - cover stale archives on a cached install

- encode absent legacy clone config as empty text - verify the actual HTTP request body and route - cover rejected null and nested config values

- require an explicit disposable test instance - retain failures while completing owned cleanup - exercise CLI backup paths through Hadoop local files - assert compression suffixes and opt-in rejection

codecov · 2026-10-04T17:18:14Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.98%. Comparing base (b066b80) to head (c3e2afb).
⚠️ Report is 246 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master     #777      +/-   ##
============================================
- Coverage     62.49%   52.98%   -9.51%     
- Complexity     1903     2161     +258     
============================================
  Files           262      336      +74     
  Lines          9541    12829    +3288     
  Branches        886     1208     +322     
============================================
+ Hits           5963     6798     +835     
- Misses         3190     5553    +2363     
- Partials        388      478      +90

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

- close the loopback request stream after reading - retain HTTP body and route assertions - verify the existing client unit suite

- bind cached archives to repository and commit - reject unverified caches without altering their contents - version the Loader CI cache key for provenance - cover cache identity and preservation with local fixtures

- Keep one active run per workflow and PR - Preserve push and manual workflow executions - Cover CI workflows on this PR target branch

- group labeler runs by PR and event - cancel older metadata updates for the same PR - preserve independent workflow runs

- remove unused Microsoft feeds from the temporary runner - retain Playwright browser dependency installation - preserve Ubuntu sources and package signature checks

- retain standard property escaping for legacy servers - encode spaces without Commons parser trimming - verify leading trailing and all-space string values - keep empty clone and scalar validation behavior

- encode non-ASCII config characters as Unicode escapes - retain whitespace and literal backslash semantics - verify ASCII bodies across three charset encodings

- merge upstream master after shared build preparation - retain upstream fixture and cache provenance fixes - preserve client compatibility changes and regressions

- preserve final build preparation files - connect upstream squash history - share synchronization across dependent branches

- absorb upstream client compatibility fixes - retain shared preparation history - align dependent branches with upstream master

- absorb finalized shared build preparation - include merged client compatibility fixes - preserve independent module changes

- place test guidance before license - retain disposable server requirements - keep the test paragraph on one line

- preserve the exact upstream master content - link the finalized client preparation history - keep organization module review diffs focused

- preserve the verified module file tree - reconcile organization and upstream client history - keep review diffs relative to the merged client

imbajin added 13 commits October 3, 2026 23:53

fix(ci): forward server source selection

ef29d9c

- connect repository and fetch refs to all server installers - reuse the shared installer for Tools and Spark validation - preserve Loader source caching with verified checkouts - isolate fixture builds and retain separate HTTPS RPC ports

fix(hubble): retain Mockito test compatibility

e9bb4d0

- preserve Mockito 2 interaction assertions with the equivalent API - align test-only Byte Buddy dependencies with Mockito 4 - retain the Spring Boot 2 and Java 11 shared baseline

fix(ci): run server fixture contract checks

e6210bc

- run shared server selection and installer tests in one CI job - document Maven 3.6.3 as the minimum source build version - report missing or ambiguous server archives before exiting

fix(ci): include client installer in path filters

6a57448

- trigger push checks for client installer changes - trigger pull request checks for the same source path - cover the installer exercised by the existing contracts

fix(client): prepare legacy graph compatibility

36afd26

- encode legacy graph configuration as properties - keep task and target tests aligned with server contracts - inherit shared compiler and coverage plugin versions - preserve Java 8 APIs in compatibility tests

fix(tools): preserve plain HDFS backups

3e5cfe1

- read plain backup streams without ZIP decoding - close invalid compressed streams before reporting errors - cover ZIP and plain backup restore round trips - allow a dedicated test server URL

fix(tools): honor auth test server URL

aa59d48

- pass the configured URL to auth backup commands - pass the same URL to auth restore commands - keep command and client requests on one test fixture

fix(ci): disable fixture checkout credentials

b2feece

- disable persisted checkout credentials in the contract job - preserve read-only source checkout permissions - retain the existing local Git fixture contracts

fix(ci): explain ambiguous server archives

9e74317

- report missing or ambiguous Loader server archives - stop before shared server startup - cover stale archives on a cached install

fix(client): send empty legacy clone config

12f6988

- encode absent legacy clone config as empty text - verify the actual HTTP request body and route - cover rejected null and nested config values

fix(tools): guard destructive backup tests

32bc165

- require an explicit disposable test instance - retain failures while completing owned cleanup - exercise CLI backup paths through Hadoop local files - assert compression suffixes and opt-in rejection

github-actions Bot added hubble hugegraph-hubble loader hugegraph-loader client hugegraph-client tools hugegraph-tools spark labels Oct 4, 2026

imbajin added 6 commits October 5, 2026 01:23

fix(client): close HTTP test request stream

8cd3766

- close the loopback request stream after reading - retain HTTP body and route assertions - verify the existing client unit suite

fix(build): verify server cache provenance

5851b41

- bind cached archives to repository and commit - reject unverified caches without altering their contents - version the Loader CI cache key for provenance - cover cache identity and preservation with local fixtures

chore(ci): cancel superseded PR runs

33708ba

- Keep one active run per workflow and PR - Preserve push and manual workflow executions - Cover CI workflows on this PR target branch

chore(ci): cancel superseded PR runs

61a0e53

- Keep one active run per workflow and PR - Preserve push and manual workflow executions - Cover CI workflows on this PR target branch

chore(ci): cancel superseded labeler runs

637b15f

- group labeler runs by PR and event - cancel older metadata updates for the same PR - preserve independent workflow runs

chore(ci): cancel superseded labeler runs

a966101

- group labeler runs by PR and event - cancel older metadata updates for the same PR - preserve independent workflow runs

imbajin mentioned this pull request Oct 4, 2026

[1.8.0] Release TODOs and cross-repository integration checklist apache/hugegraph#3242

Open

imbajin added 2 commits October 5, 2026 04:44

chore(ci): isolate Chromium dependency setup

4c737ff

- remove unused Microsoft feeds from the temporary runner - retain Playwright browser dependency installation - preserve Ubuntu sources and package signature checks

fix(client): preserve legacy config whitespace

0a85fca

- retain standard property escaping for legacy servers - encode spaces without Commons parser trimming - verify leading trailing and all-space string values - keep empty clone and scalar validation behavior

imbajin mentioned this pull request Oct 5, 2026

feat: complete Java 17 candidate SDK cutover #783

Merged

fix(client): escape legacy config unicode

c5a5d6b

- encode non-ASCII config characters as Unicode escapes - retain whitespace and literal backslash semantics - verify ASCII bodies across three charset encodings

imbajin added 7 commits October 5, 2026 22:38

chore(client): sync merged build preparation

813f155

- merge upstream master after shared build preparation - retain upstream fixture and cache provenance fixes - preserve client compatibility changes and regressions

chore: reconcile merged build preparation

4bf8332

- preserve final build preparation files - connect upstream squash history - share synchronization across dependent branches

chore: sync merged client preparation

b0542fe

- absorb upstream client compatibility fixes - retain shared preparation history - align dependent branches with upstream master

chore: sync merged upstream preparation

0aa883d

- absorb finalized shared build preparation - include merged client compatibility fixes - preserve independent module changes

docs(tools): clarify test guidance layout

4d33bc5

- place test guidance before license - retain disposable server requirements - keep the test paragraph on one line

chore: reconcile merged client history

d13dfeb

- preserve the exact upstream master content - link the finalized client preparation history - keep organization module review diffs focused

chore: align merged client ancestry

c3e2afb

- preserve the verified module file tree - reconcile organization and upstream client history - keep review diffs relative to the merged client

MrJs133 approved these changes Oct 5, 2026

View reviewed changes

github-actions Bot removed hubble hugegraph-hubble loader hugegraph-loader client hugegraph-client spark labels Oct 5, 2026

imbajin merged commit 669d8e8 into apache:master Oct 5, 2026
14 of 16 checks passed

imbajin deleted the java17-tools-prep branch October 8, 2026 16:19

Navigation Menu

Repository navigation

fix(tools): restore plain HDFS backups - #777

fix(tools): restore plain HDFS backups#777
imbajin merged 29 commits into
apache:masterfrom
hugegraph:java17-tools-prep

imbajin commented Oct 4, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Oct 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

imbajin commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose of the PR

Main Changes

Verifying these changes

Does this PR potentially affect the following parts?

Documentation Status

Uh oh!

codecov Bot commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

imbajin commented Oct 4, 2026 •

edited

Loading

codecov Bot commented Oct 4, 2026 •

edited

Loading