fix(consensus/XDPoS,core,eth/downloader): rebuild missing V2 gap snapshots

[gzliudan]

Proposed changes

A node can lose its persisted V2 gap snapshot when the process exits between
writeHeadBlock and StoreSnapshot in writeBlockWithState: the head markers are
already on disk, the snapshot is not, and nothing recreates it. getSnapshot then
fails with a leveldb "not found" error for the whole affected epoch and the node
drops out of consensus participation. Making that write ordering atomic prevents
new holes but cannot repair a database that already has one.

Add a guarded self-healing path in engine_v2.getSnapshot: when no snapshot is
stored for the gap block, rebuild it from the committed state trie at
gapHeader.Root, then persist and cache it.

The derivation itself moves into engine_v2.BuildSnapshotFromState, which
eth/downloader.generateSnapshot now also uses. The downloader already carried
its own copy of this derivation, and the unstable xdc_sort ordering it shares
with core.BlockChain.UpdateM1 must not drift: a different equal-stake order
yields a different masternode set. The shared builder also refuses to produce an
empty snapshot, which the downloader previously persisted and which would then
permanently mask the missing masternode list. A gap pivot state sync that finds
no candidates therefore now fails loudly instead of storing that empty snapshot.

The rebuild is only attempted when it is both safe and useful:

• skip when a snapshot is already stored: only a genuinely missing key is
  healed, so a decode or I/O error never overwrites a stored masternode set;
• skip at or before V2 SwitchBlock, where the initial snapshot still comes from
  Initial() rather than the state trie;
• reject numbers that are not real gap blocks, since they can originate from
  unauthenticated vote/timeout messages;
• skip gap blocks more than two epochs behind the chain head, so a peer cannot
  force trie reads and database writes for arbitrary historic gap numbers;
• skip when the chain reader does not implement the new GapStateReader
  interface, which core.BlockChain satisfies via a compile-time assertion;
• skip while the gap block itself is not imported yet, where the missing state
  is transient and retrying later succeeds.

Past that point the gap block is imported, so a rebuild that failed on its state
cannot start working later: the trie is pruned and can never come back, so the
gap block is recorded in an LRU and never retried, instead of re-reading the
trie on every vote verification. Concurrent callers for the same gap block are
collapsed with singleflight. Pruned state is logged at warn rather than error,
and everything degrades to debug while syncing, where a missing gap snapshot is
expected and not actionable.

The rebuild needs the gap block's state root to still be readable, which the
targeted case implies: a node only keeps the gap block as its head across a
restart when that root was committed. Otherwise loadLastState finds no head
state, repair() rewinds to an ancestor that has one, and re-importing the gap
block runs UpdateM1 and writes the snapshot again. What is left are roots that
are gone for good, such as offline pruning, gap blocks below a fast sync pivot,
or a side chain whose trie was collected. No later attempt can recover those,
which is what the permanent give-up above encodes.

A rebuilt snapshot is derived from the gap block state, while UpdateM1 takes
candidates from the head state and stakes from the validator contract at
"latest". Those agree on the canonical import path, where the head is the gap
block itself, but not necessarily on the reorg path, so a rebuilt masternode set
can differ from what a peer persisted through a reorg. That is accepted here:
the alternative is no snapshot at all.

Add coverage for the state-derived builder and for every getSnapshot guard, and
seed the downloader test genesis with the minimum masternode voting contract
storage the builder reads.

Types of changes

What types of changes does your code introduce to XDC network?
Put an ✅ in the boxes that apply

• build: Changes that affect the build system or external dependencies
• ci: Changes to CI configuration files and scripts
• chore: Changes that don't change source code or tests
• docs: Documentation only changes
• feat: A new feature
• fix: A bug fix
• perf: A code change that improves performance
• refactor: A code change that neither fixes a bug nor adds a feature
• revert: Revert something
• style: Changes that do not affect the meaning of the code
• test: Adding missing tests or correcting existing tests

Impacted Components

Which parts of the codebase does this PR touch?
Put an ✅ in the boxes that apply

• Consensus
• Account
• Network
• Geth
• Smart Contract
• External components
• Not sure (Please specify below)

Checklist

Put an ✅ in the boxes once you have confirmed below actions (or provide reasons on not doing so) that

• This PR has sufficient test coverage (unit/integration test) OR I have provided reason in the PR description for not having test coverage
• Tested on a private network from the genesis block and monitored the chain operating correctly for multiple epochs.
• Provide an end-to-end test plan in the PR description on how to manually test it on the devnet/testnet.
• Tested the backwards compatibility.
• Tested with XDC nodes running this version co-exist with those running the previous version.
• Relevant documentation has been updated as part of this PR
• N/A

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d121d8a2-5c4f-47f8-840a-27b57d8bcc15

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[gzliudan]

replaced by #2508