Skip to content

zlib decompression (zstd + gzip) intermittently returns corrupted output on Node 24/26 on Windows; Buffer.alloc().fill() affected too; Node 22 clean #66465

Description

@MasonZhangXM

Version

Affected: v24.16.0, v24.21.0, v26.10.0
Clean: v22.23.2 (same machine, same minute)

Platform

Windows 11 x64 (build 26200), Intel Core i9-14900HX, 32 GB RAM, NVMe.
Host is otherwise healthy: PagesPersec=0, 14.9 GB available, commit 38.6 / 63.2 GB.

Subsystem

zlib (zstd and gzip), and plain Buffer allocation.

What steps will reproduce the bug?

Self-contained script — no external files, no third-party packages. It compresses a ~360 KB payload in-process, then decompresses it N times and hashes every output. The three codecs are exercised interleaved (one iteration of each per loop) so that any time-clustered effect hits all three equally.

import zlib from 'node:zlib';
import crypto from 'node:crypto';

const N = 5000, LINES = 4000;
const payload = Buffer.from(
  Array.from({ length: LINES }, (_, i) =>
    `line-${i} session-record payload with some repeated text to make it compressible 0123456789\n`
  ).join('')
);

const packs = {
  zstd:   zlib.zstdCompressSync(payload),
  gzip:   zlib.gzipSync(payload),
  brotli: zlib.brotliCompressSync(payload),
};
const dec = {
  zstd:   (b) => zlib.zstdDecompressSync(b),
  gzip:   (b) => zlib.gunzipSync(b),
  brotli: (b) => zlib.brotliDecompressSync(b),
};
const expect = crypto.createHash('sha256').update(payload).digest('hex');

const counts = { zstd: new Map(), gzip: new Map(), brotli: new Map() };
const throws = { zstd: 0, gzip: 0, brotli: 0 };

for (let i = 0; i < N; i++) {
  for (const k of Object.keys(packs)) {
    let out;
    try { out = dec[k](packs[k]); } catch (e) { throws[k]++; continue; }
    const h = crypto.createHash('sha256').update(out).digest('hex');
    counts[k].set(h, (counts[k].get(h) || 0) + 1);
  }
}

for (const k of Object.keys(packs)) {
  const wrong = N - (counts[k].get(expect) || 0);
  console.log(`${k}: variants=${counts[k].size} mismatch=${wrong}/${N} throws=${throws[k]}`);
}

How often does it reproduce?

Every run, but the rate varies wildly between runs (0.04% – 41% observed). It comes in bursts, so small samples give random-looking answers. Use ≥5,000 iterations.

Expected behavior

Every decompressed output equals the original payload.

What actually happens

v24.16.0, N=5000, payload 366,890 B:

codec variants mismatch throws
zstd 207 209 / 5000 (4.18%) 0
gzip 1 — 189 × Z_DATA_ERROR: incorrect data check
brotli 1 0 0
  • zstd output length is always correct (366,890) — only the content differs, and no exception is thrown. Many zstd frames carry no content checksum, so this is silent.
  • gzip catches it via its own CRC32 and throws incorrect data check at a similar rate.
  • brotli is unaffected in every run I have done.

v22.23.2, same script, same machine, same minute: 0 / 5000 for all three codecs.

The same corruption happens with no zlib involved at all

const b = Buffer.alloc(366890);
b.fill(0x41);
crypto.createHash('sha256').update(b).digest('hex');

20,000 iterations, counting distinct hashes:

runtime distinct hashes
Node v22.23.2 1
Node v24.16.0 3 (19,998 / 1 / 1)
Node v24.21.0 2 (19,999 / 1)
Node v26.10.0 2 (19,999 / 1)
CPython 3.12, bytes([0x41]) * 366890 1

Why this is probably not my measurement error

gzip's own CRC32 independently reports incorrect data check at roughly the same rate as my hash comparison. That checksum is computed by zlib over the decompressed bytes, not by my script. Two independent detectors agree.

What I have already ruled out

  • Not AV / driver injection. Enumerated loaded modules of every node.exe on the box: 35 modules each, all Microsoft or node — no vendor DLL. No third-party AV kernel driver is loaded (only a network filter driver and a WinPcap-style driver, both unrelated to memory).
  • Not memory pressure. PagesPersec=0, 14.9 GB available.
  • Not corrupt input. The packed buffers are produced in-process, never written to disk, never mutated.
  • Not zstd-specific. gzip is hit too and reports it via CRC32. brotli is not.
  • Not a single build. Two independent majors (24.x and 26.x) are affected; 22.x is clean on the same hardware at the same time.

Why this matters in practice

The failure is silent for zstd when the frame has no content checksum. I only found it because an application that restores compressed session archives was intermittently failing with ZSTD_error_checksum_wrong (those frames did have a checksum) — which implies the silent variant was happening far more often than the loud one.

What I can do

I can run any additional instrumentation, ETW traces, or candidate patches on this host, and I can re-run with different iteration counts on request.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    zlibIssues and PRs related to the zlib module and its compression dependencies.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions