Skip to content

perf(@angular/build): selectively compress large entries in SQLite cache store - #34021

Open
clydin wants to merge 1 commit into
angular:mainfrom
clydin:perf/sqlite-cache-compression
Open

perf(@angular/build): selectively compress large entries in SQLite cache store#34021
clydin wants to merge 1 commit into
angular:mainfrom
clydin:perf/sqlite-cache-compression

Conversation

@clydin

@clydin clydin commented Sep 3, 2026

Copy link
Copy Markdown
Member

Transformed JavaScript files from node_modules and other cached assets stored in the persistent SQLite cache store can range from tens to hundreds of kilobytes. Because SQLite rows exceeding the page size spill into overflow b-tree pages, storing large uncompressed entries increases disk I/O and cache file bloat.

Serialized entries and binary payloads of 32 KB or greater are now selectively compressed asynchronously using raw DEFLATE at level 1. Performing compression asynchronously offloads CPU-intensive compression work to worker threads, preventing blocking of the main JavaScript event loop during builds. Smaller payloads and incompressible entries remain uncompressed.

A dedicated format column stores the encoding format (CacheFormat), distinguishing between V8 serialized and raw binary (Uint8Array) data in both uncompressed and compressed states. This allows raw binary payloads (such as transformed JavaScript files) to be stored and retrieved with zero serialization overhead and zero memory copies.

@clydin clydin added the target: minor This PR is targeted for the next minor release label Sep 3, 2026
@angular-robot angular-robot Bot added area: performance Issues related to performance area: @angular/build labels Sep 3, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces payload compression (using DEFLATE) for the SQLite cache store for entries exceeding 32 KB, which helps reduce disk usage and SQLite overflow pages. It also adds comprehensive unit tests to verify compression, decompression, legacy support, and error handling. The feedback suggests replacing Buffer.allocUnsafe with new Uint8Array to avoid triggering security scanner false positives and to allow removing the unused Buffer import.

Comment thread packages/angular/build/src/tools/esbuild/sqlite-cache-store.ts Outdated
@clydin
clydin force-pushed the perf/sqlite-cache-compression branch from c9efb9b to 775112a Compare September 3, 2026 18:52
@clydin clydin added the action: review The PR is still awaiting reviews from at least one requested reviewer label Sep 3, 2026
@clydin
clydin force-pushed the perf/sqlite-cache-compression branch 3 times, most recently from 55cc123 to 0eebe6c Compare September 3, 2026 20:28
…che store

Transformed JavaScript files from node_modules and other cached assets stored in the persistent SQLite cache store can range from tens to hundreds of kilobytes. Because SQLite rows exceeding the page size spill into overflow b-tree pages, storing large uncompressed entries increases disk I/O and cache file bloat.

Serialized entries and binary payloads of 32 KB or greater are now selectively compressed asynchronously using raw DEFLATE at level 1. Performing compression asynchronously offloads CPU-intensive compression work to worker threads, preventing blocking of the main JavaScript event loop during builds. Smaller payloads and incompressible entries remain uncompressed.

A dedicated `format` column stores the encoding format (`CacheFormat`), distinguishing between V8 serialized and raw binary (`Uint8Array`) data in both uncompressed and compressed states. This allows raw binary payloads (such as transformed JavaScript files) to be stored and retrieved with zero serialization overhead and zero memory copies.

For backward compatibility with existing databases on disk, the database schema migration adds the `format` column with a default value of 0 (`V8`), ensuring pre-existing cache entries continue to deserialize seamlessly.
@clydin
clydin force-pushed the perf/sqlite-cache-compression branch from 0eebe6c to 6bca7f7 Compare September 3, 2026 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

action: review The PR is still awaiting reviews from at least one requested reviewer area: @angular/build area: performance Issues related to performance target: minor This PR is targeted for the next minor release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant