perf(@angular/build): selectively compress large entries in SQLite cache store - #34021
Open
clydin wants to merge 1 commit into
Open
perf(@angular/build): selectively compress large entries in SQLite cache store#34021clydin wants to merge 1 commit into
clydin wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces payload compression (using DEFLATE) for the SQLite cache store for entries exceeding 32 KB, which helps reduce disk usage and SQLite overflow pages. It also adds comprehensive unit tests to verify compression, decompression, legacy support, and error handling. The feedback suggests replacing Buffer.allocUnsafe with new Uint8Array to avoid triggering security scanner false positives and to allow removing the unused Buffer import.
clydin
force-pushed
the
perf/sqlite-cache-compression
branch
from
September 3, 2026 18:52
c9efb9b to
775112a
Compare
clydin
force-pushed
the
perf/sqlite-cache-compression
branch
3 times, most recently
from
September 3, 2026 20:28
55cc123 to
0eebe6c
Compare
…che store Transformed JavaScript files from node_modules and other cached assets stored in the persistent SQLite cache store can range from tens to hundreds of kilobytes. Because SQLite rows exceeding the page size spill into overflow b-tree pages, storing large uncompressed entries increases disk I/O and cache file bloat. Serialized entries and binary payloads of 32 KB or greater are now selectively compressed asynchronously using raw DEFLATE at level 1. Performing compression asynchronously offloads CPU-intensive compression work to worker threads, preventing blocking of the main JavaScript event loop during builds. Smaller payloads and incompressible entries remain uncompressed. A dedicated `format` column stores the encoding format (`CacheFormat`), distinguishing between V8 serialized and raw binary (`Uint8Array`) data in both uncompressed and compressed states. This allows raw binary payloads (such as transformed JavaScript files) to be stored and retrieved with zero serialization overhead and zero memory copies. For backward compatibility with existing databases on disk, the database schema migration adds the `format` column with a default value of 0 (`V8`), ensuring pre-existing cache entries continue to deserialize seamlessly.
clydin
force-pushed
the
perf/sqlite-cache-compression
branch
from
September 3, 2026 20:49
0eebe6c to
6bca7f7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Transformed JavaScript files from node_modules and other cached assets stored in the persistent SQLite cache store can range from tens to hundreds of kilobytes. Because SQLite rows exceeding the page size spill into overflow b-tree pages, storing large uncompressed entries increases disk I/O and cache file bloat.
Serialized entries and binary payloads of 32 KB or greater are now selectively compressed asynchronously using raw DEFLATE at level 1. Performing compression asynchronously offloads CPU-intensive compression work to worker threads, preventing blocking of the main JavaScript event loop during builds. Smaller payloads and incompressible entries remain uncompressed.
A dedicated
formatcolumn stores the encoding format (CacheFormat), distinguishing between V8 serialized and raw binary (Uint8Array) data in both uncompressed and compressed states. This allows raw binary payloads (such as transformed JavaScript files) to be stored and retrieved with zero serialization overhead and zero memory copies.