feat: Incremental Build#1267
Closed
RandomByte wants to merge 272 commits into
Closed
Conversation
RandomByte
force-pushed
the
feat/incremental-build-4
branch
3 times, most recently
from
5224cd2 to
4904c84
Compare
RandomByte
force-pushed
the
feat/incremental-build-4
branch
from
950fc6d to
41eed91
Compare
maxreichmann
force-pushed
the
feat/incremental-build-4
branch
from
bb39565 to
2a21507
Compare
RandomByte
force-pushed
the
feat/incremental-build-4
branch
3 times, most recently
from
6233816 to
f858659
Compare
RandomByte
force-pushed
the
feat/incremental-build-4
branch
from
71db1d0 to
a2c371f
Compare
RandomByte
force-pushed
the
feat/incremental-build-4
branch
from
7364b4b to
cf43f0c
Compare
maxreichmann
force-pushed
the
feat/incremental-build-4
branch
2 times, most recently
from
252b966 to
874a943
Compare
RandomByte
commented
Feb 18, 2026
maxreichmann
force-pushed
the
feat/incremental-build-4
branch
5 times, most recently
from
df275e5 to
0345502
Compare
RandomByte
commented
Mar 5, 2026
RandomByte
force-pushed
the
feat/incremental-build-4
branch
from
9fc2509 to
20ba653
Compare
maxreichmann
force-pushed
the
feat/incremental-build-4
branch
from
20ba653 to
9fc2509
Compare
RandomByte
force-pushed
the
feat/incremental-build-4
branch
from
9fc2509 to
940376d
Compare
RandomByte
force-pushed
the
feat/incremental-build-4
branch
from
9197670 to
44d1107
Compare
maxreichmann
force-pushed
the
feat/incremental-build-4
branch
from
44d1107 to
d7c402c
Compare
Member
|
Rebased onto origin/main |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…esource.setStream Streams from older userland packages like readable-stream@2 (used by replacestream) have .pipe but no Symbol.asyncIterator, causing "stream is not async iterable" errors when the resource content is later consumed by node:stream/consumers. Also handle WHATWG TransformStream objects by unwrapping to their readable side.
The custom task reads dependencies via taskUtil.getProject().getReader() which bypasses the monitored dependencies reader. These reads are not tracked by ResourceRequestManager, so dependency changes don't invalidate the application's result cache.
Now only used in @ui5/fs
…c-generate
Replace type expressions unsupported by JSDoc's Catharsis parser:
- Union types inside Record literals in generics ({string|number|...})
- TypeScript-style import() type syntax
- Tuple types ([T, U]) inside generics
- Optional property syntax (?) inside Record generics
…ware configuration (#1367) JIRA: CPOUI5FOUNDATION-1206 --------- Co-authored-by: Copilot <copilot@github.com>
matchResourceMetadataStrict and the duplicated fast path in HashTree#upsertResources Phase 1 returned "unchanged" whenever a file's mtime matched the cached value, without consulting size or content. External operations that preserve mtime - cp -p, tar -x, rsync -t, and atomic rename of a temp file with copied stat - therefore passed silently as cache hits and produced stale build output. Verify size before the mtime-based short-circuit can return true. This matches the cheap-path semantics used by rsync's -t quick-check and git's stat-bundle compare, and closes the gap without forcing an integrity hash on every comparison. For FS-loaded resources getSize() resolves synchronously from statInfo, so the only added cost is one microtask per comparison. Remove the duplicated mtime fast path from HashTree#upsertResources and route every existing resource through Phase 2, which delegates to the now-corrected matchResourceMetadataStrict and already handles tag-only updates.
Extend matchResourceMetadataStrict to compare the resource inode against the cached inode. When both sides expose an inode and they disagree, the file has been replaced (atomic rename, cp, tar extraction, ...) — the mtime+size short-circuit is suppressed and the comparison falls through to the integrity check. Inode is now always populated by createResourceIndex; the optional includeInode flag is removed. When either side has no inode (virtual resources, caches written before this change), the inode check is skipped and behavior matches the previous mtime+size path. Falling through to integrity rather than returning false avoids unnecessary task re-execution when the replacement preserves content (e.g. an idempotent rsync). When the replacement also alters content, the integrity check correctly classifies the resource as changed, closing a stale-cache blind spot that mtime+size alone could not detect.
…changed Drop the unused matchResourceMetadata variant (no longer used) and rename matchResourceMetadataStrict to isResourceUnchanged.
Remove unused firstTruthy method, drop the stale local ResourceMetadata typedef (use existing type from HashTree.js), and strip the UI5_CACHE_PERF instrumentation.
createStat() previously returned a synthetic statInfo with ino: 0, which Resource.clone() forwarded as statInfo into the new instance. The constructor's `this.#inode ??= statInfo.ino` then set the clone's inode to 0, so memory-only resources read through Memory.byPath / Memory.byGlob advertised a fabricated inode. That defeated the inode short-circuit in the build cache's isResourceUnchanged whenever cached metadata was later compared against an FS-backed read of the same path. Set ino to undefined in createStat() so memory-only resources and their clones report getInode() === undefined, leaving the cache to fall back on size + lastModified + integrity. Add Resource tests covering inode propagation through construction, clone, and setBuffer for both memory-only and FS-backed resources.
Expose ui5DataDir as a programmatic option on ProjectGraph.build(), plumbed through ProjectBuilder, BuildContext, and CacheManager.create(). Previously the build cache location could only be configured via the UI5_DATA_DIR env var or the ~/.ui5rc config file, both of which are process-global and unsuitable for parallel test isolation. Builder integration tests now route the SQLite-backed build cache into a per-test directory via a new isolatedUi5DataDir helper. This fixes intermittent ERR_SQLITE_ERROR "database is locked" failures on Windows CI caused by parallel AVA workers contending on the shared ~/.ui5/buildCache database, and removes the UI5_DATA_DIR fallback from the builder ava.config.js.
Addresses the documentation comments from here: #1367
RandomByte
force-pushed
the
feat/incremental-build-4
branch
2 times, most recently
from
c2dc7b8 to
1041695
Compare
Port collided with port used in caching.js test
Add direct yargs builder unit tests for the --cache, --cache-mode and --output-style coerce callbacks in the build and serve commands. Drop the equivalent subprocess-based tests from base.js that exercised the same coerce logic via the real ui5 binary. The unit tests run in-process so nyc captures their coverage, restoring the cli package above its thresholds
The BuildServer constructor used to fire `watchHandler.watch(...)` without awaiting it, so `graph.serve()` could return before chokidar had fully initialised. Source-file changes made immediately afterwards were silently dropped, which surfaced as flaky cache-mode integration tests on Windows where chokidar's ReadDirectoryChangesW backend has noticeably higher startup latency than inotify/FSEvents. Move watcher setup into a `BuildServer.create()` static factory that awaits readiness, and update `ProjectGraph.serve()` to use it.
The active build promise can reject (e.g. when Cache=Force detects a stale cache and throws). When destroy() awaited that promise, the throw skipped the closeCacheManager() call below, leaving the SQLite cache.db handle open. On Windows this caused subsequent tests to fail with EBUSY when fs.rm tried to remove the buildCache directory. Wrap the active-build await in try/finally so the cache manager is released even when the build rejected.
RandomByte
force-pushed
the
feat/incremental-build-4
branch
from
1041695 to
66296d5
Compare
Member
Author
|
Will be merged via #1267 There are several known bugs that will be addressed in follow-ups before a new pre-release should be created. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implementation of RFC 0017 Incremental Build
This PR supersedes previous PoC: #1238
JIRA: CPOUI5FOUNDATION-1174