fix: better handle concurrent builds - #3301
Open
dsanders11 wants to merge 8 commits into
Open
dsanders11 wants to merge 8 commits into
dsanders11 wants to merge 8 commits into
Conversation
dsanders11
force-pushed
the
fix/concurrent-rebuilds
branch
2 times, most recently
from
April 13, 2026 00:23
49c5a3c to
dad0d06
Compare
dsanders11
marked this pull request as ready for review
April 13, 2026 01:23
Signed-off-by: David Sanders <dsanders11@ucsbalum.com>
Assisted-by: Claude Opus 4.6 Signed-off-by: David Sanders <dsanders11@ucsbalum.com>
Assisted-by: Claude Opus 4.6 Signed-off-by: David Sanders <dsanders11@ucsbalum.com>
Signed-off-by: David Sanders <dsanders11@ucsbalum.com>
Assisted-by: Claude Opus 4.6 Signed-off-by: David Sanders <dsanders11@ucsbalum.com>
dsanders11
force-pushed
the
fix/concurrent-rebuilds
branch
from
April 21, 2026 17:10
dad0d06 to
d7a1e4a
Compare
legendecas
reviewed
Apr 22, 2026
| } | ||
| }, 0) | ||
|
|
||
| await copyDirectory(srcDir, destDir) |
Member
There was a problem hiding this comment.
Would you mind adding a test that srcDir is unchanged?
Contributor
Author
There was a problem hiding this comment.
I've added a test that moveDirectory (renamed it to reflect the changed semantics) retains the contents from srcDir moved to destDir. Let me know if I misunderstood what you were asking for here and I can add more test coverage.
legendecas
reviewed
Apr 22, 2026
| const srcPath = path.join(src, entry.name) | ||
| const destPath = path.join(dest, entry.name) | ||
| try { | ||
| await fs.rename(srcPath, destPath) |
Member
There was a problem hiding this comment.
This changes the semantics to moveDirectory instead of copyDirectory.
Contributor
Author
There was a problem hiding this comment.
Happy to rename it and the files. You're right that now it's move, outside of the edge case of cross-device on Windows, which falls back to a copy and move.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
Reflects the change in functionality Assisted-by: Claude Opus 4.6
Assisted-by: Claude Opus 4.6
This was referenced Sep 5, 2026
codebytere
pushed a commit
to electron/electron
that referenced
this pull request
Sep 5, 2026
The parallel native-addon fixture builds during `yarn install` each run `node-gyp install` into the shared devdir when it is cold, and one fixture's compile can read a header while another install is rewriting it (nodejs/node-gyp#3301), e.g. `v8-isolate.h: unterminated #ifndef` on linux-arm. The pre-seed that avoids this was skipped on Linux because the containers ship a pre-warmed cache, but that cache is keyed by the Node version of the runner that built the image and the arm32v7 test image runs Node 22.15.0, so it is always cold there. Run the pre-seed everywhere; it is a no-op when the cache is warm.
codebytere
pushed a commit
to electron/electron
that referenced
this pull request
Sep 5, 2026
The parallel native-addon fixture builds during `yarn install` each run `node-gyp install` into the shared devdir when it is cold, and one fixture's compile can read a header while another install is rewriting it (nodejs/node-gyp#3301), e.g. `v8-isolate.h: unterminated #ifndef` on linux-arm. The pre-seed that avoids this was skipped on Linux because the containers ship a pre-warmed cache, but that cache is keyed by the Node version of the runner that built the image and the arm32v7 test image runs Node 22.15.0, so it is always cold there. Run the pre-seed everywhere; it is a no-op when the cache is warm.
oga35767-eng
approved these changes
Sep 20, 2026
This comment was marked as spam.
This comment was marked as spam.
legendecas
reviewed
Sep 22, 2026
| await fs.rename(srcPath, destPath) | ||
| } catch (err) { | ||
| if (RACE_ERRORS.includes(err.code)) { | ||
| // Another parallel process already placed this entry — ignore |
Member
There was a problem hiding this comment.
I don't think this is a really good idea. This could result in a folder where some of its sub-entries are stale (like on windows it's still been used resulting in EBUSY), but some are new.
A directory already at the destination is not proof the contents match.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist
npm install && npm run lint && npm testpassesDescription of change
Fixes #3165, which unfortunately was not actually fixed by #3170. I'm not sure #3170 had any effect on the issue, as my original changes in #2846 which were extended to all platforms in #3710 were targeting a specific issue on Windows, and did not aim to fix (or know of) the issue in #3165.
The second commit on this branch adds a test which repros the issue being reported in #3165, namely that header files can be seen as partially written when multiple builds are happening concurrently. A failing test run before the other commits on this branch can be seen here.
The remainder of the commits refactor
copyDirectoryto fix the issue, fix another concurrent build issue I came across during testing, and adds more test coverage. This refactor also allows for droppingexponential-backoffas a dependency.