Conversation
- In printWrap, avoid printing trailing spaces before newlines and on
empty lines.
- Remove whitespace indentation before closing parentheses on raw string
literal tool descriptions in wasm-opt and wasm-reduce.
- In printStackIR, skip Pop pseudo-instructions before emitting
indentation so that they do not produce empty lines with indentation.
The version of filecheck we currently use handles whitespace-only lines
fine, but LLVM FileCheck and newer versions of Python filecheck require
explicit `{{^ +$}}` regex matchers for such lines. It's nicer to just
not emit whitespace-only lines in the first place if we want to upgrade
our version of filecheck.
- Update update_lit_checks.py and update_help_checks.py to emit CHECK-EMPTY: for blank lines instead of empty CHECK: / CHECK-NEXT:. - Update check_line_re in update_lit_checks.py to recognize all standard FileCheck suffixes. - Regenerate help tests and manual tests with CHECK-EMPTY:.
Update requirements-dev.txt to use the modern Python filecheck package (1.0.3+).
…ro (WebAssembly#9056) This is a standard part of `fast-math` in clang and gcc: https://clang.llvm.org/docs/UsersManual.html#cmdoption-ffast-math We may have already been optimizing it, I'm not sure, but a later PR will do so, so it seems worth documenting explicitly.
Remove the PYTHONUTF8 variable from the Windows CI because the new filecheck version handles UTF-8 correcty by default.
…ly#9061) `update_lit_checks.py` doesn't always work well with wasm-split. For example, for some tests, the test CHECK lines are mixed like this after running it: ```wast ;; PRIMARY: ... ;; SECONDARY: .... ;; PRIMARY: ... ;; SECONDARY: ... ``` A follow-up PR will change these two test files' CHECK lines to be mixed when using `update_lit_checks.py`, and making them not use in that PR will make it hard to see what actually change in that PR. To make the next PR's diff tidy, this makes a few tests not use the auto-updating script. This also adds `-all` to `transtiive-globals-multi.wast` to be consistent with other `transitive-globals*.wast` tests.
…y#9058) We didn't look through the tee properly, and also did wasted work, since we can just stop at the first tee we see and use the value there.
A follow-up PR will move immutable globals to secondary modules when possible, and it will make the test expectations of `transitive-globals-multi.wast` different for mutable and immutable globals. Creating a new test in that PR will make it different to see what changes in that PR. So this PR duplicates the test to make mutable and immutable versions. Note that `$f` can't be converted to mutable because it is used in a `global.get`. This also adds mutable/immutable versions of globals to `split-module-items.wast`.
…embly#9065) `10 * MaxBinaryActions` is far too low, as CFGs can be complex enough to hit that. Use something far, far higher, unlikely to ever be seen in practice, but enough to assert instead of hanging, in case we have a bug.
This fixes typos in the codebase using codespell.
…riptor effects in the way (WebAssembly#9067)
This handles cases where the term is equal but not constant (we already handled constants before). E.g. this proves `x < y => x <= y`, which is true even though `y` is unknown.
This script runs and monitors fuzz_opt.py for up to a given number of iterations, redirecting its output to a rotating log file and printing its progress to stdout once a minute. When it detects that the fuzzer has found a bug, it prints the iteration number and the seed so the bug can be reproduced. This script is nicer to run in agent harnesses than raw fuzz_opt.py because agents can easily run the fuzzer for X iterations and can show the progress without overly polluting the context.
This will be necessary for a future optimization that turns resumes of continuations that never suspend into calls. Update the effect analysis of suspends to set the new effect and clobber global state because the suspend handler might do anything before returning. Test that the effects are analyzed as intended and that they work with global effect analysis.
…y#9076) Reduces compression time from ~67s to ~6.5s with a ~18MB / 9.5% increase in bundle size.
As proposed in WebAssembly/shared-everything-threads#119. Add parsing, printing, and validation. The optimization passes will not yet preserve the semantics of publish instructions in general; a future change will have to augment EffectAnalyzer to model its effects correctly.
GCData previously represented all allocations using a vector of Literals. Storing numeric array elements this way introduces unnecessary memory overhead and prevents efficient byte-level operations. Represent primitive numeric GC arrays using a raw byte buffer in GCData while preserving Literals storage for reference arrays and structs.
… in resume_throw_ref (WebAssembly#9082)
`llvm.sh` from the LLVM project is a standard way to get LLVM, but it downloads lots of stuff we don't need. We can just use clang from the runner in most cases. Do that, and use a newer ubuntu in the one place we need newer clang (for clang-format-21).
…#9083) - Add shared.run_parallel_tests helper with thread-safe output capture and interrupt handling - Refactor spec tests to use shared.run_parallel_tests The eventual goal here is to have all the suites (that don't parallelize themselves) using a ThreadPool to parallelize more.
Run the wasm-opt passes and print tests in parallel to speed up check.py runs. Isolate per-test temporary files and pass the debug environment explicitly to avoid overwriting other environments. This reduces runtime from 30s to 4s on a 128-core machine.
`getImmediateFallthroughPtr` needs to make sure that the fallthrough value is not modified or otherwise rendered unusable by the side effects of expressions that are evaluated after the fallthrough expression but before the top-level expression. Previously the only such expression was the br_if condition, and `getImmediateFallthroughPtr` only looked through the br_if if the condition did not have side effects that would interfere with the rest of the fallthrough expression. Make this effect analysis slightly more precise by using `orderedBefore` instead of `canReorder`. Also add similar effects analysis for the `desc` operands of `RefCast` and `BrOn` expressions, if it exists. These operands are also evaluated after the fallthrough expression. Finally, update `areConsecutiveAndEqual` in `OptimizeInstructions` to look through `RefCast` and `BrOn` expressions despite any effects that may occur in their `desc` operands; it will do its own more precise effects analysis afterward.
Add a `visitResume` in OptimizeInstructions that does the normal optimizations on null continuations and then tries to turn resumes into calls. We can do this when we are resuming a freshly allocated continuation created with a reference to a known function that GlobalEffects tells us will not suspend.
Handles this common code pattern: ```wat (if (i32.and (A) (B)) (then .. ``` We can apply both A and B in the first arm, and `!(A && B)` in the second. Also pattern-match `eqz(eqz(..))` as that is the only way to represent `!= 0` in a nested position (otherwise, just `if (local.get)` works, which we already matched).
When GTO removes fields or makes them immutable, it may introduce an immutable externref first field on the descriptor of a JS-exposed type where there was none before. That means that the described type could now have a JS-observable prototype where it did not before optimization, which makes this a misoptimization. Fix the problem by inserting an i8 placeholder first field wherever we would otherwise start exposing a prototype where there was none before. This is expected to be exceptionally rare in practice, so the extra memory use is not expected to be a real problem. Instead of adding a placeholder field, we could have inhibited optimization of the existing first field, but that would be more likely than an unaccessed placeholder field to have adverse effects in later passes. Fixes WebAssembly#9026.
One some ABIs, if a base class ends with padding, derived classes right after it can reuse that padding, placing their data there. This PR optimizes the key Expression class that way. Size changes on linux64 (common classes): | class | old size | new size | | -------- | -------- | -------- | | LocalGet | 24 | 16 | | LocalSet | 32 | 24 | | Load | 64 | 48 | | Store | 80 | 64 | | Unary | 32 | 24 | | Binary | 40 | 32 | | StructGet | 40 | 24 | | StructSet | 48 | 32 | On 5 real-world binaries I tested, this reduced peak RAM usage by 3%, 4%, 6%, 9%, 9%.
This is just not used anywhere. A long time ago asm2wasm and s2wasm used it, but both were removed. Emscripten no longer uses this after we switched to the LLVM wasm backend: https://emscripten.org/docs/tools_reference/settings_reference.html#:~:text=BINARYEN_TRAP_MODE%3A%20The%20wasm%20backend%20does%20not%20support%20a%20trap%20mode%20(it%20always%20clamps%2C%20in%20effect)%20(Valid%20values%3A%20%5B%2D1%5D)
Run the wasm2js test suite in parallel to speed up check.py runs. Isolate per-test temporary files and pass the debug environment explicitly for thread safety. This reduces runtime from 60s to 8s on a 128-core machine.
- Run native C/C++ builds, links, and runs in parallel using shared.run_parallel_tests - Remove use of `subprocess.check_call` to avoid interleaved stdout This reduces runtime from 26s to 3s on a 64-core machine.
Optimize out publishes of allocations and publishes of publishes where it is clearly not possible for there to be a write to the published object between the allocation or former publish and the outer publish. Also optimize out publishes of unshared and immutable reference types.
- Run the 6 fixed wasm-reduce tests in parallel using shared.run_parallel_tests - Isolate per-test temporary wasm and wat filenames - Pipe wasm-reduce stderr to prevent interleaved output This reduces runtime from 1m21s to 46s.
wasm-ctor-eval had a bug where return calls with non-serializable arguments would incorrectly overwrite global state before failing to serialize, leading to later crashes. Fix the problem by waiting to update global state until we are sure that the operation will be committed.
WebAssembly#9093) ## Summary Fix a parser bug where `wasm-opt` rejected valid modules containing declarative element segments with GC reftypes, failing with `parse exception: invalid tag index`. ## Problem Binaryen's binary reader (`WasmBinaryReader::readElementSegments` in `src/wasm/wasm-binary.cpp`) misread the type field of **declarative** element segments (flag `0x07`). In the binary format, a declarative segment is encoded as: ```text 0x07 reftype vec(expr) ``` where `reftype` can be a single byte (e.g. `nullref`) or a prefix byte plus a heap-type LEB (e.g. `0x63 0x02` for `(ref null 2)`). The reader only consumed a single `getU32LEB()` for the type. When the reftype was `(ref null 2)` — i.e. a prefixed heap type — the reader would consume the prefix byte as if it were the whole type, then misinterpret the following bytes as the vector length. This misaligned the stream, and downstream parsing eventually tried to resolve a value against the tag table, producing: parse exception: ```invalid tag index (at 0:51)``` Minimal repro (from the issue): ```wat (module (rec (type (array i16)) (type (sub (struct))) (type (array (mut nullexternref))) (type (sub (func (param i64)))) (type (array (mut f32))) (type (sub (array (mut v128)))) ) (elem declare (ref null 2)) (elem declare (ref null 3) (ref.null 3)) (elem declare nullref) (func (type 3) (param i64)) (func (type 3) (param i64)) ) ``` ```wasm-tools``` validate accepts this module; Binaryen rejects it. ## Fix Apply the same logic that is already used for passive and table-indexed segments to the declarative case: If usesExpressions is set (flag `0x07`), read a full reftype using `getType()`. Otherwise (flag `0x03`), read the single-byte elemkind and validate it is 0 (funcref). Previously, the declarative branch unconditionally called `getU32LEB()` for the type, which works only for flag 0x03. ```cpp if (isDeclarative) { // Declared segments are needed in wasm text and binary, but not in // Binaryen IR; skip over the segment. if (usesExpressions) { [[maybe_unused]] auto type = getType(); } else { auto elemKind = getU32LEB(); if (elemKind != 0x0) { throwError("Invalid kind (!= funcref(0)) since !usesExpressions."); } } auto num = getU32LEB(); for (Index i = 0; i < num; i++) { if (usesExpressions) { readExpression(); } else { getU32LEB(); } } continue; } ``` Why this approach: The fix is minimal and localized to the declarative branch. It reuses the existing, correct reftype-reading path rather than introducing a new helper. Declared segments are intentionally dropped from Binaryen IR (they are not needed there), so no IR representation changes are required — only the binary reader needs to consume the correct number of bytes. ## Testing Manual verification: Before the fix: ```text $ ./bin/wasm-opt repro.wasm --all-features -o /dev/null [parse exception: invalid tag index (at 0:51)] Fatal: error parsing wasm ``` After the fix: ```text $ ./bin/wasm-opt repro.wasm --all-features -o /dev/null warning: no passes specified, not doing any work ``` Cross-checked against the reference toolchain: ```text $ wasm-tools validate repro.wasm $ wasm-tools print repro.wasm (module (rec (type (;0;) (array i16)) ... ) (elem (;0;) declare (ref null 2)) (elem (;1;) declare (ref null 3) (ref.null 3)) (elem (;2;) declare nullref) ... ) ``` ```wasm-tools``` accepts the module, confirming the input is spec-compliant. New regression test: ```test/lit/binary/gc-elem-declare.wast``` exercises ```--roundtrip``` on the ```reproducer```, which forces Binaryen to write and re-read the module through the binary format. Before the fix, --roundtrip would hit the same invalid tag index error. ```text $ ./bin/binaryen-lit test/lit/binary/gc-elem-declare.wast -v PASS: Binaryen lit tests :: binary/gc-elem-declare.wast ``` **Full test suite:** ```text $ python3 check.py lit ... PASS: Binaryen lit tests :: binary/gc-elem-declare.wast (94 of 994) ... ``` All lit tests pass. **Related Issues** Fixes WebAssembly#8540. Follow-up Work None. Declared segments remain intentionally dropped from Binaryen IR, so no additional handling is needed beyond correct parsing.
…erReorder is set (WebAssembly#9096) In the neverReorder testing mode, no code can be reordered, as reordering it past branch hints leads to fuzzer errors.
Add a --jobs (-j) argument to monitor_fuzz.py to control how many instances of fuzz_opt.py are run in parallel. This can increase throughput because a single instance of fuzz_opt.py does not always saturate all cores. The instances are run in separate directories to avoid interfering with each other. To support running simultaneously in separate directories, make a few tweaks to fuzz_opt.py to make it more location-independent. For example, pass `-C` to git operations to run them explicitly in the binaryen root instead of the current directory.
Port changes from WebAssembly/acquire-release-atomics#21 and WebAssembly/acquire-release-atomics#23. After this PR the test matches the test from the proposal exactly. Once the acquire/release atomics proposal is in https://github.com/WebAssembly/testsuite, we won't have to copy this manually anymore. * Add missing `(register ...)` statements within `thread` blocks * Use unique names for threads and memories
…bly#9108) PR WebAssembly#9043 ("Avoid newly exposing prototypes in GTO") introduced prepending an i8 placeholder field at index 0 of exposed no-proto descriptors when their first field becomes prototype-exposing. When all fields of such a descriptor are kept, the new index of the last field equals the old struct's `fields.size()`. Because `updateInstructions` runs before `updateTypes` (so expression heap types still refer to the pre-optimization `Struct` definitions), and `FieldRemover` is a `PostWalker`, child `struct.get`, `struct.rmw`, and `struct.cmpxchg` instructions had their `index` mutated in-place to the new index before an enclosing `struct.new` or `struct.set` ran `ChildLocalizer` or `getResultOfFirst`. When `EffectAnalyzer` subsequently inspected those child instructions, `readsStruct` accessed `type.getStruct().fields[index]` using the old `HeapType` and the new `index`, causing an out-of-bounds assertion failure (`__n < this->size()`) or reading the wrong field's mutability. Fix this by splitting instruction updating into two passes executed in a single nested `PassRunner` invocation (`FieldRemover` followed by `IndexUpdater`), along with sequential `runOnModuleCode` runs. `FieldRemover` removes/reorders `struct.new` operands and replaces removed `struct.set`s while all struct instructions still have their old field indices matching their old `HeapType`s, and `IndexUpdater` then updates the field indices on all remaining struct instructions.
Add a pass that converts call, call_indirect, and call_ref instructions in tail position into return calls. Whether expressions are in tail position is propagated down from parent expressions to children, so we need to do a pre-order traversal instead of our normal post-order traversal. Add a bespoke PreWalker class that additionally passes `isTail` to the various expression visitors. Keep track of the exception handling depth to avoid incorrectly turning calls inside exception handlers into return calls.
Parsing tees allows us to handle more code, as it is common to see
a tee at the start of a constraint, e.g.
```wat
(if
(i32.eq ..
(local.tee $x ..)
(i32.const 42)
)
```
This is also a bugfix, as we were not handling unparsed tees before:
we need to make sure that no tee tramples a get that we parse into
a constraint. E.g.
```wat
(i32.and
(local.get $x)
(local.tee $x ..)
)
```
We cannot parse that into `$x && ..` because at the AND, we have already
trampled `$x`. This code looks for any such conflict.
Switch handlers on `resume` and `resume_throw` instructions do not have associated labels. GUFA previously assumed that all handlers would have labels, and would crash after trying to look up the target of an empty name. Fix the bug by explicitly checking for the presence of a label on the handlers.
Late in the development of the TailCall pass, we removed a run of DCE in a nested pass runner. It turns out that had been load bearing, because without it TailCall could optimize unreachable calls whose callee return types did not match the caller return type, producing invalid IR. Fix the bug by not optimizing unreachable calls, leaving them for a separate DCE pass to clean up instead. Also fix the lowering of call.without.effects when it is used as a tail call. Previously intrinsic lowering would turn such a tail call into a normal call, again producing invalid IR because the normal call is not unreachable and is not valid in all the places a tail call would be. --------- Co-authored-by: Alon Zakai <azakai@google.com>
When a new BINARYEN_FUZZ_STATS environment variable is set, collect statistics about fuzzer patterns and events. Save these statistics to a file, where they can be accumulated and updated across several fuzzer runs. The API is designed to make it easy to add new patterns and events for one-off local experiments. Include sample patterns collecting frequencies of various cast instructions as an example.
Two such instructions, one after the other, can return different things despite appearances, like a `call` or a `struct.new`.
If the replacement value is not unreachable, we still need to fix it up.
…ebAssembly#9114) Because arbitrary unshared externrefs cannot be made shared directly, lower unshared externrefs to shared i31ref table indices using a LazyTable abstraction that manages the table and runtime conversion helpers on demand. Unlike function references, externrefs are still passed across the module boundary, so wrap imported and exported functions that accept or return externrefs to convert between externrefs and table indices. Also lower any.convert_extern and extern.convert_any using imported conversion helpers, and check that table.grow succeeds (>= 0) when converting references to indices, trapping otherwise.
Alternative to WebAssembly#9085, which renamed `binaryen.none` to `binaryen.void` and added a new `binaryen.none` referring to the heap type. There were concerns that `binaryen.void` mapping to the C API `BinaryenTypeNone()` might be confusing. This PR moves all types from the top-level `binaryen.*` namespace into respective enums called `Type`, `HeapType`, and `PackedType`. This change plays nicely with AssemblyScript’s established [TypeScript typings](https://github.com/AssemblyScript/binaryen.js/blob/main/index.d.ts#L3) because now types are scoped in an enum space. This is not a breaking change for the TS typings file — e.g., a function that accepts a `Type` (previously just `number`) now accepts a member of the new `Type` enum. This change also follows well-established patterns of existing enums such as `ExpressionIds` and `Features`. `binaryen.Type.none` now uses `BinaryenTypeNone()` and `binaryen.HeapType.none` uses `BinaryenHeapTypeNone()`, so there’s no conflict across enums. This *is* a breaking change for JS/TS users: they will now have to reference types from the enum space instead of at the top level. E.g. `binaryen.i32` becomes `binaryen.Type.i32`, `binaryen.any` becomes `binaryen.HeapType.any`, `binaryen.i8` becomes `binaryen.PackedType.i8`, etc.
Return 1 instead of 0 when a bug is found to distinguish it from the case when --max-iters is reached. Return 2 in the case of other errors. This change lets this helpful bash one-liner work correctly: ```bash while (git pull origin main && ninja && ./scripts/monitor_fuzz.py -j --max-iters=1000000); do :; done ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes in
upstream/main(namely from WebAssembly#9098) caused breakages in the JS API. This PR syncs them up with your branch.