Define bounded lists

[Felix-El]

Add bounded lists (list<T, ..N>)

Closes #385.

[cpetig]

mostly indentation doubts

[lukewagner]

Thanks! This is looking generally good, a few comments:

[lukewagner]

Pre-existing, but it looks like the grammar already covers this twice: once by using <u32> (unsigned) and once with the (if maxlen > 0). We could also remove the (if maxlen > 0). But should we specify a maximum for maxlen?

[Felix-El]

For u32, zero is valid and the maxlen > 0 checks forbids zero. What would be a legitimate upper bound... i32::MAX?

[lukewagner]

There's already (just recently added) a MAX_LIST_BYTE_LENGTH = 2²⁸-1. That's just bytes, but even still, it seems like a reasonable upper bound.

[Felix-El]

Ok, defined that bound in BINARY.md but where would we check this in definitions.py ?

[cpetig]

Associated with Implement WIT Fixed-Length-Lists

[Felix-El]

Here is another shot at an ABI tweak for efficient passing of fixed/bounded lists. It is already implemented in the commit I just pushed. @cpetig 's ideas with refinement by discussion, my write-up; more versatile / less invasive compared to my previous proposal.

---

Flat ABI for Fixed and Bounded Length Lists

Motivation

The current flat ABI for fixed length lists expands all N element slots
into the core function's parameter/result list.
For large N this causes parameter overflow (exceeding
MAX_FLAT_PARAMS or MAX_FLAT_RESULTS), forcing an early fallback to
memory passing for the entire argument list when even one or two lists
are large.

If the current approach was naively adopted for bounded length lists
(which are still theory at this time), expanded form would even waste
flat-register space on uninitialized elements for partially-populated lists.

A second, independent motivation is that fixed and bounded length lists
are, unlike tuples, indexed at runtime and registers in core WebAssembly
cannot be indexed efficiently. Therefore, the expanded-element flat representation
forces register/memory (de)composition on either guest, adding to overhead.

This proposal replaces the expanded-element flat representation with a
pointer-based one for direct fixed and bounded length lists in parameters —
lists which are eligible for flat representation (not nested, immediately or
transitively, in another list).

Flat representation

Type	Flat representation
list<T> (unbounded)	[ptr, len] — pointer to len contiguous elements + actual length (unchanged, for reference)
list<T, N> (fixed)	[ptr] — one pointer to N contiguous elements
list<T, ..N> (bounded)	[ptr, len] — pointer to len contiguous elements + actual length

Memory representation

Type	Memory representation
list<T> (unbounded)	[ptr, len] — pointer to len contiguous elements + actual length (unchanged, for reference)
list<T, N> (fixed)	[e0, e1, ...] — N contiguous elements
list<T, ..N> (bounded)	[len*, e0, e1, ...] — N contiguous elements, uninitialized beyond actual length

bounded actual len: smallest integer type that can represent values up to (including) N

Lowering and Lifting

Parameters

Below explanation is for bounded length lists which carry a runtime length.
The flow is similar for fixed length lists - length just would not be passed
as it is statically known.

If all core params fit within MAX_FLAT_PARAMS, they are passed flat, otherwise
we fall back to a in-memory tuple representation (param area). This is not new.

1. Caller

   • assume list L is already constructed in guest memory as L_ptr and L_len (from API use)
   • if passing flat: pass L_ptr and L_len directly, classic unbounded list ownership semantics
   • if passing in memory: move list elements inplace into param area

2. Host, before call forwarded

   • if passing flat:
     • allocate twin list L2 in the exporting guest (optimization: bulk alloc, cache, etc.)
     • move elements between backing memories (import -> export)
     • pass L2_ptr and L2_len = L_len, classic unbounded list ownership semantics
   • if passing in memory:
     • move elements from inbound param area to outbound param area

3. Callee

   • if passing flat: take L2_ptr and L2_len from core params
   • if passing in memory: read L2_len from param area, let L2_ptr point to first inplace element in param area
   • either way: construct list from L2_ptr, L2_len for API use

Results

Memory-passing is enforced when even a single direct fixed/bounded length list is in the result type -
that's a new second switchover reason beyond exceeding MAX_FLAT_RESULTS.
A return area (ret area) is used to forward all results as an in-memory tuple.

1. Caller

   • result area is preallocated with enough room to hold all fixed/bounded length lists inline
   • result area pointer is passed as out core param

2. Host, after call forwarded

   • copy inplace elements from returned export-side ret area into import-side ret area provided by out param

3. Callee

   • construct the list by moving elements from ret area for API use

Fused allocation

When multiple direct lists exist among parameters and passing flat, the host might service
their backing (export) memory needs from a single allocation as an optimization.

[lukewagner]

Thanks for the clear write-up (and sorry for the delay replying)! That all sounds pretty good; just one suggestion: if a component exports a function with type func(p: list<u8, 4>, q: list<u8, 4>), iiuc, the above suggests there would be 2 realloc calls in the callee for p and q. I wonder if we should instead have this work like return values: everything goes into a contiguous param-buffer (so that there's always only 1 realloc call), as-if MAX_FLAT_PARAMS had been exceeded. (A component that imported this function type would work the same as you outlined.) WDYT?

[Felix-El]

if a component exports a function with type func(p: list<u8, 4>, q: list<u8, 4>), iiuc, the above suggests there would be 2 realloc calls in the callee for p and q.

I addressed that in the trailing "fused allocation" section - yes it should be done but we need not guarantee that by design. In practice, the host can alloc one big chunk for everything and hand out pointers to subobjects (list1, list2, list3, etc. correctly aligned and sized of course). The allocation will continue to be owned by the host.

I wonder if we should instead have this work like return values: everything goes into a contiguous param-buffer (so that there's always only 1 realloc call), as-if MAX_FLAT_PARAMS had been exceeded.

Passing all params in a param-area, even if callee side only, feels a bit inefficient. We should try to utilize registers for non-list value passing.
Additionally it would break the symmetry between caller core func signature and callee core func signature much further than just the current return-area result vs. out pointer story.

But that's just my 2 cents...

PS: definitions.py modifications compared between the previous/initial proposal and this one, I have to admit this one felt way less intrusive to implement (thinking of differences in definitions.py in 91a0eae vs 1fb578e over the predecessor.)

[lukewagner]

I addressed that in the trailing "fused allocation" section - yes it should be done but we need not guarantee that by design.

I think the host can do that when given a list, b/c the host can do whatever, but if we're talking about lowering a list into the guest, the realloc calls would have to happen exactly when specified, so no fusion would be possible when lowering a fixed-/bounded-length list into guest code.

Passing all params in a param-area, even if callee side only, feels a bit inefficient.

I suppose, but I would expect this to be amortized by the cost of at least 1 realloc. FWIW, I think the way realloc would get removed in a lazy-lowering future is that, when parameters aren't flat (including when there are any fixed- or bounded-length lists), the callee entry function would call a (new) task.start built-in passing a pointer to a buffer to receive all the parameters, which is sortof analogous.

[flelchuk]

I think the host can do that when given a list, b/c the host can do whatever, but if we're talking about lowering a list into the guest, the realloc calls would have to happen exactly when specified, so no fusion would be possible when lowering a fixed-/bounded-length list into guest code.

Sorry if I'm repeating myself. I might be missing some crucial C-M insight...

• This proposal leaves most of lifting/lowering as-is, including dynamic lists.
• The only special new treatment is for fixed/bounded lists.
  • Memory path: all params in param area - caller passes to host, host copies*, callee receives param area. Fixed/bounded lists are inlined.
  • Register path: fixed/bounded lists enter the host as 1-2 slots each. The host allocates export-side memory possibly as a single longlived allocation holding elements of all lists, host copies* elements and forwards to the callee also as 1-2 slots each. The callee uses memory from host-given address directly, no further realloc needed
• Return is always inlined in ret area.

What obligation would make fusion impossible?

[cpetig]

• Register path: fixed/bounded lists enter the host as 1-2 slots each. The host allocates export-side memory possibly as a single longlived allocation holding elements of all lists, host copies* elements and forwards to the callee also as 1-2 slots each. The callee uses memory from host-given address directly, no further realloc needed

My understanding is that this logic is already needed for the argument area whenever calling into the guest with more than 16 flat parameters. We simply start using this linear memory argument area earlier for spilling over the fixed/bounded list data when the rest of the arguments still fit into registers.

[cpetig]

(see WebAssembly/WASI#944 for previous status updates and reasoning)

I implemented more of this logic inside wasmtime and it needed a larger restructuring in the environ crate - because now you should allocate memory across all arguments (already default when not flattened) and pass flattened arguments pointing there. So the calculation of flattened properties needs to pass this information on. Still working on this.

[cpetig]

Opus 5.5 pointed out an interesting inconsistency in my implementation:

Does a guest imported f: func() -> list<u32,20> lower to void f(uint32_t) or to uint32_t f(uint32_t). (Guest export is clearly uint32_t f())

Felix wrote "Memory-passing is enforced when even a single direct fixed/bounded length list is in the result type - that's a new second switchover reason beyond exceeding MAX_FLAT_RESULTS."

This avoids running into the situation that an out pointer to caller-allocated memory is passed as an additional argument while still returning values, thus trading a special case (asymmetry to argument handling) for another one (unnecessary return value).

The option I had in mind was passing a "scratch buffer" to the function which would be used by the host to place array elements into. So an out buffer parameter which only is present if fixed-length and bounded lists are part of the flattened result.

If we add multi-value to the mix, this gets more interesting, returning a struct s { a: list<u32, ..20>, b: list<u32, 4>, c: u32 } would lower to (guest calling host) (uint32_t, uint32_t, uint32_t, uint32_t) f(uint32_t) or void f(uint32_t).

@lukewagner How do you feel about this? Do you value symmetry with argument handling or simplicity (no strange scratch buffer) higher? Felix and me found that both have value.

(Although in writing this I realized that the scratch buffer feels more of a strange new concept needing explanation, in this case it would be 20+4 integers versus 1+20+4+1 integers for the all-in-memory case)

PS: This proposal got rid of some special handling in the wasmtime flattening code, while introducing data structure complications for argument handling. But to me it feels more right than the simpler "consider arrays as tuples" existing strategy.

[bvisness]

I hope I haven't misunderstood anything, but I don't understand why bounded lists should be any different from plain old arbitrary-length lists at the ABI level. Putting fixed-length lists aside for a moment, the motivation for bounded lists is to avoid runtime allocations. But that just means the bindings generator has to know the maximum size of the incoming data so it can preallocate a sufficiently large chunk of memory. The actual data passed via the ABI could look exactly the same as it does today.

Yes, I realize you could pass a bounded list's data in params, but who is out here passing around bounded lists of length 16 or less? And that's assuming the list's elem type is one param; in practice I'd imagine your list bound would likely have to be 8 or 4 or 2, and who is passing around lists with a maximum size of 2?

So to me it just seems like bounded lists should in fact just be normal lists at the ABI level. Just pass them in memory, since 99% of the time the maximum length will exceed MAX_FLAT_PARAMS.

As for fixed-length lists, I understand that you'll have to copy from params into memory before indexing the list, but that seems...super incredibly normal. And the alternative is to call realloc, and surely the cost of calling realloc is far higher than the cost of copying 4 or 8 params into a stack-allocated list or tuple. All the discussion of "direct" fixed-length lists and scratch buffers and allocation fusion seems way too complicated to be justifiable. I mean the way I see it, the options are:

1. Pass fixed-length lists as tuples always. Flattening is simple. If lists are short and fit in MAX_FLAT_PARAMS, the receiver will have to copy from params into stack memory before indexing but won't need to allocate. If lists are longer, they will exceed MAX_FLAT_PARAMS and therefore require a realloc, but you naturally get just one realloc call for all such lists.
2. Pass fixed-length lists like normal lists always, realloc-ing each one. Lengths can at least be omitted from flat params. This produces lots of reallocs despite knowing all the lengths up front, which is pretty dumb.
3. Some kind of compromise where "direct" fixed lists are batched into one realloc call and flattened as pointers (again no lengths because they are known). Meanwhile, "indirect" fixed lists are...well, pick option 1 or 2 I guess, hopefully 1. If in the end this exceeds MAX_FLAT_PARAMS anyway, all the logic changes again and hopefully you can lump the fixed-list memory into the realloc for params generally. In the end you still have a realloc call, but flattening is vastly more complicated.

As I understand, option 3 is being proposed, but like...why? You're guaranteed to realloc now, and most of the flattening logic will still need to assume tuples, and you need to define "fusion". Why not just go with option 1, which is simple and probably fastest in all common cases?