diff --git a/packages/ecs/CLAUDE.md b/packages/ecs/CLAUDE.md index 91009ef6..4e2f63b0 100644 --- a/packages/ecs/CLAUDE.md +++ b/packages/ecs/CLAUDE.md @@ -25,7 +25,8 @@ Humans own the interface and agents own the implementation: agents don't add to It's a project of its own: nothing in it imports from outside it. Each type exists once, there. - **`src/implementation`** holds `createWorld` and everything behind it, the kernel compiler included. It imports types and the registry's read side from `src/interface`. -- **`src/samples`** holds one folder per sample, its page included; `pnpm sample` serves them. +- **`src/samples`** holds one folder per sample, its page included. `pnpm dev` (or `pnpm sample`, without + the type-check watch) serves them, with a sample browser at the root that has a tab for each. - `src/index.ts` exports `src/interface`, and `World` as a value too, whose `create` and `join` are the implementation's, so `import { World } from "@adobe/ecs"` gives consumers the type, `World.create` and `World.join`, the only entry points. Samples and the guide use them; tests and benchmarks may take `createWorld` from @@ -44,6 +45,12 @@ when a measured win conflicts with a style or design preference, take the win an row as low as it can be, and allocate nothing on those paths: reuse arrays, buffers and objects, share read-only empties, and avoid closures, spreads and awaits there. Measure changes with `pnpm perf` against the recorded baseline. +## Pull requests + +Each PR costs a review, a merge and a publish, so keep one open PR at a time. Add new work and fixes to it as more +commits; never open stacked or parallel PRs. Keep it rebased on main, with this package's version one above main's. +Bump it by hand in `package.json`; never run the workspace `bump` script, which bumps packages this PR didn't change. + ## Memory Decisions about working on this package live in `memory.md`; add new ones there. diff --git a/packages/ecs/README.md b/packages/ecs/README.md index 619a1ed3..b6dda888 100644 --- a/packages/ecs/README.md +++ b/packages/ecs/README.md @@ -1,5 +1,40 @@ # @adobe/ecs -A data-oriented entity component system. Systems run on the main thread, on worker threads, as WASM kernels or on the GPU. +A high-performance, multithreaded Entity Component System for web-based applications, simulations and games. + +## Features + +- **Linear, gapless structure-of-arrays component storage** + - Fixed-length values (numbers, tuples, structs) in a `SharedArrayBuffer` + - Shared across the main thread and all workers + - Read by WASM kernels without copying + - WebGPU-compatible struct layout for fast uploads + - WebGPU storage for GPU system kernels + - Synced back to shared memory only when the CPU reads it + - Strings in shared memory, readable on every thread + - Objects and other variable-length values on the thread that owns them +- **Systems** + - Plain TypeScript bodies with declared reads and writes + - Kernels compiled and checked at build time to optimized JavaScript, WASM and WGSL + - Balanced scheduling across the main thread, workers and the GPU + - Typed input, with its schema generated from the TypeScript type + - Render systems that draw straight from component storage + - Observer systems for reading state into a UI +- **Changes** + - Per-component added and changed ticks: an O(1) check per entity, with each change seen exactly once + - Filters that skip unchanged chunks and 64-row blocks + - Structural changes batched at sync points, so memory never moves while systems run + - Rules, finalizers and traits for derived components, cleanup and references +- **Queries** + - Hash, unique, sorted and keyless indexes, all updated incrementally + - Hierarchy traversal that reruns only below what changed +- **Threads** + - Worlds shared with other threads through a `MessagePort` or worker + - Joined threads run their own systems and observers, and lend their workers to the pool +- **Runtime** + - Kernels allocate nothing per row; commands, lookups and reports reuse their buffers + - Errors reported as data, in each run's report + +## Guide See the [guide](src/guide/guide.md) for how to use it. diff --git a/packages/ecs/design.md b/packages/ecs/design.md index da7d33e8..930f8fa7 100644 --- a/packages/ecs/design.md +++ b/packages/ecs/design.md @@ -28,7 +28,7 @@ Rule: anything that doesn't affect the API or correctness belongs in phase 2+. - Structure of Arrays component storage. - Numeric types, tuples and structs live in linear memory backed by `SharedArrayBuffer`, so workers can read and write them. - **Structs** (numbers, booleans, entities, fixed-length tuples and structs of those) have a fixed size and live in shared memory. **Strings** (any string, a union of string literals, either with `null`) live in shared memory too, so every thread reads them, and have no thread: a declaration giving one a `thread` is rejected, in types and at registration, and a string places no system. **Objects** (everything else: variable-length arrays, plain objects, DOM nodes, GPU objects, functions) live on one logical thread, `world` unless declared otherwise ([`threads.md`](threads.md)). - - **A string's storage:** the world's thread holds each value as it is, in a column of values, and reads and writes it there, as it does an object. While a joined thread may read strings, from the first `share` until the last thread sharing is gone, the world also keeps a record of each value in its memory, which a column of 4-byte handles names, 0 for `null` (`strings.ts`): its length in UTF-16 code units, a generation and its code units, immutable. A write makes a new record and lets the old one go; a record let go is freed at the end of the second sync point after, which no joined reader outlives: a task ends with its run, and a tail or an observer post, with its snapshot, before the second sync point after the one it reads. Two readers outlive that, so they hold records of their own: a joined thread's finalizer reads the strings its stops held from copies in the stop list, let go once the thread acknowledged them, and an observer system reading a string resource posts as row systems do, since a post holding copies of its resources is waited for by nothing. Records up to 512 bytes are carved from slabs by power-of-two size class, larger ones are the arena's blocks. A joined thread decodes a record once and keeps its text by handle, while the record's generation is the same. Without shared memory, which no other thread reads, strings are held only as values. + - **A string's storage:** the world's thread holds each value as it is, in a column of values, and reads and writes it there, as it does an object. While a joined thread may read strings, from the first `share` until the last thread sharing is gone, the world also keeps a record of each value in its memory, which a column of 4-byte handles names, 0 for `null` (`strings.ts`): its length in UTF-16 code units, a generation and its code units, immutable. A write makes a new record and lets the old one go; a record let go is freed at the end of the second sync point after, which no joined reader outlives: a task ends with its run, and a tail or an observer post, with its snapshot, before the second sync point after the one it reads. Two readers outlive that, so they hold records of their own: a joined thread's finalizer reads the strings its stops held, and the string resources it reads, from copies in the stop list, as it reads its linear resources, so every resource as of before its stops' sync point applied, let go once the thread acknowledged them, and an observer system reading a string resource posts as row systems do, since a post holding copies of its resources is waited for by nothing. Records up to 512 bytes are carved from slabs by power-of-two size class, larger ones are the arena's blocks. A joined thread decodes a record once and keeps its text by handle, while the record's generation is the same. Without shared memory, which no other thread reads, strings are held only as values. - **What reads and writes them:** any system reads strings, on any thread. Only the world's thread writes them: a system placed on a joined thread that writes one is rejected there, and a command on a joined thread gives one only its start. Kernels don't read them yet, by type and check. An index keyed by strings is kept on the world's thread, by their values as objects' are, so it's looked up only there. - Memory only reallocates at sync points. Structure is frozen between sync points, so workers never see memory move mid-phase. - Removed rows stay in place until Apply. At Apply the last row is swap-moved into the hole and the entity lookup table is updated. @@ -43,7 +43,7 @@ Rule: anything that doesn't affect the API or correctness belongs in phase 2+. - **Kernel systems (`kernel: "cpu"` or `"gpu"`) are compiled at build time.** A Vite/Rolldown plugin compiles them in its `transform` hook, after TS types are stripped and before minification, so names are never mangled and nothing is parsed at runtime. - It finds systems by resolving the import binding of the system factory. - From one parse it emits a compiled kernel (efficient JS; later WASM and WGSL) and the inferred read/write sets, which are checked against the declared signature both ways: a body can't use a key its declaration doesn't list, and a declaration can't list a key its body never uses, since that key would still narrow which entities match and hold back systems that could run alongside it. The same goes for a reduced resource the body never contributes to. Main-thread systems get the second check where their bodies index or destructure `row` and `context` only by module-scope keys, and only index or test lookup results; otherwise TypeScript's row and context types are the only check. - - **The plugin is required for kernels.** Registering a kernel system that wasn't compiled throws. + - **The plugin is required for kernels and input.** Registering a kernel system that wasn't compiled throws, as does a system whose body takes input without the schema the plugin gives. `kernelPlugin()` returns its two parts: input schemas, before TypeScript is stripped, and kernels and ids, after. - **The plugin also gives keys ids** (`compiler/identify.ts`), in the same `transform`, after the kernel transform: each top-level `const`, `let` or `var` declarator, exported or not, initialized by a call of the global `Symbol`, and `export default Symbol(...)`, becomes `identify$(Symbol(...), "#")`, with a source map (`compiler/source-map.ts`). `` is the name of the nearest `package.json` above the module that has one and the module's path in that package, without a query, so a workspace-linked package names no directory outside it; without a named package, a dependency's package name and path inside `node_modules`, or the path from the root; never an absolute path; `#default` names a default export. A function appended to the module records the id in a `Map` under the global `Symbol.for("@adobe/ecs/ids@1")`, which the registry reads at registration, so the code imports nothing and nothing new is exported. Symbols made in functions, blocks, TypeScript namespaces, class fields or CommonJS, and modules where `Symbol` or `globalThis` is bound, aren't identified. In the same way, a top-level declarator initialized by a call of `System.create`, `System.render`, `Rule.create`, `Finalizer.create` or `Finalizer.changes`, with `System`, `Rule` or `Finalizer` imported from the package under any local name, and its `export default` form, gives the call's first argument, its declaration, a composite's list of members or a system's body alone, the id `"#"`: `identify$(declaration, id)` records it in the same map, and the constructor takes it out when it registers the item, so a declaration object reused elsewhere has no id. A kernel's call wrapped by the kernel transform is identified the same. The plugin adds itself to `worker.plugins`; a second instance in one bundle does nothing. - **The application compiles every kernel,** its dependencies' too, against the one runtime it bundles. A library ships its kernels as source in its `dist`, uncompiled and unminified, so their code stays in the kernel subset and its errors stay readable. Non-kernel systems run as written on the main thread, through accessor rows over the same storage. - **Workers:** @@ -56,7 +56,7 @@ Rule: anything that doesn't affect the API or correctness belongs in phase 2+. - **Subset errors** are reported at build time. An ESLint rule shares the same validator, so errors also show in the editor. - **GPU systems.** The built-in `gpuDevice` resource, `GPUDevice | null`, is the one device the world's GPU storage lives on, for compute and render alike: GPU buffers belong to one device, so a second device means a second world. It's an ordinary resource, given as `World.create({ gpuDevice: device })` or set later by a main-thread system; the world never destroys it. After each run the world follows it: a new device gets the GPU storage and `kernel: "gpu"` systems, after what's ahead on the old device is read back; with null, GPU kernels run on the CPU again. Render systems run only while there's a device, as if their `is` listed `gpuDevice`. A lost device makes the world set `gpuDevice` to null, as its own sync point, so nothing fails: GPU kernels run on the CPU from the CPU's copies, render systems are skipped, and systems watching `changed: [gpuDevice]` see it; GPU writes not yet read back are lost with the device. - **`kernel: "gpu"` declares a GPU kernel.** It runs as a WGSL compute shader compiled from the same Program IR as WASM in a world with a device, and as a `"cpu"` kernel in one without, so it stays portable. Each rule is checked as early as it can be known: types rule out lookups, change filters and commands; the build rejects loops and anything that keeps the body from lowering to a Program; registration rejects f64 or byte values, in every world, since such a declaration can never hold. - - **Render systems** (`System.render`) run once per run on the main thread in schedule order, read-only. They record into the run's encoder over the GPU copies of the components their accessors name, which are what they read, one table per matched archetype. `Gpu.wgsl(system)` returns their shaders' declarations: the world's bind group at the group they name and a function per component (`fn position(row: u32) -> vec3`), from schemas alone, so shaders can be written before a world exists. Pipelines take `Gpu.layout(device)`, one object per device, so they can be built before a world exists; bind groups and buffers stay the world's, so they can be replaced as storage grows. + - **Render systems** (`System.render`) run once per run on the main thread in schedule order, read-only. They record into the run's encoder over the GPU copies of the components their accessors name, which are what they read, one table per matched archetype; with `order`, a sorted index, one table of every matching row in its order, whose accessors resolve an instance's position through an order block of slot and row to its archetype's columns. `Gpu.wgsl(system)` returns their shaders' declarations: the world's bind group at the group they name and a function per component (`fn position(row: u32) -> vec3`), from schemas alone, so shaders can be written before a world exists. Pipelines take `Gpu.layout(device)`, one object per device, so they can be built before a world exists; bind groups and buffers stay the world's, so they can be replaced as storage grows. - **Residency:** a separate GPU heap holds copies of only the storages GPU systems use. Each copy tracks rows the CPU changed since its upload and rows the GPU wrote since its readback. GPU systems upload what the CPU changed first, in runs of the chunks it stamped since the last upload: the components GPU systems use keep ticks, and kernels writing them keep their chunk stamps. CPU systems read back what the GPU wrote first. A sync point reads back only archetypes whose existing rows it moves or writes (despawns, adds, removes); spawns only append rows, which are uploaded alone, and a copy that outgrows its block moves on the GPU. GPU-to-GPU chains and rendering never copy back. - **Change filters:** a GPU kernel's writes stamp each written column once with the run's tick, and `changed` filters pass a whole column when its stamp is newer than their `lastRun`, so writes on the GPU cost nothing per row on the CPU. - **Ordering:** a queue write lands before recorded but unsubmitted work, so recorded work is submitted before each upload. Each run's encoder is submitted at its end, before the sync point; `run` resolves then, not when the GPU finishes. @@ -98,14 +98,13 @@ Rule: anything that doesn't affect the API or correctness belongs in phase 2+. export const move = System.create({ name: "player/move", - input: { type: "object", properties: { distance: { type: "number" } } }, components: { has: [player], write: [position] }, - }, (row, { input }) => { /* … */ }); + }, (row, context, input: { readonly distance: number }) => { /* … */ }); export const main = System.create({ name: "main", systems: [input, physics, follow, render] }); ``` - **A system is the value its constructor returns.** `System.create` and `System.render` return a system typed `System`: its `name`, and the input `run` takes. Whatever else the value holds is the implementation's and isn't typed. `System.create`'s arguments tell its forms apart: one function is a body alone, two arguments a declaration and body, one array a composite's members and one object a composite's declaration, `{ systems, before, after, during, name }`. Its one-argument forms are a single overload, so a mistake in a body alone or a composite is reported at the mistake, not as no overload matching; the plugin compiles only the two-argument form as a kernel. They exist for body type inference, a binding the plugin can find, and a value the world can run. `Finalizer.create` and `Finalizer.changes` return a finalizer typed `Finalizer`, for other finalizers' `before` and `after`, and `Rule.create` a rule typed `Rule`, both their `name` alone. Nothing runs when they're called. - - **Declaration keys mirror the body's context:** `components` and `resources` say what a system accesses, each with access modes (`read`, `write`) and filters; `lookup`, `reduce` and `input` each appear in the context under the same name. + - **Declaration keys mirror the body's context:** `components` and `resources` say what a system accesses, each with access modes (`read`, `write`) and filters; `lookup` and `reduce` each appear in the context under the same name. - **Components:** a system matches entities that have every component in `read`, `write`, `has`, `added` and `changed`, and none in `not`. - **Conditions:** `resources: { is, not }` say whether a system runs at all: only while each resource in `is` is neither `null` nor `false`, and each in `not` is. Otherwise it's skipped, which is a success, so systems ordered after it still run, and it keeps its `lastRun`. A condition's type must allow `null` or `false`, so `0` and `""` never turn a system off; `is` makes it readable, narrowed, so it isn't also read or written. Expected absence is a condition, not a failure, which is why any real failure is still reported. Finalizers have no conditions: they release what entities hold whatever the world's state. - A component in `write` is readable too, so it's never also in `read`; listing it in both is rejected in types and at registration. The same holds for resources. @@ -115,18 +114,19 @@ Rule: anything that doesn't affect the API or correctness belongs in phase 2+. - **`kernel: "cpu"`** opts a system into being compiled, so it can run in chunks across workers; **`kernel: "gpu"`** narrows the subset so it can also run on the GPU. The term comes from CUDA, WebGPU and Taichi. - It's explicit rather than inferred, so a body that leaves the subset fails the build instead of silently running on the main thread. - Type-checked: values in `components` and `lookup` must be linear (numbers, booleans, entities, fixed-length tuples, and structs of those), and resources are read-only. Presence-only lists (`has`, `not`, `added`, `changed`) can name any component. A tuple or struct a command or spawn gives must go to a linear component, which registration checks. - - The build validator checks `input`, since `Schema.ToType` doesn't produce tuples. + - A kernel's input fields are structs. A TypeScript `number` is a float, so a field the kernel uses as an integer is typed `I32` or `U32`, which the plugin reads by name. - Without `kernel`, a system is plain JS on the main thread. - **`components` is optional.** A system without it runs once per run, on the main thread, and its body takes only the context. - - **`input`** is an object schema of named fields for a per-run, read-only value, and `run` requires it exactly when it's declared. - - A system with input can't use `added` or `changed`, because its `lastRun` is shared across runs that receive different input. - - **`target`** names an input field holding an entity: the system runs on that entity's row alone, once, when it's alive and matches `components`, and on nothing otherwise, so a stale entity from an event needs no check. Its writes are in place, stamped as any row's. It needs `components`, isn't a kernel, has no `up`, `order` or `reduce`, and runs only on the world's thread. + - **Input** is the body's parameter after its context, an object of named fields for a per-run, read-only value, typed as TypeScript: `run` requires it exactly when the body takes it. It's not part of the declaration, so its type is written once, where it's used. + - The plugin reads the schema the runtime needs from the parameter's type, by its syntax, before types are stripped (`compiler/input-schemas.ts`), and passes it to `create` after the body: type literals, aliases and interfaces of the module and those it imports, `number`, `string`, `boolean`, `null`, literals, unions, arrays, tuples of one type, `ReadonlySet`, `ReadonlyMap`, `(typeof X)[keyof typeof X]` for an `as const` object of literals, the package's `Entity` and math types, and `unknown`. A name neither declared nor imported is a global class, such as `OffscreenCanvas` or `HTMLCanvasElement`, which takes any value. Any other type fails the build at the parameter. A module whose input types come from a changed file is transformed again. Without the plugin, a body taking input throws at creation, as does one not written in place. + - Its checks are on the body, after the body is inferred: a system with input can't use `added` or `changed`, because its `lastRun` is shared across runs that receive different input; a `target` names an entity field of it; an observer system takes none. A check in the declaration's position can't see the input, which TypeScript infers only from the body, after the declaration. + - **`target`** names a field of the input holding an entity: the system runs on that entity's row alone, once, when it's alive and matches `components`, and on nothing otherwise, so a stale entity from an event needs no check. Its writes are in place, stamped as any row's. It needs `components`, isn't a kernel, has no `up`, `order` or `reduce`, and runs only on the world's thread. - **A composite's input** has every field of its members' inputs, and each member receives the fields it declares, so workers get only those. A field several members declare is one value for all of them, with the same schema in each: a different type is a compile-time error at each member that differs, and creating the composite throws naming both. A field optional in one and required in another is required. - `(system name, input)` is a replicable transaction. On a GPU, input maps to a uniform buffer. - A `Blob` input can only be stored or forwarded, because its contents can't be read synchronously. - - **Bodies are `(subject, context)`:** `(row, context)` for a system with `components` and for a finalizer, `(context)` for a system without `components`, `(changes, context)` for a changes finalizer, and `(gpu, context)` for a render system. + - **Bodies are `(subject, context, input)`:** `(row, context, input)` for a system with `components`, `(context, input)` for a system without, `input` left out by a body that takes none; `(row, context)` for a finalizer, `(changes, context)` for a changes finalizer, and `(gpu, context)` for a render system. - `row`: `{ readonly entity }`, plus `read` values (read-only) and `write` values (readable and writable). No component may be named `entity`. - - `context` holds each kind under the declaration's own key, present only when the declaration declares something of that kind: `resources` (values, assignable when in `write`), `lookup` (functions by name), `reduce` (contribution functions by resource) and `input`. It also has `commands`, except in GPU kernels and render systems. Keys of different kinds never share a namespace, so nothing can collide. + - `context` holds each kind under the declaration's own key, present only when the declaration declares something of that kind: `resources` (values, assignable when in `write`), `lookup` (functions by name), `reduce` (contribution functions by resource) and `up`. It also has `commands`, except in GPU kernels and render systems. Keys of different kinds never share a namespace, so nothing can collide. - `commands`: `spawn(components)` returns the world's spawner of exactly those components, the same function for a world and set of components each time, which takes their values and returns a real `Entity`; `despawn`; `add(entity, component, value)`, the value left out for a tag, or `add(entity, components, values)`, adding or replacing and marking the components changed; and `remove(entity, component)` or `remove(entity, components)`. A spawn's or list add's values are exact against its components: a tag needs no value, a component whose type allows `null` may be left out to start at `null`, every other one is required, and leaving one out or giving `undefined` never leaves the component out; a list that isn't a tuple takes any of its components' values. The two forms of `add` and `remove` are told apart by whether the second argument is a symbol. They apply in the order they were issued. They take only components that aren't derived, which is checked in types and at runtime, where a key that isn't a component commands may name, or a list naming one twice, throws at first use before anything is queued. Values are read only by the components' keys; a string key in the values of a spawn, or of an add whose components are a list as written, an array literal or a module constant of one or of `Archetype.create`, fails the build; an add of one component takes an object literal as its value. - **Writes to the current row are direct and immediate.** Commands are only for structural changes and for writes to other entities. - **Names are labels,** optional and not unique, so two libraries can both name a system `"linked"`; namespacing them (`"physics/integrate"`) keeps labels short. A system that declares nothing is created from its body alone, `System.create(({ commands }) => …)`, which runs once per run. Ordering constraints list systems; a system can't list itself, since it doesn't exist while it's declared. @@ -149,7 +149,7 @@ Rule: anything that doesn't affect the API or correctness belongs in phase 2+. - Rows on or under a parent cycle don't run; reports don't list them yet. - **Order:** rows run depth by depth, and within a depth in groups of one archetype whose parents share one archetype, each in storage order. Groups of a depth never read each other's writes, so values don't depend on the grouping; commands and reductions follow it, deterministically. The order, each row's parent row and the children of each parent change only at sync points, since only those move rows or change references. - **Kernels read `up` straight from the parent's columns:** a call covers one group, so it binds the parent archetype's columns once, and each row reads its parent's leaf at the parent's row. A leaf's stride and offset depend only on its component's layout, so they're the same in every archetype. `up === null` holds for the whole call. - - **Filtered runs push changes down:** rows are marked from chunk ticks of their own filters and their parent reference, from the sync points' journal of entities whose parent, matching or archetype changed, and as children of parents whose read values changed; each row that runs marks its children, so a run costs a check of each chunk plus the subtrees that changed, and nothing when nothing did. + - **Filtered runs push changes down:** rows are marked from chunk and block ticks of their own filters and their parent reference, from the sync points' journal of entities whose parent, matching or archetype changed, and as children of parents whose read values changed; each row that runs marks its children, so a run costs a check of each chunk plus the subtrees that changed, and nothing when nothing did. - **`kernel: "gpu"` systems may declare `up`;** what `read` lists must be made of 4-byte scalars, as the rest of a GPU kernel's values are. - **In a main-thread body, `context.up` is one reused object,** valid only while its row runs, as `row` is. - **World** (`src/interface/world/`): @@ -196,7 +196,7 @@ Rule: anything that doesn't affect the API or correctness belongs in phase 2+. - **`within: host`:** the component exists only while its host does, whatever removes the host; transitively; in command order, so removing a host and adding it back ends its dependents. A derived host a rule keeps never leaves. A dependent added without its host is dropped. - **`reference: "owner" | "weak"`** on an entity-valued component: when the target despawns, or a reference is written to an entity that won't be alive, `owner` despawns the holder and `weak` removes the component, transitively. Only the value the plan leaves counts, so a reference pointed elsewhere in the same sync point stays. References change only through commands (systems can't list them in `components.write`), which keeps the index of holders by target exact. Without `reference`, a reference dangles and a lookup of it returns null. - **Finalizers are the only removal mechanism.** `Finalizer.create({ name, components: { read, has, not }, lookup, resources, before, after }, (row, { resources, lookup, commands }) => …)`. They run inside the sync point, on the world's thread, against the final plan and live values from before Apply, with no copying. They are for releasing resources (GPU/DOM handles), updating indexes and adjusting aggregates, and they never miss a removal. - - **On another thread:** one whose objects live on another thread than the world's, which a joined thread may fill, runs there after the sync point, with its declared struct reads copied and its thread's objects as of before Apply ([`threads.md`](threads.md)). So it issues no commands, writes no resources and looks up nothing, in types and at creation; a changes finalizer watches and reads only the world's thread's values; and finalizers on different threads can't be ordered with `before` or `after`. + - **On another thread:** one whose objects live on another thread than the world's, which a joined thread may fill, runs there after the sync point, with its declared struct and string reads and its resources copied and its thread's objects as of before Apply ([`threads.md`](threads.md)). So it issues no commands, writes no resources and looks up nothing, in types and at creation; a changes finalizer watches and reads only the world's thread's values; and finalizers on different threads can't be ordered with `before` or `after`. - **A stop:** the entity matched before the sync point and is despawned, loses a required component by any cause (a command, a rule, `within` or a reference), has one removed and added again, or gains an excluded one. An entity spawned and despawned in the same sync point never matched. - **Deferred handling:** a consumer that handles removals later appends what it needs (IDs, values) to a queue it owns during Finalize, and works through it whenever it runs. A consumer that tracks stops keeps state about the entities it saw start, and ignores stops for entities it doesn't know. - **Run once, against the final plan.** Their structural commands go to the **next** sync point. Re-planning would let a later round bring back a component an earlier round already finalized, for example a freed handle on an entity that survives. @@ -266,11 +266,11 @@ Measured with `pnpm perf` (benchmarks bundled as in production, on plain node) a - **Commands:** typed parallel arrays with explicit counts, reused per system across runs; ids resolved when issued, and each list command records its shape. Kernels issue commands through `begin`/`value`/`component` by key index, allocating nothing; the compiler gives every component of a command's list a value, `undefined` for one it leaves out, which the runtime writes as the component's start. Workers write theirs into a command ring in shared memory, structs as bytes in their storages' layout and other values as tags, which the world drains at the sync point into buffers whose struct values stay bytes in the ring, copied by word as they apply (`threads.md`). - **Kernel spawns:** staged, not commands. The compiler lists each kernel's spawn lists, and each spawn calls its list's spawner with one value per component, generated per list as main-thread spawners are. The main thread stages into batches per runner, and each chunk's buffer names the rows staged while it was current; a worker stages per task and copies its batches into its command ring after the chunk's commands. The sync point commits staged rows in merge order, right after main-thread batches and before any command, so row order doesn't depend on which thread ran a chunk. Kernel spawners reserve entities in blocks, from 8 positions doubling to 4096, so workers don't contend for the cursor; each run or task ends its block, and the sync point returns the positions it left unused to the free list with their generations. - **Sync point:** plans straight over the queued buffers into reusable typed arrays, through cached archetype edges (arrays by component id), per-pair move lists, and for each list add or remove a table from source archetype to target; archetypes grow once per sync point; common layouts are written inline rather than through layout closures. Applying groups entries by source and target archetype: each pair copies the columns its archetypes share column by column for its block of rows and writes derived defaults once per column, then each entry's values are written. Rows left are removed last entry first, so an archetype moving out whole drops its rows without copying them; a row moved into a hole that another entry still leaves takes that entry's place. -- **Chunk ticks:** every storage keeps the newest `added`/`changed` tick per 1024-row chunk, stamped with every row tick, so change filters skip unchanged chunks. Stamps every row reaches are made once per kernel call, per chunk. +- **Chunk and block ticks:** every storage keeps the newest `added`/`changed` tick per 1024-row chunk and per 64-row block, both stamped with every row tick and raised by a row copied in, in shared memory beside the row ticks, so change filters skip unchanged chunks, then the unchanged blocks of a chunk that changed: finding a chunk's passing rows costs its sixteen block ticks and the rows of the blocks that changed, not its 1024 row ticks. Writers store a row's tick, then its block's, then its chunk's, the reverse of the order filters read them. Stamps every row reaches are made once per kernel call, per block and chunk. - **Ticks only where observed:** a component has tick columns only once some registered system filters on its `added` or `changed`, and resources always do; until then nothing allocates or stamps them, and once started they stay, bounded by the components. Columns start at the current tick, so a new system, starting with `lastRun = 0`, sees every row once. Kernels also drop `changed` stamps for components nothing watches: a count per component of the registered systems filtering on its `changed` and of the finalizers' indexes following it, an index's for the world's life. A count turning positive reloads the kernels that dropped its stamps; one falling to zero reloads nothing, and kernels drop its stamps when they next load for another reason, so a filter coming and going, as a system run once does, reloads nothing. - **What a system holds lives as long as its value:** a world keeps its schedules and runners by system value, weakly, so a system nothing else references, such as one made for a single run, is collected with its runner, command buffers and `lastRun`, while a module's systems keep theirs between runs. What a runner holds beyond its JS values (reduction partials and `up` orders in the arena, lookup directories, GPU records, its hook after each sync point, its count of watched components) is released once its system is collected, through a finalization registry whose records hold nothing of the system. Runs read a schedule's runners again only after one was replaced or removed, as placements, device changes and `remove` do; a removed one reads as a runner that runs nothing, and a registered composite holding a system forgotten since registers again on its next run, keeping its other members' runners. Ids of keys, added resources and tick columns stay, bounded by the components; the registry forgets a collected item without an id, its label with it. `lifetime.test.ts` runs 10,000 new systems, each once, and finds every table and the heap as they were. - **Kernels where WASM can't compile** (a content security policy without `'wasm-unsafe-eval'`) run their compiled JS. -- **Change filters inside WASM kernels:** one call per archetype or worker chunk; chunks and rows are skipped in WASM. With SIMD, outside `up`, each aligned block of eight rows checks its ticks at once and is skipped when none passes. +- **Change filters inside WASM kernels:** one call per archetype or worker chunk; chunks, blocks and rows are skipped in WASM. An outer loop skips chunks whose ticks aren't newer, reads a passing chunk's block ticks into a mask, and finds each run of passing blocks with `ctz`; an inner loop runs the run's rows, at one comparison a row. A chunk with more than half its blocks passing is one run. With SIMD, outside `up`, each aligned group of eight rows checks its ticks at once and is skipped when none passes. - **Reductions in WASM kernels:** sums, products, minimums and maximums fold consecutive rows into four lanes in turn, so a fold doesn't wait on the one before. - **Holders:** each reference component keeps the holders of each target as linked lists by entity index, updated from the plan's reference changes at Apply, so cascades find a despawned target's holders without a scan. - **Lookups in WASM kernels:** a directory per lookup in shared memory (per archetype: a match flag, column addresses, and for `changed` each watched component's tick column, 0 without ticks), rebuilt at sync points and once a registration starts ticks; finds check index and generation against the shared entity table. A lookup with `changed` reads its system's `lastRun` after the lookup parameters, -1 once a GPU kernel stamped a whole watched column since; JS kernels on workers read it there too, and a JS kernel's result holds `changed` after its values. @@ -284,10 +284,10 @@ Measured with `pnpm perf` (benchmarks bundled as in production, on plain node) a - **The observer pass:** each list of observer systems keeps its runners, read again once runners change, and whether any looks up through an index, so an idle pass reads no index and no GPU state: `observer idle 100` about 80 ns an observer system a tick, and a run of observer systems alone skips its empty sync point: `run observer` 0.21 µs against 0.38–0.42 for `run idle`. One on a joined thread is posted only when its conditions hold and a filter may pass, by its resources' and matched chunks' ticks, into a block of its own, waking each thread once a pass; that thread reads the shared memory directly, the next writer copying what it hasn't read yet, or, for one reading only struct resources, reads copies the post holds, written into one of two blocks so the world never waits for it: `joined observer resource` 4.3–5.0 µs a tick and `joined observer 10K` 0.41–0.51 ms in node; 7.2–7.5 µs and 0.34–0.50 ms in chromium. - **Indexes:** - Keys are encoded as order-preserving words (f32 and f64 bits flipped, NaN last and -0 as +0, i32 with its sign bit flipped, booleans 0 or 1, object parts as ids interned per index), hashed by a multiply-xorshift mix, the same in JS and WASM. - - Hash indexes: two-word slots of hash and bucket beside bucket records of hash, segment and key, so a probe reads one line until a hash matches; each bucket a segment of one entries buffer with spare slots, its entries (entity index, archetype, row) in entity order; ticks by bucket for `changed`, membership's and each watched component's. Sorted indexes: blocks of at most 128 entries in key order, then entity order, listed in key order, with a fence of each block's last key and entity beside them, a Fenwick tree of block lengths for positions, and segment trees of block ticks, so a range's `changed` is O(log blocks), and the whole index's the tree's root; entries are padded to an even number of words, so a shift within a block moves 8-byte words, and a block split or dropped refreshes positions, fences and trees from its position on. An index without a key is a sorted table with no key words, its entries in entity order: an entity entering or leaving moves at most a block's entries, found by binary search by entity, and writes to what it holds by cost it nothing, since it reads no key ticks; a watched component's change stamps one block, found by chunk ticks. `keyless refresh 1M 1%` 187 ns an entity entering or leaving, `keyless spawn 1M 1%` 105 ns an entity spawned or despawned at the end of entity order, `keyless index idle` 7 ns an index, `keyless length` 5 ns a lookup reading `length` and `changed`. All in the arena. - - Kept current before each tier holding a user, from the journal and the ticks of key parts, which systems' writes and sync points' values mark per component, so an idle refresh reads no ticks: `index idle` 9 ns an index. Changed rows move their entries, short buckets one by one and long ones in one merge, sorted blocks split and dropped: `index refresh 1M 1%` 1.4 ms. Once the rows it lists pass an eighth of the index, it's built again. + - Hash indexes: two-word slots of hash and bucket beside bucket records of hash, segment and key, so a probe reads one line until a hash matches; each bucket a segment of one entries buffer with spare slots, its entries (entity index, archetype, row) in entity order; ticks by bucket for `changed`, membership's and each watched component's. A change to a bucket of at least 64 entries, in an index that isn't unique, may leave it unsorted, so moving an entity between two long buckets costs O(log bucket) to find it and O(1) amortized to move it, not a shift of the bucket: an entry leaving the part in entity order leaves a hole (archetype -1, its entity index kept for binary search) and one arriving out of order joins a tail, its position kept by entity index, leaving it by swap-remove; the record's length counts entries, not holes, so `length` and `changed` read no order. An unsorted bucket is sorted, its tail sorted aside, its holes closed and the tail merged in from the end, when a list on the world's thread first reads it by position, when the index's arrays are handed to a kernel or another thread (`writeIndexParameters`), when its segment is full and has holes, closing them in place rather than growing it, and once its holes and tail pass four times its entries; a segment an unsorted bucket outgrows gets half its length more spare slots, so sorts to make room are O(1) amortized; long moves copy 8-byte words, since shared memory copies an element at a time. `bucket move 100K length` 5.5 µs a move between two buckets of 50K and a run reading both lengths, against 423 µs kept in order; `bucket move 100K in order` 69 µs, both buckets sorted again for a read of their first rows. Sorted indexes: blocks of at most 128 entries in key order, then entity order, listed in key order, with a fence of each block's last key and entity beside them, a Fenwick tree of block lengths for positions, and segment trees of block ticks, so a range's `changed` is O(log blocks), and the whole index's the tree's root; entries are padded to an even number of words, so a shift within a block moves 8-byte words, and a block split or dropped refreshes positions, fences and trees from its position on. An index without a key is a sorted table with no key words, its entries in entity order: an entity entering or leaving moves at most a block's entries, found by binary search by entity, and writes to what it holds by cost it nothing, since it reads no key ticks; a watched component's change stamps one block, found by chunk ticks. `keyless refresh 1M 1%` 187 ns an entity entering or leaving, `keyless spawn 1M 1%` 105 ns an entity spawned or despawned at the end of entity order, `keyless index idle` 7 ns an index, `keyless length` 5 ns a lookup reading `length` and `changed`. All in the arena. + - Kept current before each tier holding a user, from the journal and the ticks of key parts, which systems' writes and sync points' values mark per component, so an idle refresh reads no ticks: `index idle` 9 ns an index. Changed rows move their entries, short buckets kept in entity order and long ones left unsorted, sorted blocks split and dropped: `index refresh 1M 1%` 1.4 ms. Once the rows it lists pass an eighth of the index, it's built again. - Built again in four phases over fixed pieces of entity indexes and fixed partitions of the slots, so the index is the same whatever the number of workers: counting each piece's members by partition, listing them into partition order through code generated for the key's leaves, grouping each partition's keys into buckets, and filling its range of the table. Workers claim phases through their task rings; with no pool the world's thread runs them: `index rebuild 1M` 27 ms on the world's thread, 6.7 ms across nine workers. Sorted indexes are built on the world's thread, by LSD radix sort over 16-bit digits. - - Main-thread lookups: the plugin rewrites each call into a site of its own, literal tuples and comparisons flattened into scalar arguments; a hash site's encoding, hash and probe are generated code, a loop where a content security policy forbids it. Lists, and the rows they give, share one hidden class each, their accessors the same functions reading their state through `this`; a site's lookup makes its rows ahead of `at`, for a list's first 256 positions and a sorted list's last 256, and on the world's thread rows check one count of replaced arrays rather than their archetype's version: `unique lookup 1M` 44 ns a lookup, `children lookup` 13 ns a child, `sorted range 1M` 0.56 µs a range of ten, `sorted page 100K` 5 µs a change against 1.1 ms for `select page 100K`. + - Main-thread lookups: the plugin rewrites each call into a site of its own, literal tuples and comparisons flattened into scalar arguments; a hash site's encoding, hash and probe are generated code, a loop where a content security policy forbids it. Lists, and the rows they give, share one hidden class each, their accessors the same functions reading their state through `this`; a site's lookup makes its rows ahead of `at`, for a list's first 256 positions and a sorted list's last 256, and rows check one count of replaced arrays rather than their archetype's version, on a joined thread settled once a run: `unique lookup 1M` 44 ns a lookup, `children lookup` 13 ns a child, `sorted range 1M` 0.56 µs a range of ten, `sorted page 100K` 5 µs a change against 1.1 ms for `select page 100K`. - Kernels compile each lookup call into a probe the world binds to its index's layout; WASM probes in straight-line code, reading the index's arrays at addresses each run's parameters give, on the world's thread and pool workers; joined threads read a record of them the world writes before each task: `grid neighbors 100K` 3.7 µs a row on the world's thread, 179 ns a row in a kernel across the pool. - `order`: main-thread bodies walk the index's entries; filtered runs sort passing rows' positions while few pass and scan past an eighth; kernels run lists of rows as `up`'s do: `order run 100K` 31 ns a row against 28 in storage order. - **`up`:** @@ -295,7 +295,7 @@ Measured with `pnpm perf` (benchmarks bundled as in production, on plain node) a - Each system keeps a tree by entity index (parent, level, children as linked lists), read from the journal; levels settle by walking up to a root or a settled entity, which also finds cycles. - The order: groups of one depth, archetype and parent archetype, as segments of one arena buffer with spare slots, patched in two phases by binary search and block moves within a group, so rows keep storage order. - Rebuilt in full, in O(rows), when the journal dropped entries a system hadn't read, or when more than a quarter of the order's rows would move. - - Filtered runs mark entities into lists by depth, from chunk ticks and the journal; a depth with few marks is sorted, one with many is scanned in order. + - Filtered runs mark entities into lists by depth, from chunk and block ticks and the journal; a depth with few marks is sorted, one with many is scanned in order. - WASM kernels run a group's row list, with the parent archetype's column addresses as parameters. - Workers take one depth at a time as claimed pieces, with a barrier per depth; depths too small to split run on the main thread without a wait. - The GPU runs a dispatch per group, in order, within one pass; row lists live in the heap and upload only when the order changes. diff --git a/packages/ecs/followups.md b/packages/ecs/followups.md index 566e8c1e..1028b718 100644 --- a/packages/ecs/followups.md +++ b/packages/ecs/followups.md @@ -4,7 +4,6 @@ Open items awaiting a decision or work. Remove an item once it's decided or done ## Queued (decided, not started) - ## Decisions for the maintainers - **An index lookup's `changed: true` is a change filter,** as a resource in `changed` is: when the index changed every @@ -35,12 +34,9 @@ Open items awaiting a decision or work. Remove an item once it's decided or done per-row object such as a texture's bind group, and a row body can't learn its GPU row. A consumer drawing an image per row keeps its own instance buffer, 4,000 draws in 2.1 ms a tick. Possible: tables giving their entities in GPU row order. Not needed until a consumer's profile says so. -- **Paint order across render tables:** a render system draws each matching archetype as its own table in row order, - so 2D content of several kinds (shapes, images, paths) can't be blended back to front without the consumer keeping - its own sorted instance list. Proposal: `order: index` on `System.render`, giving one table spanning its archetypes - whose `bind` sets a buffer of the index's rows in sorted order with each archetype's column base addresses; every - column already lives in one heap, so accessors resolve a sorted instance to its row. Not needed while consumers draw - every kind with one shader from their own list. +- **Descending sorted indexes:** an ordered render reads its index front to back, so a consumer painting back to front + maps `instance_index` to `ecs_count() - 1u - instance_index`, which costs nothing. Possible: a `descending` option on + sorted indexes. Not needed until a consumer's profile or readability says so. - **Report retention:** the last 256 runs keep their reports (`retainedRuns` in `reports.ts`), a fixed internal constant; a later run takes the oldest one's entity. Recommended: keep it internal until a consumer needs another; then a world option or resource. @@ -82,27 +78,33 @@ Open items awaiting a decision or work. Remove an item once it's decided or done - **An observer system on a joined thread that reads a string resource doesn't post copies of its resources:** nothing waits for such posts, and a handle there could outlive its record, so it posts as row systems do. Recommended: accept. +## API optimizations + +The plugin is required, so it can derive what consumers now restate. Each is a contract change, reviewed one at a +time; inferred values are emitted as the same literals the runtime takes today, so none costs anything at runtime. + +1. **Declarations from the augmentation.** The pre-strip pass reads `interface Components`, `Resources` and `Traits` + and emits the `register` calls: the schema from the type (`F32`, `I32`, `U32` for precision and width), the traits + as declared, and defaults as a trait. About 200 calls in consumer code restate their augmentation. +2. **Names from bindings.** `name` on systems, composites, indexes and finalizers defaults to the plugin's id + (`#`), which 218 of 325 names restate; `Symbol` descriptions likewise. +3. **Reductions declared once.** A system's `reduce` drops the operator, which is the resource's `reduce` trait, and + its keys come from the body's `reduce[key](…)` calls. +4. **Kernels by default.** A body in the kernel subset compiles as a `"cpu"` kernel unflagged; `kernel: "cpu"` + asserts it, `"gpu"` chooses the backend, `false` runs it as written. Needs a measured rule for small runs, and + accepts results that aren't bit-exact with plain JS. +5. **Access inferred from bodies.** `components`, `resources`, `lookup` and `up` read and write lists come from the + body where `usedBy` follows every use, explicit otherwise. Bodies' rows are then typed over every component, so + it needs the planned ESLint rule to keep errors in the editor. + ## Findings from the todo sample (`src/samples/todo/README.md`) 2. **Observer systems can't declare `up`**, and `order` can't be combined with `up`, so views as separate entities don't work as bindings. Proposal: allow `up` in observer systems, since it only reads; let `order` take an index over the entity a reference names. -4. **Low-cardinality hash indexes cost O(bucket) per change.** Buckets are kept in entity order and shifted on removal: - one toggle costs 26, 62 and 400 µs at 1K, 10K and 100K. Proposal: unordered buckets with swap-remove and a position - map, or a count-only index. -5. **Change filters scan the whole dirty chunk** (1,024 row ticks), once per system that filters on `changed`. - Proposal: a second level of change ticks, one per 32–64 rows. 6. **Nothing awaits the next tick.** Tests watch `clock` with an observer system. Proposal: a test helper in `src/observe`. -## Findings from strings in shared memory - -- **A joined finalizer read a resource of its thread as undefined:** in `remote-finalizers.test.ts`, "that throw are in - a later run's report", the finalizer's `resources[tone]`, an object of its thread, is undefined for the stops of a - sync point soon after the thread joined, where the same finalizer reads it in the other cases; the fixture reads it - optionally now. The test checks no tone, so whether it was undefined before, as a string, wasn't seen. Worth finding - out whether a remote finalizer can run before its thread's resources reach its mirror. - ## Features to discuss - **Computed index keys:** an index keyed on a value computed from a component, such as a struct field @@ -120,26 +122,18 @@ Open items awaiting a decision or work. Remove an item once it's decided or done ## Performance -- **The filtered `up` benchmarks are slower than this machine's record at `e3c30aa1`:** `up one root changed` +49%, - `up filtered after a reparent` +45%, `up small tree in a big world` +79%. They're the same at `c6bb3c83`, before - strings, so the drift is older. Worth bisecting from `e3c30aa1`. - -- **A joined thread's first tail task once never finished,** under heavy machine load (two test suites at once): the - world waited on `channel.idle()` with task 1 posted and not done (`threads.browser.test.ts`, tails "see one tick's - values"). It didn't recur in 25 runs without the extra load, and the base failed once on another threads test in the - same comparison. Worth a stress run of the joined-thread suites under load. - -- **`transfer`, chromium:** its median varied with machine load. Check it alone on a quiet machine before the next - record. -- **`joined observer 10K`:** checking archetype versions once per post instead of per row would save some of the - thread's run (see perf.md "Next"). +- **A run allocates about 3.6 KB even for an empty system,** in node: the run path's async functions (`executeTiers`, + `perform`, `execute`, `runWith`), their promises and a few small objects. Make the path allocate nothing when nothing + waits, and measure with a sampling heap profile. +- **`transfer`, chromium:** three runs on a quiet machine gave 19.9, 22.5 and 16.1 ms, so its spread of about 20% is + its own, not the machine's. Find what varies (likely the time a worker takes to wake) before treating its changes + as signal. ## Earlier open work - Index step 5 items. - Starting the pool for large index builds. - `order` on workers. -- Typed transfers (Phase 7). - The `Finalizer` namespace. - Chromium timer clamps. - The `transfer` cost. diff --git a/packages/ecs/package.json b/packages/ecs/package.json index 8c605e03..be1bf7b7 100644 --- a/packages/ecs/package.json +++ b/packages/ecs/package.json @@ -1,6 +1,6 @@ { "name": "@adobe/ecs", - "version": "0.1.0", + "version": "0.1.1", "description": "Adobe ECS", "license": "MIT", "type": "module", @@ -30,7 +30,7 @@ ], "scripts": { "build": "cp ../../LICENSE . && tsc -b", - "dev": "tsc -b -w --preserveWatchOutput", + "dev": "tsc -b -w --preserveWatchOutput & vite --config vite.sample.config.ts", "typecheck": "tsc -b", "test": "pnpm exec playwright install chromium && vitest --run", "test:perf": "pnpm exec playwright install chromium && vitest --run --config vitest.perf.config.ts", diff --git a/packages/ecs/src/exports.test.ts b/packages/ecs/src/exports.test.ts index 161675a2..cc204203 100644 --- a/packages/ecs/src/exports.test.ts +++ b/packages/ecs/src/exports.test.ts @@ -116,8 +116,8 @@ declare module "@adobe/ecs" { } Component.register(health, { type: { type: "number" } }); export const both = System.create({ components: { read: [health], write: [health] } }, () => {}); -const near = System.create({ input: { type: "object", properties: { distance: { type: "number" } } } }, () => {}); -const far = System.create({ input: { type: "object", properties: { distance: { type: "string" } } } }, () => {}); +const near = System.create({}, (_context, input: { readonly distance: number }) => { void input; }); +const far = System.create({}, (_context, input: { readonly distance: string }) => { void input; }); export const listed = System.create([near, far]); export const declared = System.create({ systems: [near, far], after: [both] }); `; diff --git a/packages/ecs/src/guide/08-input.test.ts b/packages/ecs/src/guide/08-input.test.ts index 7b78cf13..e7bc7f05 100644 --- a/packages/ecs/src/guide/08-input.test.ts +++ b/packages/ecs/src/guide/08-input.test.ts @@ -15,20 +15,18 @@ Component.register(health, { type: { type: "number" } }); let player = Entity.none; const spawn = System.create(({ commands }) => { player = commands.spawn([health])({ [health]: 10 }); }); -// input is an object schema of named fields; each run passes a value of it. +// A body's input is its parameter after its context, an object of named fields; each run passes a value of it. const hit = System.create({ components: { write: [health] }, - input: { type: "object", properties: { amount: { type: "number" } } }, -}, (row, { input }) => { +}, (row, _context, input: { readonly amount: number }) => { row[health] -= input.amount; }); // target names an input field holding an entity: the system runs on that entity's row alone, or on none once it's gone. const heal = System.create({ components: { write: [health] }, - input: { type: "object", properties: { entity: Entity.schema, amount: { type: "number" } } }, target: "entity", -}, (row, { input }) => { +}, (row, _context, input: { readonly entity: Entity; readonly amount: number }) => { row[health] += input.amount; }); diff --git a/packages/ecs/src/guide/13-indexes.test.ts b/packages/ecs/src/guide/13-indexes.test.ts index dbc539f1..9c1d22f5 100644 --- a/packages/ecs/src/guide/13-indexes.test.ts +++ b/packages/ecs/src/guide/13-indexes.test.ts @@ -51,7 +51,7 @@ const list = System.create({ observer: true, components: { read: [player] }, ord // A unique index gives one row, or null. const byServerId = Index.create({ key: [serverId], unique: true }); let found: number | null = null; -const find = System.create({ input: { type: "object", properties: { id: { type: "integer" } } }, lookup: { byId: { index: byServerId } } }, ({ input, lookup }) => { +const find = System.create({ lookup: { byId: { index: byServerId } } }, ({ lookup }, input: { readonly id: number }) => { found = lookup.byId(input.id)?.entity ?? null; }); diff --git a/packages/ecs/src/guide/guide.md b/packages/ecs/src/guide/guide.md index 4554c0e3..e0f39521 100644 --- a/packages/ecs/src/guide/guide.md +++ b/packages/ecs/src/guide/guide.md @@ -35,7 +35,8 @@ How to use the ECS. Each excerpt comes from a test in this folder; the link unde ## Setup ```ts -// vite.config.ts: compiles kernels, checks each system's declaration against its body, and gives keys and systems ids. +// vite.config.ts: compiles kernels, checks each system's declaration against its body, reads the schema of each +// body's input from its type, and gives keys and systems ids. import { kernelPlugin } from "@adobe/ecs/compiler"; export default defineConfig({ plugins: [kernelPlugin()] }); ``` @@ -337,20 +338,18 @@ const step = System.create({ resources: { is: [map], not: [paused], write: [step ### 8. Input ```ts -// input is an object schema of named fields; each run passes a value of it. +// A body's input is its parameter after its context, an object of named fields; each run passes a value of it. const hit = System.create({ components: { write: [health] }, - input: { type: "object", properties: { amount: { type: "number" } } }, -}, (row, { input }) => { +}, (row, _context, input: { readonly amount: number }) => { row[health] -= input.amount; }); // target names an input field holding an entity: the system runs on that entity's row alone, or on none once it's gone. const heal = System.create({ components: { write: [health] }, - input: { type: "object", properties: { entity: Entity.schema, amount: { type: "number" } } }, target: "entity", -}, (row, { input }) => { +}, (row, _context, input: { readonly entity: Entity; readonly amount: number }) => { row[health] += input.amount; }); @@ -359,7 +358,7 @@ await world.run(hit, { amount: 3 }); await world.run(heal, { entity: player, amount: 1 }); ``` -[`08-input.test.ts`](08-input.test.ts#L19-L49) +[`08-input.test.ts`](08-input.test.ts#L18-L43) ### 9. Rules @@ -503,7 +502,7 @@ const list = System.create({ observer: true, components: { read: [player] }, ord // A unique index gives one row, or null. const byServerId = Index.create({ key: [serverId], unique: true }); let found: number | null = null; -const find = System.create({ input: { type: "object", properties: { id: { type: "integer" } } }, lookup: { byId: { index: byServerId } } }, ({ input, lookup }) => { +const find = System.create({ lookup: { byId: { index: byServerId } } }, ({ lookup }, input: { readonly id: number }) => { found = lookup.byId(input.id)?.entity ?? null; }); ``` diff --git a/packages/ecs/src/implementation/compiler/access.ts b/packages/ecs/src/implementation/compiler/access.ts index 6ae9f6b8..b2d6b0a7 100644 --- a/packages/ecs/src/implementation/compiler/access.ts +++ b/packages/ecs/src/implementation/compiler/access.ts @@ -50,9 +50,8 @@ export const access = (value: Value, key: Key, node: Node, scope: Scope, optiona if (name === "lookup") return { kind: "lookups" }; if (name === "reduce") return { kind: "reducers" }; if (name === "commands") return { kind: "commands" }; - if (name === "input" && signature.input) return { kind: "dynamic", expression: { kind: "args", path: [] } }; if (name === "up" && scope.unit.up !== null) return { kind: "found", lookup: signature.lookups.length, expression: local(scope.unit.up) }; - return fail(node, "the context holds the declared resources, lookup, reduce, input and up, and commands"); + return fail(node, "the context holds the declared resources, lookup, reduce and up, and commands"); case "resources": { const index = key.kind === "binding" ? signature.resources.indexOf(key.name) : -1; if (key.kind !== "binding" || index < 0) return fail(node, `${key.kind === "binding" ? key.name : "resources"} is not in resources read`); diff --git a/packages/ecs/src/implementation/compiler/columns.ts b/packages/ecs/src/implementation/compiler/columns.ts index e9b259bc..5deb8474 100644 --- a/packages/ecs/src/implementation/compiler/columns.ts +++ b/packages/ecs/src/implementation/compiler/columns.ts @@ -19,12 +19,14 @@ export const fieldColumn = (unit: Unit, component: string, path: Path): number = export const fieldRead = (unit: Unit, component: string, path: Path): Expression => ({ kind: "column", column: fieldColumn(unit, component, path) }); -/** A component's changed column, immediately followed by its dense chunk column. */ -export const changedColumns = (unit: Unit, component: string): { readonly column: number; readonly chunks: number } => { +/** A component's changed column, immediately followed by its dense chunk and block columns. */ +export const changedColumns = (unit: Unit, component: string): { readonly column: number; readonly chunks: number; readonly blocks: number } => { const changed = column(unit, `changed ${component}`, `{ kind: "changed", component: ${component} }`); const chunks = column(unit, `chunks ${component}`, `{ kind: "chunks", component: ${component} }`); + const blocks = column(unit, `blocks ${component}`, `{ kind: "blocks", component: ${component} }`); unit.chunks.add(chunks); - return { column: changed, chunks }; + unit.blocks.add(blocks); + return { column: changed, chunks, blocks }; }; /** A scalar leaf of a component of the row's parent, read through `up`. */ diff --git a/packages/ecs/src/implementation/compiler/compile-kernel.ts b/packages/ecs/src/implementation/compiler/compile-kernel.ts index 96637b0a..605ff984 100644 --- a/packages/ecs/src/implementation/compiler/compile-kernel.ts +++ b/packages/ecs/src/implementation/compiler/compile-kernel.ts @@ -12,14 +12,15 @@ import { localName, printStatements } from './print.js'; import { toProgram } from './program.js'; import { serialize } from './serialize.js'; import { unconditionalChunks } from './stamps.js'; -import { chunkShift, type Program } from '../world/kernel.js'; +import { blockShift, chunkShift, type Program } from '../world/kernel.js'; import { rootScope } from './scope.js'; import { signature } from './signature.js'; import { rowLabel, statement } from './statement.js'; import { createUnit } from './unit.js'; import type { Value } from './value.js'; -const parameterValues: readonly Value[] = [{ kind: "row" }, { kind: "context" }]; +/** A kernel body takes its row, its context and its input, the run's arguments. */ +const parameterValues: readonly Value[] = [{ kind: "row" }, { kind: "context" }, { kind: "dynamic", expression: { kind: "args", path: [] } }]; /** * One compiled kernel: `description(url)` is the source of its `KernelDescription`, with keys as the declaration's @@ -46,7 +47,7 @@ export const compileKernel = (declaration: ObjectExpression, body: Node, module: return fail(body, "a kernel body must be an inline function"); } if (body.async || body.generator) return fail(body, "a kernel body can't be async or a generator"); - if (body.params.length > 2) return fail(body, "a kernel body takes row and context"); + if (body.params.length > 3) return fail(body, "a kernel body takes its row, context and input"); const lists = createLists(module, link, [declaration, body]); const unit = createUnit(signature(declaration, module.bindings, lists), module, lists, expression, statement, link, target); const scope = rootScope(unit, false); @@ -59,7 +60,7 @@ export const compileKernel = (declaration: ObjectExpression, body: Node, module: statement({ type: "ExpressionStatement", expression: body.body, start: body.body.start, end: body.body.end }, scope, rows); } const statements = [...resolve(prelude), ...resolve(rows)]; - for (const chunks of unconditionalChunks(resolve(rows))) unit.stampedPerCall.add(chunks); + for (const [chunks, blocks] of unconditionalChunks(resolve(rows))) unit.stampedPerCall.set(chunks, blocks); const kernel = printStatements(unit, resolve(rows)); const preludeCode = printStatements(unit, resolve(prelude)); const accumulators = unit.accumulators.map(entry => entry.local); @@ -76,7 +77,7 @@ export const compileKernel = (declaration: ObjectExpression, body: Node, module: } const contributed = new Set(unit.accumulators.flatMap(entry => reduced[entry.reduced]?.resource ?? [])); requireExact(declaration, { query: unit.used, resources: { read: unit.used.resources, write: new Set(), reduce: contributed }, lookups: unit.used.lookups, up: unit.used.up }, module.bindings, lists); - const strided = columns.flatMap((_, index) => unit.chunks.has(index) ? [] : [index]); + const strided = columns.flatMap((_, index) => unit.chunks.has(index) || unit.blocks.has(index) ? [] : [index]); const locals = [ ...columns.map((_, index) => `const c${index} = columns[${index}];`), ...resources.map((_, index) => `const r${index} = resources[${index}];`), @@ -124,11 +125,13 @@ export const compileKernel = (declaration: ObjectExpression, body: Node, module: ...locals, preludeCode, ...accumulators.map((local, k) => `let ${localName(unit, local)} = partials[partial + ${k}];`), - // A kernel that runs a list of rows, which needn't be consecutive, stamps each row's own chunk. - ...listed ? [] : [...unit.stampedPerCall].map(chunks => - `if (start < end) for (let k = start >> ${chunkShift}, last = (end - 1) >> ${chunkShift}; k <= last; k++) c${chunks}[k] = tick;`), + // A kernel that runs a list of rows, which needn't be consecutive, stamps each row's own block and chunk. + ...listed ? [] : [...unit.stampedPerCall].flatMap(([chunks, blocks]) => [ + `if (start < end) for (let k = start >> ${blockShift}, last = (end - 1) >> ${blockShift}; k <= last; k++) c${blocks}[k] = tick;`, + `if (start < end) for (let k = start >> ${chunkShift}, last = (end - 1) >> ${chunkShift}; k <= last; k++) c${chunks}[k] = tick;`, + ]), listed - ? `${rowLabel}: for (let n = start; n < end; n++) { const i = rows[n], p = parents[n]; ${[...strided.map(index => `const x${index} = i * s${index} + o${index};`), ...[...unit.stampedPerCall].map(chunks => `c${chunks}[i >> ${chunkShift}] = tick;`)].join(" ")}` + ? `${rowLabel}: for (let n = start; n < end; n++) { const i = rows[n], p = parents[n]; ${[...strided.map(index => `const x${index} = i * s${index} + o${index};`), ...[...unit.stampedPerCall].map(([chunks, blocks]) => `c${blocks}[i >> ${blockShift}] = tick; c${chunks}[i >> ${chunkShift}] = tick;`)].join(" ")}` : `${rowLabel}: for (let ${["i = start", ...strided.map(index => `x${index} = start * s${index} + o${index}`)].join(", ")}; i < end; ${["i++", ...strided.map(index => `x${index} += s${index}`)].join(", ")}) {`, kernel, "}", diff --git a/packages/ecs/src/implementation/compiler/identify.test.ts b/packages/ecs/src/implementation/compiler/identify.test.ts index ef71e9db..f6149e95 100644 --- a/packages/ecs/src/implementation/compiler/identify.test.ts +++ b/packages/ecs/src/implementation/compiler/identify.test.ts @@ -4,7 +4,7 @@ import { parseAst } from 'vite'; import { idOfKey, takeIdOfDeclaration } from '../../interface/registry/ids.js'; import { estreeOf } from './estree.js'; import { identify as identifyModule, moduleIdOf } from './identify.js'; -import { attachKernelPath } from './paths.js'; +import { ownCopy } from './paths.js'; import { decodeMappings } from './source-map.js'; const parse = (code: string) => estreeOf(parseAst(code)); @@ -104,7 +104,7 @@ describe("identify, for systems, rules and finalizers", () => { expect(typeof declaration === "object" && declaration !== null ? [Reflect.get(declaration, "name"), takeIdOfDeclaration(declaration), takeIdOfDeclaration(declaration)] : declaration).toEqual(["move", "src/keys.ts#move", undefined]); }); test("identifies a kernel's call the kernel transform wrapped", async () => { - const result = await identify(`import { attachKernel as attachKernel$ } from ${JSON.stringify(attachKernelPath)};\nimport { System } from "@adobe/ecs";\nconst grow = attachKernel$(System.create({ name: "grow", kernel: "cpu" }, () => {}), {});`); + const result = await identify(`import { attachKernel as attachKernel$ } from ${JSON.stringify(ownCopy.attachKernel)};\nimport { System } from "@adobe/ecs";\nconst grow = attachKernel$(System.create({ name: "grow", kernel: "cpu" }, () => {}), {});`); expect(declared(result?.code)).toEqual([[`{ name: "grow", kernel: "cpu" }`, "src/keys.ts#grow"]]); }); test("follows the namespace to the module that exports it, under any local name", async () => { diff --git a/packages/ecs/src/implementation/compiler/identify.ts b/packages/ecs/src/implementation/compiler/identify.ts index a2e20589..b51acc34 100644 --- a/packages/ecs/src/implementation/compiler/identify.ts +++ b/packages/ecs/src/implementation/compiler/identify.ts @@ -5,7 +5,7 @@ import { posix } from 'node:path'; import { idsName } from '../../interface/registry/ids.js'; import { children, isFunction } from './estree.js'; import { moduleScope } from './module-scope.js'; -import { attachKernelPath } from './paths.js'; +import { isAttachKernelModule } from './paths.js'; import { insertInto, type Insertion, type SourceMap } from './source-map.js'; export type Parse = (code: string) => Program; @@ -76,7 +76,7 @@ const localsOf = async (program: Program, resolve: ResolveNamespace): Promise<{ for (const specifier of statement.specifiers) { if (specifier.type !== "ImportSpecifier" || specifier.imported.type !== "Identifier") continue; const imported = specifier.imported.name; - if (statement.source.value === attachKernelPath && imported === "attachKernel") attach.add(specifier.local.name); + if (imported === "attachKernel" && isAttachKernelModule(statement.source.value)) attach.add(specifier.local.name); else if (isNamespace(imported) && await resolve(statement.source.value, imported)) namespaces.set(specifier.local.name, imported); } } diff --git a/packages/ecs/src/implementation/compiler/input-schemas.test.ts b/packages/ecs/src/implementation/compiler/input-schemas.test.ts new file mode 100644 index 00000000..cbf6fda8 --- /dev/null +++ b/packages/ecs/src/implementation/compiler/input-schemas.test.ts @@ -0,0 +1,154 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, test } from 'vitest'; +import { parseAst } from 'vite'; +import { passInputSchemas, type InputHost } from './input-schemas.js'; + +/** Files by path; `@adobe/ecs` is the package, which also exports `System`. */ +const hostOf = (files: { readonly [path: string]: string }): InputHost => ({ + parse: (code, file) => parseAst(code, { lang: "ts" }, file), + resolve: async specifier => specifier.startsWith("./") && `/src/${specifier.slice(2).replace(/\.js$/, ".ts")}` in files ? `/src/${specifier.slice(2).replace(/\.js$/, ".ts")}` : null, + load: async file => files[file] ?? "", + isPackage: async specifier => specifier === "@adobe/ecs", + isSystem: async specifier => specifier === "@adobe/ecs", +}); + +const header = `import { System, type Entity, type Vec2, type F32, type I32 } from "@adobe/ecs";\n`; + +/** The schemas the pass gives each system of `body`, in order. */ +const schemasOf = async (body: string, files: { readonly [path: string]: string } = {}): Promise => { + const code = header + body; + const result = await passInputSchemas(code, "/src/main.ts", hostOf(files)); + if (result === null) return null; + if ("failure" in result) return { failure: result.failure.message }; + return [...result.code.matchAll(/, (\{.*\})\);?$/gm)].map(match => JSON.parse(match[1] ?? "null")); +}; + +describe("passInputSchemas", () => { + test("gives a system the schema of its body's parameter after its context", async () => { + expect(await schemasOf(`System.create({}, (_context, input: { readonly count: number; label: string; on: boolean; none: null }) => {});`)).toEqual([ + { type: "object", properties: { count: { type: "number" }, label: { type: "string" }, on: { type: "boolean" }, none: { type: "null" } } }, + ]); + }); + test("finds the input after the row when the declaration has components", async () => { + expect(await schemasOf(`System.create({ components: { read: [a] } }, (row, context, input: { dx: number }) => {});`)).toEqual([ + { type: "object", properties: { dx: { type: "number" } } }, + ]); + expect(await schemasOf(`System.create({ components: { read: [a] } }, (row, context) => {});`)).toBe(null); + }); + test("takes a declaration not written in place's input from a third parameter, or a second with a type", async () => { + expect(await schemasOf([ + `System.create(declaration, (row, context, input: { a: number }) => {});`, + `System.create(declaration, (context, input: { b: number }) => {});`, + `System.create(declaration, (row, context) => {});`, + ].join("\n"))).toEqual([{ type: "object", properties: { a: { type: "number" } } }, { type: "object", properties: { b: { type: "number" } } }]); + }); + test("passes nothing for a body without input", async () => { + expect(await schemasOf(`System.create({ resources: { read: [a] } }, ({ resources }) => {});\nSystem.create(context => {});`)).toBe(null); + }); + test("reads optional fields, literals, unions, arrays, tuples, sets and maps", async () => { + expect(await schemasOf(`System.create({}, (_c, input: { + mode?: "add" | "remove"; + level: 1 | -1; + tags: readonly string[]; + ids: ReadonlyArray; + pair: [number, number]; + seen: ReadonlySet; + names: ReadonlyMap; + maybe: string | null; + }) => {});`)).toEqual([{ + type: "object", + properties: { + mode: { anyOf: [{ const: "add" }, { const: "remove" }] }, + level: { anyOf: [{ const: 1 }, { const: -1 }] }, + tags: { type: "array", items: { type: "string" } }, + ids: { type: "array", items: { type: "integer", format: "entity" } }, + pair: { type: "array", items: { type: "number" }, minItems: 2, maxItems: 2 }, + seen: { type: "set", items: { type: "string" } }, + names: { type: "map", keys: { type: "integer", format: "entity" }, values: { type: "string" } }, + maybe: { anyOf: [{ type: "string" }, { type: "null" }] }, + }, + required: ["level", "tags", "ids", "pair", "seen", "names", "maybe"], + }]); + }); + test("gives the package's types their schemas", async () => { + expect(await schemasOf(`System.create({}, (_c, input: { entity: Entity; at: Vec2; cell: Vec2.I32; dt: F32; steps: I32 }) => {});`)).toEqual([{ + type: "object", + properties: { + entity: { type: "integer", format: "entity" }, + at: { type: "array", items: { type: "number", precision: 1 }, minItems: 2, maxItems: 2 }, + cell: { type: "array", items: { type: "integer", minimum: -2147483648, maximum: 2147483647 }, minItems: 2, maxItems: 2 }, + dt: { type: "number", precision: 1 }, + steps: { type: "integer", minimum: -2147483648, maximum: 2147483647 }, + }, + }]); + }); + test("takes any value for a global class or unknown", async () => { + expect(await schemasOf(`System.create({}, (_c, input: { canvas: OffscreenCanvas; bitmap: ImageBitmap; element: HTMLCanvasElement; anything: unknown }) => {});`)).toEqual([ + { type: "object", properties: { canvas: { allOf: [] }, bitmap: { allOf: [] }, element: { allOf: [] }, anything: { allOf: [] } } }, + ]); + }); + test("reads the values of an `as const` object, here or imported, as a union of their literals", async () => { + const files = { "/src/align.ts": `export const Align = { start: "start", end: "end", again: "start" } as const;\nexport type Align = (typeof Align)[keyof typeof Align];` }; + expect(await schemasOf([ + `import { Align } from "./align.js";`, + `const Level = { low: 1, high: -1, on: true } as const satisfies Record;`, + `type Level = (typeof Level)[keyof typeof Level];`, + `System.create({}, (_c, input: { align: Align; level: Level; inline: (typeof Level)[keyof typeof Level] }) => {});`, + ].join("\n"), files)).toEqual([{ + type: "object", + properties: { + align: { anyOf: [{ const: "start" }, { const: "end" }] }, + level: { anyOf: [{ const: 1 }, { const: -1 }, { const: true }] }, + inline: { anyOf: [{ const: 1 }, { const: -1 }, { const: true }] }, + }, + }]); + }); + test("follows aliases and interfaces, here and in modules imported", async () => { + const files = { + "/src/types.ts": `export type Move = { readonly by: number };\nexport interface Named { readonly name: string }\nexport { type Tagged } from "./more.js";\nexport * from "./all.js";`, + "/src/more.ts": `export interface Tagged extends Base { readonly tag: string }\ninterface Base { readonly base: boolean }`, + "/src/all.ts": `export type Spread = { readonly spread: number };`, + }; + expect(await schemasOf([ + `import type { Move, Named, Tagged, Spread } from "./types.js";`, + `type Local = Move & Named;`, + `System.create({}, (_c, input: Local) => {});`, + `System.create({}, (_c, input: Tagged) => {});`, + `System.create({}, (_c, input: Spread) => {});`, + ].join("\n"), files)).toEqual([ + { type: "object", properties: { by: { type: "number" }, name: { type: "string" } } }, + { type: "object", properties: { base: { type: "boolean" }, tag: { type: "string" } } }, + { type: "object", properties: { spread: { type: "number" } } }, + ]); + }); + test("names the module of a type it can't read", async () => { + const failed = await passInputSchemas(`${header}import type { Bad } from "./bad.js";\nSystem.create({}, (_c, input: Bad) => {});`, "/src/main.ts", hostOf({ "/src/bad.ts": `export type Bad = Pick;` })); + expect(failed !== null && "failure" in failed ? failed.failure.frames.map(frame => [frame.file, frame.line]) : failed).toEqual([["/src/bad.ts", 1]]); + expect(failed !== null && "failure" in failed ? failed.dependencies : failed).toEqual(["/src/bad.ts"]); + }); + test("fails, saying why, for what an input type can't be", async () => { + const failure = async (body: string) => { + const schemas = await schemasOf(body); + return typeof schemas === "object" && schemas !== null && "failure" in schemas ? schemas.failure : schemas; + }; + expect(await failure(`System.create({}, (_c, input) => {});`)).toContain("has a type"); + expect(await failure(`System.create({}, (_c, input: { a?: number | undefined }) => {});`)).toContain("optional"); + expect(await failure(`System.create({}, (_c, input: Pick) => {});`)).toContain("type arguments"); + expect(await failure(`type Tree = { readonly child: Tree };\nSystem.create({}, (_c, input: Tree) => {});`)).toContain("refers to itself"); + expect(await failure(`type Box = { readonly value: T };\nSystem.create({}, (_c, input: Box) => {});`)).toContain("generic"); + expect(await failure(`System.create({}, (_c, input: [number, string]) => {});`)).toContain("one type"); + expect(await failure(`System.create({}, (_c, input: number) => {});`)).toContain("object of named fields"); + expect(await failure(`const Loose = { a: "a" };\nSystem.create({}, (_c, input: { a: (typeof Loose)[keyof typeof Loose] }) => {});`)).toContain("as const"); + expect(await failure(`const Nested = { a: { b: 1 } } as const;\nSystem.create({}, (_c, input: { a: (typeof Nested)[keyof typeof Nested] }) => {});`)).toContain("literals"); + expect(await failure(`System.create({}, (_c, input: { a: Big["x"] }) => {});`)).toContain("keyof typeof"); + expect(await failure(`System.create({}, (_c, ...rest: [{ a: number }]) => {});`)).toContain("rest parameter"); + }); + test("inserts each schema after the body, mapping the rest of the line back", async () => { + const result = await passInputSchemas(`${header}System.create({}, (_c, input: { a: number }) => {}); const after = 1;`, "/src/main.ts", hostOf({})); + expect(result !== null && "code" in result ? result.code.split("\n")[1] : result).toBe(`System.create({}, (_c, input: { a: number }) => {}, {"type":"object","properties":{"a":{"type":"number"}}}); const after = 1;`); + expect(result !== null && "map" in result ? result.map.mappings.length > 0 : result).toBe(true); + }); + test("leaves calls of another System", async () => { + expect(await passInputSchemas(`import { System } from "elsewhere";\nSystem.create({}, (_c, input: { a: number }) => {});`, "/src/main.ts", hostOf({}))).toBe(null); + }); +}); diff --git a/packages/ecs/src/implementation/compiler/input-schemas.ts b/packages/ecs/src/implementation/compiler/input-schemas.ts new file mode 100644 index 00000000..9d4656f3 --- /dev/null +++ b/packages/ecs/src/implementation/compiler/input-schemas.ts @@ -0,0 +1,450 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import type { parseAst } from 'vite'; +import { Entity } from '../../interface/entity/entity.js'; +import * as math from '../../interface/math/index.js'; +import type { Schema } from '../../interface/schema/schema.js'; +import type { Failure } from './failure.js'; +import { insertInto, type SourceMap } from './source-map.js'; + +/** A TypeScript module's syntax, types included. */ +export type TypedProgram = ReturnType; +type Statement = TypedProgram["body"][number]; +type Alias = Extract; +type Interface = Extract; +type Import = Extract; +type Declarator = Extract["declarations"][number]; +type TypeNode = Alias["typeAnnotation"]; +type Member = Interface["body"]["body"][number]; +type Named = Extract["typeName"]; +/** Any node of a module, by the fields this pass reads. */ +type Node = { readonly type: string; readonly start: number; readonly end: number }; + +/** Parses a TypeScript module, types included. */ +export type ParseTyped = (code: string, file: string) => TypedProgram; +/** The file an import specifier resolves to from `importer`, or null when it's none this pass can read. */ +export type ResolveFile = (specifier: string, importer: string) => Promise; +/** A file's code. */ +export type LoadFile = (file: string) => Promise; +/** Whether an import specifier, resolved from `importer`, is one of the package's public modules. */ +export type IsPackage = (specifier: string, importer: string) => Promise; +/** Whether an import specifier, resolved from `importer`, is a module of the package exporting `System`. */ +export type IsSystem = (specifier: string, importer: string) => Promise; + +export type InputHost = { readonly parse: ParseTyped; readonly resolve: ResolveFile; readonly load: LoadFile; readonly isPackage: IsPackage; readonly isSystem: IsSystem }; + +/** Code with each system's input schema passed to its `create`, the files whose types it read, or why it failed. */ +export type InputResult = + | { readonly code: string; readonly map: SourceMap; readonly dependencies: readonly string[] } + | { readonly failure: Failure; readonly dependencies: readonly string[] }; + +/** Thrown inside the pass, and returned as its failure. */ +type Thrown = { readonly inputFailure: Failure }; +const isThrown = (value: unknown): value is Thrown => typeof value === "object" && value !== null && "inputFailure" in value; + +/** A value of any type: a global class, such as the platform's, or `unknown`. */ +const anything: Schema = { allOf: [] }; + +const supported = "a type literal, an alias or interface of one, `number`, `string`, `boolean`, `null`, a literal, a union, an array, a tuple of one type, `ReadonlySet`, `ReadonlyMap`, the values of an `as const` object, the package's `Entity` and math types, a global class or `unknown`"; + +/** The schema of the package's type at `path`, such as `["Vec2", "I32"]`: its namespace's `schema`. */ +const packageSchema = (path: readonly string[]): Schema | undefined => { + const [first, ...rest] = path; + let value: unknown = first === "Entity" ? Entity : first === undefined ? undefined : Reflect.get(math, first); + for (const name of rest) value = typeof value === "object" && value !== null ? Reflect.get(value, name) : undefined; + const schema: unknown = typeof value === "object" && value !== null ? Reflect.get(value, "schema") : undefined; + return isSchema(schema) ? schema : undefined; +}; + +// The package's own schemas are its exported constants. +const isSchema = (value: unknown): value is Schema => typeof value === "object" && value !== null; + +/** A parsed module: its declarations of types and its top-level constants by name, and its imports by local name. */ +type Module = { + readonly file: string; + readonly code: string; + readonly types: ReadonlyMap; + readonly constants: ReadonlyMap; + readonly imports: ReadonlyMap; + readonly exports: readonly Statement[]; +}; + +const declarationOf = (statement: Statement): Alias | Interface | null => { + const declared = statement.type === "ExportNamedDeclaration" ? statement.declaration : statement; + return declared?.type === "TSTypeAliasDeclaration" || declared?.type === "TSInterfaceDeclaration" ? declared : null; +}; + +/** The top-level constants a statement declares. */ +const constantsOf = (statement: Statement): readonly Declarator[] => { + const declared = statement.type === "ExportNamedDeclaration" ? statement.declaration : statement; + return declared?.type === "VariableDeclaration" && declared.kind === "const" ? declared.declarations.filter(declarator => declarator.id.type === "Identifier") : []; +}; + +const nameOfDeclarator = (declarator: Declarator): string => declarator.id.type === "Identifier" ? declarator.id.name : ""; + +const importsOf = (statement: Import): [string, { readonly source: string; readonly imported: string | null }][] => + statement.specifiers.map(specifier => [specifier.local.name, { + source: statement.source.value, + // A namespace import names the module itself. + imported: specifier.type === "ImportSpecifier" ? (specifier.imported.type === "Identifier" ? specifier.imported.name : specifier.imported.value) + : specifier.type === "ImportDefaultSpecifier" ? "default" : null, + }]); + +const moduleOf = (file: string, code: string, program: TypedProgram): Module => { + const types = new Map(); + const constants = new Map(); + const imports = new Map(); + for (const statement of program.body) { + const declared = declarationOf(statement); + if (declared !== null) types.set(declared.id.name, declared); + for (const declarator of constantsOf(statement)) constants.set(nameOfDeclarator(declarator), declarator); + if (statement.type === "ImportDeclaration") for (const [local, entry] of importsOf(statement)) imports.set(local, entry); + } + return { file, code, types, constants, imports, exports: program.body.filter(statement => statement.type === "ExportNamedDeclaration" || statement.type === "ExportAllDeclaration") }; +}; + +/** The line and column of an offset, 1-based line, 0-based column, as a failure's position. */ +const positionOf = (code: string, offset: number): { readonly line: number; readonly column: number } => { + const before = code.slice(0, offset); + const line = before.split("\n").length; + return { line, column: offset - (before.lastIndexOf("\n") + 1) }; +}; + +/** The module being transformed, and where in it the parameter whose type is being converted is. */ +type Site = { readonly root: string; at: number }; + +/** Fails at `node`, or for a type of another module at the parameter, with a frame naming where in that module. */ +const failAt = (site: Site, module: Module, node: Node, message: string): never => { + const { line, column } = positionOf(module.code, node.start); + const inRoot = module.file === site.root; + const thrown: Thrown = { inputFailure: { message, position: inRoot ? node.start : site.at, frames: inRoot ? [] : [{ name: "input type", file: module.file, line, column }] } }; + throw thrown; +}; + +/** Converts input types to schemas, reading the modules they're declared in through `host`. */ +const createConverter = (host: InputHost, root: string) => { + const site: Site = { root, at: 0 }; + const fail = (module: Module, node: Node, message: string): never => failAt(site, module, node, message); + const modules = new Map>(); + const read = new Set(); + const load = (file: string): Promise => { + const found = modules.get(file); + if (found !== undefined) return found; + const loading = (async () => { + read.add(file); + const code = await host.load(file); + return moduleOf(file, code, host.parse(code, file)); + })(); + modules.set(file, loading); + return loading; + }; + + /** + * What `name` names in `module`'s exports, of `kind`, following re-exports: a type's declaration, or a constant's + * declarator; null when it exports none by that name. + */ + const exported = async (module: Module, name: string, kind: (module: Module) => ReadonlyMap, seen: Set): Promise<{ readonly module: Module; readonly declaration: T } | "package" | null> => { + const key = `${module.file}#${name}`; + if (seen.has(key)) return null; + seen.add(key); + // A module's own declaration comes before what it re-exports. + const own = kind(module).get(name); + if (own !== undefined) return { module, declaration: own }; + for (const statement of module.exports) { + if (statement.type === "ExportNamedDeclaration") { + for (const specifier of statement.specifiers) { + const exportedName = specifier.exported.type === "Identifier" ? specifier.exported.name : specifier.exported.value; + if (exportedName !== name) continue; + const localName = specifier.local.type === "Identifier" ? specifier.local.name : specifier.local.value; + if (statement.source !== null) return await fromSource(module, statement.source.value, localName, kind, seen); + const local = kind(module).get(localName); + if (local !== undefined) return { module, declaration: local }; + const imported = module.imports.get(localName); + if (imported !== undefined && imported.imported !== null) return await fromSource(module, imported.source, imported.imported, kind, seen); + } + } + else if (statement.type === "ExportAllDeclaration" && statement.exported === null) { + const found = await fromSource(module, statement.source.value, name, kind, seen); + if (found !== null) return found; + } + } + return null; + }; + + const fromSource = async (module: Module, source: string, name: string, kind: (module: Module) => ReadonlyMap, seen: Set): Promise<{ readonly module: Module; readonly declaration: T } | "package" | null> => { + if (await host.isPackage(source, module.file)) return "package"; + const file = await host.resolve(source, module.file); + if (file === null) return null; + const target = await load(file); + return target === null ? null : await exported(target, name, kind, seen); + }; + const typesOf = (module: Module): ReadonlyMap => module.types; + const constantsIn = (module: Module): ReadonlyMap => module.constants; + + const nameParts = (name: Named): readonly string[] => + name.type === "Identifier" ? [name.name] : name.type === "TSQualifiedName" ? [...nameParts(name.left), name.right.name] : []; + + /** The schema of a type, read in `module`; `active` holds the declarations being converted, so recursion fails. */ + const convert = async (type: TypeNode, module: Module, active: ReadonlySet): Promise => { + switch (type.type) { + case "TSNumberKeyword": return { type: "number" }; + case "TSStringKeyword": return { type: "string" }; + case "TSBooleanKeyword": return { type: "boolean" }; + case "TSNullKeyword": return { type: "null" }; + case "TSUnknownKeyword": return anything; + case "TSParenthesizedType": return await convert(type.typeAnnotation, module, active); + case "TSTypeOperator": + return type.operator === "readonly" ? await convert(type.typeAnnotation, module, active) : fail(module, type, `\`${type.operator}\` isn't an input type: an input is ${supported}`); + case "TSLiteralType": { + const { literal } = type; + if (literal.type === "Literal" && (typeof literal.value === "string" || typeof literal.value === "number" || typeof literal.value === "boolean")) return { const: literal.value }; + if (literal.type === "UnaryExpression" && literal.operator === "-" && literal.argument.type === "Literal" && typeof literal.argument.value === "number") return { const: -literal.argument.value }; + return fail(module, type, "an input's literal type is a string, number or boolean"); + } + case "TSUnionType": { + const types = type.types.filter(member => member.type !== "TSUndefinedKeyword"); + if (types.length !== type.types.length) return fail(module, type, "an input field that may be missing is optional (`field?:`) rather than `| undefined`"); + const schemas = await Promise.all(types.map(member => convert(member, module, active))); + return { anyOf: schemas }; + } + case "TSArrayType": return { type: "array", items: await convert(type.elementType, module, active) }; + case "TSTupleType": { + const elements = type.elementTypes; + const first = elements[0]; + if (first === undefined) return fail(module, type, "an input's tuple has elements"); + const schemas = await Promise.all(elements.map(element => element.type === "TSOptionalType" || element.type === "TSRestType" || element.type === "TSNamedTupleMember" + ? fail(module, element, "an input's tuple has a fixed length, and its elements no names") : convert(element, module, active))); + const items = schemas[0]; + if (items === undefined || schemas.some(schema => JSON.stringify(schema) !== JSON.stringify(items))) return fail(module, type, "an input's tuple has elements of one type"); + return { type: "array", items, minItems: elements.length, maxItems: elements.length }; + } + case "TSTypeLiteral": return await objectOf(type.members, [], module, active, type); + case "TSIntersectionType": { + const parts = await Promise.all(type.types.map(member => convert(member, module, active))); + const properties: { [field: string]: Schema } = {}; + const required: string[] = []; + let optional = false; + for (const part of parts) { + if (!("type" in part) || part.type !== "object") return fail(module, type, "an input's intersection joins object types"); + for (const [field, schema] of Object.entries(part.properties)) properties[field] = schema; + if (part.required === undefined) required.push(...Object.keys(part.properties)); + else { + optional = true; + required.push(...part.required); + } + } + return optional ? { type: "object", properties, required } : { type: "object", properties }; + } + case "TSTypeReference": return await referenced(type, module, active); + case "TSIndexedAccessType": return await valuesOf(type, module); + default: return fail(module, type, `an input is ${supported}`); + } + }; + + const objectOf = async (members: readonly Member[], inherited: readonly Schema[], module: Module, active: ReadonlySet, at: Node): Promise => { + const properties: { [field: string]: Schema } = {}; + const required: string[] = []; + let optional = false; + for (const schema of inherited) { + if (!("type" in schema) || schema.type !== "object") return fail(module, at, "an input interface extends object types"); + for (const [field, property] of Object.entries(schema.properties)) properties[field] = property; + if (schema.required === undefined) required.push(...Object.keys(schema.properties)); + else { + optional = true; + required.push(...schema.required); + } + } + for (const member of members) { + if (member.type !== "TSPropertySignature" || member.computed) return fail(module, member, "an input's fields are named properties"); + const { key } = member; + const field = key.type === "Identifier" ? key.name : key.type === "Literal" && typeof key.value === "string" ? key.value : fail(module, member, "an input's fields are named properties"); + if (member.typeAnnotation === null) return fail(module, member, `input field "${field}" has a type`); + properties[field] = await convert(member.typeAnnotation.typeAnnotation, module, active); + if (member.optional) optional = true; + else required.push(field); + } + return optional ? { type: "object", properties, required } : { type: "object", properties }; + }; + + const declared = async (declaration: Alias | Interface, module: Module, active: ReadonlySet, at: Node): Promise => { + if (active.has(declaration)) return fail(module, at, `input type ${declaration.id.name} refers to itself`); + if (declaration.typeParameters !== null) return fail(module, at, `input type ${declaration.id.name} is generic, which an input type isn't`); + const inner = new Set(active).add(declaration); + if (declaration.type === "TSTypeAliasDeclaration") return await convert(declaration.typeAnnotation, module, inner); + const inherited = await Promise.all(declaration.extends.map(async heritage => { + const { expression } = heritage; + if (expression.type !== "Identifier" || heritage.typeArguments !== null) return fail(module, heritage, "an input interface extends named, non-generic types"); + return await named([expression.name], module, inner, heritage); + })); + return await objectOf(declaration.body.body, inherited, module, inner, declaration); + }; + + /** The schema of the type `path` names in `module`. */ + const named = async (path: readonly string[], module: Module, active: ReadonlySet, at: Node): Promise => { + const [first, ...rest] = path; + if (first === undefined) return fail(module, at, `an input is ${supported}`); + const local = module.types.get(first); + if (local !== undefined && rest.length === 0) return await declared(local, module, active, at); + const imported = module.imports.get(first); + if (imported !== undefined) { + if (await host.isPackage(imported.source, module.file)) { + const schema = packageSchema(imported.imported === null ? rest : [imported.imported, ...rest]); + return schema ?? fail(module, at, `${path.join(".")} is no type of the package an input can take: it takes Entity and the math types`); + } + if (imported.imported === null || rest.length > 0) return fail(module, at, `input type ${path.join(".")} is read through a namespace, which this pass doesn't follow; import it by name`); + const found = await fromSource(module, imported.source, imported.imported, typesOf, new Set()); + if (found === "package") { + return packageSchema([imported.imported]) ?? fail(module, at, `${imported.imported} is no type of the package an input can take: it takes Entity and the math types`); + } + if (found === null) return fail(module, at, `input type ${first} isn't a type alias or interface its module exports, which this pass can read`); + return await declared(found.declaration, found.module, active, at); + } + // Neither declared nor imported, it's a global, which TypeScript checked exists: a class, such as the platform's. + return anything; + }; + + const referenced = async (type: Extract, module: Module, active: ReadonlySet): Promise => { + const path = nameParts(type.typeName); + const parameters = type.typeArguments?.params ?? []; + const [only] = path; + if (path.length === 1 && only !== undefined && !module.types.has(only) && !module.imports.has(only)) { + const [first, second] = parameters; + if ((only === "ReadonlyArray" || only === "Array") && parameters.length === 1 && first !== undefined) return { type: "array", items: await convert(first, module, active) }; + if ((only === "ReadonlySet" || only === "Set") && parameters.length === 1 && first !== undefined) return { type: "set", items: await convert(first, module, active) }; + if ((only === "ReadonlyMap" || only === "Map") && parameters.length === 2 && first !== undefined && second !== undefined) { + return { type: "map", keys: await convert(first, module, active), values: await convert(second, module, active) }; + } + } + if (parameters.length > 0) return fail(module, type, `input type ${path.join(".")} takes type arguments, which an input type doesn't`); + return await named(path, module, active, type); + }; + + /** `(typeof X)[keyof typeof X]`: the values of the `as const` object `X`, as a union of their literals. */ + const valuesOf = async (type: Extract, module: Module): Promise => { + const queried = (node: TypeNode): string | null => { + const inner = node.type === "TSParenthesizedType" ? node.typeAnnotation : node; + return inner.type === "TSTypeQuery" && inner.typeArguments === null && inner.exprName.type === "Identifier" ? inner.exprName.name : null; + }; + const name = queried(type.objectType); + const index = type.indexType.type === "TSTypeOperator" && type.indexType.operator === "keyof" ? queried(type.indexType.typeAnnotation) : null; + if (name === null || index !== name) return fail(module, type, `an input's indexed access type is \`(typeof X)[keyof typeof X]\`, the values of an \`as const\` object`); + const imported = module.imports.get(name); + const local = module.constants.get(name); + const found = local !== undefined ? { module, declaration: local } + : imported !== undefined && imported.imported !== null ? await fromSource(module, imported.source, imported.imported, constantsIn, new Set()) : null; + if (found === null || found === "package") return fail(module, type, `${name} isn't a constant its module declares or exports, which this pass can read`); + const { declaration } = found; + let init = declaration.init; + let constant = false; + while (init !== null && (init.type === "TSAsExpression" || init.type === "TSSatisfiesExpression" || init.type === "ParenthesizedExpression")) { + if (init.type === "TSAsExpression" && init.typeAnnotation.type === "TSTypeReference" && init.typeAnnotation.typeName.type === "Identifier" && init.typeAnnotation.typeName.name === "const") constant = true; + init = init.expression; + } + if (init === null || init.type !== "ObjectExpression" || !constant) return fail(found.module, declaration, `${name} is an object declared \`as const\`, whose values are literals`); + const values: (string | number | boolean)[] = []; + for (const property of init.properties) { + const value = property.type === "Property" ? property.value : null; + const literal = value?.type === "Literal" && (typeof value.value === "string" || typeof value.value === "number" || typeof value.value === "boolean") ? value.value + : value?.type === "UnaryExpression" && value.operator === "-" && value.argument.type === "Literal" && typeof value.argument.value === "number" ? -value.argument.value + : undefined; + if (literal === undefined) return fail(found.module, property, `${name}'s values are string, number or boolean literals`); + if (!values.includes(literal)) values.push(literal); + } + return { anyOf: values.map(value => ({ const: value })) }; + }; + + return { convert, moduleOf, read, site, fail }; +}; + +type Expression = Extract["expression"]; +type Call = Extract; +type FunctionNode = Extract; + +/** Every node under `node`, in source order. */ +const each = (node: unknown, visit: (node: Node) => void): void => { + if (Array.isArray(node)) { + for (const item of node) each(item, visit); + return; + } + if (typeof node !== "object" || node === null || typeof Reflect.get(node, "type") !== "string") return; + if (isNode(node)) visit(node); + for (const key of Object.keys(node)) if (key !== "parent") each(Reflect.get(node, key), visit); +}; + +const isNode = (value: object): value is Node => typeof Reflect.get(value, "start") === "number" && typeof Reflect.get(value, "end") === "number"; + +// Each call's arguments are its own expressions. +const isCall = (node: Node): node is Call => node.type === "CallExpression"; + +const unwrapped = (node: Call["arguments"][number]): Call["arguments"][number] => + node.type === "ParenthesizedExpression" ? unwrapped(node.expression) : node; + +const isFunction = (node: Call["arguments"][number]): node is FunctionNode => node.type === "ArrowFunctionExpression" || node.type === "FunctionExpression"; + +/** Whether a declaration written in place declares `components`; null when it isn't written in place. */ +const declaresComponents = (declaration: Call["arguments"][number]): boolean | null => { + if (declaration.type !== "ObjectExpression") return null; + return declaration.properties.some(property => property.type === "Property" && !property.computed + && ((property.key.type === "Identifier" && property.key.name === "components") || (property.key.type === "Literal" && property.key.value === "components"))); +}; + +/** + * Passes each `System.create(declaration, body)` the schema of its body's input, the type of its parameter after its + * context, as a third argument: `create` takes the schema from there. The type is read from its syntax, through aliases + * and interfaces of the module and those it imports, so it needs no type checker. Returns null when no system here + * takes input. + */ +export const passInputSchemas = async (code: string, file: string, host: InputHost): Promise => { + if (!code.includes("System") || !code.includes(".create(")) return null; + const converter = createConverter(host, file); + const { fail } = converter; + const program = host.parse(code, file); + const module = moduleOf(file, code, program); + const systems = new Set(); + for (const [local, entry] of module.imports) { + if (entry.imported === "System" && await host.isSystem(entry.source, file)) systems.add(local); + } + if (systems.size === 0) return null; + const calls: Call[] = []; + each(program.body, node => { + if (!isCall(node)) return; + const { callee } = node; + if (callee.type === "MemberExpression" && !callee.computed && callee.object.type === "Identifier" && systems.has(callee.object.name) + && callee.property.type === "Identifier" && callee.property.name === "create" && node.arguments.length === 2) calls.push(node); + }); + const insertions: { readonly at: number; readonly text: string }[] = []; + try { + for (const call of calls) { + const [declaration, second] = call.arguments; + if (declaration === undefined || second === undefined) continue; + const body = unwrapped(second); + if (!isFunction(body)) continue; + const components = declaresComponents(declaration); + // For a declaration not written in place, a third parameter is the input, as is a second one with a type: a + // context is rarely given one. + const before = components === null ? (body.params.length > 2 || annotationOf(body.params[1]) === null ? 2 : 1) : components ? 2 : 1; + const parameter = body.params[before]; + if (parameter === undefined) continue; + if (parameter.type === "RestElement") fail(module, parameter, "a body's input is one parameter, not a rest parameter"); + if (parameter.type === "AssignmentPattern") fail(module, parameter, "a body's input has no default"); + const annotation = annotationOf(parameter); + if (annotation === null) return fail(module, parameter, "a body's input parameter has a type, which its schema is made from"); + converter.site.at = parameter.start; + const schema = await converter.convert(annotation, module, new Set()); + if (!("type" in schema) || schema.type !== "object") fail(module, parameter, "a body's input is an object of named fields"); + insertions.push({ at: second.end, text: `, ${JSON.stringify(schema)}` }); + } + } catch (error) { + if (isThrown(error)) return { failure: error.inputFailure, dependencies: [...converter.read] }; + throw error; + } + if (insertions.length === 0) return null; + return { ...insertInto(code, insertions, file), dependencies: [...converter.read] }; +}; + +type Parameter = FunctionNode["params"][number]; + +const annotationOf = (parameter: Parameter | undefined): TypeNode | null => { + if (parameter === undefined || parameter.type === "TSParameterProperty") return null; + const annotation = parameter.type === "RestElement" ? parameter.typeAnnotation : parameter.typeAnnotation; + return annotation === null || annotation === undefined ? null : annotation.typeAnnotation; +}; diff --git a/packages/ecs/src/implementation/compiler/ir.ts b/packages/ecs/src/implementation/compiler/ir.ts index f7444e69..3a2f53b7 100644 --- a/packages/ecs/src/implementation/compiler/ir.ts +++ b/packages/ecs/src/implementation/compiler/ir.ts @@ -55,8 +55,8 @@ export type Statement = /** Declares `let` locals that every path assigns before reading. */ | { readonly kind: "declare"; readonly locals: readonly number[] } | { readonly kind: "store"; readonly column: number; readonly value: Expression } - /** Stamps a changed column at the row and its chunk column at the row's chunk. */ - | { readonly kind: "stamp"; readonly column: number; readonly chunks: number } + /** Stamps a changed column at the row, and its chunk and block columns at the row's chunk and block. */ + | { readonly kind: "stamp"; readonly column: number; readonly chunks: number; readonly blocks: number } | { readonly kind: "if"; readonly test: Expression; readonly consequent: Out; readonly alternate: Out } | { readonly kind: "block"; readonly label: number; readonly body: Out } | { readonly kind: "loop"; readonly label: number; readonly body: Out } diff --git a/packages/ecs/src/implementation/compiler/paths.ts b/packages/ecs/src/implementation/compiler/paths.ts index 4dddc4af..00c5a96a 100644 --- a/packages/ecs/src/implementation/compiler/paths.ts +++ b/packages/ecs/src/implementation/compiler/paths.ts @@ -1,44 +1,170 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. /// -import { extname } from 'node:path'; +import { readFileSync, realpathSync } from 'node:fs'; +import { dirname, extname, isAbsolute } from 'node:path'; import { fileURLToPath } from 'node:url'; -const here = fileURLToPath(import.meta.url); -const extension = extname(here); +const packageName = "@adobe/ecs"; const posix = (path: string): string => path.replace(/\\/g, "/"); -const packageRoot = posix(fileURLToPath(new URL("../../..", import.meta.url))).replace(/\/$/, ""); +const fileOf = (id: string): string => posix(id.split("?")[0] ?? id); -/** The runtime hook emitted code imports: the source file when running from `src`, the built one from `dist`. */ -export const attachKernelPath = posix(fileURLToPath(new URL(`../world/attach-kernel${extension}`, import.meta.url))); +/** + * The files of one copy of the package that compiled code imports. A copy is judged by the files an importer resolves + * into it, so emitted code always binds to the copy, and the registry, the importer itself uses. + */ +export type PackageCopy = { + /** The runtime hook kernels attach through. */ + readonly attachKernel: string; + /** The runtime hook for spawns and adds whose values pass positionally. */ + readonly preparedSpawn: string; + /** The runtime hook for index lookups whose keys pass positionally. */ + readonly preparedLookup: string; + /** The package root's module, for kernels that import the package when a build treats it as external. */ + readonly index: string; +}; -/** The runtime hook for spawns whose values pass positionally. */ -export const prepareSpawnPath = posix(fileURLToPath(new URL(`../world/prepared-spawn${extension}`, import.meta.url))); +/** The root of the copy of the package containing each directory, or null; found once per directory. */ +const roots = new Map(); -/** The runtime hook for index lookups whose keys pass positionally. */ -export const prepareLookupPath = posix(fileURLToPath(new URL(`../world/prepared-lookup${extension}`, import.meta.url))); +/** The name the package.json in a directory gives, or null when it has none that parses. */ +const packageNameAt = (directory: string): unknown => { + try { + const json: unknown = JSON.parse(readFileSync(`${directory}/package.json`, "utf8")); + return typeof json === "object" && json !== null ? Reflect.get(json, "name") : undefined; + } + catch { + // No package.json here, or none that parses. + return null; + } +}; -const intrinsicModules: ReadonlySet = new Set(["src/interface/math/intrinsic.ts", "src/interface/math/intrinsic.js", "dist/interface/math/intrinsic.js"].map(path => `${packageRoot}/${path}`)); +const isPackageRoot = (directory: string): boolean => packageNameAt(directory) === packageName; -/** Whether a file is the module math functions declare their WGSL builtins with. */ -export const isIntrinsicModule = (file: string): boolean => intrinsicModules.has(posix(file)); +const rootOfDirectory = (directory: string): string | null => { + const known = roots.get(directory); + if (known !== undefined) return known; + const name = packageNameAt(directory); + const parent = dirname(directory); + // The nearest package.json owns the file, whichever package it names. + const root = name === packageName ? directory : name !== null || parent === directory ? null : rootOfDirectory(parent); + roots.set(directory, root); + return root; +}; -/** The package root's module, for kernels that import the package when a build treats it as external. */ -export const packageIndex = posix(fileURLToPath(new URL(`../../index${extension}`, import.meta.url))); +/** A file's path in the copy of the package containing it, as `src/…` or `dist/…`, or null outside every copy. */ +const pathInCopy = (file: string): { readonly root: string; readonly path: string } | null => { + if (!isAbsolute(file)) return null; + const root = rootOfDirectory(dirname(file)); + return root === null ? null : { root, path: file.slice(root.length + 1) }; +}; -const systemModules: ReadonlySet = new Set([ - "src/interface/system/system.ts", "src/interface/system/system.js", "dist/interface/system/system.js", - "src/interface/index.ts", "src/interface/index.js", "dist/interface/index.js", "src/index.ts", "src/index.js", "dist/index.js", -].map(path => `${packageRoot}/${path}`)); +const copies = new Map(); -/** Whether a resolved module id is this package's `System` namespace, directly or through the package root. */ -export const isSystemModule = (id: string): boolean => systemModules.has(posix(id.split("?")[0] ?? id)); +/** The copy at `root`, laid out as its `src` with `extension`, or as its built `dist`. */ +const copyAt = (root: string, source: boolean, extension: string): PackageCopy => { + const key = `${root}\0${source ? extension : ""}`; + const known = copies.get(key); + if (known !== undefined) return known; + const prefix = source ? `${root}/src` : `${root}/dist`; + const suffix = source ? extension : ".js"; + const copy: PackageCopy = { + attachKernel: `${prefix}/implementation/world/attach-kernel${suffix}`, + preparedSpawn: `${prefix}/implementation/world/prepared-spawn${suffix}`, + preparedLookup: `${prefix}/implementation/world/prepared-lookup${suffix}`, + index: `${prefix}/index${suffix}`, + }; + copies.set(key, copy); + return copy; +}; -const roots = ["src/interface/index.ts", "src/interface/index.js", "dist/interface/index.js", "src/index.ts", "src/index.js", "dist/index.js"]; +const here = posix(fileURLToPath(import.meta.url)); +const ownRoot = posix(fileURLToPath(new URL("../../..", import.meta.url))).replace(/\/$/, ""); + +/** The copy this compiler runs from: its sources when running from `src`, its `dist` otherwise. */ +export const ownCopy: PackageCopy = copyAt(ownRoot, here.startsWith(`${ownRoot}/src/`), extname(here)); + +/** + * The copy at `root`, in the layout of a file in it. This compiler's own copy keeps the compiler's layout, so modules + * importing its sources and its `dist` share the runtime the compiler runs with. + */ +const copyIn = (root: string, path: string, extension: string): PackageCopy => + root === ownRoot ? ownCopy : copyAt(root, path.startsWith("src/"), extension); + +/** The copy a file belongs to, or null outside every copy. */ +const copyOfFile = (file: string): PackageCopy | null => { + const found = pathInCopy(file); + return found === null ? null : copyIn(found.root, found.path, extname(file)); +}; + +/** The copy installed for each directory's modules, found once per directory. */ +const installed = new Map(); + +const installedFrom = (directory: string): PackageCopy | null => { + const known = installed.get(directory); + if (known !== undefined) return known; + let copy: PackageCopy | null = null; + const candidate = `${directory}/node_modules/${packageName}`; + if (isPackageRoot(candidate)) { + copy = copyIn(posix(realpathSync(candidate)), "dist/", ".js"); + } else { + const parent = dirname(directory); + copy = parent === directory ? null : installedFrom(parent); + } + installed.set(directory, copy); + return copy; +}; + +/** + * The copy an importer that can't resolve the package through the build uses: the copy it is part of, or else the one + * installed nearest it, or else this compiler's own. + */ +export const copyForImporter = (importer: string): PackageCopy => { + const file = fileOf(importer); + return copyOfFile(file) ?? installedFrom(dirname(file)) ?? ownCopy; +}; + +const layouts = (module: string, extensions: readonly string[] = [".ts", ".js"]): readonly string[] => + [...extensions.map(extension => `src/${module}${extension}`), `dist/${module}.js`]; + +const intrinsicModules: ReadonlySet = new Set(layouts("interface/math/intrinsic")); + +/** Whether a file is the module math functions declare their WGSL builtins with, in any copy of the package. */ +export const isIntrinsicModule = (file: string): boolean => { + const posixFile = posix(file); + if (!posixFile.includes("/intrinsic.")) return false; + const found = pathInCopy(posixFile); + return found !== null && intrinsicModules.has(found.path); +}; + +const attachKernelModules: ReadonlySet = new Set(layouts("implementation/world/attach-kernel")); + +/** Whether an import specifier is the runtime hook kernels attach through, in any copy of the package. */ +export const isAttachKernelModule = (specifier: string): boolean => { + if (!specifier.includes("/attach-kernel.")) return false; + const found = pathInCopy(fileOf(specifier)); + return found !== null && attachKernelModules.has(found.path); +}; + +const rootModules = [...layouts("interface/index"), ...layouts("index")]; const namespaceModules: { readonly [namespace: string]: ReadonlySet } = Object.fromEntries( - [["System", "system/system"], ["Rule", "rule/rule"], ["Finalizer", "finalizer/finalizer"], ["Index", "indexes/index-type"]].map(([namespace, module]) => [namespace, new Set( - [`src/interface/${module}.ts`, `src/interface/${module}.js`, `dist/interface/${module}.js`, ...roots].map(path => `${packageRoot}/${path}`), - )]), + [["System", "system/system"], ["Rule", "rule/rule"], ["Finalizer", "finalizer/finalizer"], ["Index", "indexes/index-type"]] + .map(([namespace, module]) => [namespace, new Set([...layouts(`interface/${module}`), ...rootModules])]), ); -/** Whether a resolved module id is the module of this package's `namespace`, directly or through the package root. */ -export const isNamespaceModule = (id: string, namespace: string): boolean => namespaceModules[namespace]?.has(posix(id.split("?")[0] ?? id)) ?? false; +/** + * The copy of the package whose module of `namespace` a resolved module id is, directly or through the package root, + * in the id's layout; null when the id is no such module. + */ +export const namespaceCopyOf = (id: string, namespace: string): PackageCopy | null => { + const modules = namespaceModules[namespace]; + if (modules === undefined) return null; + const file = fileOf(id); + const found = pathInCopy(file); + return found !== null && modules.has(found.path) ? copyIn(found.root, found.path, extname(file)) : null; +}; + +/** Whether a resolved module id is one of the package's public modules, in any copy: its root or its interface. */ +export const isInterfaceModule = (id: string): boolean => { + const found = pathInCopy(fileOf(id)); + return found !== null && (found.path.startsWith("src/interface/") || found.path.startsWith("dist/interface/") || rootModules.includes(found.path)); +}; diff --git a/packages/ecs/src/implementation/compiler/plugin.test.ts b/packages/ecs/src/implementation/compiler/plugin.test.ts index 9dea11dc..28ce647b 100644 --- a/packages/ecs/src/implementation/compiler/plugin.test.ts +++ b/packages/ecs/src/implementation/compiler/plugin.test.ts @@ -1,11 +1,11 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. /// -import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; +import { mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; -import { join } from 'node:path'; +import { dirname, join, relative, resolve } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; import { afterAll, expect, test } from 'vitest'; -import { build, createServer, type Rollup, type ViteDevServer } from 'vite'; +import { build, createServer, parseAst, transformWithOxc, type Rollup, type ViteDevServer } from 'vite'; import type { KernelDescription, KernelModule } from '../world/kernel.js'; import { kernelPlugin } from './plugin.js'; @@ -44,6 +44,17 @@ import { score } from "./linked-keys.js"; const kernel = System.create({ name: "linked", kernel: "cpu", components: { write: [score] } }, (row) => { row[score] = increment(scaled(row.entity)); }); export const description = kernelOf(kernel); `); +/** Systems taking input, of a type another TypeScript module declares, which the test edits. */ +const inputs = join(directory, "inputs.ts"); +const inputTypes = join(directory, "input-types.ts"); +const writeInputTypes = (field: string) => writeFileSync(inputTypes, `import type { Entity } from ${source("../../interface/entity/entity.ts")};\nexport type Push = { readonly entity: Entity; readonly ${field}: number };\n`); +writeInputTypes("by"); +writeFileSync(inputs, `import { System } from ${source("../../interface/system/system.ts")}; +import type { Push } from "./input-types.ts"; +const push = System.create({ name: "inputs/push" }, (_context, input: Push) => { void input; }); +const label = System.create({ name: "inputs/label" }, (_context, input: { readonly text?: string }) => { void input; }); +export const schemas = [push, label].map(system => Reflect.get(system, "input")); +`); /** A kernel spawning through a list another module declares, which the test edits. */ const listing = join(directory, "listing.js"); const listed = join(directory, "listed.js"); @@ -61,21 +72,6 @@ import { Listed } from "./listed.js"; const kernel = System.create({ name: "listing", kernel: "cpu", components: { read: [score] } }, (row, { commands }) => { commands.spawn(Listed)({ [score]: row[score] }); }); export const description = kernelOf(kernel); `); -/** An application importing a kernel a library ships uncompiled, from its `dist`. */ -const application = join(directory, "application.js"); -mkdirSync(join(directory, "node_modules/@adobe"), { recursive: true }); -symlinkSync(fileURLToPath(new URL("../../..", import.meta.url)), join(directory, "node_modules/@adobe/ecs")); -mkdirSync(join(directory, "node_modules/kernel-library/dist"), { recursive: true }); -writeFileSync(join(directory, "node_modules/kernel-library/package.json"), JSON.stringify({ name: "kernel-library", type: "module", exports: "./dist/index.js" })); -writeFileSync(join(directory, "node_modules/kernel-library/dist/index.js"), `import { Component, System } from "@adobe/ecs"; -const score = Symbol("score"); -Component.register(score, { type: { type: "number" } }); -export const kernel = System.create({ name: "library", kernel: "cpu", components: { write: [score] } }, (row) => { row[score] = row.entity * 2; }); -`); -writeFileSync(application, `import { kernelOf } from ${source("../world/attach-kernel.ts")}; -import { kernel } from "kernel-library"; -export const description = kernelOf(kernel); -`); /** Two kernels in one module, writing different functions of the entity. */ const pair = join(directory, "pair.js"); writeFileSync(pair, `import { System } from ${source("../../interface/system/system.ts")}; @@ -145,6 +141,47 @@ const externalList = join(directory, "external-list.js"); writeFileSync(externalList, externalKernel("externalList", "Pair")); afterAll(() => rmSync(directory, { recursive: true, force: true })); +const own = fileURLToPath(new URL("../../..", import.meta.url)); +/** The modules an installed copy's code starts from: its root, its worker, and those compiled code imports. */ +const copyEntries = ["index", "implementation/world/worker", "implementation/world/attach-kernel", "implementation/world/prepared-spawn", "implementation/world/prepared-lookup"]; +let builtDist: Promise> | null = null; + +/** Every relative module a module's code imports or re-exports from. */ +const relativeImports = (code: string): readonly string[] => parseAst(code).body.flatMap(statement => + (statement.type === "ImportDeclaration" || statement.type === "ExportAllDeclaration" || statement.type === "ExportNamedDeclaration") + && typeof statement.source?.value === "string" && statement.source.value.startsWith(".") ? [statement.source.value] : []); + +/** + * The package's `dist` code by path within it, built from the sources as `tsc` emits each module on its own: the + * modules the entries reach at runtime, so no test reads a previous build. Built once per test file. + */ +const distFromSources = (): Promise> => builtDist ??= (async () => { + const sources = join(own, "src"); + const files = new Map(); + const visit = async (file: string): Promise => { + const path = relative(sources, file).replace(/\.ts$/, ".js"); + if (files.has(path)) return; + files.set(path, ""); + const { code } = await transformWithOxc(readFileSync(file, "utf8"), file, { lang: "ts" }); + files.set(path, code); + await Promise.all(relativeImports(code).map(specifier => visit(resolve(dirname(file), specifier.replace(/\.js$/, ".ts"))))); + }; + await Promise.all(copyEntries.map(entry => visit(join(sources, `${entry}.ts`)))); + return files; +})(); + +/** Installs a built copy of the package in a project's `node_modules`, a directory of its own as an install makes. */ +const installCopy = async (project: string): Promise => { + const copy = join(project, "node_modules/@adobe/ecs"); + mkdirSync(copy, { recursive: true }); + writeFileSync(join(copy, "package.json"), readFileSync(join(own, "package.json"))); + for (const [path, code] of await distFromSources()) { + mkdirSync(dirname(join(copy, "dist", path)), { recursive: true }); + writeFileSync(join(copy, "dist", path), code); + } + return copy; +}; + /** Runs a one-column kernel writing a function of the entity, for entities 3 and 4. */ const runs = (module: KernelModule, description: KernelDescription): readonly number[] => { const entities = Float64Array.from([3, 4]); @@ -280,9 +317,26 @@ test("serve compiles kernels again when a list they import changes", async () => }); test("an application's build compiles the kernels a library ships uncompiled", async () => { - const outDir = join(directory, "dist-application"); + // An application importing a kernel a library ships uncompiled, from its `dist`, with the package installed. + const project = join(directory, "shipped"); + const copy = await installCopy(project); + writeFileSync(join(project, "package.json"), JSON.stringify({ name: "shipped-app", type: "module" })); + mkdirSync(join(project, "node_modules/kernel-library/dist"), { recursive: true }); + writeFileSync(join(project, "node_modules/kernel-library/package.json"), JSON.stringify({ name: "kernel-library", type: "module", exports: "./dist/index.js" })); + writeFileSync(join(project, "node_modules/kernel-library/dist/index.js"), `import { Component, System } from "@adobe/ecs"; +const score = Symbol("score"); +Component.register(score, { type: { type: "number" } }); +export const kernel = System.create({ name: "library", kernel: "cpu", components: { write: [score] } }, (row) => { row[score] = row.entity * 2; }); +`); + // The test hook comes from the installed copy, which the library's compiled kernel attaches to. + const application = join(project, "application.js"); + writeFileSync(application, `import { kernelOf } from ${JSON.stringify(join(copy, "dist/implementation/world/attach-kernel.js"))}; +import { kernel } from "kernel-library"; +export const description = kernelOf(kernel); +`); + const outDir = join(project, "dist"); const output = await build({ - configFile: false, logLevel: "silent", root: directory, plugins: [kernelPlugin()], + configFile: false, logLevel: "silent", root: project, plugins: [kernelPlugin()], build: { outDir, minify: false, lib: { entry: application, formats: ["es"], fileName: "main" } }, }); const main = (Array.isArray(output) ? output : [output]).flatMap((result: Rollup.RollupOutput | Rollup.RollupWatcher) => "output" in result ? result.output : []) @@ -374,3 +428,61 @@ test("a build takes a name from a package external to it as one key, and registr expect(description.keys.map(String)).toEqual(["Symbol(heat)"]); await expect(import(/* @vite-ignore */ url(externalList))).rejects.toThrow(`System "externalList" names a list of components where a command takes one component; its kernel's build couldn't read the list, so name it from a module the build reads, or as an array literal.`); }); + +test("compiled code imports the runtime from the copy of the package its module imports, not the plugin's own", async () => { + // A project whose installed copy of the package is a directory of its own, as a lockfile's copy is beside a linked one. + const project = join(directory, "copied"); + await installCopy(project); + writeFileSync(join(project, "package.json"), JSON.stringify({ name: "copy-app", type: "module" })); + const module = join(project, "copied.js"); + writeFileSync(module, `import { System, Component } from "@adobe/ecs"; +const score = Symbol("score"); +Component.register(score, { type: { type: "number" } }); +export const kernel = System.create({ name: "copied", kernel: "cpu", components: { write: [score] } }, (row) => { row[score] = row.entity * 2; }); +export const spawner = System.create({ name: "copiedSpawner", components: { read: [score] } }, (row, { commands }) => { commands.spawn([score])({ [score]: row[score] }); }); +`); + const server = await createServer({ + configFile: false, logLevel: "silent", root: project, plugins: [kernelPlugin()], + appType: "custom", server: { middlewareMode: true, hmr: false, watch: null }, optimizeDeps: { noDiscovery: true }, + }); + try { + const code = (await server.transformRequest(module))?.code ?? ""; + const imported = [...code.matchAll(/^import .* from "([^"]+)";$/gm)].map(match => match[1] ?? ""); + // The server serves files under its root by their path from it: here, the project's copy. + expect(imported.filter(path => path.includes("/implementation/world/"))).toEqual([ + "/node_modules/@adobe/ecs/dist/implementation/world/attach-kernel.js", + "/node_modules/@adobe/ecs/dist/implementation/world/prepared-spawn.js", + ]); + expect(code).not.toContain(own); + // The copy's System namespace is recognized, so its declarations take their ids. + expect(code).toContain(`"copy-app/copied.js#kernel"`); + } finally { + await server.close(); + } +}); + +test("serve passes each system the schema of its body's input, from types other modules declare, and again when they change", async () => { + try { + await serve(async server => { + expect((await server.ssrLoadModule(inputs)).schemas).toEqual([ + { type: "object", properties: { entity: { type: "integer", format: "entity" }, by: { type: "number" } } }, + { type: "object", properties: { text: { type: "string" } }, required: [] }, + ]); + writeInputTypes("to"); + server.watcher.emit("change", inputTypes); + const [push] = (await server.ssrLoadModule(inputs)).schemas; + expect(push).toEqual({ type: "object", properties: { entity: { type: "integer", format: "entity" }, to: { type: "number" } } }); + }); + } finally { + writeInputTypes("by"); + } +}); + +test("a build fails at a system's input whose type can't be one", async () => { + const entry = join(directory, "bad-input.ts"); + writeFileSync(entry, `import { System } from ${source("../../interface/system/system.ts")};\nexport const bad = System.create({}, (_context, input: { readonly when: Big["when"] }) => { void input; });\n`); + await expect(build({ + configFile: false, logLevel: "silent", root: directory, plugins: [kernelPlugin()], + build: { outDir: join(directory, "dist-bad-input"), minify: false, lib: { entry, formats: ["es"], fileName: "main" } }, + })).rejects.toThrow("(typeof X)[keyof typeof X]"); +}); diff --git a/packages/ecs/src/implementation/compiler/plugin.ts b/packages/ecs/src/implementation/compiler/plugin.ts index 928f760b..af3b8398 100644 --- a/packages/ecs/src/implementation/compiler/plugin.ts +++ b/packages/ecs/src/implementation/compiler/plugin.ts @@ -9,7 +9,8 @@ import { describeFailure } from './failure.js'; import { identify as identifyModule, moduleIdOf, type PackageNameOf } from './identify.js'; import { buildId, kernelHash, parseKernelId, serveUrl } from './kernel-id.js'; import { createLinker, type LinkHost, type Linker } from './link.js'; -import { isNamespaceModule, isSystemModule, packageIndex } from './paths.js'; +import { passInputSchemas, type InputHost } from './input-schemas.js'; +import { copyForImporter, isInterfaceModule, namespaceCopyOf } from './paths.js'; import { locateThroughMap } from './source-map.js'; import { transformKernels } from './transform.js'; @@ -56,7 +57,7 @@ const buildHost = (resolver: () => Rollup.PluginContext): LinkHost => ({ const resolved = await resolver().resolve(specifier, importer, { skipSelf: true }); if (resolved === null) return { error: `${specifier} can't be resolved from ${importer}` }; if (resolved.external) { - return specifier === packageName ? packageIndex : { error: `${specifier} is external to the build, so kernels can't inline from it`, unreadable: true }; + return specifier === packageName ? copyForImporter(importer).index : { error: `${specifier} is external to the build, so kernels can't inline from it`, unreadable: true }; } const id = resolved.id.split("?")[0] ?? resolved.id; return id.includes("\0") || !isAbsolute(id) ? { error: `${specifier} isn't a file` } : id; @@ -93,7 +94,81 @@ const inlineKernels = (code: string, kernels: readonly string[], hashes: readonl JSON.stringify(`data:text/javascript;base64,${Buffer.from(source).toString("base64")}`), ), code); -export const kernelPlugin = (): Plugin => { +const typed = /\.[cm]?tsx?$/; + +/** + * Passes each system's `create` the schema of its body's input, from the type of the body's parameter after its + * context (`input-schemas.ts`). It runs before TypeScript is stripped, which the type needs. + */ +const inputPlugin = (): Plugin => { + let repeated = false; + let server: ViteDevServer | undefined; + /** Modules whose input types each file declares, and the files each module's input types were read from. */ + const dependents = new Map>(); + const dependencies = new Map(); + const depend = (parent: string, files: readonly string[]): void => { + for (const file of dependencies.get(parent) ?? []) dependents.get(file)?.delete(parent); + dependencies.set(parent, files); + for (const file of files) { + const parents = dependents.get(file) ?? new Set(); + parents.add(parent); + dependents.set(file, parents); + } + }; + const plugin: Plugin = { + name: `${packageName}/input`, + enforce: "pre", + configResolved: config => { + repeated = config.plugins.find(other => other.name === plugin.name) !== plugin; + }, + configureServer: devServer => { + server = devServer; + }, + // A module whose input types come from a changed file is transformed again, for its schemas. + watchChange(id) { + for (const parent of dependents.get(id) ?? []) { + for (const module of server?.moduleGraph.getModulesByFile(parent) ?? []) server?.moduleGraph.invalidateModule(module); + } + }, + async transform(code, id) { + const file = id.split("?")[0] ?? id; + if (repeated || id.includes("\0") || !typed.test(file)) return null; + const { parseAst } = await import("vite"); + const resolved = async (specifier: string, importer: string) => await this.resolve(specifier, importer, { skipSelf: true }); + const host: InputHost = { + parse: (source, path) => parseAst(source, { lang: path.endsWith("x") ? "tsx" : "ts" }, path), + resolve: async (specifier, importer) => { + const found = await resolved(specifier, importer); + if (found === null || found.external) return null; + const path = found.id.split("?")[0] ?? found.id; + return path.includes("\0") || !isAbsolute(path) ? null : path; + }, + load: file => readFile(file, "utf8"), + isPackage: async (specifier, importer) => { + const found = await resolved(specifier, importer); + return found !== null && (found.external ? specifier === packageName : isInterfaceModule(found.id)); + }, + isSystem: async (specifier, importer) => { + const found = await resolved(specifier, importer); + return found !== null && (found.external ? specifier === packageName : namespaceCopyOf(found.id, "System") !== null); + }, + }; + const result = await passInputSchemas(code, file, host); + const read = result === null ? [] : result.dependencies.filter(dependency => dependency !== file); + for (const dependency of read) this.addWatchFile(dependency); + depend(file, read); + if (result === null) return null; + if ("failure" in result) return this.error(describeFailure(result.failure), result.failure.position); + return { code: result.code, map: result.map }; + }, + }; + return plugin; +}; + +/** The kernel plugin's two parts: input schemas, before TypeScript is stripped, and kernels and ids, after. */ +export const kernelPlugin = (): Plugin[] => [inputPlugin(), compilePlugin()]; + +const compilePlugin = (): Plugin => { let serve = true; let root = process.cwd(); /** Whether an earlier instance of the plugin is in the same bundle. */ @@ -144,7 +219,7 @@ export const kernelPlugin = (): Plugin => { name: `${packageName}/kernel`, enforce: "post", // Dep pre-bundling breaks the runtime's `new Worker(new URL("./worker.js", import.meta.url))`. - config: () => ({ optimizeDeps: { exclude: [packageName] }, worker: { plugins: () => [kernelPlugin()] } }), + config: () => ({ optimizeDeps: { exclude: [packageName] }, worker: { plugins: () => kernelPlugin() } }), configResolved: config => { serve = config.command === "serve"; root = config.root; @@ -191,14 +266,14 @@ export const kernelPlugin = (): Plugin => { const parse = (input: string) => estreeOf(this.parse(input)); const identify = (input: string) => identifyModule(input, moduleIdOf(id, root, packageNameOf), parse, async (source, namespace) => { const resolved = await this.resolve(source, id); - return resolved !== null && (resolved.external ? source === packageName : isNamespaceModule(resolved.id, namespace)); + return resolved !== null && (resolved.external ? source === packageName : namespaceCopyOf(resolved.id, namespace) !== null); }); const ssr = options?.ssr === true; const linker = linkerFor(ssr, this); const hashes: string[] = []; const transformed = await transformKernels(code, id, parse, async source => { const resolved = await this.resolve(source, id); - return resolved !== null && isSystemModule(resolved.id); + return resolved === null ? null : namespaceCopyOf(resolved.id, "System"); }, (index, identity) => { const hash = kernelHash(identity); hashes[index] = hash; diff --git a/packages/ecs/src/implementation/compiler/print.ts b/packages/ecs/src/implementation/compiler/print.ts index 2201e6ba..90f6514e 100644 --- a/packages/ecs/src/implementation/compiler/print.ts +++ b/packages/ecs/src/implementation/compiler/print.ts @@ -1,5 +1,5 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. -import { chunkShift, type Path } from '../world/kernel.js'; +import { blockShift, chunkShift, type Path } from '../world/kernel.js'; import type { Expression, Statement } from './ir.js'; import { resolve, type Out } from './out.js'; import { rowLabel } from './statement.js'; @@ -69,7 +69,7 @@ const printStatement = (unit: Unit, statement: Statement, labels: Labels): strin case "store": return `c${statement.column}[x${statement.column}] = ${value(statement.value)};`; case "stamp": { const row = `c${statement.column}[x${statement.column}] = tick;`; - return unit.stampedPerCall.has(statement.chunks) ? row : `${row}\nc${statement.chunks}[i >> ${chunkShift}] = tick;`; + return unit.stampedPerCall.has(statement.chunks) ? row : `${row}\nc${statement.blocks}[i >> ${blockShift}] = tick;\nc${statement.chunks}[i >> ${chunkShift}] = tick;`; } case "if": { const alternate = printStatements(unit, statement.alternate, labels); diff --git a/packages/ecs/src/implementation/compiler/signature.ts b/packages/ecs/src/implementation/compiler/signature.ts index 7dbcf071..c2ed0608 100644 --- a/packages/ecs/src/implementation/compiler/signature.ts +++ b/packages/ecs/src/implementation/compiler/signature.ts @@ -31,7 +31,6 @@ export type Signature = { readonly up: Up | null; /** Whether its rows run in an index's order, as a list, as `up`'s do. */ readonly ordered: boolean; - readonly input: boolean; /** Whether it filters on `added` or `changed` components, or on changed resources. */ readonly filtered: boolean; }; @@ -94,7 +93,6 @@ export const signature = (declaration: ObjectExpression, module: ModuleScope, li let resources: readonly string[] = []; let reduced: readonly Reduced[] = []; let changedResources: readonly string[] = []; - let input = false; let up: Up | null = null; let ordered = false; const lookups: Lookup[] = []; @@ -145,9 +143,6 @@ export const signature = (declaration: ObjectExpression, module: ModuleScope, li up = { through: through ?? fail(property, "up needs through"), read }; break; } - case "input": - input = true; - break; // The index is a value the world resolves; only that rows run as a list changes how it compiles. case "order": ordered = true; @@ -167,5 +162,5 @@ export const signature = (declaration: ObjectExpression, module: ModuleScope, li if (both !== undefined) fail(declaration, `${both} is in both components read and write; write already makes it readable`); const rowFiltered = (components.get("added")?.length ?? 0) + (components.get("changed")?.length ?? 0) > 0; if (up !== null && ordered) fail(declaration, "a kernel can't use both up and order, which both decide the order of its rows"); - return { read: new Set(components.get("read")), write: new Set(write), resources, reduced, lookups, up, ordered, input, filtered: changedResources.length > 0 || rowFiltered }; + return { read: new Set(components.get("read")), write: new Set(write), resources, reduced, lookups, up, ordered, filtered: changedResources.length > 0 || rowFiltered }; }; diff --git a/packages/ecs/src/implementation/compiler/stamps.ts b/packages/ecs/src/implementation/compiler/stamps.ts index 5db1c3cb..28b1e579 100644 --- a/packages/ecs/src/implementation/compiler/stamps.ts +++ b/packages/ecs/src/implementation/compiler/stamps.ts @@ -12,14 +12,14 @@ const exits = (statement: Statement): boolean => { }; /** - * Chunk columns stamped on every row: by a top-level stamp that no earlier `next` can skip. Every row in - * `[start, end)` runs the body, so these chunks can be stamped once per call instead of per row. + * Chunk columns stamped on every row, with their block columns: by a top-level stamp that no earlier `next` can skip. + * Every row in `[start, end)` runs the body, so these chunks and blocks can be stamped once per call instead of per row. */ -export const unconditionalChunks = (rows: readonly Statement[]): ReadonlySet => { - const chunks = new Set(); +export const unconditionalChunks = (rows: readonly Statement[]): ReadonlyMap => { + const chunks = new Map(); for (const statement of rows) { if (exits(statement)) break; - if (statement.kind === "stamp") chunks.add(statement.chunks); + if (statement.kind === "stamp") chunks.set(statement.chunks, statement.blocks); } return chunks; }; diff --git a/packages/ecs/src/implementation/compiler/transform.test.ts b/packages/ecs/src/implementation/compiler/transform.test.ts index bd257aca..449b7d82 100644 --- a/packages/ecs/src/implementation/compiler/transform.test.ts +++ b/packages/ecs/src/implementation/compiler/transform.test.ts @@ -5,10 +5,10 @@ import { readFile } from 'node:fs/promises'; import { dirname, resolve } from 'node:path'; import { describe, expect, test } from 'vitest'; import { parseAst } from 'vite'; -import { chunkShift, type KernelDescription, type KernelModule, type Program } from '../world/kernel.js'; +import { blockShift, chunkShift, type KernelDescription, type KernelModule, type Program } from '../world/kernel.js'; import { transformKernels } from './transform.js'; import { createLinker, type LinkHost } from './link.js'; -import { packageIndex } from './paths.js'; +import { ownCopy } from './paths.js'; import { locateThroughMap } from './source-map.js'; import { estreeOf } from './estree.js'; import { Mat4x4, Quat, Vec3 } from '../../interface/math/index.js'; @@ -31,7 +31,7 @@ const target = "target"; /** Modules by absolute path, for tests of imports: real files for anything not in `files`, and "ecs" for the package. */ const host = (files: { readonly [file: string]: string }): LinkHost => ({ resolve: async (specifier, importer) => { - if (specifier === "ecs") return packageIndex; + if (specifier === "ecs") return ownCopy.index; if (specifier === "external") return { error: "external is external to the build", unreadable: true }; if (!specifier.startsWith(".")) return { error: `${specifier} can't be resolved from ${importer}` }; const path = resolve(dirname(importer), specifier); @@ -48,7 +48,7 @@ const host = (files: { readonly [file: string]: string }): LinkHost => ({ }); const transform = (source: string, files: { readonly [file: string]: string } = {}) => transformKernels(`${header}${source}`, "/test.js", parse, - async specifier => specifier === "ecs", index => JSON.stringify(`kernel:${index}`), createLinker(host(files), parse)); + async specifier => specifier === "ecs" ? ownCopy : null, index => JSON.stringify(`kernel:${index}`), createLinker(host(files), parse)); type Compiled = KernelDescription & KernelModule; @@ -196,8 +196,10 @@ const interpret = (program: Program, columns: readonly (Float64Array | Uint32Arr case "stamp": { const column = columns[statement.column]; const chunks = columns[statement.chunks]; + const blocks = columns[statement.blocks]; if (column) column[at(statement.column, row)] = tick; if (chunks) chunks[row >> chunkShift] = tick; + if (blocks) blocks[row >> blockShift] = tick; return normal; } case "if": return execute(evaluate(statement.test) ? statement.consequent : statement.alternate); @@ -248,6 +250,7 @@ const run = (description: Compiled, rows: number, fields: Fields, options: Optio if (column.kind === "entity") return Float64Array.from({ length: rows }, (_, row) => 100 + row); if (column.kind === "changed") return new Uint32Array(rows); if (column.kind === "chunks") return new Float64Array(Math.ceil(rows / 2 ** chunkShift)); + if (column.kind === "blocks") return new Float64Array(Math.ceil(rows / 2 ** blockShift)); const component = String(column.component); const values = fields[component]?.[JSON.stringify(column.path)] ?? new Array(rows).fill(0); const stride = interleaved[component]?.stride; @@ -579,7 +582,7 @@ describe("context", () => { }); test("optional lookup access, destructured context groups and input", async () => { - const kernel = await compile(`const kernel = System.create({ name: "a", kernel: "cpu", input: { type: "number" }, components: { read: [target], write: [score] }, resources: { read: [delta] }, lookup: { targets: { read: [mass] } } }, (row, { resources: { [delta]: d }, lookup: { targets }, input }) => { + const kernel = await compile(`const kernel = System.create({ name: "a", kernel: "cpu", components: { read: [target], write: [score] }, resources: { read: [delta] }, lookup: { targets: { read: [mass] } } }, (row, { resources: { [delta]: d }, lookup: { targets } }, input) => { row[score] = (targets(row[target])?.[mass] ?? 0) + d + input; });`); const found = (entity: number) => entity === 1 ? [3] : null; @@ -587,9 +590,9 @@ describe("context", () => { }); test("context groups reached through the context parameter", async () => { - const kernel = await compile(`const kernel = System.create({ name: "a", kernel: "cpu", input: { type: "number" }, components: { read: [target], write: [score] }, resources: { read: [delta] }, lookup: { targets: { read: [mass] } } }, (row, context) => { + const kernel = await compile(`const kernel = System.create({ name: "a", kernel: "cpu", components: { read: [target], write: [score] }, resources: { read: [delta] }, lookup: { targets: { read: [mass] } } }, (row, context, input) => { const { resources } = context; - row[score] = (context.lookup.targets(row[target])?.[mass] ?? 0) + resources[delta] + context.input; + row[score] = (context.lookup.targets(row[target])?.[mass] ?? 0) + resources[delta] + input; });`); const found = (entity: number) => entity === 1 ? [3] : null; expect(run(kernel, 2, { target: { "[]": [1, 2] } }, { lookups: [found], resources: [10], args: 100, program: false }).field("score")).toEqual([113, 110]); @@ -700,7 +703,7 @@ describe("context", () => { describe("program", () => { test("input and resource paths are scalars in the program", async () => { - const kernel = await compile(`const kernel = System.create({ name: "a", kernel: "cpu", input: { type: "object" }, components: { write: [score] }, resources: { read: [delta] } }, (row, { resources: { [delta]: d }, input }) => { + const kernel = await compile(`const kernel = System.create({ name: "a", kernel: "cpu", components: { write: [score] }, resources: { read: [delta] } }, (row, { resources: { [delta]: d } }, input) => { const [, y] = d; row[score] = y * input.scale + Math.max(input.scale, Math.PI) + (input.flag ? 1 : 0); });`); @@ -834,7 +837,7 @@ describe("subset", () => { test("non-kernel systems that may reach their commands and other modules are left untouched", async () => { expect(await transform(`const s = System.create({ name: "a", components: { read: [mass] } }, (row, context) => { const f = () => [row, context]; });`)).toBeNull(); - expect(await transformKernels(`import { System } from "other";\nSystem.create({ name: "a", kernel: "cpu" }, () => {});`, "/x.js", parse, async () => false, () => "")).toBeNull(); + expect(await transformKernels(`import { System } from "other";\nSystem.create({ name: "a", kernel: "cpu" }, () => {});`, "/x.js", parse, async () => null, () => "")).toBeNull(); }); }); @@ -849,7 +852,7 @@ describe("terminal bodies", () => { test.each([ ["no context", "(row)", "console.log(row);", `{ name: "a", components: { has: [mass] } }`], ["a context destructured without commands", "({ resources })", "console.log(resources);", `{ name: "a" }`], - ["a context used only through its other properties", "(context)", "console.log(context.input);", `{ name: "a", input: { type: "number" } }`], + ["a context used only through its other properties", "(context)", "console.log(context.resources);", `{ name: "a", resources: { read: [delta] } }`], ["a context reaching resources and lookups", "(row, context)", "console.log(context.resources, context.lookup, row);", `{ name: "a", components: { has: [mass] } }`], ["nested destructuring", "({ resources: { [delta]: d } })", "console.log(d);", `{ name: "a", resources: { read: [delta] } }`], ])("mark a body with %s", async (_, parameters, body, declaration) => { @@ -879,7 +882,7 @@ describe("terminal bodies", () => { expect(alone !== null && "code" in alone ? alone.code : "").toContain("System.create(terminal$(() =>"); const kernel = await transform(`const kernel = System.create({ name: "k", kernel: "cpu", components: { write: [score] } }, (row) => { row[score] = 1; });`); expect(kernel !== null && "code" in kernel ? kernel.code : "").not.toContain("terminal$"); - const finalizer = await transformKernels(`import { Finalizer } from "ecs";\nconst f = Finalizer.create({ components: { read: [] } }, () => {});`, "/x.js", parse, async () => true, () => ""); + const finalizer = await transformKernels(`import { Finalizer } from "ecs";\nconst f = Finalizer.create({ components: { read: [] } }, () => {});`, "/x.js", parse, async () => ownCopy, () => ""); expect(finalizer).toBeNull(); }); }); @@ -1335,7 +1338,7 @@ describe("declared access", () => { expect(await failure(system("resources: { read: [delta], write: [score] }", "(context) => { context.resources[score] = 1; }"))).toBe(unread); expect(await failure(system("resources: { read: [delta], write: [score] }", "({ resources }) => { resources[score] = 1; }"))).toBe(unread); expect(await failure(system("resources: { read: [delta], write: [score] }", "(context) => { const { resources } = context; resources[score] = 1; }"))).toBe(unread); - expect(await failure(system("resources: { read: [delta] }, input: { type: \"number\" }", "({ input }) => { console.log(input); }"))).toBe(unread); + expect(await failure(system("resources: { read: [delta] }", "(_context, input) => { console.log(input); }"))).toBe(unread); expect(await failure(system("resources: { write: [score, mass] }", "({ resources }) => { resources[score] = 1; }"))).toBe("mass is in resources write but never written; remove it, or move it to resources read if it's only read"); expect(await failure(system("components: { has: [mass] }, resources: { read: [delta], write: [score] }", "(_, context) => { context.resources[score] = context.resources[delta]; }"))).toBe("compiled"); @@ -1383,7 +1386,7 @@ describe("declared access", () => { }); test("finalizers are checked as main-thread bodies, in files that import only Finalizer", async () => { - const finalizing = (source: string) => transformKernels(source, "/finalizer.js", parse, async specifier => specifier === "ecs", index => JSON.stringify(`kernel:${index}`)); + const finalizing = (source: string) => transformKernels(source, "/finalizer.js", parse, async specifier => specifier === "ecs" ? ownCopy : null, index => JSON.stringify(`kernel:${index}`)); const failed = async (source: string) => { const result = await finalizing(source); return result !== null && "failure" in result ? result.failure.message : "compiled"; diff --git a/packages/ecs/src/implementation/compiler/transform.ts b/packages/ecs/src/implementation/compiler/transform.ts index 1bb93bb9..544f4b8c 100644 --- a/packages/ecs/src/implementation/compiler/transform.ts +++ b/packages/ecs/src/implementation/compiler/transform.ts @@ -5,7 +5,7 @@ import { children, isFunction } from './estree.js'; import { fail, failureOf, type Failure } from './failure.js'; import { isPending, type Link, type Linker } from './link.js'; import { createModule, locateIn, type Module } from './module-scope.js'; -import { attachKernelPath, prepareLookupPath, prepareSpawnPath } from './paths.js'; +import type { PackageCopy } from './paths.js'; import { spawnSites } from './spawns.js'; import { kernelOf } from './signature.js'; import { usedBy } from './body-access.js'; @@ -16,8 +16,11 @@ import { lookupSites } from './lookups.js'; import { terminalBodiesName } from '../world/terminal.js'; export type Parse = (code: string) => Program; -/** Whether an import specifier, resolved from the module being transformed, is the package's, which exports `System` and `Finalizer`. */ -export type ResolveSystemImport = (source: string) => Promise; +/** + * The copy of the package an import specifier, resolved from the module being transformed, is a module of that exports + * `System` and `Finalizer`; null when it's no such module. Compiled code imports the runtime from that copy. + */ +export type ResolveSystemImport = (source: string) => Promise; /** * The JS expression for kernel `index`'s module url. `identity` is everything the description and module are * compiled to (the description without its url, plus the module source), so it can version the url. @@ -34,17 +37,21 @@ export type TransformResult = type Edit = { readonly start: number; readonly end: number; readonly text: string }; /** The local names of the package's `System` and `Finalizer` namespaces, by the namespace each names. */ -const systemLocals = async (program: Program, resolveSystemImport: ResolveSystemImport): Promise> => { +/** The module's local names of `System` and `Finalizer`, and the copy of the package the first of them comes from. */ +const systemLocals = async (program: Program, resolveSystemImport: ResolveSystemImport): Promise<{ readonly locals: ReadonlyMap; readonly copy: PackageCopy | null }> => { const locals = new Map(); + let copy: PackageCopy | null = null; for (const statement of program.body) { if (statement.type !== "ImportDeclaration" || typeof statement.source.value !== "string") continue; const specifiers = statement.specifiers.flatMap(specifier => specifier.type === "ImportSpecifier" && specifier.imported.type === "Identifier" && (specifier.imported.name === "System" || specifier.imported.name === "Finalizer") ? [{ local: specifier.local.name, imported: specifier.imported.name }] : []); - if (specifiers.length > 0 && await resolveSystemImport(statement.source.value)) { + const resolved = specifiers.length > 0 ? await resolveSystemImport(statement.source.value) : null; + if (resolved !== null) { + copy ??= resolved; for (const { local, imported } of specifiers) locals.set(local, imported === "System" ? "System" : "Finalizer"); } } - return locals; + return { locals, copy }; }; type Created = "system" | "finalizer" | "changes"; @@ -128,8 +135,8 @@ export const transformKernels = async ( ): Promise => { if ((!code.includes("System") && !code.includes("Finalizer")) || id.includes("\0")) return null; const program = parse(code); - const locals = await systemLocals(program, resolveSystemImport); - if (locals.size === 0) return null; + const { locals, copy } = await systemLocals(program, resolveSystemImport); + if (copy === null) return null; const file = id.split("?")[0] ?? id; const module = createModule(file, program, locateIn(code)); linker?.add(module); @@ -230,10 +237,10 @@ export const transformKernels = async ( let result = code; for (const edit of [...edits].sort((a, b) => b.start - a.start)) result = result.slice(0, edit.start) + edit.text + result.slice(edit.end); const imports = [ - ...kernels.length > 0 ? [`import { attachKernel as ${attach} } from ${JSON.stringify(attachKernelPath)};`] : [], - ...spawning ? [`import { prepareSpawn as ${prepare} } from ${JSON.stringify(prepareSpawnPath)};`] : [], - ...adding ? [`import { prepareAdd as ${prepareAddName} } from ${JSON.stringify(prepareSpawnPath)};`] : [], - ...looking ? [`import { prepareLookup as ${prepareLookupName} } from ${JSON.stringify(prepareLookupPath)};`] : [], + ...kernels.length > 0 ? [`import { attachKernel as ${attach} } from ${JSON.stringify(copy.attachKernel)};`] : [], + ...spawning ? [`import { prepareSpawn as ${prepare} } from ${JSON.stringify(copy.preparedSpawn)};`] : [], + ...adding ? [`import { prepareAdd as ${prepareAddName} } from ${JSON.stringify(copy.preparedSpawn)};`] : [], + ...looking ? [`import { prepareLookup as ${prepareLookupName} } from ${JSON.stringify(copy.preparedLookup)};`] : [], ]; const marking = terminals ? [`function ${terminal}(body) { const set = Symbol.for(${JSON.stringify(terminalBodiesName)}); (globalThis[set] || (globalThis[set] = new WeakSet())).add(body); return body; }`] : []; return { code: [...imports, result, ...prepared, ...marking].join("\n"), kernels, dependencies: [...dependencies] }; diff --git a/packages/ecs/src/implementation/compiler/unit.ts b/packages/ecs/src/implementation/compiler/unit.ts index 71ef3575..c222785b 100644 --- a/packages/ecs/src/implementation/compiler/unit.ts +++ b/packages/ecs/src/implementation/compiler/unit.ts @@ -31,8 +31,13 @@ export type Unit = { readonly columnIndex: Map; /** Chunk columns: dense, indexed by `i >> chunkShift` rather than a strided row index. */ readonly chunks: Set; - /** Chunk columns stamped once per call before the row loop, so their per-row stamps are dropped. */ - readonly stampedPerCall: Set; + /** Block columns: dense, indexed by `i >> blockShift`. */ + readonly blocks: Set; + /** + * Chunk columns stamped once per call before the row loop, with their block columns, so their per-row stamps are + * dropped. + */ + readonly stampedPerCall: Map; /** Columns read where a boolean's identity is observable, so they need conversion. */ readonly booleans: Set; /** Every value assigned to each local, to propagate observation. */ @@ -80,7 +85,7 @@ export const createUnit = (signature: Signature, module: Module, lists: Lists, e const up = signature.up === null ? null : counter++; if (up !== null) names.set(up, "up"); return { - signature, module, lists, link, target, columns: [], columnIndex: new Map(), chunks: new Set(), stampedPerCall: new Set(), booleans: new Set(), assignments: new Map(), observed: new Set(), names, shapes: new Set(), reducedShapes: new Set(), keys: [], structuredValues: new Set(), spawns: [], accumulators: [], active: new Set(), + signature, module, lists, link, target, columns: [], columnIndex: new Map(), chunks: new Set(), blocks: new Set(), stampedPerCall: new Map(), booleans: new Set(), assignments: new Map(), observed: new Set(), names, shapes: new Set(), reducedShapes: new Set(), keys: [], structuredValues: new Set(), spawns: [], accumulators: [], active: new Set(), used: { read: new Set(), write: new Set(), resources: new Set(), lookups: new Map(), up: new Set() }, up, upColumns: [], upColumnIndex: new Map(), upBooleans: new Set(), probes: [], rowLeaves: [], handles: new Set(), next: () => counter++, expression, statement, diff --git a/packages/ecs/src/implementation/world/archetype.ts b/packages/ecs/src/implementation/world/archetype.ts index 1af69b6d..5f954509 100644 --- a/packages/ecs/src/implementation/world/archetype.ts +++ b/packages/ecs/src/implementation/world/archetype.ts @@ -2,7 +2,7 @@ import type { Entity } from '../../interface/entity/entity.js'; import type { Layout, Views } from '../../interface/registry/layout.js'; import { allocateArray, allocateViews, emptyViews, freeArray, freeViews, growArray } from './arrays.js'; -import { chunkShift } from './kernel.js'; +import { blockShift, chunkShift } from './kernel.js'; import type { Memory } from './memory.js'; import { nullTag, startTag, type MoveLog } from './move-log.js'; import type { StringStore } from './strings.js'; @@ -40,6 +40,12 @@ export type Storage = { /** The newest `added` and `changed` tick in each chunk of `2 ** chunkShift` rows, so change filters skip unchanged chunks. */ chunkAdded: Float64Array; chunkChanged: Float64Array; + /** + * The newest `added` and `changed` tick in each block of `2 ** blockShift` rows, so change filters skip the + * unchanged blocks of a chunk that changed: a filter's scan costs what changed, not the chunk. + */ + blockAdded: Float64Array; + blockChanged: Float64Array; /** * The newest tick at which every row may have changed, as a GPU kernel's writes do, so stamping them costs * nothing per row: change filters pass the whole column when it's newer than their `lastRun`. @@ -73,21 +79,32 @@ export const writeOf = (layout: Layout): { readonly write: number; readonly unit /** Chunks covering `capacity` rows. */ const chunksFor = (capacity: number): number => Math.max(1, Math.ceil(capacity / 2 ** chunkShift)); +/** Blocks covering those chunks, so every row of a chunk has a block. */ +const blocksFor = (capacity: number): number => chunksFor(capacity) << (chunkShift - blockShift); /** What stamping a row's ticks writes: a storage's, or a joined thread's view of one. */ -export type Stamped = Pick; +export type Stamped = Pick; -/** Stamps a row's `changed` tick and its chunk's. Ticks only increase, so the chunk's is the newest. */ +/** + * Stamps a row's `changed` tick, its block's and its chunk's. Ticks only increase, so theirs are the newest. Every + * writer stores the row, then the block, then the chunk, the reverse of the order filters read them, so a filter on + * another thread that finds a chunk newer finds its block and row newer too. + */ export const stampChanged = (storage: Stamped, row: number, tick: number): void => { if (!storage.ticked) return; storage.changed[row] = tick; + storage.blockChanged[row >> blockShift] = tick; storage.chunkChanged[row >> chunkShift] = tick; }; -export const stampAdded = (storage: Stamped, row: number, tick: number): void => { - if (!storage.ticked) return; - storage.added[row] = tick; - storage.chunkAdded[row >> chunkShift] = tick; +/** + * Raises the block and chunk ticks over rows `start` to `end - 1` to at least `tick`: for ticks written to those rows + * in bulk, or copied in from rows of other chunks. + */ +export const raiseRange = (chunks: Float64Array, blocks: Float64Array, start: number, end: number, tick: number): void => { + if (start >= end) return; + for (let block = start >> blockShift, last = (end - 1) >> blockShift; block <= last; block++) if (blocks[block] < tick) blocks[block] = tick; + for (let chunk = start >> chunkShift, last = (end - 1) >> chunkShift; chunk <= last; chunk++) if (chunks[chunk] < tick) chunks[chunk] = tick; }; /** A table of the entities that have exactly `components`. Rows are dense: `0` to `count`. */ @@ -156,6 +173,8 @@ const allocateTicks = (memory: Memory, storage: Storage, capacity: number, tick: storage.changed = storage.layout.kind === "constant" ? storage.added : allocateArray(memory, Float64Array, capacity).fill(tick); storage.chunkAdded = allocateArray(memory, Float64Array, chunksFor(capacity)).fill(tick); storage.chunkChanged = storage.layout.kind === "constant" ? storage.chunkAdded : allocateArray(memory, Float64Array, chunksFor(capacity)).fill(tick); + storage.blockAdded = allocateArray(memory, Float64Array, blocksFor(capacity)).fill(tick); + storage.blockChanged = storage.layout.kind === "constant" ? storage.blockAdded : allocateArray(memory, Float64Array, blocksFor(capacity)).fill(tick); }; /** Views of a new column, for a linear layout's or a string's `capacity` rows; a string's handles start at 0. */ @@ -181,6 +200,8 @@ const createStorage = (memory: Memory, strings: StringStore | null, component: n changed: noTicks, chunkAdded: noTicks, chunkChanged: noTicks, + blockAdded: noTicks, + blockChanged: noTicks, columnChanged: 0, remote: null, tags: noTags, @@ -262,6 +283,10 @@ const grow = (memory: Memory, archetype: Archetype, capacity = archetype.capacit const used = storage.chunkAdded.length; storage.chunkAdded = growArray(memory, storage.chunkAdded, Float64Array, chunks, used).fill(0, used); storage.chunkChanged = shared ? storage.chunkAdded : growArray(memory, storage.chunkChanged, Float64Array, chunks, used).fill(0, used); + const blocks = blocksFor(capacity); + const usedBlocks = storage.blockAdded.length; + storage.blockAdded = growArray(memory, storage.blockAdded, Float64Array, blocks, usedBlocks).fill(0, usedBlocks); + storage.blockChanged = shared ? storage.blockAdded : growArray(memory, storage.blockChanged, Float64Array, blocks, usedBlocks).fill(0, usedBlocks); } } archetype.entities = growArray(memory, archetype.entities, Float64Array, capacity, count); @@ -330,7 +355,10 @@ export const copyCell = (from: Storage, fromRow: number, to: Storage, toRow: num const changed = from.changed[fromRow]; to.added[toRow] = added; to.changed[toRow] = changed; - // A chunk's tick is the newest of its rows, so a row moving in can raise it. + // A block's and a chunk's tick are the newest of their rows, so a row moving in can raise them. + const block = toRow >> blockShift; + if (added > to.blockAdded[block]) to.blockAdded[block] = added; + if (changed > to.blockChanged[block]) to.blockChanged[block] = changed; const chunk = toRow >> chunkShift; if (added > to.chunkAdded[chunk]) to.chunkAdded[chunk] = added; if (changed > to.chunkChanged[chunk]) to.chunkChanged[chunk] = changed; diff --git a/packages/ecs/src/implementation/world/async-loading.test.ts b/packages/ecs/src/implementation/world/async-loading.test.ts index 7236d81a..e99347e5 100644 --- a/packages/ecs/src/implementation/world/async-loading.test.ts +++ b/packages/ecs/src/implementation/world/async-loading.test.ts @@ -67,9 +67,8 @@ const request = System.create({ // 3. A completed load arrives as a run of its own. A slot whose token still matches gets the mesh; a stale one releases it. const apply = System.create({ name: "asyncLoading/apply", - input: { type: "object", properties: { entity: Entity.schema, token: { type: "integer" }, handle: { type: "integer" } } }, lookup: { slots: { read: [meshSlot] } }, -}, ({ input, lookup, commands }) => { +}, ({ lookup, commands }, input: { readonly entity: Entity; readonly token: number; readonly handle: number }) => { if (lookup.slots(input.entity)?.[meshSlot] === input.token) commands.add(input.entity, meshReady, input.handle); else loaded.delete(input.handle); }); @@ -81,7 +80,7 @@ const spawn = System.create({ name: "asyncLoading/spawn" }, ({ commands }) => { commands.spawn([meshSource])({ [meshSource]: "ship.glb" }); commands.spawn([meshSource])({ [meshSource]: "rock.glb" }); }); -const hide = System.create({ name: "asyncLoading/hide", input: { type: "object", properties: { entity: Entity.schema } } }, ({ input, commands }) => { +const hide = System.create({ name: "asyncLoading/hide" }, ({ commands }, input: { readonly entity: Entity }) => { commands.add(input.entity, hidden); }); diff --git a/packages/ecs/src/implementation/world/batches.ts b/packages/ecs/src/implementation/world/batches.ts index fed392c9..07166eb5 100644 --- a/packages/ecs/src/implementation/world/batches.ts +++ b/packages/ecs/src/implementation/world/batches.ts @@ -2,7 +2,7 @@ import { ensureCapacity, type Archetype } from './archetype.js'; import type { CommandBuffer } from './buffer.js'; import { indexOf } from './entities.js'; -import { chunkShift } from './kernel.js'; +import { blockShift, chunkShift } from './kernel.js'; import { createBatch, type SpawnBatch } from './staging.js'; import { archetypeOf, type State } from './state.js'; import { gainsOf } from './structure.js'; @@ -82,7 +82,9 @@ const commitRows = (state: State, batch: SpawnBatch, first: number, n: number, t if (!storage.ticked) continue; storage.added.fill(tick, start, start + n); if (storage.changed !== storage.added) storage.changed.fill(tick, start, start + n); - // Ticks only increase, so the newest tick in each chunk the rows reach is this one. + // Ticks only increase, so the newest tick in each chunk and block the rows reach is this one. + storage.blockAdded.fill(tick, start >> blockShift, ((start + n - 1) >> blockShift) + 1); + if (storage.blockChanged !== storage.blockAdded) storage.blockChanged.fill(tick, start >> blockShift, ((start + n - 1) >> blockShift) + 1); storage.chunkAdded.fill(tick, start >> chunkShift, ((start + n - 1) >> chunkShift) + 1); if (storage.chunkChanged !== storage.chunkAdded) storage.chunkChanged.fill(tick, start >> chunkShift, ((start + n - 1) >> chunkShift) + 1); } diff --git a/packages/ecs/src/implementation/world/change-blocks.browser.test.ts b/packages/ecs/src/implementation/world/change-blocks.browser.test.ts new file mode 100644 index 00000000..6dcba44a --- /dev/null +++ b/packages/ecs/src/implementation/world/change-blocks.browser.test.ts @@ -0,0 +1,84 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, it } from "vitest"; +import { Component } from "../../interface/component/component.js"; +import { Entity } from "../../interface/entity/entity.js"; +import { gpuDevice } from "../../interface/resources/gpu-device.js"; +import { workers } from "../../interface/resources/workers.js"; +import { System } from "../../interface/system/system.js"; +import { createWorld, join } from "./create.js"; + +// A GPU kernel stamps a whole column once; before a joined thread's systems filter on it, its rows, chunks and blocks +// take the stamp, so the thread's change filters find every row, and then only rows written after. + +const value = Symbol("changeBlocksGpuTest/value"); +const at = Symbol("changeBlocksGpuTest/at"); +const note = Symbol("changeBlocksGpuTest/note"); + +declare module "@adobe/ecs" { + interface Threads { + changeBlocksGpuTest: true; + } + interface Components { + [value]: { type: number }; + [at]: { type: number }; + [note]: { type: { readonly text: string } | null; thread: "changeBlocksGpuTest" }; + } +} +Component.register(value, { type: { type: "number", precision: 1 } }); +Component.register(at, { type: { type: "integer", minimum: 0, maximum: 0x7FFFFFFF } }); +const text = { type: "object", properties: { text: { type: "string" } } } as const; +Component.register(note, { type: { anyOf: [text, { type: "null" }], default: { text: "none" } }, thread: "changeBlocksGpuTest" }); + +const rows = 4500; +let entities: Entity[] = []; +const seed = System.create({ name: "changeBlocksGpuTest/seed" }, ({ commands }) => { + entities = []; + for (let i = 0; i < rows; i++) entities.push(commands.spawn([value, at, note])({ [value]: 0, [at]: i })); +}); +const raise = System.create({ name: "changeBlocksGpuTest/raise", kernel: "gpu", components: { write: [value] } }, row => { + row[value] = row[value] + 1; +}); +const setValue = System.create({ name: "changeBlocksGpuTest/setValue", target: "entity", components: { write: [value] } }, (row, _context, input: { readonly entity: Entity }) => { + row[value] += 1; +}); +let found: number[] = []; +const there = System.create({ name: "changeBlocksGpuTest/there", components: { read: [at, note], changed: [value] } }, row => { if (row[note] !== undefined) found.push(row[at]); }); + +const device = async (): Promise => { + const adapter = await navigator.gpu.requestAdapter(); + if (adapter === null) throw new Error("No WebGPU adapter."); + return adapter.requestDevice(); +}; + +describe("chunk and block ticks after a GPU kernel's column stamp", () => { + it("let a joined thread's filters find every row a GPU kernel wrote, then a row the CPU wrote", async () => { + const world = createWorld({ [workers]: 0, [gpuDevice]: await device() }); + const channel = new MessageChannel(); + world.share(channel.port1); + const remote = join(channel.port2, { threads: ["changeBlocksGpuTest"] }); + try { + await new Promise(resolve => { + const stop = world.threads(threads => { + if (threads.at(-1)?.status !== "joined") return; + queueMicrotask(() => stop()); + resolve(); + }); + }); + await world.run(seed); + await world.run(there); + await world.run(raise); + found = []; + await world.run(there); + expect(found.length).toBe(rows); + const row = 2 * 1024 + 5 * 64 + 9; + await world.run(setValue, { entity: entities[row] }); + found = []; + await world.run(there); + expect(found).toEqual([row]); + } + finally { + await remote.dispose(); + await world.dispose(); + } + }); +}); diff --git a/packages/ecs/src/implementation/world/change-blocks.cases.ts b/packages/ecs/src/implementation/world/change-blocks.cases.ts new file mode 100644 index 00000000..f8a219e3 --- /dev/null +++ b/packages/ecs/src/implementation/world/change-blocks.cases.ts @@ -0,0 +1,14 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, it } from "vitest"; +import type * as Fixture from "./change-blocks.fixture.js"; + +/** Chunk and block ticks on workers: the cases, over the fixture built for node or from source. */ +export const changeBlocksCases = (load: () => Promise) => describe("chunk and block ticks on workers", () => { + for (const wasm of [false, true]) { + it(`let readers find the rows a kernel on workers wrote (${wasm ? "WASM" : "JS"} kernels)`, async () => { + const fixture = await load(); + const expected = [5, 2048 + 5 * 64 + 9, 4499]; + expect(await fixture.foundOnWorkers(wasm)).toEqual({ kernel: expected, main: expected }); + }); + } +}); diff --git a/packages/ecs/src/implementation/world/change-blocks.fixture.ts b/packages/ecs/src/implementation/world/change-blocks.fixture.ts new file mode 100644 index 00000000..d475bb20 --- /dev/null +++ b/packages/ecs/src/implementation/world/change-blocks.fixture.ts @@ -0,0 +1,73 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { Component } from "../../interface/component/component.js"; +import type { I32 } from "../../interface/math/i32/index.js"; +import { workers } from "../../interface/resources/workers.js"; +import { System } from "../../interface/system/system.js"; +import { readEach, select } from "./reads.test-helper.js"; +import { createWorld } from "./create.js"; + +/** Scenarios of chunk and block ticks on workers, run built in node and from source in chromium. */ + +const value = Symbol("changeBlocksWorkersTest/value"); +const at = Symbol("changeBlocksWorkersTest/at"); +const seen = Symbol("changeBlocksWorkersTest/seen"); + +declare module "@adobe/ecs" { + export interface Components { + [value]: { type: number }; + [at]: { type: number }; + [seen]: { type: number }; + } +} + +Component.register(value, { type: { type: "number" } }); +Component.register(at, { type: { type: "integer", minimum: 0, maximum: 0x7FFFFFFF } }); +Component.register(seen, { type: { type: "number" } }); + +const rows = 4500; + +const seed = System.create({ name: "changeBlocksWorkersTest/seed" }, ({ commands }) => { + for (let i = 0; i < rows; i++) commands.spawn([value, at, seen])({ [value]: 0, [at]: i, [seen]: 0 }); +}); + +/** Writes the rows whose `at` is one of three, stamping row by row. */ +const writeSome = System.create({ name: "changeBlocksWorkersTest/writeSome", kernel: "cpu", components: { read: [at], write: [value] } }, (row, _context, input: { readonly a: I32; readonly b: I32; readonly c: I32 }) => { + const here = row[at]; + if (here === input.a || here === input.b || here === input.c) row[value] += 1; +}); + +/** Counts each run of the rows whose value changed. */ +const watch = System.create({ name: "changeBlocksWorkersTest/watch", kernel: "cpu", components: { write: [seen], changed: [value] } }, row => { + row[seen] += 1; +}); + +let found: number[] = []; +const onMain = System.create({ name: "changeBlocksWorkersTest/onMain", components: { read: [at], changed: [value] } }, row => { found.push(row[at]); }); + +/** + * The rows a kernel on workers and a system on the world's thread find after a kernel on workers wrote three rows far + * apart: in the first block, a middle block of a middle chunk, and the last row. + */ +export const foundOnWorkers = async (wasm: boolean): Promise<{ readonly kernel: readonly number[]; readonly main: readonly number[] }> => { + const world = createWorld({ [workers]: 2 }, { wasm, minimumChunk: 64 }); + try { + await world.run(seed); + await world.run(watch); + await world.run(onMain); + const all = await select(world, { required: [seen] }); + const before = await readEach(world, all, seen); + await world.run(writeSome, { a: 5, b: 2048 + 5 * 64 + 9, c: rows - 1 }); + await world.run(watch); + found = []; + await world.run(onMain); + const after = await readEach(world, all, seen); + const ats = await readEach(world, all, at); + return { + kernel: all.flatMap((_, k) => after[k] !== before[k] ? [ats[k] ?? -1] : []).sort((a, b) => a - b), + main: [...found].sort((a, b) => a - b), + }; + } + finally { + await world.dispose(); + } +}; diff --git a/packages/ecs/src/implementation/world/change-blocks.test.ts b/packages/ecs/src/implementation/world/change-blocks.test.ts new file mode 100644 index 00000000..add37fb4 --- /dev/null +++ b/packages/ecs/src/implementation/world/change-blocks.test.ts @@ -0,0 +1,280 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, it } from "vitest"; +import { Archetype } from "../../interface/archetype/archetype.js"; +import { Component } from "../../interface/component/component.js"; +import { Entity } from "../../interface/entity/entity.js"; +import { Index } from "../../interface/indexes/index-type.js"; +import type { I32 } from "../../interface/math/i32/index.js"; +import { workers } from "../../interface/resources/workers.js"; +import { System } from "../../interface/system/system.js"; +import type { World } from "../../interface/world/world.js"; +import { createWorld, join } from "./create.js"; +import { blockShift, chunkShift } from "./kernel.js"; +import { readEach, select } from "./reads.test-helper.js"; + +// Change filters skip chunks and blocks whose ticks aren't newer than their `lastRun`, so every writer of a row's tick +// raises its chunk's and block's. Each case writes a few rows far apart in an archetype of many chunks, through one +// writer, and every kind of reader must find exactly those rows. + +const value = Symbol("changeBlocksTest/value"); +const at = Symbol("changeBlocksTest/at"); +const seen = Symbol("changeBlocksTest/seen"); +const group = Symbol("changeBlocksTest/group"); +const tagged = Symbol("changeBlocksTest/tagged"); +const note = Symbol("changeBlocksTest/note"); +const parent = Symbol("changeBlocksTest/parent"); +const extra = Symbol("changeBlocksTest/extra"); +const height = Symbol("changeBlocksTest/height"); + +declare module "@adobe/ecs" { + interface Threads { + changeBlocksTest: true; + } + interface Components { + [value]: { type: number }; + [at]: { type: number }; + [seen]: { type: number }; + [group]: { type: number }; + [tagged]: { type: true }; + [note]: { type: { readonly text: string } | null; thread: "changeBlocksTest" }; + [parent]: { type: Entity; reference: "owner" }; + [extra]: { type: number }; + [height]: { type: number }; + } +} +Component.register(value, { type: { type: "number" } }); +Component.register(at, { type: { type: "integer", minimum: 0, maximum: 0x7FFFFFFF } }); +Component.register(seen, { type: { type: "number" } }); +Component.register(group, { type: { type: "integer", minimum: 0, maximum: 1000 } }); +Component.register(tagged, { type: { const: true } }); +const text = { type: "object", properties: { text: { type: "string" } } } as const; +Component.register(note, { type: { anyOf: [text, { type: "null" }], default: { text: "none" } }, thread: "changeBlocksTest" }); +Component.register(parent, { type: Entity.schema, reference: "owner" }); +Component.register(extra, { type: { type: "number" } }); +Component.register(height, { type: { type: "number" } }); + +/** Rows in the archetype: five chunks, the last partial. */ +const rows = 4500; +const plain = Archetype.create([value, at, seen, group]); +const marked = Archetype.create([value, at, seen, group, tagged]); +const noted = Archetype.create([value, at, note]); + +let entities: Entity[] = []; +const seed = System.create({ name: "changeBlocksTest/seed" }, ({ commands }) => { + entities = []; + for (let i = 0; i < rows; i++) entities.push(commands.spawn(plain)({ [value]: 0, [at]: i, [seen]: 0, [group]: 0 })); + // Rows an entity tagged later moves after, so it lands far into that archetype. + for (let i = rows; i < 2 * rows; i++) entities.push(commands.spawn(marked)({ [value]: 0, [at]: i, [seen]: 0, [group]: 0, [tagged]: true })); +}); + +// Writers. +const setValue = System.create({ name: "changeBlocksTest/setValue", target: "entity", components: { write: [value] } }, (row, _context, input: { readonly entity: Entity }) => { + row[value] += 1; +}); +const replace = System.create({ name: "changeBlocksTest/replace" }, ({ commands }, input: { readonly entities: readonly Entity[] }) => { + for (const entity of input.entities) commands.add(entity, value, 7); +}); +const despawn = System.create({ name: "changeBlocksTest/despawn" }, ({ commands }, input: { readonly entities: readonly Entity[] }) => { + for (const entity of input.entities) commands.despawn(entity); +}); +const tag = System.create({ name: "changeBlocksTest/tag" }, ({ commands }, input: { readonly entities: readonly Entity[] }) => { + for (const entity of input.entities) commands.add(entity, tagged); +}); +const spawnOne = System.create({ name: "changeBlocksTest/spawnOne" }, ({ commands }, input: { readonly at: number }) => { + commands.spawn(plain)({ [value]: 0, [at]: input.at, [seen]: 0, [group]: 0 }); +}); +/** Writes the rows whose `at` is in a list of up to four, which stamps row by row. */ +const writeSome = System.create({ name: "changeBlocksTest/writeSome", kernel: "cpu", components: { read: [at], write: [value] } }, (row, _context, input: { readonly a: I32; readonly b: I32; readonly c: I32; readonly d: I32 }) => { + const here = row[at]; + if (here === input.a || here === input.b || here === input.c || here === input.d) row[value] += 1; +}); +/** Writes every row, which stamps chunks and blocks once per call. */ +const writeAll = System.create({ name: "changeBlocksTest/writeAll", kernel: "cpu", components: { write: [value] } }, row => { + row[value] += 1; +}); +const setGroup = System.create({ name: "changeBlocksTest/setGroup", target: "entity", components: { write: [group] } }, (row, _context, input: { readonly entity: Entity }) => { + row[group] = 1; +}); + +// Readers. +let found: number[] = []; +const onMain = System.create({ name: "changeBlocksTest/onMain", components: { read: [at], changed: [value] } }, row => { found.push(row[at]); }); +const observed = System.create({ name: "changeBlocksTest/observed", observer: true, components: { read: [at], changed: [value] } }, row => { found.push(row[at]); }); +const ofTwo = System.create({ name: "changeBlocksTest/ofTwo", components: { read: [at], changed: [group, value] } }, row => { found.push(row[at]); }); +const inKernel = System.create({ name: "changeBlocksTest/inKernel", kernel: "cpu", components: { write: [seen], changed: [value] } }, row => { row[seen] += 1; }); +const inUp = System.create({ name: "changeBlocksTest/inUp", components: { read: [at], changed: [value] }, up: { through: parent, read: [value] } }, row => { found.push(row[at]); }); +const byGroup = Index.create({ name: "changeBlocksTest/byGroup", key: [group] }); +let grouped: number[] = []; +const inGroup = System.create({ name: "changeBlocksTest/inGroup", lookup: { byGroup: { index: byGroup } } }, ({ lookup }) => { + const list = lookup.byGroup(1); + grouped = Array.from({ length: list.length }, (_, i) => list.at(i).entity); +}); + +// On a joined thread, by the object they read: a writer, and readers of its writes and of the world's thread's. +const seedNoted = System.create({ name: "changeBlocksTest/seedNoted" }, ({ commands }) => { + entities = []; + for (let i = 0; i < rows; i++) entities.push(commands.spawn(noted)({ [value]: 0, [at]: i })); +}); +const noteSome = System.create({ name: "changeBlocksTest/noteSome", components: { read: [at], write: [note] } }, (row, _context, input: { readonly ats: readonly number[] }) => { + if (input.ats.includes(row[at])) row[note] = { text: "noted" }; +}); +const notesThere = System.create({ name: "changeBlocksTest/notesThere", components: { read: [at, note], changed: [note] } }, row => { if (row[note] !== undefined) found.push(row[at]); }); +const valuesThere = System.create({ name: "changeBlocksTest/valuesThere", components: { read: [at, note], changed: [value] } }, row => { if (row[note] !== undefined) found.push(row[at]); }); + +/** The `at` of the rows a main-thread reader ran, in order. */ +const ranOn = async (world: World, reader: System): Promise => { + found = []; + await world.run(reader); + return [...found].sort((a, b) => a - b); +}; + +/** The `at` of the rows the kernel reader ran, by the `seen` it raised. */ +const ranInKernel = async (world: World): Promise => { + const all = await select(world, { required: [seen] }); + const before = await readEach(world, all, seen); + await world.run(inKernel); + const after = await readEach(world, all, seen); + const ats = await readEach(world, all, at); + return all.flatMap((_, k) => after[k] !== before[k] ? [ats[k] ?? -1] : []).sort((a, b) => a - b); +}; + +const readers = { onMain, observed, ofTwo, inUp } as const; + +/** Every reader's rows after `write`, each having run once before, so each sees only what `write` changed. */ +const seenAfter = async (world: World, write: () => Promise): Promise<{ readonly [reader: string]: readonly number[] }> => { + for (const reader of Object.values(readers)) await world.run(reader); + await world.run(inKernel); + await write(); + const result: { [reader: string]: readonly number[] } = {}; + for (const [name, reader] of Object.entries(readers)) result[name] = await ranOn(world, reader); + result.inKernel = await ranInKernel(world); + return result; +}; + +const every = (expected: readonly number[]) => ({ onMain: expected, observed: expected, ofTwo: expected, inUp: expected, inKernel: expected }); + +/** Rows far apart: in the first block, a middle block of a middle chunk, and the last chunk's last block. */ +const spread = [5, (2 << chunkShift) + (5 << blockShift) + 9, rows - 1] as const; + +describe("chunk and block ticks", () => { + for (const wasm of [false, true]) { + const label = wasm ? "WASM" : "JS"; + const inline = () => createWorld({ [workers]: 0 }, { wasm }); + + it(`let readers find rows a main-thread system wrote (${label} kernels)`, async () => { + await using world = inline(); + await world.run(seed); + expect(await seenAfter(world, async () => { + for (const row of spread) await world.run(setValue, { entity: entities[row] }); + })).toEqual(every(spread)); + }); + + it(`let readers find rows a kernel wrote, row by row and once per call (${label} kernels)`, async () => { + await using world = inline(); + await world.run(seed); + expect(await seenAfter(world, () => world.run(writeSome, { a: spread[0], b: spread[1], c: spread[2], d: -1 }))).toEqual(every(spread)); + const all = Array.from({ length: 2 * rows }, (_, k) => k); + expect(await seenAfter(world, () => world.run(writeAll))).toEqual(every(all)); + }); + + it(`let readers find rows commands wrote, moved, swapped into and spawned (${label} kernels)`, async () => { + await using world = inline(); + await world.run(seed); + expect(await seenAfter(world, () => world.run(replace, { entities: spread.map(row => entities[row]) }))).toEqual(every(spread)); + // The last row, changed, is swapped into the first block's despawned row. + const last = rows - 1; + expect(await seenAfter(world, async () => { + await world.run(setValue, { entity: entities[last] }); + await world.run(despawn, { entities: [entities[3]] }); + })).toEqual(every([last])); + // A changed row moves to the end of another archetype, past its first chunks. + const moved = spread[1]; + expect(await seenAfter(world, async () => { + await world.run(setValue, { entity: entities[moved] }); + await world.run(tag, { entities: [entities[moved]] }); + })).toEqual(every([moved])); + expect(await seenAfter(world, () => world.run(spawnOne, { at: 9999 }))).toEqual(every([9999])); + }); + } + + it("let a filter of two components find a row only the other component changed in", async () => { + await using world = createWorld({ [workers]: 0 }); + await world.run(seed); + await world.run(ofTwo); + await world.run(setGroup, { entity: entities[spread[1]] }); + expect(await ranOn(world, ofTwo)).toEqual([spread[1]]); + }); + + it("let systems on a joined thread find rows written there and on the world's thread", async () => { + const world = createWorld({ [workers]: 0 }); + const channel = new MessageChannel(); + world.share(channel.port1); + const remote = join(channel.port2, { threads: ["changeBlocksTest"] }); + try { + await new Promise(resolve => { + const stop = world.threads(threads => { + if (threads.at(-1)?.status !== "joined") return; + queueMicrotask(() => stop()); + resolve(); + }); + }); + await world.run(seedNoted); + await world.run(valuesThere); + await world.run(notesThere); + for (const row of spread) await world.run(setValue, { entity: entities[row] }); + expect(await ranOn(world, valuesThere)).toEqual(spread); + await world.run(noteSome, { ats: [spread[1], spread[2]] }); + expect(await ranOn(world, notesThere)).toEqual([spread[1], spread[2]]); + } + finally { + await remote.dispose(); + await world.dispose(); + } + }); + + it("let an index refresh find a key written in a block of a chunk", async () => { + await using world = createWorld({ [workers]: 0 }); + await world.run(seed); + await world.run(inGroup); + expect(grouped).toEqual([]); + await world.run(setGroup, { entity: entities[spread[1]] }); + await world.run(inGroup); + expect(grouped).toEqual([entities[spread[1]]]); + }); + + it("let an added filter find a component a command added, and that row once swapped into an old block", async () => { + await using world = createWorld({ [workers]: 0 }); + await world.run(seedExtra); + await world.run(addedExtra); + // Rows `rows` and `rows + 1` gain the component and land after the archetype's rows, in its last block. + await world.run(addExtra, { entity: entities[rows] }); + expect(await ranOn(world, addedExtra)).toEqual([rows]); + // Its last row, just added, moves into the first block's despawned row. + await world.run(addExtra, { entity: entities[rows + 1] }); + await world.run(despawn, { entities: [entities[3]] }); + expect(await ranOn(world, addedExtra)).toEqual([rows + 1]); + }); + + it("let filters find what a JS kernel reading up wrote on every row", async () => { + await using world = createWorld({ [workers]: 0 }, { wasm: false }); + await world.run(seedExtra); + await world.run(heights); + await world.run(raiseHeights); + expect((await ranOn(world, heights)).length).toBe(rows + 2); + }); +}); + +// Rows that have `extra` from the start, then two that gain it; and a JS `up` kernel writing every row's height. +const seedExtra = System.create({ name: "changeBlocksTest/seedExtra" }, ({ commands }) => { + entities = []; + for (let i = 0; i < rows; i++) entities.push(commands.spawn([at, extra, height])({ [at]: i, [extra]: 0, [height]: 0 })); + for (let i = rows; i < rows + 2; i++) entities.push(commands.spawn([at, height])({ [at]: i, [height]: 0 })); +}); +const addExtra = System.create({ name: "changeBlocksTest/addExtra" }, ({ commands }, input: { readonly entity: Entity }) => { + commands.add(input.entity, extra, 1); +}); +const addedExtra = System.create({ name: "changeBlocksTest/addedExtra", components: { read: [at], added: [extra] } }, row => { found.push(row[at]); }); +const raiseHeights = System.create({ name: "changeBlocksTest/raiseHeights", kernel: "cpu", components: { write: [height] }, up: { through: parent, read: [height] } }, (row, { up }) => { + row[height] = up === null ? 1 : up[height] + 1; +}); +const heights = System.create({ name: "changeBlocksTest/heights", components: { read: [at], changed: [height] } }, row => { found.push(row[at]); }); diff --git a/packages/ecs/src/implementation/world/check-kernel.ts b/packages/ecs/src/implementation/world/check-kernel.ts index 38e974cb..da60b0cf 100644 --- a/packages/ecs/src/implementation/world/check-kernel.ts +++ b/packages/ecs/src/implementation/world/check-kernel.ts @@ -42,7 +42,7 @@ export const checkKernel = (leaf: Leaf, description: KernelDescription): void => for (const column of description.columns) { if (column.kind === "entity") continue; if (!queried.has(column.component)) fail(`"${labelOf(column.component)}" isn't in its components' read or write.`); - if (column.kind === "changed" || column.kind === "chunks") { + if (column.kind === "changed" || column.kind === "chunks" || column.kind === "blocks") { if (!written.has(column.component)) fail(`writes "${labelOf(column.component)}", which isn't in its components' write.`); continue; } diff --git a/packages/ecs/src/implementation/world/command-rings.fixture.ts b/packages/ecs/src/implementation/world/command-rings.fixture.ts index 2fdfdb17..2f0a65e0 100644 --- a/packages/ecs/src/implementation/world/command-rings.fixture.ts +++ b/packages/ecs/src/implementation/world/command-rings.fixture.ts @@ -49,7 +49,7 @@ Component.register(name, { type: { anyOf: [{ type: "string" }, { type: "null" }] Component.register(maybe, { type: { anyOf: [{ type: "string" }, { type: "null" }] } }); Component.register(tally, { type: { anyOf: [{ type: "number" }, { type: "null" }] } }); -const seed = System.create({ name: "commandRingsTest/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const seed = System.create({ name: "commandRingsTest/seed" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) commands.spawn([index])({ [index]: i }); }); @@ -60,9 +60,8 @@ const seed = System.create({ name: "commandRingsTest/seed", input: { type: "obje const burst = System.create({ name: "commandRingsTest/burst", kernel: "cpu", - input: { type: "object", properties: { shift: { type: "number" } } }, components: { read: [index] }, -}, (row, { input, commands }) => { +}, (row, { commands }, input: { readonly shift: number }) => { const i = row[index]; commands.add(row.entity, weight, -1); commands.add(row.entity, weight, i + input.shift); @@ -106,7 +105,7 @@ export const overflow = (count: number): Promise<{ readonly rows: readonly strin }; /** Rows targeting a live entity, but for the last, whose target never lived. */ -const targets = System.create({ name: "commandRingsTest/targets", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const targets = System.create({ name: "commandRingsTest/targets" }, ({ commands }, input: { readonly count: number }) => { const first = commands.spawn([index])({ [index]: 0 }); for (let i = 1; i < input.count; i++) commands.spawn([index, target])({ [index]: i, [target]: i === input.count - 1 ? Entity.none : first }); }); diff --git a/packages/ecs/src/implementation/world/composite-input.test.ts b/packages/ecs/src/implementation/world/composite-input.test.ts index c41b86b0..2e7372ef 100644 --- a/packages/ecs/src/implementation/world/composite-input.test.ts +++ b/packages/ecs/src/implementation/world/composite-input.test.ts @@ -7,17 +7,14 @@ import { workers } from "../../interface/resources/workers.js"; const seen: { readonly system: string; readonly input: unknown }[] = []; const move = System.create({ name: "compositeInputTest/move", - input: { type: "object", properties: { distance: { type: "number" } } }, -}, ({ input }) => { seen.push({ system: "move", input: { ...input } }); }); +}, (_context, input: { readonly distance: number }) => { seen.push({ system: "move", input: { ...input } }); }); const step = System.create({ name: "compositeInputTest/step", - input: { type: "object", properties: { distance: { type: "number" }, dt: { type: "number" } } }, -}, ({ input }) => { seen.push({ system: "step", input: { ...input } }); }); +}, (_context, input: { readonly distance: number; readonly dt: number }) => { seen.push({ system: "step", input: { ...input } }); }); const idle = System.create({ name: "compositeInputTest/idle" }, () => {}); const label = System.create({ name: "compositeInputTest/label", - input: { type: "object", properties: { distance: { type: "string" } } }, -}, ({ input }) => { void input.distance; }); +}, (_context, input: { readonly distance: string }) => { void input.distance; }); describe("composite input", () => { it("passes each member the fields of the run's input that it declares", async () => { @@ -31,8 +28,13 @@ describe("composite input", () => { expect(() => System.create({ name: "compositeInputTest/conflict", systems: [move, label] })).toThrow('Systems "compositeInputTest/move" and "compositeInputTest/label" declare input field "distance" with different schemas.'); }); - it("rejects an input that isn't an object of named fields, for untyped declarations", () => { - // @ts-expect-error input is an object schema - expect(() => System.create({ name: "compositeInputTest/scalar", input: { type: "number" } }, () => {})).toThrow('System "compositeInputTest/scalar": input is an object schema of named fields.'); + it("rejects a body taking input without the schema the plugin gives from its type, as one not written in place has", () => { + const counting = (_context: unknown, input: { readonly count: number }): void => { void input.count; }; + expect(() => System.create({ name: "compositeInputTest/unbuilt" }, counting)).toThrow('System "compositeInputTest/unbuilt" takes input, whose schema the plugin gives from the type of its body\'s parameter: write its body in place and build it with the kernel plugin.'); + }); + + it("rejects an input schema that isn't an object of named fields", () => { + const create: (...parameters: unknown[]) => unknown = System.create; + expect(() => create({ name: "compositeInputTest/scalar" }, (_context: unknown, _input: unknown) => {}, { type: "number" })).toThrow('System "compositeInputTest/scalar": input is an object schema of named fields.'); }); }); diff --git a/packages/ecs/src/implementation/world/create.test.ts b/packages/ecs/src/implementation/world/create.test.ts index 113efae0..b9eaee3c 100644 --- a/packages/ecs/src/implementation/world/create.test.ts +++ b/packages/ecs/src/implementation/world/create.test.ts @@ -110,7 +110,7 @@ describe("World.create", () => { it("applies the net structural change of a sync point", async () => { let first = Entity.none; let second = Entity.none; - const edit = System.create({ name: "createTest/edit/net", input: { type: "object", properties: { action: { type: "string" } } } }, ({ input, commands }) => { + const edit = System.create({ name: "createTest/edit/net" }, ({ commands }, input: { readonly action: string }) => { if (input.action === "spawn") { first = commands.spawn([velocity])({ [velocity]: 1 }); second = commands.spawn([velocity])({ [velocity]: 2 }); @@ -134,7 +134,7 @@ describe("World.create", () => { it("reuses a despawned index only after a sync point, with a new generation", async () => { const spawned: Entity[] = []; - const edit = System.create({ name: "createTest/edit/reuse", input: { type: "object", properties: { action: { type: "string" } } } }, ({ input, commands }) => { + const edit = System.create({ name: "createTest/edit/reuse" }, ({ commands }, input: { readonly action: string }) => { if (input.action === "spawn") spawned.push(commands.spawn([velocity])({ [velocity]: spawned.length })); else for (const entity of spawned) commands.despawn(entity); }); @@ -153,7 +153,7 @@ describe("World.create", () => { it("keeps rows consistent across swap-removes and archetype growth", async () => { const spawned: Entity[] = []; - const edit = System.create({ name: "createTest/edit/rows", input: { type: "object", properties: { action: { type: "string" } } } }, ({ input, commands }) => { + const edit = System.create({ name: "createTest/edit/rows" }, ({ commands }, input: { readonly action: string }) => { if (input.action === "spawn") for (let i = 0; i < 100; i++) spawned.push(commands.spawn([velocity, label])({ [velocity]: i, [label]: `${i}` })); else for (let i = 0; i < 100; i += 3) commands.remove(spawned[i], label); }); @@ -173,7 +173,7 @@ describe("World.create", () => { await world.run(spawnPlain); const [entity] = await select(world, { required: [velocity] }); expect(await read(world, entity, frozen)).toBe(null); - const freeze = System.create({ name: "createTest/freeze", input: { type: "object", properties: { entity: Entity.schema } } }, ({ input, commands }) => { + const freeze = System.create({ name: "createTest/freeze" }, ({ commands }, input: { readonly entity: Entity }) => { commands.add(input.entity, frozen, true); }); await world.run(freeze, { entity }); @@ -297,7 +297,7 @@ describe("World.create", () => { it("sees a change moved into an older chunk by a swap-remove", async () => { const spawned: Entity[] = []; - const edit = System.create({ name: "createTest/edit/swap", input: { type: "object", properties: { action: { type: "string" } } } }, ({ input, commands }) => { + const edit = System.create({ name: "createTest/edit/swap" }, ({ commands }, input: { readonly action: string }) => { if (input.action === "spawn") for (let i = 0; i < 2100; i++) spawned.push(commands.spawn([velocity])({ [velocity]: i })); else commands.despawn(spawned[5]); }); diff --git a/packages/ecs/src/implementation/world/entity-lookup-workers.fixture.ts b/packages/ecs/src/implementation/world/entity-lookup-workers.fixture.ts index 4b81b6d7..8f744f1c 100644 --- a/packages/ecs/src/implementation/world/entity-lookup-workers.fixture.ts +++ b/packages/ecs/src/implementation/world/entity-lookup-workers.fixture.ts @@ -73,7 +73,7 @@ const seed = System.create({ name: "entityLookupWorkersTest/seed" }, ({ commands }); /** Writes some weights, marks and unmarks some targets, despawns some and spawns others: a different change each round. */ -const stir = System.create({ name: "entityLookupWorkersTest/stir", input: { type: "object", properties: { round: { type: "integer" } } }, components: { write: [weight] } }, (row, { input, commands }) => { +const stir = System.create({ name: "entityLookupWorkersTest/stir", components: { write: [weight] } }, (row, { commands }, input: { readonly round: number }) => { const w = row[weight]; if ((w + input.round) % 5 === 0) row[weight] = w + 1000; if ((w + input.round) % 11 === 0) commands.add(row.entity, mark, 2); diff --git a/packages/ecs/src/implementation/world/entity-lookups.test-helper.ts b/packages/ecs/src/implementation/world/entity-lookups.test-helper.ts index e2da4b0a..54375d60 100644 --- a/packages/ecs/src/implementation/world/entity-lookups.test-helper.ts +++ b/packages/ecs/src/implementation/world/entity-lookups.test-helper.ts @@ -114,8 +114,7 @@ export const touch = System.create({ name: "entityLookupsTest/touch", components /** Changes the subject by command: `action` names how. */ export const edit = System.create({ name: "entityLookupsTest/edit", - input: { type: "object", properties: { action: { type: "string" } } }, -}, ({ input, commands }) => { +}, ({ commands }, input: { readonly action: string }) => { const { subject } = scene; switch (input.action) { case "spawn": diff --git a/packages/ecs/src/implementation/world/finalizers.test.ts b/packages/ecs/src/implementation/world/finalizers.test.ts index e5b700bd..a4f4e2d5 100644 --- a/packages/ecs/src/implementation/world/finalizers.test.ts +++ b/packages/ecs/src/implementation/world/finalizers.test.ts @@ -76,7 +76,7 @@ Component.register(reloaded, { type: { type: "number" } }); Rule.create({ name: "finalizersTest/slots", components: { has: [source] }, derive: [slot] }); let steps: ((commands: System.Commands) => void)[] = []; -const edit = System.create({ name: "finalizersTest/edit", input: { type: "object", properties: { step: { type: "integer" } } } }, ({ input, commands }) => { +const edit = System.create({ name: "finalizersTest/edit" }, ({ commands }, input: { readonly step: number }) => { steps[input.step]?.(commands); }); const step = (world: ReturnType) => (run: (commands: System.Commands) => void) => { diff --git a/packages/ecs/src/implementation/world/finalizers.ts b/packages/ecs/src/implementation/world/finalizers.ts index 59e63052..19a4e5ae 100644 --- a/packages/ecs/src/implementation/world/finalizers.ts +++ b/packages/ecs/src/implementation/world/finalizers.ts @@ -15,8 +15,9 @@ import type { Queued } from './synchronize.js'; import type { Essential, ReadonlyValue } from '../../interface/component/value.js'; import type { Leaf } from '../../interface/finalizer/created.js'; import { newestKeyOf, threadOf } from '../../interface/registry/index.js'; +import type { Layout } from '../../interface/registry/layout.js'; import type { TaskChannel } from './remote-systems.js'; -import { writeStops, type StopsOf } from './stop-lists.js'; +import { resourceOffsetsOf, writeStops, type StopsOf } from './stop-lists.js'; import { acquireLookups, refreshUses, type IndexLookups } from './indexes.js'; import { indexWatchesOf } from './lookup-keys.js'; @@ -112,8 +113,30 @@ export const placeFinalizer = (state: State, runner: FinalizerRunner, channel: T copies.push(layout.kind === "string"); } } + // Its string and linear resources, in the order the thread's context reads them, copied as of before apply. + const stringResources: number[] = []; + const linearResources: number[] = []; + const resourceStrides: number[] = []; + const stringStarts: unknown[] = []; + const linearStarts: unknown[] = []; + const resourceLayouts: Layout[] = []; + for (const key of new Set(finalizer.declaration.resources?.read ?? [])) { + const id = idOf(state, key); + const layout = state.layouts[id]; + if (layout.kind === "string") { + stringResources.push(id); + stringStarts.push(state.starts[id]); + } + else if (layout.kind === "linear") { + linearResources.push(id); + resourceStrides.push(layout.stride); + linearStarts.push(state.starts[id]); + resourceLayouts.push(layout); + } + } + const resourceOffsets = resourceOffsetsOf(stringResources.length, resourceStrides); const index = channel.placeFinalizer(state, finalizer); - runner.remote = { finalizer, index, objects, linear, strides, copies, entries: [], channel }; + runner.remote = { finalizer, index, objects, linear, strides, copies, stringResources, linearResources, resourceStrides, resourceOffsets, stringStarts, linearStarts, resourceLayouts, entries: [], channel }; }; const matches = (runner: FinalizerRunner, archetype: Archetype): boolean => { @@ -382,7 +405,11 @@ export const runFinalizers = (state: State, queued: readonly Queued[], born: Map } if (state.claimed && staged(state)) { const { plan } = state; - const source = { entity: plan.entity, from: plan.from, despawned: plan.despawned, archetypes: state.archetypes, row: state.entities.row, holds: state.holds }; + const resource = state.resourceEntity < 0 ? -1 : indexOf(state.resourceEntity); + const source = { + entity: plan.entity, from: plan.from, despawned: plan.despawned, archetypes: state.archetypes, row: state.entities.row, holds: state.holds, + resourceArchetype: resource < 0 ? -1 : state.entities.archetype[resource], resourceRow: resource < 0 ? 0 : state.entities.row[resource], + }; for (const channel of state.channels) writeStops(channel.stops, tick, source); } } diff --git a/packages/ecs/src/implementation/world/generated-code.md b/packages/ecs/src/implementation/world/generated-code.md index bf079d67..e56be80a 100644 --- a/packages/ecs/src/implementation/world/generated-code.md +++ b/packages/ecs/src/implementation/world/generated-code.md @@ -21,7 +21,7 @@ import { readFile } from 'node:fs/promises'; import { dirname, resolve } from 'node:path'; import { transformKernels } from '../implementation/compiler/transform.js'; import { createLinker } from '../implementation/compiler/link.js'; -import { packageIndex } from '../implementation/compiler/paths.js'; +import { ownCopy } from '../implementation/compiler/paths.js'; import { estreeOf } from '../implementation/compiler/estree.js'; import { generateWgsl } from '../implementation/world/wgsl.js'; import { generateWasm, type WasmColumn } from '../implementation/world/wasm.js'; @@ -36,7 +36,7 @@ const move = System.create({ name: "move", kernel: "gpu", components: { read: [v // Resolves the package to its source and relative imports to files, stripping types as the plugin does. const linker = createLinker({ resolve: async (specifier, importer) => { - if (specifier === "@adobe/ecs") return packageIndex; + if (specifier === "@adobe/ecs") return ownCopy.index; const path = resolve(dirname(importer), specifier); return existsSync(path.replace(/\.js$/, ".ts")) ? path.replace(/\.js$/, ".ts") : path; }, @@ -45,7 +45,7 @@ const move = System.create({ name: "move", kernel: "gpu", components: { read: [v return { code: file.endsWith(".ts") ? (await (await import("vite")).transformWithOxc(code, file, { lang: "ts" })).code : code }; }, }, parse); - const result = await transformKernels(source, "/move.js", parse, async s => s === "@adobe/ecs", i => JSON.stringify(`kernel:${i}`), linker); + const result = await transformKernels(source, "/move.js", parse, async s => s === "@adobe/ecs" ? ownCopy : null, i => JSON.stringify(`kernel:${i}`), linker); if (result === null || "failure" in result) throw new Error(JSON.stringify(result)); console.log(result.code); console.log(result.kernels[0]); @@ -112,9 +112,9 @@ fn main(@builtin(global_invocation_id) id: vec3u, @builtin(num_workgroups) group Calls nest as builtins do: `Vec3.scale(Vec3.normalize(Vec3.cross(p, v)), Vec3.dot(p, v))` emits `cross_6 = cross(…)`, `normalize_19 = normalize(cross_6)`, `dot_26 = dot(…)` and `scale_31 = (normalize_19 * dot_26)`. -### WASM (499 bytes, excerpts from `wasm2wat`) +### WASM (557 bytes, excerpts from `wasm2wat`) -Three loops: chunk stamps, four rows at a time, then one row at a time for the rest. +Four loops: block stamps, chunk stamps, four rows at a time, then one row at a time for the rest. ```wat ;; Four rows: 48 bytes per component, three v128 loads each. diff --git a/packages/ecs/src/implementation/world/index-buckets.test.ts b/packages/ecs/src/implementation/world/index-buckets.test.ts new file mode 100644 index 00000000..666a12a1 --- /dev/null +++ b/packages/ecs/src/implementation/world/index-buckets.test.ts @@ -0,0 +1,284 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, it } from "vitest"; +import { Component } from "../../interface/component/component.js"; +import type { Entity } from "../../interface/entity/entity.js"; +import { Index } from "../../interface/indexes/index-type.js"; +import { workers } from "../../interface/resources/workers.js"; +import { System } from "../../interface/system/system.js"; +import { createWorld, type CreatedWorld } from "./create.js"; + +// Long buckets of a hash index, which a change may leave unsorted until a list is read by position: every reader still +// gets each bucket's rows in entity order. + +const group = Symbol("indexBucketsTest/group"); +const serial = Symbol("indexBucketsTest/serial"); +const hidden = Symbol("indexBucketsTest/hidden"); +const extra = Symbol("indexBucketsTest/extra"); +const reads = Symbol("indexBucketsTest/reads"); +const sum = Symbol("indexBucketsTest/sum"); +const sumByHand = Symbol("indexBucketsTest/sumByHand"); + +declare module "@adobe/ecs" { + interface Components { + [group]: { type: number }; + [serial]: { type: number }; + [hidden]: { type: true }; + [extra]: { type: true }; + [reads]: { type: number }; + [sum]: { type: number }; + [sumByHand]: { type: number }; + } +} +Component.register(group, { type: { type: "integer", minimum: 0, maximum: 2 } }); +Component.register(serial, { type: { type: "integer", minimum: 0, maximum: 10000000 } }); +Component.register(hidden, { type: { const: true } }); +Component.register(extra, { type: { const: true } }); +Component.register(reads, { type: { type: "integer", minimum: 0, maximum: 2 } }); +Component.register(sum, { type: { type: "number" } }); +Component.register(sumByHand, { type: { type: "number" } }); + +const groups = [0, 1, 2]; +const byGroup = Index.create({ name: "indexBucketsTest/byGroup", key: [group], components: { has: [serial], not: [hidden] } }); + +/** What a reader saw: each bucket's length, or each bucket's rows as entity index and serial, in order. */ +let lengths: number[] = []; +let rows: (readonly [number, number])[][] = []; +const readLengths = System.create({ name: "indexBucketsTest/readLengths", lookup: { byGroup: { index: byGroup } } }, ({ lookup }) => { + lengths = groups.map(value => lookup.byGroup(value).length); +}); +const readRows = System.create({ name: "indexBucketsTest/readRows", lookup: { byGroup: { index: byGroup, read: [serial] } } }, ({ lookup }) => { + rows = groups.map(value => { + const list = lookup.byGroup(value); + const found: (readonly [number, number])[] = []; + for (let i = 0; i < list.length; i++) { + const row = list.at(i); + found.push([row.entity >>> 0, row[serial]]); + } + return found; + }); +}); + +/** Each reader row's bucket's serials, each times its position plus one: a sum that tells the order. */ +const orderSum = System.create({ + name: "indexBucketsTest/orderSum", kernel: "cpu", + components: { read: [reads], write: [sum] }, + lookup: { byGroup: { index: byGroup, read: [serial] } }, +}, (row, { lookup }) => { + const list = lookup.byGroup(row[reads]); + let total = 0; + for (let i = 0; i < list.length; i++) total += (i + 1) * list.at(i)[serial]; + row[sum] = total; +}); +const orderSumByHand = System.create({ + name: "indexBucketsTest/orderSumByHand", + components: { read: [reads], write: [sumByHand] }, + lookup: { byGroup: { index: byGroup, read: [serial] } }, +}, (row, { lookup }) => { + const list = lookup.byGroup(row[reads]); + let total = 0; + for (let i = 0; i < list.length; i++) total += (i + 1) * list.at(i)[serial]; + row[sumByHand] = total; +}); +let sums: { kernel: number[]; byHand: number[] } = { kernel: [], byHand: [] }; +const readSums = System.create({ name: "indexBucketsTest/readSums", components: { read: [reads, sum, sumByHand] } }, row => { + sums.kernel[row[reads]] = row[sum]; + sums.byHand[row[reads]] = row[sumByHand]; +}); +const spawnReaders = System.create({ name: "indexBucketsTest/spawnReaders" }, ({ commands }) => { + for (const value of groups) commands.spawn([reads, sum, sumByHand])({ [reads]: value, [sum]: 0, [sumByHand]: 0 }); +}); + +/** An entity as the test keeps it, and the changes a round makes. */ +type Held = { group: number; readonly serial: number; hidden: boolean }; +type Change = + | { readonly kind: "spawn"; readonly group: number; readonly serial: number; readonly extra: boolean } + | { readonly kind: "despawn" | "hide" | "show" | "extra" | "plain"; readonly entity: Entity } + | { readonly kind: "group"; readonly entity: Entity; readonly group: number }; + +let spawned: Entity[] = []; +const apply = System.create({ name: "indexBucketsTest/apply" }, ({ commands }, input: { readonly changes: readonly Change[] }) => { + spawned = []; + for (const change of input.changes) { + switch (change.kind) { + case "spawn": + spawned.push(change.extra + ? commands.spawn([group, serial, extra])({ [group]: change.group, [serial]: change.serial, [extra]: true }) + : commands.spawn([group, serial])({ [group]: change.group, [serial]: change.serial })); + break; + case "despawn": commands.despawn(change.entity); break; + case "hide": commands.add(change.entity, hidden); break; + case "show": commands.remove(change.entity, hidden); break; + case "extra": commands.add(change.entity, extra); break; + case "plain": commands.remove(change.entity, extra); break; + case "group": commands.add(change.entity, group, change.group); break; + } + } +}); + +/** A seeded generator of numbers in [0, 1). */ +const generator = (seed: number): (() => number) => { + let state = seed >>> 0; + return () => { + state = (state + 0x6D2B79F5) >>> 0; + let t = state; + t = Math.imul(t ^ (t >>> 15), t | 1); + t ^= t + Math.imul(t ^ (t >>> 7), t | 61); + return ((t ^ (t >>> 14)) >>> 0) / 4294967296; + }; +}; + +/** The world and what it holds, as the test keeps it: entities by entity, and the next serial. */ +type Scenario = { readonly world: CreatedWorld; readonly held: Map; next: number }; + +const applyChanges = async (scenario: Scenario, changes: readonly Change[]): Promise => { + await scenario.world.run(apply, { changes }); + let k = 0; + for (const change of changes) { + if (change.kind === "spawn") scenario.held.set(spawned[k++], { group: change.group, serial: change.serial, hidden: false }); + else if (change.kind === "despawn") scenario.held.delete(change.entity); + else if (change.kind === "hide" || change.kind === "show") { + const found = scenario.held.get(change.entity); + if (found !== undefined) found.hidden = change.kind === "hide"; + } + else if (change.kind === "group") { + const found = scenario.held.get(change.entity); + if (found !== undefined) found.group = change.group; + } + } +}; + +const spawnMany = (scenario: Scenario, count: number, groupOf: (k: number) => number): Change[] => + Array.from({ length: count }, (_, k): Change => ({ kind: "spawn", group: groupOf(k), serial: scenario.next++, extra: k % 3 === 0 })); + +/** Each bucket's rows as the index must give them: by entity index. */ +const expectedRows = (scenario: Scenario): (readonly [number, number])[][] => groups.map(value => [...scenario.held] + .filter(([, held]) => held.group === value && !held.hidden) + .map(([entity, held]): readonly [number, number] => [entity >>> 0, held.serial]) + .sort((a, b) => a[0] - b[0])); + +const orderSumOf = (bucket: readonly (readonly [number, number])[]): number => bucket.reduce((total, [, value], i) => total + (i + 1) * value, 0); + +const checkRows = async (scenario: Scenario): Promise => { + await scenario.world.run(readRows); + expect(rows).toEqual(expectedRows(scenario)); +}; +const checkLengths = async (scenario: Scenario): Promise => { + await scenario.world.run(readLengths); + expect(lengths).toEqual(expectedRows(scenario).map(bucket => bucket.length)); +}; + +/** A round of random changes: entities moved between buckets, hidden and shown, given a tag and plain again, despawned and spawned. */ +const randomChanges = (scenario: Scenario, next: () => number, count: number): Change[] => { + const entities = [...scenario.held.keys()]; + const pick = (): Entity => entities[Math.floor(next() * entities.length)]; + const used = new Set(); + const changes: Change[] = []; + for (let k = 0; k < count; k++) { + const entity = pick(); + const choice = next(); + if (choice < 0.08) { + changes.push({ kind: "spawn", group: Math.floor(next() * 3), serial: scenario.next++, extra: next() < 0.5 }); + continue; + } + // One change an entity a round, so the test's model applies them as the sync point does. + if (used.has(entity)) continue; + used.add(entity); + const held = scenario.held.get(entity); + if (held === undefined) continue; + if (choice < 0.6) changes.push({ kind: "group", entity, group: (held.group + 1 + Math.floor(next() * 2)) % 3 }); + else if (choice < 0.7) changes.push({ kind: "despawn", entity }); + else if (choice < 0.8) changes.push({ kind: held.hidden ? "show" : "hide", entity }); + else changes.push({ kind: next() < 0.5 ? "extra" : "plain", entity }); + } + return changes; +}; + +const createScenario = (wasm = true): Scenario => ({ world: createWorld({ [workers]: 0 }, { wasm }), held: new Map(), next: 0 }); + +describe("a hash index's long buckets", () => { + it.each([[1], [2], [3], [4]])("give their rows in entity order after random changes, read by length and by position, seed %i", async seed => { + const scenario = createScenario(); + await using _world = scenario.world; + const next = generator(seed); + await applyChanges(scenario, spawnMany(scenario, 600, k => k % 3)); + await checkRows(scenario); + for (let round = 0; round < 80; round++) { + await applyChanges(scenario, randomChanges(scenario, next, Math.floor(next() * 60))); + const mode = Math.floor(next() * 4); + if (mode === 0 || mode === 2) await checkLengths(scenario); + if (mode === 1 || mode === 2) await checkRows(scenario); + } + await checkRows(scenario); + expect(expectedRows(scenario).every(bucket => bucket.length >= 64)).toBe(true); + }); + it("give their rows in entity order after an entity leaves and comes back, and one moves between buckets each run", async () => { + const scenario = createScenario(); + await using _world = scenario.world; + await applyChanges(scenario, spawnMany(scenario, 300, k => k % 2)); + await checkRows(scenario); + const entities = [...scenario.held.keys()]; + for (let k = 0; k < 400; k++) { + const entity = entities[(k * 37) % entities.length]; + const held = scenario.held.get(entity); + if (held === undefined) continue; + await applyChanges(scenario, [k % 5 === 0 ? { kind: held.hidden ? "show" : "hide", entity } : { kind: "group", entity, group: 1 - held.group }]); + if (k % 7 === 0) await checkRows(scenario); + else await checkLengths(scenario); + } + await checkRows(scenario); + }); + it("give their rows in entity order after a refresh that builds the index again, and changes after it", async () => { + const scenario = createScenario(); + await using _world = scenario.world; + await applyChanges(scenario, spawnMany(scenario, 6000, k => k % 3)); + await checkLengths(scenario); + const next = generator(9); + for (let round = 0; round < 6; round++) { + await applyChanges(scenario, randomChanges(scenario, next, 30)); + await checkLengths(scenario); + // More changes than a refresh patches: the index is built again, its unsorted buckets with it. + const everyOther = [...scenario.held].filter((_, k) => k % 2 === round % 2).map(([entity, held]): Change => ({ kind: "group", entity, group: (held.group + 1) % 3 })); + await applyChanges(scenario, [...randomChanges(scenario, next, 30).filter(change => change.kind === "spawn"), ...everyOther]); + await checkLengths(scenario); + await applyChanges(scenario, randomChanges(scenario, next, 30)); + await checkRows(scenario); + } + }); + it("give their rows in entity order after a build in phases, and changes after it", async () => { + const scenario = createScenario(); + await using _world = scenario.world; + // Enough entities for a build in phases (`rebuildPhased`). + await applyChanges(scenario, spawnMany(scenario, 70_000, k => k % 2)); + await checkLengths(scenario); + const next = generator(11); + for (let round = 0; round < 4; round++) { + await applyChanges(scenario, randomChanges(scenario, next, 50)); + await checkLengths(scenario); + await applyChanges(scenario, randomChanges(scenario, next, 50)); + await checkRows(scenario); + } + expect((await scenario.world.tables()).phasedBuilds).toBeGreaterThan(0); + }); +}); + +describe.each([[true], [false]])("kernels reading a hash index's long buckets, WASM %s", wasm => { + it("read their rows in entity order, as a main-thread system does, after changes left them unsorted", async () => { + const scenario = createScenario(wasm); + await using _world = scenario.world; + await scenario.world.run(spawnReaders); + await applyChanges(scenario, spawnMany(scenario, 600, k => k % 3)); + const next = generator(5); + for (let round = 0; round < 12; round++) { + await applyChanges(scenario, randomChanges(scenario, next, 40)); + // Read by length only, so the buckets stay unsorted for the kernel. + await checkLengths(scenario); + await scenario.world.run(orderSum); + await scenario.world.run(orderSumByHand); + sums = { kernel: [], byHand: [] }; + await scenario.world.run(readSums); + const expected = expectedRows(scenario).map(orderSumOf); + expect(sums.kernel).toEqual(expected); + expect(sums.byHand).toEqual(expected); + } + }); +}); diff --git a/packages/ecs/src/implementation/world/index-changed.test.ts b/packages/ecs/src/implementation/world/index-changed.test.ts index 8340898c..f3f11696 100644 --- a/packages/ecs/src/implementation/world/index-changed.test.ts +++ b/packages/ecs/src/implementation/world/index-changed.test.ts @@ -36,24 +36,23 @@ const inOrder = Index.create({ name: "indexChangedTest/inOrder", key: [group], s const weighted = Index.create({ name: "indexChangedTest/weighted", key: [], components: { has: [weight] } }); const one = { type: "object", properties: { entity: Entity.schema, value: { type: "number" } } } as const; -const entityOnly = { type: "object", properties: { entity: Entity.schema } } as const; let spawned: Entity[] = []; -const spawn = System.create({ name: "indexChangedTest/spawn", input: { type: "object", properties: { groups: { type: "array", items: { type: "integer" } } } } }, ({ input, commands }) => { +const spawn = System.create({ name: "indexChangedTest/spawn" }, ({ commands }, input: { readonly groups: readonly number[] }) => { for (const value of input.groups) spawned.push(commands.spawn([group, weight, total])({ [group]: value, [weight]: 1, [total]: 0 })); }); -const setGroup = System.create({ name: "indexChangedTest/setGroup", input: one, target: "entity", components: { write: [group] } }, (row, { input }) => { +const setGroup = System.create({ name: "indexChangedTest/setGroup", target: "entity", components: { write: [group] } }, (row, _context, input: { readonly entity: Entity; readonly value: number }) => { row[group] = input.value; }); -const setWeight = System.create({ name: "indexChangedTest/setWeight", input: one, target: "entity", components: { write: [weight] } }, (row, { input }) => { +const setWeight = System.create({ name: "indexChangedTest/setWeight", target: "entity", components: { write: [weight] } }, (row, _context, input: { readonly entity: Entity; readonly value: number }) => { row[weight] = input.value; }); -const despawn = System.create({ name: "indexChangedTest/despawn", input: entityOnly, target: "entity", components: { has: [group] } }, (row, { commands }) => { +const despawn = System.create({ name: "indexChangedTest/despawn", target: "entity", components: { has: [group] } }, (row, { commands }, _input: { readonly entity: Entity }) => { commands.despawn(row.entity); }); -const dropWeight = System.create({ name: "indexChangedTest/dropWeight", input: entityOnly, target: "entity", components: { has: [weight] } }, (row, { commands }) => { +const dropWeight = System.create({ name: "indexChangedTest/dropWeight", target: "entity", components: { has: [weight] } }, (row, { commands }, _input: { readonly entity: Entity }) => { commands.remove(row.entity, weight); }); -const pause = System.create({ name: "indexChangedTest/pause", input: { type: "object", properties: { paused: { type: "boolean" } } }, resources: { write: [paused] } }, ({ input, resources }) => { +const pause = System.create({ name: "indexChangedTest/pause", resources: { write: [paused] } }, ({ resources }, input: { readonly paused: boolean }) => { resources[paused] = input.paused; }); diff --git a/packages/ecs/src/implementation/world/index-hash.test.ts b/packages/ecs/src/implementation/world/index-hash.test.ts new file mode 100644 index 00000000..bd2fdf44 --- /dev/null +++ b/packages/ecs/src/implementation/world/index-hash.test.ts @@ -0,0 +1,34 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, it } from "vitest"; +import { bucketFor, coverEntities, createHashTable, entryWords, insertEntry, lengthOf, removeEntry, sortBucket, startOf } from "./index-hash.js"; +import { hashWords } from "./index-keys.js"; +import { createMemory } from "./memory.js"; + +describe("a hash table's unsorted buckets", () => { + it("are listed once each, however often one is sorted and left unsorted again while another stays unsorted", () => { + const table = createHashTable(createMemory(), 1); + coverEntities(table, 400); + const key = new Uint32Array(1); + const bucketOf = (value: number): number => { + key[0] = value; + return bucketFor(table, key, 0, hashWords(key, 0, 1), 0); + }; + const buckets = [bucketOf(0), bucketOf(1)]; + for (let entity = 1; entity <= 300; entity++) insertEntry(table, buckets[entity % 2], entity, 0, entity, 0, true); + // One entity leaves the first bucket once: it stays unsorted. + removeEntry(table, buckets[0], 2, 1, true); + let out = false; + for (let run = 0; run < 500; run++) { + // An entity of the second bucket leaves it or comes back, and a read by position sorts it. + if (out) insertEntry(table, buckets[1], 151, 0, 151, run, true); + else removeEntry(table, buckets[1], 151, run, true); + out = !out; + if (table.tails[buckets[1]] >= 0) sortBucket(table, buckets[1]); + } + expect(table.unsortedListed).toBeLessThanOrEqual(2); + sortBucket(table, buckets[0]); + const entities = (bucket: number): number[] => Array.from({ length: lengthOf(table, bucket) }, (_, k) => table.entries[(startOf(table, bucket) + k) * entryWords]); + expect(entities(buckets[0])).toEqual(Array.from({ length: 149 }, (_, k) => 2 * k + 4)); + expect(entities(buckets[1])).toEqual(Array.from({ length: 150 }, (_, k) => 2 * k + 1)); + }); +}); diff --git a/packages/ecs/src/implementation/world/index-hash.ts b/packages/ecs/src/implementation/world/index-hash.ts index b0b55d36..26b476d7 100644 --- a/packages/ecs/src/implementation/world/index-hash.ts +++ b/packages/ecs/src/implementation/world/index-hash.ts @@ -18,9 +18,19 @@ const freedCapacity = 0xFFFFFFFF; /** * A hash index: an open-addressing table of buckets by key, linear probing, at most half full, each slot a key's hash * and its bucket, so a probe reads one line until a hash matches; each bucket a record of its hash, segment and key, - * its segment of one entries buffer with spare slots holding its entries in entity order. Every array is in the arena, - * so other threads read it where the world's thread wrote it. By entity index, the bucket it's filed in, so its old - * bucket is found without its old key. + * its segment of one entries buffer with spare slots holding its entries in entity order. Every array readers use is + * in the arena, so other threads read it where the world's thread wrote it. By entity index, the bucket it's filed in, + * so its old bucket is found without its old key. + * + * A change to a bucket of at least `unsortedMinimum` entries, in an index that isn't unique, may leave it unsorted: + * its entries in entity order up to its tail, with holes where entries left (archetype -1, the entity index kept for + * binary search), then a tail of entries appended in any order, each one's position kept by entity index. Its + * record's length counts its entries, not its holes, so reading `length` needs no order. It's sorted before anything + * reads its entries by position: on the world's thread by a list's `at`, and every unsorted bucket before the index's + * arrays are handed to a kernel or another thread; when its segment is full and has holes, which sorting closes in + * place; and once its holes and tail pass four times its entries. A segment an unsorted bucket outgrows gets half its + * length more spare slots, so sorts to make room come once every so many changes, O(1) amortized each. What only + * unsorted buckets use is the world's thread's alone, outside the arena. */ export type HashTable = { readonly memory: Memory; @@ -48,16 +58,42 @@ export type HashTable = { size: number; /** By component a lookup watches: by bucket, the newest tick its value may have changed in a row there. */ readonly watched: Map; + /** How many buckets are unsorted, and a list holding each of them once, and some sorted since; by bucket, 1 when listed. */ + unsorted: number; + unsortedBuckets: Int32Array; + unsortedListed: number; + listed: Uint8Array; + /** + * By bucket id: where its tail starts, or -1 when it's sorted; the slots its entries and holes take; and a position + * no hole is before. + */ + tails: Int32Array; + extents: Int32Array; + holes: Int32Array; + /** By entity index: its position in its bucket's tail, valid only while it's there. */ + positionOf: Int32Array; + /** Scratch for sorting a tail: twice its words. */ + spare: Int32Array; + /** The entries as 8-byte words, and a scratch as 8- and 4-byte words, for long moves (`moveWide`). */ + wide: Float64Array; + scratch: Float64Array; + scratchWords: Int32Array; }; /** What lookups read of a hash table: on the world's thread the table, on a joined thread its views of it. */ -export type HashView = Pick & { readonly watched: ReadonlyMap }; +export type HashView = Pick & { readonly watched: ReadonlyMap }; + +/** Whether a view is the world's table itself, whose unsorted buckets a list sorts before reading them by position. */ +export const isHashTable = (view: HashView): view is HashTable => "tails" in view; const minimumSlots = 16; /** Slots the table keeps for each bucket at least: it's at most half full. */ const slotsPerBucket = 2; +/** Entries a bucket holds at least before a change may leave it unsorted; a shorter one shifts its entries. */ +export const unsortedMinimum = 64; + /** Whether `slots` slots hold `buckets` buckets. */ export const slotsHold = (slots: number, buckets: number): boolean => slotsPerBucket * buckets + 2 <= slots; @@ -78,6 +114,18 @@ export const createHashTable = (memory: Memory, words: number): HashTable => ({ bucketOf: allocateArray(memory, Int32Array, 64).fill(-1), size: 0, watched: new Map(), + unsorted: 0, + unsortedBuckets: new Int32Array(8), + unsortedListed: 0, + listed: new Uint8Array(8), + tails: new Int32Array(8).fill(-1), + extents: new Int32Array(8), + holes: new Int32Array(8), + positionOf: new Int32Array(64), + spare: new Int32Array(0), + wide: new Float64Array(0), + scratch: new Float64Array(0), + scratchWords: new Int32Array(0), }); /** Frees every array, once nothing reads the table again. */ @@ -96,10 +144,19 @@ const regrow = (memory: Memor return grown; }; +/** A copy of `array`, `length` long, the rest filled with `fill`. */ +const grown = (array: Int32Array, length: number, fill: number): Int32Array => { + const copy = new Int32Array(length); + copy.set(array); + if (fill !== 0) copy.fill(fill, array.length); + return copy; +}; + /** Grows what's by entity index to cover `length` indexes. */ export const coverEntities = (table: HashTable, length: number): void => { if (table.bucketOf.length >= length) return; table.bucketOf = regrow(table.memory, table.bucketOf, Int32Array, Math.max(length, 2 * table.bucketOf.length), table.bucketOf.length, -1); + table.positionOf = grown(table.positionOf, table.bucketOf.length, 0); }; /** Grows what's by bucket id to hold `count` buckets. */ @@ -111,6 +168,12 @@ export const coverBuckets = (table: HashTable, count: number): void => { table.records = regrow(memory, table.records, Uint32Array, length * stride, capacity * stride, 0); table.ticks = regrow(memory, table.ticks, Float64Array, length, capacity, 0); for (const [component, ticks] of table.watched) table.watched.set(component, regrow(memory, ticks, Float64Array, length, capacity, 0)); + table.tails = grown(table.tails, length, -1); + table.extents = grown(table.extents, length, 0); + table.holes = grown(table.holes, length, 0); + const listed = new Uint8Array(length); + listed.set(table.listed); + table.listed = listed; }; /** A bucket's segment's first entry, length and capacity. */ @@ -203,29 +266,110 @@ const allocateSegment = (table: HashTable, capacity: number): number => { return start; }; -/** Gives a bucket room for `length` entries, moving its segment to the end of the buffer when it doesn't fit. */ -const reserve = (table: HashTable, bucket: number, length: number): void => { +/** The slots a bucket's entries take, with an unsorted one's holes. */ +const extentOf = (table: HashTable, bucket: number): number => table.tails[bucket] < 0 ? lengthOf(table, bucket) : table.extents[bucket]; + +/** + * Gives a bucket room for `slots` slots, moving its segment to the end of the buffer when they don't fit; an unsorted + * one's gets half as many again spare, since its tail grows into them. + */ +const reserve = (table: HashTable, bucket: number, slots: number): void => { const base = bucket * table.stride; - if (table.records[base + recordCapacity] >= length) return; - const capacity = capacityFor(length); + if (table.records[base + recordCapacity] >= slots) return; + const capacity = capacityFor(slots) + (table.tails[bucket] >= 0 ? slots >> 1 : 0); + const kept = extentOf(table, bucket); const start = allocateSegment(table, capacity); - const { entries, records } = table; - const from = records[base + recordStart] * entryWords; - entries.copyWithin(start * entryWords, from, from + records[base + recordLength] * entryWords); + const { records } = table; + move(table, start * entryWords, records[base + recordStart] * entryWords, kept * entryWords); table.garbage += records[base + recordCapacity]; records[base + recordStart] = start; records[base + recordCapacity] = capacity; }; +/** The table's entries as 8-byte words, made again when its entries were. */ +const wideOf = (table: HashTable): Float64Array => { + const { entries } = table; + // Its last 8-byte word may take 4 bytes past the entries, which are in its block: blocks are 64-byte aligned. + const length = (entries.length + 1) >> 1; + if (table.wide.buffer !== entries.buffer || table.wide.byteOffset !== entries.byteOffset || table.wide.length !== length) { + table.wide = new Float64Array(entries.buffer, entries.byteOffset, length); + } + return table.wide; +}; + +/** Scratch of at least `words` 4-byte words, as 4- and 8-byte words. */ +const scratchOf = (table: HashTable, words: number): Float64Array => { + if (2 * table.scratch.length < words) { + table.scratch = new Float64Array(Math.max((words + 1) >> 1, 2 * table.scratch.length)); + table.scratchWords = new Int32Array(table.scratch.buffer); + } + return table.scratch; +}; + /** - * Moves `words` of entries, a multiple of `entryWords`, from `from` to `to` within one buffer: by hand, an entry a step, - * when few, which a native call would cost more than. + * Words a move by an odd number of words takes at least before it goes through the scratch, which costs two views of + * it; a shorter one copies a word at a time, allocating nothing. A move this long takes about a microsecond, the views + * tens of nanoseconds. */ -const move = (entries: Int32Array, to: number, from: number, words: number): void => { - if (words > 96) { +const oddWideMinimum = 1024; + +/** + * Moves `words` of entries from `from` to `to` in a table's buffer. Shared memory copies an element at a time, so a + * long move copies 8-byte words, 8 times faster than 4-byte ones: directly when both ends have the same parity, an + * edge word by hand; else, past `oddWideMinimum`, through a scratch, which shifts them by a word, the target's edge + * words kept. + */ +const moveWide = (table: HashTable, to: number, from: number, words: number): void => { + const { entries } = table; + if (!table.memory.shared) { entries.copyWithin(to, from, from + words); return; } + const wide = wideOf(table); + if (((to ^ from) & 1) === 0) { + const odd = from & 1; + const even = (words - odd) & ~1; + const rest = words - odd - even; + if (to < from) { + if (odd === 1) entries[to] = entries[from]; + wide.copyWithin((to + odd) >> 1, (from + odd) >> 1, (from + odd + even) >> 1); + if (rest === 1) entries[to + words - 1] = entries[from + words - 1]; + } + else { + if (rest === 1) entries[to + words - 1] = entries[from + words - 1]; + wide.copyWithin((to + odd) >> 1, (from + odd) >> 1, (from + odd + even) >> 1); + if (odd === 1) entries[to] = entries[from]; + } + return; + } + if (words < oddWideMinimum) { + if (to < from) for (let k = 0; k < words; k++) entries[to + k] = entries[from + k]; + else for (let k = words - 1; k >= 0; k--) entries[to + k] = entries[from + k]; + return; + } + const sourceFirst = from & ~1; + const sourceEnd = (from + words + 1) & ~1; + const targetFirst = to & ~1; + const targetEnd = (to + words + 1) & ~1; + const scratch = scratchOf(table, Math.max(sourceEnd - sourceFirst, targetEnd - targetFirst) + 2); + const scratchWords = table.scratchWords; + scratch.set(wide.subarray(sourceFirst >> 1, sourceEnd >> 1)); + scratchWords.copyWithin(to - targetFirst, from - sourceFirst, from - sourceFirst + words); + if ((to & 1) === 1) scratchWords[0] = entries[targetFirst]; + if (((to + words) & 1) === 1) scratchWords[targetEnd - targetFirst - 1] = entries[to + words]; + wide.set(scratch.subarray(0, (targetEnd - targetFirst) >> 1), targetFirst >> 1); +}; + +/** + * Moves `words` of entries, a multiple of `entryWords`, from `from` to `to` within a table's buffer: by hand, an entry + * a step, when few, which a native call would cost more than. + */ +const move = (table: HashTable, to: number, from: number, words: number): void => { + const { entries } = table; + if (words > 96) { + moveWide(table, to, from, words); + return; + } if (to < from) { for (let k = 0; k < words; k += 3) { entries[to + k] = entries[from + k]; @@ -242,15 +386,16 @@ const move = (entries: Int32Array, to: number, from: number, words: number): voi } }; -/** Where entity index `index` is in a bucket, or where it would go, by binary search: `~position` when it isn't there. */ -export const positionIn = (table: HashTable, bucket: number, index: number): number => { - const { entries } = table; - const base = startOf(table, bucket); +/** + * Where entity index `index` is among the `end` entries in entity order from entry `start`, holes included, or where it + * would go, by binary search: `~position` when it isn't there. + */ +const search = (entries: Int32Array, start: number, end: number, index: number): number => { let low = 0; - let high = lengthOf(table, bucket); + let high = end; while (low < high) { const middle = (low + high) >>> 1; - const at = entries[(base + middle) * entryWords]; + const at = entries[(start + middle) * entryWords]; if (at < index) low = middle + 1; else if (at > index) high = middle; else return middle; @@ -258,9 +403,24 @@ export const positionIn = (table: HashTable, bucket: number, index: number): num return ~low; }; +/** Where entity index `index`'s entry is in a bucket, or -1 when it isn't there. */ +const findEntry = (table: HashTable, bucket: number, index: number): number => { + const { entries } = table; + const start = startOf(table, bucket); + const tail = table.tails[bucket]; + if (tail < 0) { + const position = search(entries, start, lengthOf(table, bucket), index); + return position < 0 ? -1 : position; + } + const position = table.positionOf[index]; + if (position >= tail && position < table.extents[bucket] && entries[(start + position) * entryWords] === index) return position; + const found = search(entries, start, tail, index); + return found >= 0 && entries[(start + found) * entryWords + 1] >= 0 ? found : -1; +}; + /** Updates where entity index `index` is: its archetype and row, in the bucket it's filed in. */ export const relocate = (table: HashTable, bucket: number, index: number, archetype: number, row: number): void => { - const position = positionIn(table, bucket, index); + const position = findEntry(table, bucket, index); if (position < 0) return; const at = (startOf(table, bucket) + position) * entryWords; table.entries[at + 1] = archetype; @@ -273,82 +433,299 @@ const countEmpty = (table: HashTable, before: number, after: number): void => { else if (before > 0 && after === 0) table.empty++; }; -/** Removes entity index `index` from a bucket, stamping it. */ -export const removeEntry = (table: HashTable, bucket: number, index: number, tick: number): void => { - const position = positionIn(table, bucket, index); - if (position < 0) return; +/** Empties the list of unsorted buckets, each one listed no longer. */ +const clearList = (table: HashTable): void => { + const list = table.unsortedBuckets; + for (let k = 0; k < table.unsortedListed; k++) table.listed[list[k]] = 0; + table.unsortedListed = 0; +}; + +/** Marks a sorted bucket unsorted, with an empty tail and no holes, listing it unless it's listed still. */ +const unsort = (table: HashTable, bucket: number): void => { + const length = lengthOf(table, bucket); + table.tails[bucket] = length; + table.extents[bucket] = length; + table.holes[bucket] = length; + table.unsorted++; + if (table.listed[bucket] === 1) return; + let listed = table.unsortedListed; + const list = table.unsortedBuckets; + // The list drops the buckets sorted since they were listed once they're most of it, so each listing costs O(1) + // amortized. + if (listed >= 2 * table.unsorted + 8) { + let kept = 0; + for (let k = 0; k < listed; k++) { + if (table.tails[list[k]] >= 0) list[kept++] = list[k]; + else table.listed[list[k]] = 0; + } + listed = kept; + } + if (listed === list.length) table.unsortedBuckets = grown(list, 2 * listed, 0); + table.unsortedBuckets[listed] = bucket; + table.unsortedListed = listed + 1; + table.listed[bucket] = 1; +}; + +/** Marks an unsorted bucket sorted. */ +const sorted = (table: HashTable, bucket: number): void => { + table.tails[bucket] = -1; + if (--table.unsorted === 0) clearList(table); +}; + +/** + * After a change to an unsorted bucket: sorted when it's empty or in order, or once its holes and tail pass four times + * its entries and 64 more, which bounds what a bucket nothing reads by position keeps. + */ +const settle = (table: HashTable, bucket: number): void => { + const length = lengthOf(table, bucket); + const extent = table.extents[bucket]; + const tail = table.tails[bucket]; + if (length === 0 || (extent === length && tail === extent)) sorted(table, bucket); + else if (2 * extent - length - tail > (length << 2) + unsortedMinimum) sortBucket(table, bucket); +}; + +/** Removes entity index `index` from a bucket, stamping it; a long one may be left unsorted when `unsorting`. */ +export const removeEntry = (table: HashTable, bucket: number, index: number, tick: number, unsorting: boolean): void => { const record = bucket * table.stride; const length = table.records[record + recordLength]; - const base = table.records[record + recordStart] * entryWords; - move(table.entries, base + position * entryWords, base + (position + 1) * entryWords, (length - position - 1) * entryWords); + const start = table.records[record + recordStart]; + const { entries } = table; + const tail = table.tails[bucket]; + if (tail < 0) { + const position = search(entries, start, length, index); + if (position < 0) return; + if (position < length - 1) { + if (!unsorting || length < unsortedMinimum) { + const base = start * entryWords; + move(table, base + position * entryWords, base + (position + 1) * entryWords, (length - position - 1) * entryWords); + } + else { + unsort(table, bucket); + entries[(start + position) * entryWords + 1] = -1; + table.holes[bucket] = position; + } + } + } + else { + const extent = table.extents[bucket]; + const position = table.positionOf[index]; + if (position >= tail && position < extent && entries[(start + position) * entryWords] === index) { + // Out of the tail, its last entry taking its place. + const at = (start + position) * entryWords; + const last = (start + extent - 1) * entryWords; + if (at !== last) { + entries[at] = entries[last]; + entries[at + 1] = entries[last + 1]; + entries[at + 2] = entries[last + 2]; + table.positionOf[entries[at]] = position; + } + table.extents[bucket] = extent - 1; + } + else { + const found = search(entries, start, tail, index); + if (found < 0 || entries[(start + found) * entryWords + 1] < 0) return; + if (found === tail - 1 && tail === extent) { + table.tails[bucket] = tail - 1; + table.extents[bucket] = extent - 1; + table.holes[bucket] = Math.min(table.holes[bucket], tail - 1); + } + else { + entries[(start + found) * entryWords + 1] = -1; + table.holes[bucket] = Math.min(table.holes[bucket], found); + } + } + } table.records[record + recordLength] = length - 1; table.ticks[bucket] = tick; table.size--; countEmpty(table, length, length - 1); + if (table.tails[bucket] >= 0) settle(table, bucket); }; -/** Inserts entity index `index` at its archetype and row into a bucket, in entity order, stamping it. */ -export const insertEntry = (table: HashTable, bucket: number, index: number, archetype: number, row: number, tick: number): void => { - const record = bucket * table.stride; - const length = table.records[record + recordLength]; - reserve(table, bucket, length + 1); - const found = positionIn(table, bucket, index); - const position = found < 0 ? ~found : found; - const base = table.records[record + recordStart] * entryWords; +/** Appends an entry to an unsorted bucket's tail. */ +const appendTail = (table: HashTable, bucket: number, index: number, archetype: number, row: number): void => { + const extent = table.extents[bucket]; + reserve(table, bucket, extent + 1); + const at = (startOf(table, bucket) + extent) * entryWords; const { entries } = table; - move(entries, base + (position + 1) * entryWords, base + position * entryWords, (length - position) * entryWords); - const at = base + position * entryWords; entries[at] = index; entries[at + 1] = archetype; entries[at + 2] = row; + table.positionOf[index] = extent; + table.extents[bucket] = extent + 1; +}; + +/** + * Inserts entity index `index`, which the bucket doesn't hold, at its archetype and row, stamping it: in entity order, + * or when `unsorting`, into a long bucket's tail, or the hole it left. + */ +export const insertEntry = (table: HashTable, bucket: number, index: number, archetype: number, row: number, tick: number, unsorting: boolean): void => { + const record = bucket * table.stride; + const length = table.records[record + recordLength]; + // A full unsorted bucket with holes closes them in place rather than moving its segment to grow it. + if (table.tails[bucket] >= 0 && table.extents[bucket] > length && table.extents[bucket] >= table.records[record + recordCapacity]) sortBucket(table, bucket); + const tail = table.tails[bucket]; + if (tail < 0) { + reserve(table, bucket, length + 1); + const start = table.records[record + recordStart]; + const { entries } = table; + const after = length === 0 || entries[(start + length - 1) * entryWords] < index; + if (after || !unsorting || length < unsortedMinimum) { + const found = after ? ~length : search(entries, start, length, index); + const position = found < 0 ? ~found : found; + const base = start * entryWords; + move(table, base + (position + 1) * entryWords, base + position * entryWords, (length - position) * entryWords); + const at = base + position * entryWords; + entries[at] = index; + entries[at + 1] = archetype; + entries[at + 2] = row; + } + else { + unsort(table, bucket); + appendTail(table, bucket, index, archetype, row); + } + } + else { + const start = table.records[record + recordStart]; + const { entries } = table; + const extent = table.extents[bucket]; + const found = search(entries, start, tail, index); + if (found >= 0) { + // The hole it left. + const at = (start + found) * entryWords; + entries[at + 1] = archetype; + entries[at + 2] = row; + } + else if (~found === tail && tail === extent) { + // After every entry in order, with no tail: still in order. + reserve(table, bucket, extent + 1); + const at = (startOf(table, bucket) + extent) * entryWords; + table.entries[at] = index; + table.entries[at + 1] = archetype; + table.entries[at + 2] = row; + if (table.holes[bucket] === tail) table.holes[bucket] = tail + 1; + table.tails[bucket] = tail + 1; + table.extents[bucket] = extent + 1; + } + else appendTail(table, bucket, index, archetype, row); + } table.records[record + recordLength] = length + 1; table.ticks[bucket] = tick; table.size++; countEmpty(table, length, length + 1); + if (table.tails[bucket] >= 0) settle(table, bucket); +}; + +/** Copies entry `from` of `source` to entry `to` of `target`. */ +const copyEntry = (target: Int32Array, to: number, source: Int32Array, from: number): void => { + target[to * entryWords] = source[from * entryWords]; + target[to * entryWords + 1] = source[from * entryWords + 1]; + target[to * entryWords + 2] = source[from * entryWords + 2]; }; /** - * Applies many removals and insertions to one bucket in one merge: `removed` entity indexes and `added` entries - * (`entryWords` each), both in entity order. Keeps the bucket's entries in entity order. + * Sorts `count` entries by entity index, the first `count * entryWords` words of `spare`, which holds twice that: runs + * of eight by insertion, then merged in passes between its halves. */ -export const mergeEntries = (table: HashTable, bucket: number, removed: Int32Array, removedCount: number, added: Int32Array, addedCount: number, scratch: Int32Array, tick: number): void => { - const record = bucket * table.stride; - const length = table.records[record + recordLength]; - const result = length - removedCount + addedCount; - const base = table.records[record + recordStart] * entryWords; - // The bucket's current entries, copied out, merged back with the insertions, skipping the removals. - scratch.set(table.entries.subarray(base, base + length * entryWords)); - reserve(table, bucket, result); - const target = table.entries; - let to = table.records[record + recordStart] * entryWords; - let r = 0; - let a = 0; - for (let k = 0; k < length; k++) { - const index = scratch[k * entryWords]; - while (r < removedCount && removed[r] < index) r++; - while (a < addedCount && added[a * entryWords] < index) { - target[to++] = added[a * entryWords]; - target[to++] = added[a * entryWords + 1]; - target[to++] = added[a * entryWords + 2]; - a++; +const sortEntries = (spare: Int32Array, count: number): void => { + const run = 8; + for (let low = 0; low < count; low += run) { + for (let k = low + 1, high = Math.min(count, low + run); k < high; k++) { + const index = spare[k * entryWords]; + const archetype = spare[k * entryWords + 1]; + const row = spare[k * entryWords + 2]; + let at = k; + while (at > low && spare[(at - 1) * entryWords] > index) { + copyEntry(spare, at, spare, at - 1); + at--; + } + spare[at * entryWords] = index; + spare[at * entryWords + 1] = archetype; + spare[at * entryWords + 2] = row; + } + } + let from = 0; + let to = count; + for (let width = run; width < count; width *= 2) { + for (let low = 0; low < count; low += 2 * width) { + const middle = Math.min(low + width, count); + const high = Math.min(low + 2 * width, count); + let a = low; + let b = middle; + for (let k = low; k < high; k++) { + if (b >= high || (a < middle && spare[(from + a) * entryWords] <= spare[(from + b) * entryWords])) copyEntry(spare, to + k, spare, from + a++); + else copyEntry(spare, to + k, spare, from + b++); + } } - if (r < removedCount && removed[r] === index) { - r++; + const swap = from; + from = to; + to = swap; + } + if (from !== 0) spare.copyWithin(0, from * entryWords, (from + count) * entryWords); +}; + +/** + * Puts an unsorted bucket's entries in entity order again: its tail sorted aside, its holes closed from the first, and + * the tail merged in from the end, so only the entries past the first hole and the first tail entry's place move. + */ +export const sortBucket = (table: HashTable, bucket: number): void => { + const tail = table.tails[bucket]; + const extent = table.extents[bucket]; + const start = startOf(table, bucket); + const { entries } = table; + const count = extent - tail; + if (table.spare.length < 2 * count * entryWords) table.spare = new Int32Array(Math.max(2 * count * entryWords, 2 * table.spare.length)); + const { spare } = table; + for (let k = 0; k < count; k++) copyEntry(spare, k, entries, start + tail + k); + sortEntries(spare, count); + // Holes closed, a run of entries at a time; past the last hole, the rest in one move. + let left = extent - lengthOf(table, bucket); + let write = Math.min(table.holes[bucket], tail); + let read = write; + while (read < tail) { + if (left === 0) { + if (write !== read) move(table, (start + write) * entryWords, (start + read) * entryWords, (tail - read) * entryWords); + write += tail - read; + break; + } + if (entries[(start + read) * entryWords + 1] < 0) { + read++; + left--; continue; } - target[to++] = index; - target[to++] = scratch[k * entryWords + 1]; - target[to++] = scratch[k * entryWords + 2]; + let end = read + 1; + while (end < tail && entries[(start + end) * entryWords + 1] >= 0) end++; + if (write !== read) move(table, (start + write) * entryWords, (start + read) * entryWords, (end - read) * entryWords); + write += end - read; + read = end; } - for (; a < addedCount; a++) { - target[to++] = added[a * entryWords]; - target[to++] = added[a * entryWords + 1]; - target[to++] = added[a * entryWords + 2]; + // The tail merged in from the end: each of its entries, from the last, after the run of entries above it. + let kept = write; + for (let k = count - 1; k >= 0; k--) { + const index = spare[k * entryWords]; + const found = search(entries, start, kept, index); + const position = found < 0 ? ~found : found; + const to = position + k + 1; + if (kept > position) move(table, (start + to) * entryWords, (start + position) * entryWords, (kept - position) * entryWords); + copyEntry(entries, start + position + k, spare, k); + kept = position; } - table.records[record + recordLength] = result; - table.ticks[bucket] = tick; - table.size += result - length; - countEmpty(table, length, result); + sorted(table, bucket); +}; + +/** Sorts every unsorted bucket, before the table's arrays are handed to a reader that reads entries by position. */ +export const sortHashTable = (table: HashTable): void => { + const list = table.unsortedBuckets; + // Sorting the last unsorted one empties the list. + for (let k = 0; k < table.unsortedListed; k++) if (table.tails[list[k]] >= 0) sortBucket(table, list[k]); +}; + +/** Forgets every unsorted bucket, before the table is built again. */ +export const forgetUnsorted = (table: HashTable): void => { + const list = table.unsortedBuckets; + for (let k = 0; k < table.unsortedListed; k++) table.tails[list[k]] = -1; + table.unsorted = 0; + clearList(table); }; /** @@ -374,8 +751,9 @@ export const tidyHashTable = (table: HashTable, tick: number): void => { if (table.garbage > Math.max(1024, table.used - table.garbage)) compact(table); }; -/** Lays every bucket's segment out again, one after another, with their spare slots. */ +/** Lays every bucket's segment out again, one after another, with their spare slots, each sorted first. */ const compact = (table: HashTable): void => { + sortHashTable(table); const { records, stride } = table; let total = 0; for (let bucket = 0; bucket < table.count; bucket++) if (!isFreed(table, bucket)) total += capacityFor(records[bucket * stride + recordLength]); @@ -405,6 +783,7 @@ const compact = (table: HashTable): void => { */ export const loadHashTable = (table: HashTable, count: number, entries: Int32Array, keys: Uint32Array, hashes: Uint32Array, bucketOfEntry: Int32Array, tick: number, entityCount: number): void => { const { words, stride } = table; + forgetUnsorted(table); coverEntities(table, entityCount); table.bucketOf.fill(-1); // Sized for as many buckets as it last held, growing as more are found, so the table is as small as its keys need. diff --git a/packages/ecs/src/implementation/world/index-joined.test.ts b/packages/ecs/src/implementation/world/index-joined.test.ts index 60bb57e1..8aadce65 100644 --- a/packages/ecs/src/implementation/world/index-joined.test.ts +++ b/packages/ecs/src/implementation/world/index-joined.test.ts @@ -25,6 +25,9 @@ const rank = Symbol("indexJoinedTest/rank"); const label = Symbol("indexJoinedTest/label"); const name = Symbol("indexJoinedTest/name"); const title = Symbol("indexJoinedTest/title"); +const band = Symbol("indexJoinedTest/band"); +const bandReader = Symbol("indexJoinedTest/bandReader"); +const marked = Symbol("indexJoinedTest/marked"); declare module "@adobe/ecs" { export interface Threads { @@ -40,6 +43,9 @@ declare module "@adobe/ecs" { [label]: { type: { readonly text: string }; thread: "indexJoinedTest" }; [name]: { type: { readonly text: string }; thread: "indexJoinedTest" }; [title]: { type: string }; + [band]: { type: number }; + [bandReader]: { type: true }; + [marked]: { type: true }; } } Component.register(cell, { type: { type: "array", items: { type: "integer", minimum: -1000, maximum: 1000 }, minItems: 2, maxItems: 2 } }); @@ -54,6 +60,9 @@ const blank = { text: "" }; Component.register(label, { type: { ...text, default: blank }, thread: "indexJoinedTest" }); Component.register(name, { type: text, thread: "indexJoinedTest" }); Component.register(title, { type: { type: "string" } }); +Component.register(band, { type: { type: "integer", minimum: 0, maximum: 1 } }); +Component.register(bandReader, { type: { const: true } }); +Component.register(marked, { type: { const: true } }); const grid = Index.create({ name: "indexJoinedTest/grid", key: [cell], components: { has: [weight] } }); const serials = Index.create({ name: "indexJoinedTest/serials", key: [serial], unique: true, components: { has: [weight] } }); @@ -61,6 +70,65 @@ const lanes = Index.create({ name: "indexJoinedTest/lanes", key: [lane, rank], s const names = Index.create({ name: "indexJoinedTest/names", key: [name] }); const titles = Index.create({ name: "indexJoinedTest/titles", key: [title] }); const weighted = Index.create({ name: "indexJoinedTest/weighted", key: [], components: { has: [weight] } }); +/** Two long buckets, which changes leave unsorted until a list is read by position. */ +const bands = Index.create({ name: "indexJoinedTest/bands", key: [band], components: { has: [serial] } }); + +/** On the joined thread: the serials of a band's rows, in entity order. */ +const describeBand = System.create({ + name: "indexJoinedTest/describeBand", + components: { read: [target], write: [label], has: [bandReader] }, + lookup: { inBand: { index: bands, read: [serial] } }, +}, (row, { lookup }) => { + const list = lookup.inBand(row[target]); + const serials: number[] = []; + for (let i = 0; i < list.length; i++) serials.push(list.at(i)[serial]); + row[label] = { text: `${row[target]}:${serials.join(",")}` }; +}); +const seedBands = System.create({ name: "indexJoinedTest/seedBands" }, ({ commands }) => { + for (let k = 0; k < 300; k++) commands.spawn([band, serial])({ [band]: k % 2, [serial]: k }); + for (let k = 0; k < 2; k++) commands.spawn([target, label, bandReader])({ [target]: k, [label]: blank }); +}); +/** Moves some entities to the other band, despawns some and spawns others. */ +const stirBands = System.create({ name: "indexJoinedTest/stirBands", components: { read: [serial], write: [band] } }, (row, { commands }, input: { readonly round: number }) => { + if ((row[serial] + input.round) % 7 === 0) row[band] = 1 - row[band]; + if ((row[serial] + input.round) % 29 === 0) commands.despawn(row.entity); + if (row[serial] % 31 === input.round) commands.spawn([band, serial])({ [band]: row[serial] % 2, [serial]: 1000 + row[serial] * 10 + input.round }); +}); +/** An observer system of the joined thread's labels: the serials of each reader's band, in entity order, by reader. */ +let observedBands: string[] = []; +const observeBands = System.create({ + name: "indexJoinedTest/observeBands", observer: true, + components: { read: [target], has: [bandReader, label] }, + lookup: { inBand: { index: bands, read: [serial] } }, +}, (row, { lookup }) => { + const list = lookup.inBand(row[target]); + const serials: number[] = []; + for (let i = 0; i < list.length; i++) serials.push(list.at(i)[serial]); + observedBands[row[target]] = serials.join(","); +}); +/** Seeds the bands, then observes them where `holder` runs observer systems, around changes; what it saw each time. */ +const observedBandScenario = async (world: CreatedWorld, holder: World): Promise => { + const seen: (readonly string[])[] = []; + await world.run(seedBands); + for (let round = 0; round < 6; round++) { + await world.run(stirBands, { round }); + observedBands = []; + await holder.run(observeBands); + seen.push([...observedBands]); + } + return seen; +}; +/** Seeds the bands, then reads them on the thread around changes; every label after each read. */ +const bandScenario = async (world: CreatedWorld, holder: World): Promise => { + const seen: (readonly string[])[] = []; + await world.run(seedBands); + for (let round = 0; round < 6; round++) { + await world.run(describeBand); + seen.push(await labels(holder, world)); + await world.run(stirBands, { round }); + } + return seen; +}; /** On the joined thread, in the tier before the lookups, so a task of consecutive tiers there would see the index before it: moves some cells. */ const shift = System.create({ name: "indexJoinedTest/shift", components: { read: [serial], write: [cell, label] } }, row => { @@ -116,14 +184,14 @@ const seed = System.create({ name: "indexJoinedTest/seed" }, ({ commands }) => { }); /** Moves cells, changes weights and ranks, despawns some and spawns others. */ -const stir = System.create({ name: "indexJoinedTest/stir", input: { type: "object", properties: { round: { type: "integer" } } }, components: { read: [serial], write: [cell, weight] } }, (row, { input, commands }) => { +const stir = System.create({ name: "indexJoinedTest/stir", components: { read: [serial], write: [cell, weight] } }, (row, { commands }, input: { readonly round: number }) => { const [x, y] = row[cell]; if ((row[serial] + input.round) % 9 === 0) row[cell] = [(x + 1) % 13, y]; if ((row[serial] + input.round) % 11 === 0) row[weight] = row[weight] + 1; if ((row[serial] + input.round) % 23 === 0) commands.despawn(row.entity); if (row[serial] % 50 === input.round) commands.spawn([cell, weight, serial, target, label])({ [cell]: [input.round, 3], [weight]: 2, [serial]: 1000 + row[serial], [target]: row[serial], [label]: blank }); }); -const rerank = System.create({ name: "indexJoinedTest/rerank", input: { type: "object", properties: { round: { type: "integer" } } }, components: { write: [rank] } }, (row, { input }) => { +const rerank = System.create({ name: "indexJoinedTest/rerank", components: { write: [rank] } }, (row, _context, input: { readonly round: number }) => { if ((row[rank] + input.round) % 5 === 0) row[rank] = (row[rank] * 3 + 1) % 97; }); @@ -211,6 +279,81 @@ const joinedStatus = (world: CreatedWorld): Promise => new Promise(resolve }); }); +/** Entities a keyless and a unique index hold, each with this thread's label and a string title. */ +const seedHeld = System.create({ name: "indexJoinedTest/seedHeld" }, ({ commands }, input: { readonly from: number; readonly count: number }) => { + for (let k = input.from; k < input.from + input.count; k++) commands.spawn([serial, weight, label, title])({ [serial]: k, [weight]: k % 5, [label]: blank, [title]: `t${k}` }); +}); +/** On the joined thread: every label, by serial and round. */ +const relabelHeld = System.create({ name: "indexJoinedTest/relabelHeld", components: { read: [serial], write: [label], has: [title] } }, (row, _context, input: { readonly round: number }) => { + row[label] = { text: `${row[serial]}.${input.round}` }; +}); +/** On the world's thread: every weight and title, by round. */ +const rewriteHeld = System.create({ name: "indexJoinedTest/rewriteHeld", components: { read: [serial], write: [weight, title] } }, (row, _context, input: { readonly round: number }) => { + row[weight] = row[serial] * 10 + input.round; + row[title] = `t${row[serial]}.${input.round}`; +}); +/** Moves every held entity to the other archetype, marked on one run and not on the next. */ +const markHeld = System.create({ name: "indexJoinedTest/markHeld", components: { has: [title], not: [marked] } }, (row, { commands }) => { + commands.add(row.entity, marked); +}); +const unmarkHeld = System.create({ name: "indexJoinedTest/unmarkHeld", components: { has: [title, marked] } }, (row, { commands }) => { + commands.remove(row.entity, marked); +}); +const held = Index.create({ name: "indexJoinedTest/held", key: [], components: { has: [serial, weight, label, title] } }); +const heldSerials = Index.create({ name: "indexJoinedTest/heldSerials", key: [serial], unique: true, components: { has: [weight, label, title] } }); +/** What `observeHeld` read last. */ +let observedHeld = ""; +/** An observer system reading every held row through the keyless index, and serial 3's through the unique one. */ +const observeHeld = System.create({ + name: "indexJoinedTest/observeHeld", observer: true, + lookup: { all: { index: held, read: [serial, weight, label, title] }, bySerial: { index: heldSerials, read: [weight, label, title] } }, +}, ({ lookup }) => { + const all = lookup.all(); + const read: string[] = []; + for (let i = 0; i < all.length; i++) { + const row = all.at(i); + read.push(`${row[serial]}:${row[weight]}:${row[label].text}:${row[title]}`); + } + const one = lookup.bySerial(3); + observedHeld = `${read.sort().join(",")}|${one === null ? "none" : `${one[weight]}:${one[label].text}:${one[title]}`}`; +}); +/** + * Observes the held rows where `holder` runs observer systems, around what makes the rows a lookup gave stale: arrays + * grown, values and strings rewritten, the thread's labels rewritten, and every row moved past what its log holds. + */ +const heldScenario = async (world: CreatedWorld, holder: World): Promise => { + const seen: string[] = []; + const observe = async (): Promise => { + await holder.run(observeHeld); + seen.push(observedHeld); + }; + await world.run(seedHeld, { from: 0, count: 8 }); + await world.run(relabelHeld, { round: 0 }); + await observe(); + // Past the archetype's capacity, without growing the memory. + await world.run(seedHeld, { from: 8, count: 12 }); + await observe(); + await world.run(rewriteHeld, { round: 1 }); + await world.run(relabelHeld, { round: 1 }); + await observe(); + await world.run(seedHeld, { from: 20, count: 600 }); + await observe(); + // Both archetypes and the reports' grown, so lapping the log again replaces no arrays: only the mirror's rebuild + // makes views again. + const lap = async (): Promise => { + for (let k = 0; k < 30; k++) await world.run(k % 2 === 0 ? markHeld : unmarkHeld); + await world.run(dropReports); + }; + await lap(); + await observe(); + await lap(); + await observe(); + await world.run(relabelHeld, { round: 2 }); + await world.run(rewriteHeld, { round: 2 }); + await observe(); + return seen; +}; + describe("systems on a joined thread looking up through indexes", () => { it("find what the world's own systems find: hash buckets, unique rows, sorted ranges and changes", async () => { const inline = createWorld({ [workers]: 0 }); @@ -230,6 +373,48 @@ describe("systems on a joined thread looking up through indexes", () => { await remote.dispose(); await world.dispose(); }); + it("read a hash index's long buckets in entity order after changes left them unsorted, as the world's own systems do", async () => { + const inline = createWorld({ [workers]: 0 }); + const alone = await bandScenario(inline, inline); + await inline.dispose(); + const world = createWorld({ [workers]: 0 }); + const channel = new MessageChannel(); + world.share(channel.port1); + const remote = join(channel.port2, { threads: ["indexJoinedTest"] }); + await joinedStatus(world); + expect(await bandScenario(world, remote)).toEqual(alone); + expect(alone.every(round => round.length === 2 && round.every(text => text.split(",").length > 100))).toBe(true); + await remote.dispose(); + await world.dispose(); + }); + it("observe a hash index's long buckets in entity order after changes left them unsorted, as the world's own observer systems do", async () => { + const inline = createWorld({ [workers]: 0 }); + const alone = await observedBandScenario(inline, inline); + await inline.dispose(); + const world = createWorld({ [workers]: 0 }); + const channel = new MessageChannel(); + world.share(channel.port1); + const remote = join(channel.port2, { threads: ["indexJoinedTest"] }); + await joinedStatus(world); + expect(await observedBandScenario(world, remote)).toEqual(alone); + expect(alone.every(round => round.length === 2 && round.every(text => text.split(",").length > 100))).toBe(true); + await remote.dispose(); + await world.dispose(); + }); + it("read rows as of each run's sync point once arrays grew, values, strings and the thread's objects changed, and its log lapped", async () => { + const inline = createWorld({ [workers]: 0 }); + const alone = await heldScenario(inline, inline); + await inline.dispose(); + const world = createWorld({ [workers]: 0 }); + const channel = new MessageChannel(); + world.share(channel.port1); + const remote = join(channel.port2, { threads: ["indexJoinedTest"] }); + await joinedStatus(world); + expect(await heldScenario(world, remote)).toEqual(alone); + expect(alone.at(-1)).toContain("|32:3.2:t3.2"); + await remote.dispose(); + await world.dispose(); + }); it("release the index and its records once the last system using it there is removed", async () => { const driver = backToBack(); const world = createWorld({ [workers]: 0, [always]: true }, { loop: driver }); diff --git a/packages/ecs/src/implementation/world/index-kernels.test.ts b/packages/ecs/src/implementation/world/index-kernels.test.ts index 53cb5a70..980831ff 100644 --- a/packages/ecs/src/implementation/world/index-kernels.test.ts +++ b/packages/ecs/src/implementation/world/index-kernels.test.ts @@ -127,11 +127,11 @@ const watch = System.create({ if (lookup.near([row[cell][0], row[cell][1]]).changed) row[changes] = row[changes] + 1; }); -const spawnGrid = System.create({ name: "indexKernelsTest/spawnGrid", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const spawnGrid = System.create({ name: "indexKernelsTest/spawnGrid" }, ({ commands }, input: { readonly count: number }) => { for (let k = 0; k < input.count; k++) commands.spawn([cell, weight, total, count, changes, serial])({ [cell]: [(k * 7) % 5, (k * 3) % 4], [weight]: k % 9, [total]: 0, [count]: 0, [changes]: 0, [serial]: k }); }); -const spawnLanes = System.create({ name: "indexKernelsTest/spawnLanes", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const spawnLanes = System.create({ name: "indexKernelsTest/spawnLanes" }, ({ commands }, input: { readonly count: number }) => { for (let k = 0; k < input.count; k++) commands.spawn([lane, rank, ranked, count, seen])({ [lane]: k % 3, [rank]: (k * 37) % 101, [ranked]: true, [count]: 0, [seen]: Entity.none }); }); diff --git a/packages/ecs/src/implementation/world/index-keyless.test.ts b/packages/ecs/src/implementation/world/index-keyless.test.ts index fa71c8a7..552b45cb 100644 --- a/packages/ecs/src/implementation/world/index-keyless.test.ts +++ b/packages/ecs/src/implementation/world/index-keyless.test.ts @@ -76,7 +76,7 @@ const expected = async (world: CreatedWorld): Promise<{ readonly entities: reado all: [...await select(world, {})].sort(byIndex), }); -const spawn = System.create({ name: "indexKeylessTest/spawn", input: { type: "object", properties: { weights: { type: "array", items: { type: "number" } } } } }, ({ input, commands }) => { +const spawn = System.create({ name: "indexKeylessTest/spawn" }, ({ commands }, input: { readonly weights: readonly number[] }) => { for (const value of input.weights) commands.spawn([weight, color])({ [weight]: value, [color]: 0 }); }); diff --git a/packages/ecs/src/implementation/world/index-lookups.ts b/packages/ecs/src/implementation/world/index-lookups.ts index 2ca976e1..83a85063 100644 --- a/packages/ecs/src/implementation/world/index-lookups.ts +++ b/packages/ecs/src/implementation/world/index-lookups.ts @@ -1,7 +1,7 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. import type { Scalar } from '../../interface/registry/layout.js'; import { encodeScalar, encodeValue, hashWords, orderF32, orderF64, partWords, type KeyPart } from './index-keys.js'; -import { entryWords, findBucket, lengthOf, recordKey, recordLength, recordStart, startOf, type HashView } from './index-hash.js'; +import { entryWords, findBucket, isHashTable, lengthOf, recordKey, recordLength, recordStart, sortBucket, startOf, type HashView } from './index-hash.js'; import { blockAt, blockBase, bound, firstPosition, treeMaximum, type SortedView } from './index-sorted.js'; import { sharedRows, type Row, type RowArchetype, type RowHost } from './rows.js'; import { idOf, type Key, type State } from './state.js'; @@ -41,10 +41,10 @@ export type IndexLookup = { readonly watched: readonly number[]; readonly since: Since; /** - * How many times the world's archetypes' arrays were replaced, while which rows bound to them stay current: on the - * world's thread. Null on a joined thread, whose rows check their archetype's version each time. + * How many times archetypes' arrays were replaced, while which rows bound to them stay current: the world's count on + * its thread; on a joined thread, its directory's, moved on once a run finds any archetype's views changed. */ - readonly replaced: { readonly count: number } | null; + readonly replaced: { readonly count: number }; }; /** The lookup behind each function bodies are given, for the plugin's call sites. */ @@ -59,6 +59,8 @@ type Pool = { readonly rows: (Row | undefined)[]; readonly archetypes: number[]; readonly versions: number[]; + /** By position: the archetype it was bound to, since a joined thread's view of one is made again when it changes. */ + readonly bound: (LookupArchetype | undefined)[]; readonly make: (archetype: RowArchetype) => Row; /** The lookup's `replaced` count when every row was last checked against its archetype's version. */ replaced: number; @@ -72,7 +74,7 @@ const makers = new WeakMap Row>(); const poolOf = (lookup: IndexLookup): Pool => { let make = makers.get(lookup); if (make === undefined) makers.set(lookup, make = sharedRows(lookup.rows, lookup.reads, lookup.readIds)); - return { rows: [], archetypes: [], versions: [], make, replaced: lookup.replaced?.count ?? 0, ahead: 0 }; + return { rows: [], archetypes: [], versions: [], bound: [], make, replaced: lookup.replaced.count, ahead: 0 }; }; /** Binds row `n` of a pool to an archetype, making it first if the pool has none there yet. */ @@ -83,12 +85,14 @@ const bindRow = (pool: Pool, n: number, archetypeId: number, archetype: LookupAr rows.push(undefined); pool.archetypes.push(-1); pool.versions.push(-1); + pool.bound.push(undefined); } let found = rows[n]; if (found === undefined) found = rows[n] = pool.make(archetype); else found.bind(archetype); pool.archetypes[n] = archetypeId; pool.versions[n] = archetype.version; + pool.bound[n] = archetype; return found; }; @@ -101,18 +105,22 @@ const madeAhead = 256; * body inlines. */ const growPool = (lookup: IndexLookup, pool: Pool, length: number, archetypeId: number): void => { - pool.ahead = length; + // A first entry that's an unsorted bucket's hole, archetype -1, names none: a later lookup makes them. const archetype = lookup.archetypeAt(archetypeId); if (archetype === undefined) return; + pool.ahead = length; const end = Math.min(length, madeAhead); for (let n = pool.rows.length; n < end; n++) bindRow(pool, n, archetypeId, archetype); }; -/** Binds again the rows whose archetype's arrays were replaced since the pool last checked, on the world's thread. */ +/** Whether row `n` of a pool is bound to an archetype as it is now: the same view, its arrays not replaced since. */ +const boundTo = (pool: Pool, n: number, archetype: LookupArchetype): boolean => pool.bound[n] === archetype && pool.versions[n] === archetype.version; + +/** Binds again the rows whose archetype's arrays were replaced since the pool last checked. */ const refreshPool = (lookup: IndexLookup, pool: Pool, replaced: { readonly count: number }): void => { for (let n = 0; n < pool.rows.length; n++) { const archetype = pool.rows[n] === undefined ? undefined : lookup.archetypeAt(pool.archetypes[n]); - if (archetype !== undefined && archetype.version !== pool.versions[n]) bindRow(pool, n, pool.archetypes[n], archetype); + if (archetype !== undefined && !boundTo(pool, n, archetype)) bindRow(pool, n, pool.archetypes[n], archetype); } pool.replaced = replaced.count; }; @@ -120,27 +128,22 @@ const refreshPool = (lookup: IndexLookup, pool: Pool, replaced: { readonly count /** Binds row `n` again to an entry's archetype, or makes it, when its archetype or that's arrays changed. */ const reboundAt = (lookup: IndexLookup, pool: Pool, n: number, archetypeId: number, row: number): object | undefined => { const { replaced } = lookup; - if (replaced !== null && pool.replaced !== replaced.count) refreshPool(lookup, pool, replaced); + if (pool.replaced !== replaced.count) refreshPool(lookup, pool, replaced); const archetype = lookup.archetypeAt(archetypeId); if (archetype === undefined) return undefined; const current = pool.rows[n]; - const found = current !== undefined && pool.archetypes[n] === archetypeId && pool.versions[n] === archetype.version ? current : bindRow(pool, n, archetypeId, archetype); + const found = current !== undefined && pool.archetypes[n] === archetypeId && boundTo(pool, n, archetype) ? current : bindRow(pool, n, archetypeId, archetype); found.index = row; return found.value; }; -/** Whether row `n` of a pool is bound to its archetype's current arrays, on a joined thread. */ -const currentVersion = (lookup: IndexLookup, pool: Pool, n: number, archetypeId: number): boolean => - pool.versions[n] === lookup.archetypeAt(archetypeId)?.version; - /** - * Points row `n` of a pool at an entry's archetype and row, binding it again only when its archetype or that's arrays - * changed: on the world's thread, when any archetype's were replaced since the pool last checked. + * Points row `n` of a pool at an entry's archetype and row, binding it again only when its archetype changed, or any + * archetype's arrays were replaced since the pool last checked. */ const rowAt = (lookup: IndexLookup, pool: Pool, n: number, archetypeId: number, row: number): object | undefined => { const found = pool.rows[n]; - const { replaced } = lookup; - if (found === undefined || pool.archetypes[n] !== archetypeId || (replaced === null ? !currentVersion(lookup, pool, n, archetypeId) : pool.replaced !== replaced.count)) return reboundAt(lookup, pool, n, archetypeId, row); + if (found === undefined || pool.archetypes[n] !== archetypeId || pool.replaced !== lookup.replaced.count) return reboundAt(lookup, pool, n, archetypeId, row); found.index = row; return found.value; }; @@ -192,6 +195,8 @@ const hashAt = function (this: HashList, position: number): object | undefined { const { bucket, hash } = found; const record = bucket * hash.stride; if (bucket < 0 || !(position >= 0 && position < hash.records[record + recordLength])) return undefined; + // On the world's thread, a bucket a change left unsorted is sorted by the first read by position. + if (hash.unsorted !== 0 && isHashTable(hash) && hash.tails[bucket] >= 0) sortBucket(hash, bucket); const at = (hash.records[record + recordStart] + position) * entryWords; return rowAt(found.lookup, found.pool, position, hash.entries[at + 1], hash.entries[at + 2]); }; @@ -308,8 +313,8 @@ const generatedFind = (lookup: IndexLookup, hash: HashView, shape: Shape, list: ...lookup.unique ? [ `if (bucket < 0 || records[bucket * ${hash.stride} + ${recordLength}] === 0) return null;`, `const at = records[bucket * ${hash.stride} + ${recordStart}] * ${entryWords};`, - "const archetypeId = table.entries[at + 1], archetype = archetypeAt(archetypeId), row = pool.rows[0];", - "if (row === undefined || archetype === undefined || pool.archetypes[0] !== archetypeId || pool.versions[0] !== archetype.version) return rowAt(lookup, pool, 0, archetypeId, table.entries[at + 2]);", + "const archetypeId = table.entries[at + 1], row = pool.rows[0];", + "if (row === undefined || pool.archetypes[0] !== archetypeId || pool.replaced !== replaced.count) return rowAt(lookup, pool, 0, archetypeId, table.entries[at + 2]);", "row.index = table.entries[at + 2];", "return row.value;", ] : [ @@ -320,9 +325,9 @@ const generatedFind = (lookup: IndexLookup, hash: HashView, shape: Shape, list: "};", ].join("\n"); try { - const factory = new Function("key", "table", "found", "list", "pool", "lookup", "rowAt", "growPool", "encodeValue", "parts", "orderF32", "orderF64", "f32", "f64", "u32", "u8", "archetypeAt", body); + const factory = new Function("key", "table", "found", "list", "pool", "lookup", "rowAt", "growPool", "encodeValue", "parts", "orderF32", "orderF64", "f32", "f64", "u32", "u8", "replaced", body); // The factory returns the function its body above makes. - const made: (...values: unknown[]) => unknown = factory(new Uint32Array(Math.max(1, words)), hash, found, list, found.pool, lookup, rowAt, growPool, encodeValue, index.parts, orderF32, orderF64, f32, f64, u32, u8, lookup.archetypeAt); + const made: (...values: unknown[]) => unknown = factory(new Uint32Array(Math.max(1, words)), hash, found, list, found.pool, lookup, rowAt, growPool, encodeValue, index.parts, orderF32, orderF64, f32, f64, u32, u8, lookup.replaced); return made; } catch { diff --git a/packages/ecs/src/implementation/world/index-order.test.ts b/packages/ecs/src/implementation/world/index-order.test.ts index 23df92ec..9fedb072 100644 --- a/packages/ecs/src/implementation/world/index-order.test.ts +++ b/packages/ecs/src/implementation/world/index-order.test.ts @@ -57,7 +57,7 @@ const nudge = System.create({ name: "indexOrderTest/nudge", components: { write: }); /** Entities in two archetypes, interleaved in the index's order, some without the weight the index requires. */ -const seed = System.create({ name: "indexOrderTest/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const seed = System.create({ name: "indexOrderTest/seed" }, ({ commands }, input: { readonly count: number }) => { for (let k = 0; k < input.count; k++) { const values = { [lane]: k % 3, [rank]: (k * 37) % 101, [weight]: k }; if (k % 5 === 4) commands.spawn([lane, rank])({ [lane]: k % 3, [rank]: (k * 37) % 101 }); diff --git a/packages/ecs/src/implementation/world/index-sorted.ts b/packages/ecs/src/implementation/world/index-sorted.ts index 6cfee390..b0a3dcfc 100644 --- a/packages/ecs/src/implementation/world/index-sorted.ts +++ b/packages/ecs/src/implementation/world/index-sorted.ts @@ -585,12 +585,16 @@ const split = (table: SortedTable, k: number, tick: number): void => { reindexFrom(table, k, table.blocks); }; -/** Updates where entity index `entity` is: its archetype and row, at its entry. */ -export const relocateSorted = (table: SortedTable, entity: number, archetype: number, row: number): void => { +/** Updates where entity index `entity` is: its archetype and row, at its entry. Returns whether either changed. */ +export const relocateSorted = (table: SortedTable, entity: number, archetype: number, row: number): boolean => { const base = entryOf(table, entity); - if (base < 0) return; - table.entries[base + table.words + 1] = archetype; - table.entries[base + table.words + 2] = row; + if (base < 0) return false; + const at = base + table.words; + const { entries } = table; + if (entries[at + 1] === archetype && entries[at + 2] === row) return false; + entries[at + 1] = archetype; + entries[at + 2] = row; + return true; }; /** diff --git a/packages/ecs/src/implementation/world/index-workers.cases.ts b/packages/ecs/src/implementation/world/index-workers.cases.ts index 801f401d..b19a7f82 100644 --- a/packages/ecs/src/implementation/world/index-workers.cases.ts +++ b/packages/ecs/src/implementation/world/index-workers.cases.ts @@ -15,4 +15,13 @@ export const indexWorkersCases = (load: () => Promise) => descri // Targets' tallies through the index without a key: its length in millions, then a stride of its weights. expect(three.rows.filter(row => /^\[\d+,-?\d+,\d{7,}\]$/.test(row)).length).toBeGreaterThan(900); }); + it.each([[true], [false]])("read a hash index's long buckets in entity order after changes left them unsorted, WASM %s", async wasm => { + const fixture = await load(); + for (const threads of [0, 3]) { + const orders = await fixture.bandOrders(threads, wasm); + expect(orders.kernel).toEqual(orders.byHand); + expect(orders.kernel.every(round => round[0] > 0)).toBe(true); + if (threads > 0) expect(orders.tasks).toBeGreaterThan(0); + } + }); }); diff --git a/packages/ecs/src/implementation/world/index-workers.fixture.ts b/packages/ecs/src/implementation/world/index-workers.fixture.ts index 8eb066e9..e2f4d400 100644 --- a/packages/ecs/src/implementation/world/index-workers.fixture.ts +++ b/packages/ecs/src/implementation/world/index-workers.fixture.ts @@ -21,6 +21,9 @@ const seen = Symbol("indexWorkersTest/seen"); const serial = Symbol("indexWorkersTest/serial"); const target = Symbol("indexWorkersTest/target"); const tally = Symbol("indexWorkersTest/tally"); +const band = Symbol("indexWorkersTest/band"); +const bandSum = Symbol("indexWorkersTest/bandSum"); +const bandSumByHand = Symbol("indexWorkersTest/bandSumByHand"); declare module "@adobe/ecs" { interface Components { @@ -35,6 +38,9 @@ declare module "@adobe/ecs" { [serial]: { type: number }; [target]: { type: number }; [tally]: { type: number }; + [band]: { type: number }; + [bandSum]: { type: number }; + [bandSumByHand]: { type: number }; } } Component.register(cell, { type: { type: "array", items: { type: "integer", minimum: -1000, maximum: 1000 }, minItems: 2, maxItems: 2 } }); @@ -48,6 +54,9 @@ Component.register(seen, { type: Entity.schema }); Component.register(serial, { type: { type: "integer", minimum: 0, maximum: 1000000 } }); Component.register(target, { type: { type: "integer", minimum: 0, maximum: 1000000 } }); Component.register(tally, { type: { type: "number" } }); +Component.register(band, { type: { type: "integer", minimum: 0, maximum: 1 } }); +Component.register(bandSum, { type: { type: "number" } }); +Component.register(bandSumByHand, { type: { type: "number" } }); const grid = Index.create({ name: "indexWorkersTest/grid", key: [cell], components: { has: [weight] } }); const lanes = Index.create({ name: "indexWorkersTest/lanes", key: [lane, rank], sorted: true }); @@ -137,15 +146,15 @@ const seed = System.create({ name: "indexWorkersTest/seed" }, ({ commands }) => }); /** Moves some cells, changes some weights and ranks, despawns some serials and spawns more: a different input each time. */ -const stir = System.create({ name: "indexWorkersTest/stir", input: { type: "object", properties: { round: { type: "integer" } } }, components: { write: [cell, weight] } }, (row, { input }) => { +const stir = System.create({ name: "indexWorkersTest/stir", components: { write: [cell, weight] } }, (row, _context, input: { readonly round: number }) => { const [x, y] = row[cell]; if ((x + y + input.round) % 11 === 0) row[cell] = [(x + 1) % 31, y]; if ((x * y + input.round) % 13 === 0) row[weight] = row[weight] + 1; }); -const rerank = System.create({ name: "indexWorkersTest/rerank", input: { type: "object", properties: { round: { type: "integer" } } }, components: { write: [rank] } }, (row, { input }) => { +const rerank = System.create({ name: "indexWorkersTest/rerank", components: { write: [rank] } }, (row, _context, input: { readonly round: number }) => { if ((row[rank] + input.round) % 7 === 0) row[rank] = (row[rank] * 3 + 1) % 1009; }); -const churn = System.create({ name: "indexWorkersTest/churn", input: { type: "object", properties: { round: { type: "integer" } } }, components: { read: [serial] } }, (row, { input, commands }) => { +const churn = System.create({ name: "indexWorkersTest/churn", components: { read: [serial] } }, (row, { commands }, input: { readonly round: number }) => { if ((row[serial] + input.round) % 17 === 0) commands.despawn(row.entity); if (row[serial] % 101 === input.round) { commands.spawn([cell, weight, total, count, changes, serial])({ [cell]: [input.round, row[serial] % 29], [weight]: 4, [total]: 0, [count]: 0, [changes]: 0, [serial]: 3000 + row[serial] + input.round }); @@ -190,3 +199,66 @@ export const simulate = async (threads: number, wasm: boolean): Promise<{ readon await ecs.dispose(); } }; + +/** Two long buckets, which changes leave unsorted until something reads them by position. */ +const bands = Index.create({ name: "indexWorkersTest/bands", key: [band], components: { has: [serial] } }); + +/** Each row's band's serials, each times its position plus one: a sum that tells the order. */ +const bandOrder = System.create({ + name: "indexWorkersTest/bandOrder", kernel: "cpu", + components: { read: [target], write: [bandSum] }, + lookup: { inBand: { index: bands, read: [serial] } }, +}, (row, { lookup }) => { + const list = lookup.inBand(row[target] % 2); + let sum = 0; + for (let i = 0; i < list.length; i++) sum += (i + 1) * list.at(i)[serial]; + row[bandSum] = sum; +}); +const bandOrderByHand = System.create({ + name: "indexWorkersTest/bandOrderByHand", + components: { read: [target], write: [bandSumByHand] }, + lookup: { inBand: { index: bands, read: [serial] } }, +}, (row, { lookup }) => { + const list = lookup.inBand(row[target] % 2); + let sum = 0; + for (let i = 0; i < list.length; i++) sum += (i + 1) * list.at(i)[serial]; + row[bandSumByHand] = sum; +}); +const seedBands = System.create({ name: "indexWorkersTest/seedBands" }, ({ commands }) => { + for (let k = 0; k < 600; k++) commands.spawn([band, serial])({ [band]: k % 2, [serial]: k }); + for (let k = 0; k < 512; k++) commands.spawn([target, bandSum, bandSumByHand])({ [target]: k, [bandSum]: 0, [bandSumByHand]: 0 }); +}); +/** Moves some entities to the other band, despawns some and spawns others. */ +const stirBands = System.create({ name: "indexWorkersTest/stirBands", components: { read: [serial], write: [band] } }, (row, { commands }, input: { readonly round: number }) => { + if ((row[serial] + input.round) % 7 === 0) row[band] = 1 - row[band]; + if ((row[serial] + input.round) % 29 === 0) commands.despawn(row.entity); + if (row[serial] % 31 === input.round) commands.spawn([band, serial])({ [band]: row[serial] % 2, [serial]: 1000 + row[serial] * 10 + input.round }); +}); + +/** + * Over rounds of changes, what a kernel reading the bands in order wrote, and what a main-thread system reading them + * after it wrote, each round's sorted; and the run tasks posted to workers. + */ +export const bandOrders = async (threads: number, wasm: boolean): Promise<{ readonly kernel: readonly (readonly number[])[]; readonly byHand: readonly (readonly number[])[]; readonly tasks: number }> => { + const counting = countingWorkers(); + const ecs = createWorld({ [workers]: threads }, { minimumChunk: 64, wasm, pool: counting.platform }); + try { + await ecs.run(seedBands); + const posted = (): number => counting.posted("run") + counting.posted("window"); + const before = posted(); + const kernel: number[][] = []; + const byHand: number[][] = []; + for (let round = 0; round < 6; round++) { + await ecs.run(stirBands, { round }); + await ecs.run(bandOrder); + await ecs.run(bandOrderByHand); + const values = await Promise.all((await select(ecs, { required: [target] })).map(entity => readEntity(ecs, entity, [bandSum, bandSumByHand]))); + kernel.push(values.map(found => found?.[bandSum] ?? -1).sort((a, b) => a - b)); + byHand.push(values.map(found => found?.[bandSumByHand] ?? -1).sort((a, b) => a - b)); + } + return { kernel, byHand, tasks: posted() - before }; + } + finally { + await ecs.dispose(); + } +}; diff --git a/packages/ecs/src/implementation/world/indexes.test.ts b/packages/ecs/src/implementation/world/indexes.test.ts index 3638409c..fe6a9629 100644 --- a/packages/ecs/src/implementation/world/indexes.test.ts +++ b/packages/ecs/src/implementation/world/indexes.test.ts @@ -47,7 +47,7 @@ const byServerId = Index.create({ name: "indexesTest/byServerId", key: [serverId const byLabel = Index.create({ name: "indexesTest/byLabel", key: [label] }); const children = Index.create({ name: "indexesTest/children", key: [owner] }); -const spawnCells = System.create({ name: "indexesTest/spawnCells", input: { type: "object", properties: { cells: { type: "array", items: { type: "array", items: { type: "integer" } } } } } }, ({ input, commands }) => { +const spawnCells = System.create({ name: "indexesTest/spawnCells" }, ({ commands }, input: { readonly cells: readonly (readonly number[])[] }) => { input.cells.forEach(([x, y], k) => commands.spawn([cell, weight])({ [cell]: [x ?? 0, y ?? 0], [weight]: k })); }); @@ -56,9 +56,8 @@ let seen: { readonly entities: readonly Entity[]; readonly weights: readonly num const look = System.create({ name: "indexesTest/look", - input: { type: "object", properties: { x: { type: "integer" }, y: { type: "integer" } } }, lookup: { near: { index: grid, read: [weight] } }, -}, ({ input, lookup }) => { +}, ({ lookup }, input: { readonly x: number; readonly y: number }) => { const near = lookup.near([input.x, input.y]); const entities: Entity[] = []; const weights: number[] = []; @@ -141,7 +140,7 @@ describe("hash indexes on the main thread", () => { it("key strings by value, and entities by their generation too", async () => { await using world = createWorld({ [workers]: 0 }); let found: readonly number[] = []; - const named = System.create({ input: { type: "object", properties: { name: { type: "string" } } }, lookup: { named: { index: byLabel } } }, ({ input, lookup }) => { + const named = System.create({ lookup: { named: { index: byLabel } } }, ({ lookup }, input: { readonly name: string }) => { const list = lookup.named(input.name); found = Array.from({ length: list.length }, (_, i) => list.at(i).entity); }); @@ -157,7 +156,7 @@ describe("hash indexes on the main thread", () => { expect(found).toEqual([]); const parent = labeled[1]; let held: readonly Entity[] = []; - const kids = System.create({ input: { type: "object", properties: { of: { type: "number" } } }, lookup: { of: { index: children } } }, ({ input, lookup }) => { + const kids = System.create({ lookup: { of: { index: children } } }, ({ lookup }, input: { readonly of: number }) => { const list = lookup.of(Entity.none); void list; const of = lookup.of(input.of === 0 ? parent : Entity.none); @@ -177,7 +176,7 @@ describe("hash indexes on the main thread", () => { describe("unique indexes", () => { let known: Entity | null = null; - const find = System.create({ input: { type: "object", properties: { id: { type: "integer" } } }, lookup: { known: { index: byServerId } } }, ({ input, lookup }) => { + const find = System.create({ lookup: { known: { index: byServerId } } }, ({ lookup }, input: { readonly id: number }) => { known = lookup.known(input.id)?.entity ?? null; }); it("give the row of a key or null, and report two entities with one key as an index failure, keeping the lower", async () => { @@ -346,9 +345,8 @@ const foldNear = System.create({ /** Moves each row whose weight is below `share` of 1000 to the cell `cell` gives it, so keys change during runs. */ const rekey = System.create({ - name: "indexesTest/rekey", input: { type: "object", properties: { share: { type: "number" }, seed: { type: "integer" } } }, - components: { read: [weight], write: [cell] }, before: [probe], -}, (row, { input }) => { + name: "indexesTest/rekey", components: { read: [weight], write: [cell] }, before: [probe], +}, (row, _context, input: { readonly share: number; readonly seed: number }) => { if ((row[weight] * 7919 + input.seed) % 1000 < input.share * 1000) row[cell] = cells[(row[weight] + input.seed) % cells.length]; }); @@ -423,13 +421,12 @@ const ranksOf = (list: { readonly length: number; readonly changed: boolean; at( }); describe("sorted indexes on the main thread", () => { - const spawnRanks = System.create({ input: { type: "object", properties: { ranks: { type: "array", items: { type: "number" } } } } }, ({ input, commands }) => { + const spawnRanks = System.create({ }, ({ commands }, input: { readonly ranks: readonly number[] }) => { for (const value of input.ranks) commands.spawn([rank])({ [rank]: value }); }); const query = System.create({ - input: { type: "object", properties: { kind: { type: "string" }, value: { type: "number" }, other: { type: "number" } } }, lookup: { ranks: { index: byRank, read: [rank] } }, - }, ({ input, lookup }) => { + }, ({ lookup }, input: { readonly kind: string; readonly value: number; readonly other: number }) => { const { value, other } = input; switch (input.kind) { case "all": ranked = ranksOf(lookup.ranks()); break; @@ -572,9 +569,8 @@ const probeSorted = System.create({ name: "indexesTest/probeSorted", lookup: { l }); }); const rerank = System.create({ - name: "indexesTest/rerank", input: { type: "object", properties: { share: { type: "number" }, seed: { type: "integer" } } }, - components: { read: [weight], write: [rank] }, before: [probeSorted], -}, (row, { input }) => { + name: "indexesTest/rerank", components: { read: [weight], write: [rank] }, before: [probeSorted], +}, (row, _context, input: { readonly share: number; readonly seed: number }) => { if ((row[weight] * 7919 + input.seed) % 1000 < input.share * 1000) row[rank] = (row[weight] * 31 + input.seed * 17) % 1000; }); diff --git a/packages/ecs/src/implementation/world/indexes.ts b/packages/ecs/src/implementation/world/indexes.ts index df866c16..f7c23bd3 100644 --- a/packages/ecs/src/implementation/world/indexes.ts +++ b/packages/ecs/src/implementation/world/indexes.ts @@ -5,11 +5,12 @@ import type { Failure } from '../../interface/report/failure.js'; import type { SystemLookup } from '../../interface/system/access.js'; import type { Archetype, Storage } from './archetype.js'; import { readCell } from './cells.js'; -import { chunkShift } from './kernel.js'; +import { blockShift, chunkShift } from './kernel.js'; +import { forEachNewerRow } from './ranges.js'; import { journalEnd } from './journal.js'; import { encodeStored, hashWords, keyPartsOf, missing, release as releaseInterned, retain, type KeyPart } from './index-keys.js'; import { - bucketFor, coverBuckets, coverEntities, createHashTable, entryWords, insertEntry, lengthOf, loadHashTable, mergeEntries, place, placeAll, recordHash, + bucketFor, coverBuckets, coverEntities, createHashTable, entryWords, insertEntry, lengthOf, loadHashTable, forgetUnsorted, place, placeAll, recordHash, recordKey, rehash, relocate, slotsHold, releaseHashTable, removeEntry, startOf, tidyHashTable, unwatchHashComponent, watchHashComponent, type HashTable, } from './index-hash.js'; import { allocateArray, freeArray } from './arrays.js'; @@ -51,6 +52,11 @@ export type WorldIndex = { * it was built; 0 before. */ changed: number; + /** + * For a sorted index, the tick of the last refresh at which an entry moved to another archetype or row with its key + * and place unchanged, or it was built; 0 before. + */ + moved: number; /** By watched component: how many lookups watch it, and the tick its ticks were read up to. */ readonly watchers: Map; readonly scanned: Map; @@ -64,8 +70,6 @@ export type WorldIndex = { pending: Int32Array; pendingArchetypes: Int32Array; pendingRows: Int32Array; - /** What an incremental refresh changes, by bucket: removals' entity indexes and insertions' entries. */ - operations: Int32Array; /** A unique index's insertions of a refresh, as bucket and entity index pairs. */ inserted: Int32Array; /** @@ -75,8 +79,6 @@ export type WorldIndex = { readonly built: { entries: Int32Array; keys: Uint32Array; hashes: Uint32Array; buckets: Int32Array; order: Int32Array; spare: Int32Array; readonly histogram: Int32Array }; /** What a build across the pool lists, in the arena, kept for the next; null before the first. */ pooled: Pooled | null; - /** A merge's scratch, kept for the next: by bucket, its operations' start; operations in bucket order; removals, insertions and a segment. */ - merging: { starts: Int32Array; order: Int32Array; removed: Int32Array; added: Int32Array; segment: Int32Array }; }; /** A build across the pool's arrays in the arena (`index-build.ts`): by entry, by piece, by partition, and its plan. */ @@ -124,6 +126,7 @@ export const acquireIndex = (state: State, definition: IndexDefinition): WorldIn position: -1, refreshed: 0, changed: 0, + moved: 0, watchers: new Map(), scanned: new Map(), hash: sorted ? null : createHashTable(state.memory, words), @@ -134,11 +137,9 @@ export const acquireIndex = (state: State, definition: IndexDefinition): WorldIn pending: new Int32Array(64), pendingArchetypes: new Int32Array(64), pendingRows: new Int32Array(64), - operations: new Int32Array(64), inserted: new Int32Array(64), built: { entries: new Int32Array(0), keys: new Uint32Array(0), hashes: new Uint32Array(0), buckets: new Int32Array(0), order: new Int32Array(0), spare: new Int32Array(0), histogram: new Int32Array(sorted && words > 0 ? 65536 : 0) }, pooled: null, - merging: { starts: new Int32Array(0), order: new Int32Array(0), removed: new Int32Array(0), added: new Int32Array(0), segment: new Int32Array(0) }, }; state.indexes.set(definition, index); state.journal.readers++; @@ -352,14 +353,11 @@ const pend = (index: WorldIndex, entity: number, archetype: number, row: number, /** Calls `each` with every row of a storage whose `changed` tick is after `since`, a whole column a GPU kernel wrote included. */ const changedRows = (archetype: Archetype, storage: Storage, since: number, each: (row: number) => void): void => { - const whole = storage.columnChanged > since; - const { changed, chunkChanged } = storage; - for (let chunk = 0, count = Math.ceil(archetype.count / 2 ** chunkShift); chunk < count; chunk++) { - if (!whole && !(chunkChanged[chunk] > since)) continue; - for (let row = chunk << chunkShift, end = Math.min(archetype.count, (chunk + 1) << chunkShift); row < end; row++) { - if (whole || changed[row] > since) each(row); - } + if (storage.columnChanged > since) { + for (let row = 0; row < archetype.count; row++) each(row); + return; } + forEachNewerRow(storage.changed, storage.chunkChanged, storage.blockChanged, archetype.count, since, each); }; /** Whether a component may have been written since `since`: by a system, or by a command's value at a sync point. */ @@ -434,6 +432,7 @@ const finishRefresh = async (state: State, index: WorldIndex, end: number, tick: /** Builds an index again: in phases, across the pool when there is one, for a large hash index keyed by structs; else here. */ const rebuildAny = (state: State, index: WorldIndex, tick: number, failures: Failure[], pool: Pool | null): Promise | null => { index.changed = tick; + index.moved = tick; const phased = rebuildPhased(state, index, tick, failures, pool); if (phased !== false) return phased; rebuild(state, index, tick, failures); @@ -468,10 +467,15 @@ const refresh = (state: State, index: WorldIndex, end: number, keyed: boolean, t const storage = archetype.storageById[part]; if (storage === undefined || !storage.ticked || !writtenSince(state, part, since)) continue; const whole = storage.columnChanged > since; - const { changed, chunkChanged } = storage; + const { changed, chunkChanged, blockChanged } = storage; for (let chunk = 0, chunks = Math.ceil(archetype.count / 2 ** chunkShift); chunk < chunks; chunk++) { if (!whole && !(chunkChanged[chunk] > since)) continue; for (let row = chunk << chunkShift, end = Math.min(archetype.count, (chunk + 1) << chunkShift); row < end; row++) { + // A block no row of which changed is passed over whole. + if (!whole && (row & (2 ** blockShift - 1)) === 0 && !(blockChanged[row >> blockShift] > since)) { + row += 2 ** blockShift - 1; + continue; + } if (!whole && !(changed[row] > since)) continue; count = marking ? pend(index, rows[row] >>> 0, id, row, count) : list(index, rows[row] >>> 0, id, row, count); if (count > most) return rebuildAny(state, index, tick, failures, pool); @@ -488,38 +492,17 @@ const refresh = (state: State, index: WorldIndex, end: number, keyed: boolean, t }; /** - * A bucket at least this long takes a refresh's operations in one merge, rather than a shift each; shorter ones take - * each as it comes. - */ -const mergeMinimum = 64; - -/** - * Files each pending entity in the bucket of its key, or none: an operation applies at once on a short bucket, and on - * a long one waits to merge with the bucket's others. A unique index's buckets holding two entries once every - * operation applied are conflicts. + * Files each pending entity in the bucket of its key, or none. A long bucket of an index that isn't unique may be left + * unsorted (`index-hash.ts`), so a change to it costs O(1) amortized; a unique index's stay sorted, and its buckets + * holding two entries once every change applied are conflicts. */ const patch = (state: State, index: WorldIndex, hash: HashTable, count: number, tick: number, failures: Failure[]): void => { const { pending, key, words, pendingArchetypes, pendingRows } = index; const { entities } = state; coverEntities(hash, entities.length); - // Each waiting operation: bucket, entity index, archetype id or -1 for a removal, row. - let waiting = 0; - const wait = (bucket: number, entity: number, archetype: number, row: number): void => { - if (4 * (waiting + 1) > index.operations.length) index.operations = fit(index.operations, 4 * (waiting + 1)); - const at = 4 * waiting++; - index.operations[at] = bucket; - index.operations[at + 1] = entity; - index.operations[at + 2] = archetype; - index.operations[at + 3] = row; - }; + const unsorting = !index.unique; // A unique index's insertions, as bucket and entity index pairs, checked once all apply. let inserted = 0; - const remove = (bucket: number, entity: number): void => { - index.changed = tick; - holdKey(hash, index, bucket, false); - if (lengthOf(hash, bucket) >= mergeMinimum) wait(bucket, entity, -1, 0); - else removeEntry(hash, bucket, entity, tick); - }; for (let k = 0; k < count; k++) { const entity = pending[k]; const scanned = pendingArchetypes[k]; @@ -527,7 +510,9 @@ const patch = (state: State, index: WorldIndex, hash: HashTable, count: number, const old = hash.bucketOf[entity]; if (archetypeId < 0 || index.member[archetypeId] !== 1) { if (old < 0) continue; - remove(old, entity); + index.changed = tick; + holdKey(hash, index, old, false); + removeEntry(hash, old, entity, tick, unsorting); hash.bucketOf[entity] = -1; continue; } @@ -544,9 +529,11 @@ const patch = (state: State, index: WorldIndex, hash: HashTable, count: number, } if (!fresh) holdKey(hash, index, bucket, true); index.changed = tick; - if (old >= 0) remove(old, entity); - if (lengthOf(hash, bucket) >= mergeMinimum) wait(bucket, entity, archetypeId, row); - else insertEntry(hash, bucket, entity, archetypeId, row, tick); + if (old >= 0) { + holdKey(hash, index, old, false); + removeEntry(hash, old, entity, tick, unsorting); + } + insertEntry(hash, bucket, entity, archetypeId, row, tick, unsorting); hash.bucketOf[entity] = bucket; if (index.unique) { if (2 * (inserted + 1) > index.inserted.length) index.inserted = fit(index.inserted, 2 * (inserted + 1)); @@ -554,7 +541,6 @@ const patch = (state: State, index: WorldIndex, hash: HashTable, count: number, index.inserted[2 * inserted++ + 1] = entity; } } - if (waiting > 0) merge(index, hash, waiting, tick); for (let k = 0; k < inserted; k++) { const bucket = index.inserted[2 * k]; if (lengthOf(hash, bucket) > 1 && hash.bucketOf[index.inserted[2 * k + 1]] === bucket) conflict(state, index, hash, bucket, index.inserted[2 * k + 1], failures); @@ -597,7 +583,7 @@ const patchSorted = (state: State, index: WorldIndex, sorted: SortedTable, count const fresh = index.objects && unknownValue(index, key); if (fresh) encodeRow(index, archetype, row, key, 0, true); if (old >= 0 && !fresh && sameStoredKey(sorted, old, key, 0)) { - relocateSorted(sorted, entity, archetypeId, row); + if (relocateSorted(sorted, entity, archetypeId, row)) index.moved = tick; continue; } if (objects && !fresh) holdWords(index, key, 0, true); @@ -615,59 +601,6 @@ const patchSorted = (state: State, index: WorldIndex, sorted: SortedTable, count for (let k = 0; k < inserted; k++) sortedConflict(state, index, sorted, index.inserted[k], failures); }; -/** Applies the operations long buckets wait with, each bucket's in one merge, in entity order. */ -const merge = (index: WorldIndex, hash: HashTable, count: number, tick: number): void => { - const operations = index.operations; - const merging = index.merging; - // By bucket, counted then placed, in the order they came; then each bucket's by entity index. - if (merging.starts.length < hash.count + 1) merging.starts = new Int32Array(Math.max(hash.count + 1, 2 * merging.starts.length)); - if (merging.order.length < count) { - const length = Math.max(count, 2 * merging.order.length); - merging.order = new Int32Array(length); - merging.removed = new Int32Array(length); - merging.added = new Int32Array(length * entryWords); - } - const { starts, order, removed, added } = merging; - starts.fill(0, 0, hash.count + 1); - for (let k = 0; k < count; k++) starts[operations[4 * k] + 1]++; - for (let bucket = 0; bucket < hash.count; bucket++) starts[bucket + 1] += starts[bucket]; - for (let k = 0; k < count; k++) order[starts[operations[4 * k]]++] = k; - let scratch = merging.segment; - for (let start = 0; start < count;) { - const bucket = operations[4 * order[start]]; - let end = start + 1; - while (end < count && operations[4 * order[end]] === bucket) end++; - // Insertion sort by entity index: a bucket's operations of one refresh are few. - for (let k = start + 1; k < end; k++) { - const moved = order[k]; - const entity = operations[4 * moved + 1]; - let at = k; - while (at > start && operations[4 * order[at - 1] + 1] > entity) { - order[at] = order[at - 1]; - at--; - } - order[at] = moved; - } - let removals = 0; - let insertions = 0; - for (let k = start; k < end; k++) { - const at = 4 * order[k]; - if (operations[at + 2] < 0) removed[removals++] = operations[at + 1]; - else { - added[insertions * entryWords] = operations[at + 1]; - added[insertions * entryWords + 1] = operations[at + 2]; - added[insertions * entryWords + 2] = operations[at + 3]; - insertions++; - } - } - const length = lengthOf(hash, bucket) * entryWords; - if (scratch.length < length) scratch = new Int32Array(Math.max(length, 2 * scratch.length)); - mergeEntries(hash, bucket, removed, removals, added, insertions, scratch, tick); - start = end; - } - merging.segment = scratch; -}; - /** Builds an index from every entity it holds, in entity order. */ const rebuild = (state: State, index: WorldIndex, tick: number, failures: Failure[]): void => { refreshMembers(state, index); @@ -875,6 +808,7 @@ const startBuild = (state: State, index: WorldIndex, hash: HashTable, total: num const entityEnd = entities.length; const pieces = Math.ceil(entityEnd / pieceEntities); coverEntities(hash, entityEnd); + forgetUnsorted(hash); // Sized for as many buckets as it last held, as `loadHashTable` sizes it. rehash(hash, Math.min(total, Math.max(16, hash.count - hash.freed.length - hash.empty))); const pooled = pooledFor(state, index, total, pieces); @@ -1060,6 +994,8 @@ const sortedOrder = (index: WorldIndex, count: number, keys: Uint32Array): Int32 order.set(ordered); return order; } + // Entries already in key order, as a load in document order lists them, need no sort. + if (inKeyOrder(keys, count, words)) return order; const histogram = built.histogram; for (let w = words - 1; w >= 0; w--) { for (let shift = 0; shift < 32; shift += 16) { @@ -1087,6 +1023,21 @@ const sortedOrder = (index: WorldIndex, count: number, keys: Uint32Array): Int32 return order; }; +/** Whether each of `count` entries' keys, `words` each, is at least the one before, comparing words from the first. */ +const inKeyOrder = (keys: Uint32Array, count: number, words: number): boolean => { + for (let k = 1; k < count; k++) { + const previous = (k - 1) * words; + const next = k * words; + for (let w = 0; w < words; w++) { + const a = keys[previous + w]; + const b = keys[next + w]; + if (a < b) break; + if (a > b) return false; + } + } + return true; +}; + /** Stamps the buckets holding rows whose watched component changed since it was last read, with `tick`. */ const scanWatched = (state: State, index: WorldIndex, component: number, tick: number): void => { const since = index.scanned.get(component) ?? tick; diff --git a/packages/ecs/src/implementation/world/input.ts b/packages/ecs/src/implementation/world/input.ts index 67b65ac5..fab08996 100644 --- a/packages/ecs/src/implementation/world/input.ts +++ b/packages/ecs/src/implementation/world/input.ts @@ -3,7 +3,7 @@ import type { Leaf } from '../../interface/system/leaf.js'; /** The fields of a run's input that a leaf declares, so other members' fields aren't copied to its workers. */ export const inputFor = (leaf: Leaf): (input: unknown) => unknown => { - const { input } = leaf.declaration; + const { input } = leaf; if (input === undefined) return () => undefined; const fields = Object.keys(input.properties); return value => { diff --git a/packages/ecs/src/implementation/world/intrinsics.browser.test.ts b/packages/ecs/src/implementation/world/intrinsics.browser.test.ts index e35812ef..b60844c3 100644 --- a/packages/ecs/src/implementation/world/intrinsics.browser.test.ts +++ b/packages/ecs/src/implementation/world/intrinsics.browser.test.ts @@ -189,9 +189,7 @@ const spawn = System.create({ name: "intrinsicsTest/spawn" }, ({ commands }) => } }); -const operation = { type: "object", properties: { op: { type: "number" } } } as const; - -const vec2f = System.create({ name: "intrinsicsTest/vec2f", kernel: "gpu", input: operation, components: { read: [f2a, f2b, f2p], write: [f2o] } }, (row, { input: { op } }) => { +const vec2f = System.create({ name: "intrinsicsTest/vec2f", kernel: "gpu", components: { read: [f2a, f2b, f2p], write: [f2o] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[f2a]; const b = row[f2b]; const p = row[f2p]; @@ -236,7 +234,7 @@ const vec2f = System.create({ name: "intrinsicsTest/vec2f", kernel: "gpu", input else if (op === 38) row[f2o] = Vec2.faceforward(a, b, p); }); -const vec2fScalar = System.create({ name: "intrinsicsTest/vec2fScalar", kernel: "gpu", input: operation, components: { read: [f2a, f2b], write: [fo] } }, (row, { input: { op } }) => { +const vec2fScalar = System.create({ name: "intrinsicsTest/vec2fScalar", kernel: "gpu", components: { read: [f2a, f2b], write: [fo] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[f2a]; const b = row[f2b]; if (op === 0) row[fo] = Vec2.dot(a, b); @@ -244,7 +242,7 @@ const vec2fScalar = System.create({ name: "intrinsicsTest/vec2fScalar", kernel: else if (op === 2) row[fo] = Vec2.length(a); }); -const vec2i = System.create({ name: "intrinsicsTest/vec2i", kernel: "gpu", input: operation, components: { read: [i2a, i2b], write: [i2o] } }, (row, { input: { op } }) => { +const vec2i = System.create({ name: "intrinsicsTest/vec2i", kernel: "gpu", components: { read: [i2a, i2b], write: [i2o] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[i2a]; const b = row[i2b]; if (op === 0) row[i2o] = Vec2.I32.add(a, b); @@ -265,7 +263,7 @@ const vec2i = System.create({ name: "intrinsicsTest/vec2i", kernel: "gpu", input else if (op === 15) row[i2o] = Vec2.I32.clamp(a, [-1000, -1000], [1000, 1000]); }); -const vec2u = System.create({ name: "intrinsicsTest/vec2u", kernel: "gpu", input: operation, components: { read: [u2a, u2b], write: [u2o] } }, (row, { input: { op } }) => { +const vec2u = System.create({ name: "intrinsicsTest/vec2u", kernel: "gpu", components: { read: [u2a, u2b], write: [u2o] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[u2a]; const b = row[u2b]; if (op === 0) row[u2o] = Vec2.U32.add(a, b); @@ -284,7 +282,7 @@ const vec2u = System.create({ name: "intrinsicsTest/vec2u", kernel: "gpu", input else if (op === 13) row[u2o] = Vec2.U32.clamp(a, [1000, 1000], [100000, 100000]); }); -const vec3f = System.create({ name: "intrinsicsTest/vec3f", kernel: "gpu", input: operation, components: { read: [f3a, f3b, f3p], write: [f3o] } }, (row, { input: { op } }) => { +const vec3f = System.create({ name: "intrinsicsTest/vec3f", kernel: "gpu", components: { read: [f3a, f3b, f3p], write: [f3o] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[f3a]; const b = row[f3b]; const p = row[f3p]; @@ -331,7 +329,7 @@ const vec3f = System.create({ name: "intrinsicsTest/vec3f", kernel: "gpu", input else if (op === 40) row[f3o] = Vec3.faceforward(a, b, p); }); -const vec3fScalar = System.create({ name: "intrinsicsTest/vec3fScalar", kernel: "gpu", input: operation, components: { read: [f3a, f3b], write: [fo] } }, (row, { input: { op } }) => { +const vec3fScalar = System.create({ name: "intrinsicsTest/vec3fScalar", kernel: "gpu", components: { read: [f3a, f3b], write: [fo] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[f3a]; const b = row[f3b]; if (op === 0) row[fo] = Vec3.dot(a, b); @@ -339,7 +337,7 @@ const vec3fScalar = System.create({ name: "intrinsicsTest/vec3fScalar", kernel: else if (op === 2) row[fo] = Vec3.length(a); }); -const vec3i = System.create({ name: "intrinsicsTest/vec3i", kernel: "gpu", input: operation, components: { read: [i3a, i3b], write: [i3o] } }, (row, { input: { op } }) => { +const vec3i = System.create({ name: "intrinsicsTest/vec3i", kernel: "gpu", components: { read: [i3a, i3b], write: [i3o] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[i3a]; const b = row[i3b]; if (op === 0) row[i3o] = Vec3.I32.add(a, b); @@ -361,7 +359,7 @@ const vec3i = System.create({ name: "intrinsicsTest/vec3i", kernel: "gpu", input else if (op === 16) row[i3o] = Vec3.I32.clamp(a, [-1000, -1000, -1000], [1000, 1000, 1000]); }); -const vec3u = System.create({ name: "intrinsicsTest/vec3u", kernel: "gpu", input: operation, components: { read: [u3a, u3b], write: [u3o] } }, (row, { input: { op } }) => { +const vec3u = System.create({ name: "intrinsicsTest/vec3u", kernel: "gpu", components: { read: [u3a, u3b], write: [u3o] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[u3a]; const b = row[u3b]; if (op === 0) row[u3o] = Vec3.U32.add(a, b); @@ -381,7 +379,7 @@ const vec3u = System.create({ name: "intrinsicsTest/vec3u", kernel: "gpu", input else if (op === 14) row[u3o] = Vec3.U32.clamp(a, [1000, 1000, 1000], [100000, 100000, 100000]); }); -const vec4f = System.create({ name: "intrinsicsTest/vec4f", kernel: "gpu", input: operation, components: { read: [f4a, f4b, f4p], write: [f4o] } }, (row, { input: { op } }) => { +const vec4f = System.create({ name: "intrinsicsTest/vec4f", kernel: "gpu", components: { read: [f4a, f4b, f4p], write: [f4o] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[f4a]; const b = row[f4b]; const p = row[f4p]; @@ -427,7 +425,7 @@ const vec4f = System.create({ name: "intrinsicsTest/vec4f", kernel: "gpu", input else if (op === 39) row[f4o] = Vec4.faceforward(a, b, p); }); -const vec4fScalar = System.create({ name: "intrinsicsTest/vec4fScalar", kernel: "gpu", input: operation, components: { read: [f4a, f4b], write: [fo] } }, (row, { input: { op } }) => { +const vec4fScalar = System.create({ name: "intrinsicsTest/vec4fScalar", kernel: "gpu", components: { read: [f4a, f4b], write: [fo] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[f4a]; const b = row[f4b]; if (op === 0) row[fo] = Vec4.dot(a, b); @@ -435,7 +433,7 @@ const vec4fScalar = System.create({ name: "intrinsicsTest/vec4fScalar", kernel: else if (op === 2) row[fo] = Vec4.length(a); }); -const vec4i = System.create({ name: "intrinsicsTest/vec4i", kernel: "gpu", input: operation, components: { read: [i4a, i4b], write: [i4o] } }, (row, { input: { op } }) => { +const vec4i = System.create({ name: "intrinsicsTest/vec4i", kernel: "gpu", components: { read: [i4a, i4b], write: [i4o] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[i4a]; const b = row[i4b]; if (op === 0) row[i4o] = Vec4.I32.add(a, b); @@ -457,7 +455,7 @@ const vec4i = System.create({ name: "intrinsicsTest/vec4i", kernel: "gpu", input else if (op === 16) row[i4o] = Vec4.I32.clamp(a, [-1000, -1000, -1000, -1000], [1000, 1000, 1000, 1000]); }); -const vec4u = System.create({ name: "intrinsicsTest/vec4u", kernel: "gpu", input: operation, components: { read: [u4a, u4b], write: [u4o] } }, (row, { input: { op } }) => { +const vec4u = System.create({ name: "intrinsicsTest/vec4u", kernel: "gpu", components: { read: [u4a, u4b], write: [u4o] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[u4a]; const b = row[u4b]; if (op === 0) row[u4o] = Vec4.U32.add(a, b); @@ -477,7 +475,7 @@ const vec4u = System.create({ name: "intrinsicsTest/vec4u", kernel: "gpu", input else if (op === 14) row[u4o] = Vec4.U32.clamp(a, [1000, 1000, 1000, 1000], [100000, 100000, 100000, 100000]); }); -const mat4x4 = System.create({ name: "intrinsicsTest/mat4x4", kernel: "gpu", input: operation, components: { read: [ma, mb], write: [mo] } }, (row, { input: { op } }) => { +const mat4x4 = System.create({ name: "intrinsicsTest/mat4x4", kernel: "gpu", components: { read: [ma, mb], write: [mo] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[ma]; const b = row[mb]; if (op === 0) row[mo] = Mat4x4.multiply(a, b); @@ -487,18 +485,18 @@ const mat4x4 = System.create({ name: "intrinsicsTest/mat4x4", kernel: "gpu", inp else if (op === 4) row[mo] = Mat4x4.scale(a, 0.5); }); -const mat4x4Vector = System.create({ name: "intrinsicsTest/mat4x4Vector", kernel: "gpu", input: operation, components: { read: [ma, f4a], write: [f4o] } }, (row, { input: { op } }) => { +const mat4x4Vector = System.create({ name: "intrinsicsTest/mat4x4Vector", kernel: "gpu", components: { read: [ma, f4a], write: [f4o] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[ma]; const b = row[f4a]; if (op === 0) row[f4o] = Mat4x4.multiplyVec4(a, b); }); -const mat4x4Scalar = System.create({ name: "intrinsicsTest/mat4x4Scalar", kernel: "gpu", input: operation, components: { read: [ma], write: [fo] } }, (row, { input: { op } }) => { +const mat4x4Scalar = System.create({ name: "intrinsicsTest/mat4x4Scalar", kernel: "gpu", components: { read: [ma], write: [fo] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[ma]; if (op === 0) row[fo] = Mat4x4.determinant(a); }); -const quat = System.create({ name: "intrinsicsTest/quat", kernel: "gpu", input: operation, components: { read: [qa, qb], write: [qo] } }, (row, { input: { op } }) => { +const quat = System.create({ name: "intrinsicsTest/quat", kernel: "gpu", components: { read: [qa, qb], write: [qo] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[qa]; const b = row[qb]; if (op === 0) row[qo] = Quat.add(a, b); @@ -507,14 +505,14 @@ const quat = System.create({ name: "intrinsicsTest/quat", kernel: "gpu", input: else if (op === 3) row[qo] = Quat.negate(a); }); -const quatScalar = System.create({ name: "intrinsicsTest/quatScalar", kernel: "gpu", input: operation, components: { read: [qa, qb], write: [fo] } }, (row, { input: { op } }) => { +const quatScalar = System.create({ name: "intrinsicsTest/quatScalar", kernel: "gpu", components: { read: [qa, qb], write: [fo] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[qa]; const b = row[qb]; if (op === 0) row[fo] = Quat.dot(a, b); else if (op === 1) row[fo] = Quat.length(a); }); -const u32 = System.create({ name: "intrinsicsTest/u32", kernel: "gpu", input: operation, components: { read: [ua], write: [uo] } }, (row, { input: { op } }) => { +const u32 = System.create({ name: "intrinsicsTest/u32", kernel: "gpu", components: { read: [ua], write: [uo] } }, (row, _context, { op }: { readonly op: number }) => { const a = row[ua]; if (op === 0) row[uo] = U32.countLeadingZeros(a); else if (op === 1) row[uo] = U32.countOneBits(a); diff --git a/packages/ecs/src/implementation/world/join.test.ts b/packages/ecs/src/implementation/world/join.test.ts index 86250b91..c6ad5721 100644 --- a/packages/ecs/src/implementation/world/join.test.ts +++ b/packages/ecs/src/implementation/world/join.test.ts @@ -41,17 +41,17 @@ Component.register(words, { type: { type: "set", items: { type: "string" } } }); Resource.register(counts, { type: { type: "map", keys: { type: "string" }, values: { type: "number" } }, default: new Map() }); /** Adds its struct input to `total`, and its object input's names to `names`. */ -const add = System.create({ name: "joinTest/add", input: { type: "object", properties: { amount: { type: "number" } } }, resources: { write: [total] } }, ({ input, resources }) => { +const add = System.create({ name: "joinTest/add", resources: { write: [total] } }, ({ resources }, input: { readonly amount: number }) => { resources[total] = resources[total] * 10 + input.amount; }); -const append = System.create({ name: "joinTest/append", input: { type: "object", properties: { name: { type: "string" } } }, resources: { write: [names] } }, ({ input, resources }) => { +const append = System.create({ name: "joinTest/append", resources: { write: [names] } }, ({ resources }, input: { readonly name: string }) => { resources[names] = [...resources[names], input.name]; }); -const spawn = System.create({ name: "joinTest/spawn", input: { type: "object", properties: { x: { type: "number" } } } }, ({ input, commands }) => { +const spawn = System.create({ name: "joinTest/spawn" }, ({ commands }, input: { readonly x: number }) => { commands.spawn([point, label, marked])({ [point]: [input.x, -input.x], [label]: `p${input.x}` }); }); /** Counts each word of its input's set in `counts`, and spawns an entity holding the set. */ -const tally = System.create({ name: "joinTest/tally", input: { type: "object", properties: { words: { type: "set", items: { type: "string" } } } }, resources: { write: [counts] } }, ({ input, resources, commands }) => { +const tally = System.create({ name: "joinTest/tally", resources: { write: [counts] } }, ({ resources, commands }, input: { readonly words: ReadonlySet }) => { const next = new Map(resources[counts]); for (const word of input.words) next.set(word, (next.get(word) ?? 0) + 1); resources[counts] = next; diff --git a/packages/ecs/src/implementation/world/join.ts b/packages/ecs/src/implementation/world/join.ts index 49e39d8b..28bbd8fa 100644 --- a/packages/ecs/src/implementation/world/join.ts +++ b/packages/ecs/src/implementation/world/join.ts @@ -220,7 +220,7 @@ export const joinWorld = (port: Parameters[0], options: { readon const base = slotsWord + slot * slotWords; control[base + slotStatus] = pending; control[base + slotNumber] = number; - const schema = definition.kind === "composite" ? undefined : definition.declaration.input; + const schema = definition.kind === "composite" ? undefined : definition.input; const layout = fitsInline(schema, input) ? inputLayoutOf(definition, schema) : null; if (layout !== null && layout.kind === "linear") { layout.write(views[slot], 0, input); @@ -410,7 +410,7 @@ export const joinWorld = (port: Parameters[0], options: { readon current: () => { if (replayed === null) return true; const state = replay(replayed); - return state !== "held" && (state !== "lapped" || rebuild(replayed)); + return state !== "held" && (state !== "lapped" || rebuild(replayed) === "rebuilt"); }, }); if (message.tasks !== null && mirror !== null) { diff --git a/packages/ecs/src/implementation/world/joined-observer.test.ts b/packages/ecs/src/implementation/world/joined-observer.test.ts new file mode 100644 index 00000000..6b7856f2 --- /dev/null +++ b/packages/ecs/src/implementation/world/joined-observer.test.ts @@ -0,0 +1,58 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, it } from "vitest"; +import { Component } from "../../interface/component/component.js"; +import { definitionOf } from "../../interface/registry/index.js"; +import { isLeaf } from "../../interface/registry/schedule.js"; +import { System } from "../../interface/system/system.js"; +import { createBuffer } from "./buffer.js"; +import { createCommands } from "./commands.js"; +import { cancelled, createJoinedObserverRunner, createObserverTarget, postObserver, settleObserverKeys, settlePosts } from "./joined-observer.js"; +import { createState, idOf } from "./state.js"; +import { synchronize } from "./synchronize.js"; +import { doneWord, progressWord, snapshotted, tailChunk } from "./tail-blocks.js"; + +// The world's side of an observer system on a joined thread, with this test standing for the thread: its claims and +// done word, in the order a loaded machine may give them. + +const amount = Symbol("joinedObserverTest/amount"); +declare module "@adobe/ecs" { + export interface Components { + [amount]: { type: number }; + } +} +Component.register(amount, { type: { type: "number" } }); + +/** What the observer system read, were it run: this test stands for the thread, so it never runs. */ +const seen: number[] = []; +const watch = System.create({ name: "joinedObserverTest/watch", observer: true, components: { read: [amount] } }, row => { seen.push(row[amount]); }); + +describe("an observer system on a joined thread", () => { + it("is waited for, not cancelled, by a sync point after a writer copied the rest of a post the thread claimed partway", async () => { + const state = createState(true); + const target = { buffer: createBuffer() }; + const commands = createCommands(state, target); + for (let k = 0; k < 200; k++) commands.spawn([amount])({ [amount]: k }); + state.queued.push({ buffer: target.buffer, guarded: true }); + synchronize(state); + const leaf = definitionOf(watch); + if (leaf === undefined || !isLeaf(leaf)) throw new Error("The observer system has no leaf."); + const observer = createJoinedObserverRunner(state, leaf, createObserverTarget(1, "joinedObserverTest", () => {}, () => {})).joined; + if (observer === null) throw new Error("The runner stands for no observer system on a joined thread."); + expect(postObserver(state, [], [observer], 0, null)).toBeNull(); + const { control } = observer; + // The thread claims the head and first rows, and reads them. + expect(Atomics.add(control, progressWord, tailChunk)).toBe(0); + Atomics.store(control, doneWord, tailChunk); + // A writer of what it reads copies the rest; the thread hasn't claimed again yet. + expect(settleObserverKeys(state, [idOf(state, amount)], [])).toBeNull(); + const settled = settlePosts(state); + expect(settled).not.toBeNull(); + // The thread's next claim reads the rest from the copy, and it finishes. + const claimed = Atomics.add(control, progressWord, tailChunk); + expect(claimed >= snapshotted && claimed < cancelled).toBe(true); + Atomics.store(control, doneWord, observer.end); + Atomics.notify(control, doneWord); + await settled; + expect(state.observer.posts).toEqual([]); + }); +}); diff --git a/packages/ecs/src/implementation/world/joining.test-helper.ts b/packages/ecs/src/implementation/world/joining.test-helper.ts index 6f5c9f0f..79ea705a 100644 --- a/packages/ecs/src/implementation/world/joining.test-helper.ts +++ b/packages/ecs/src/implementation/world/joining.test-helper.ts @@ -31,13 +31,13 @@ Resource.register(total, { type: { type: "number", default: 0 } }); Resource.register(names, { type: { type: "array", items: { type: "string" }, default: [] } }); /** Appends a digit to `total`, so the order of runs shows in it. */ -export const add = System.create({ name: "joiningTest/add", input: { type: "object", properties: { amount: { type: "number" } } }, resources: { write: [total] } }, ({ input, resources }) => { +export const add = System.create({ name: "joiningTest/add", resources: { write: [total] } }, ({ resources }, input: { readonly amount: number }) => { resources[total] = resources[total] * 10 + input.amount; }); -export const append = System.create({ name: "joiningTest/append", input: { type: "object", properties: { name: { type: "string" } } }, resources: { write: [names] } }, ({ input, resources }) => { +export const append = System.create({ name: "joiningTest/append", resources: { write: [names] } }, ({ resources }, input: { readonly name: string }) => { resources[names] = [...resources[names], input.name]; }); -export const spawn = System.create({ name: "joiningTest/spawn", input: { type: "object", properties: { x: { type: "number" } } } }, ({ input, commands }) => { +export const spawn = System.create({ name: "joiningTest/spawn" }, ({ commands }, input: { readonly x: number }) => { commands.spawn([point, label, marked])({ [point]: [input.x, -input.x], [label]: `p${input.x}` }); }); export const fail = System.create({ name: "joiningTest/fail" }, () => { throw new Error("joiningTest failure"); }); diff --git a/packages/ecs/src/implementation/world/kernel-indexes.ts b/packages/ecs/src/implementation/world/kernel-indexes.ts index f34109b8..7c178d0a 100644 --- a/packages/ecs/src/implementation/world/kernel-indexes.ts +++ b/packages/ecs/src/implementation/world/kernel-indexes.ts @@ -4,7 +4,7 @@ import { labelOf, watchedBy } from '../../interface/registry/index.js'; import type { KernelDescription } from './kernel.js'; import { partWords } from './index-keys.js'; import { sortedStride } from './index-sorted.js'; -import { recordKey } from './index-hash.js'; +import { recordKey, sortHashTable } from './index-hash.js'; import type { WorldIndex } from './indexes.js'; import type { IndexReaderSetup, IndexSource, RowLeaf } from './index-readers.js'; import { idOf, type Key, type State } from './state.js'; @@ -102,11 +102,16 @@ export const kernelIndexesOf = (state: State, description: KernelDescription, lo }; }); -/** Writes a lookup through an index's WASM parameters into `out` from `at`, as of now: its arrays' addresses, counts and ticks. */ +/** + * Writes a lookup through an index's WASM parameters into `out` from `at`, as of now: its arrays' addresses, counts and + * ticks. A hash index's unsorted buckets are sorted first, since its reader reads entries by position where it can't + * sort them. + */ export const writeIndexParameters = (bound: Pick, since: number, out: number[], at: number): number => { const { hash, sorted } = bound.index; let next = at; if (hash !== null) { + if (hash.unsorted !== 0) sortHashTable(hash); out[next++] = hash.slots.byteOffset; out[next++] = (hash.slots.length >> 1) - 1; out[next++] = hash.records.byteOffset; @@ -133,7 +138,10 @@ export const writeIndexParameters = (bound: Pick { const { hash, sorted } = bound.index; source.since = since; diff --git a/packages/ecs/src/implementation/world/kernel-spawns.test.ts b/packages/ecs/src/implementation/world/kernel-spawns.test.ts index e3bb8b3f..97820ed9 100644 --- a/packages/ecs/src/implementation/world/kernel-spawns.test.ts +++ b/packages/ecs/src/implementation/world/kernel-spawns.test.ts @@ -40,7 +40,7 @@ Component.register(position, { type: { type: "array", items: { type: "number", p const spawned = Archetype.create([value, made]); -const seed = System.create({ name: "kernelSpawnsTest/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const seed = System.create({ name: "kernelSpawnsTest/seed" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) commands.spawn([index])({ [index]: i }); }); diff --git a/packages/ecs/src/implementation/world/kernel.ts b/packages/ecs/src/implementation/world/kernel.ts index 2aa23f9e..e6344376 100644 --- a/packages/ecs/src/implementation/world/kernel.ts +++ b/packages/ecs/src/implementation/world/kernel.ts @@ -20,13 +20,20 @@ export type Column = /** Float64Array of the component's changed ticks, stamped with `tick` on each write. */ | { readonly kind: "changed"; readonly component: Key } /** - * Float64Array of the component's newest changed tick per chunk of `chunkRows` rows, so change filters can skip - * unchanged chunks. Every `changed` column is followed by one, and each stamp sets both: `chunks[row >> chunkShift] = tick`. + * Float64Array of the component's newest changed tick per chunk of `2 ** chunkShift` rows, so change filters can + * skip unchanged chunks. Every `changed` column is followed by one, then by its `blocks` column. */ - | { readonly kind: "chunks"; readonly component: Key }; + | { readonly kind: "chunks"; readonly component: Key } + /** + * Float64Array of the component's newest changed tick per block of `2 ** blockShift` rows, so change filters skip + * the unchanged blocks of a chunk that changed. A stamp sets all three: the row's tick, its chunk's and its block's. + */ + | { readonly kind: "blocks"; readonly component: Key }; /** Rows per chunk of change ticks is `2 ** chunkShift`. */ export const chunkShift = 10; +/** Rows per block of change ticks is `2 ** blockShift`: a chunk holds `2 ** (chunkShift - blockShift)` blocks. */ +export const blockShift = 6; /** A leaf of a resource a kernel reduces into, folded with `operator`. */ export type Reduced = { readonly resource: Key; readonly operator: Reduction; readonly path: Path }; @@ -146,8 +153,11 @@ export namespace Program { | { readonly kind: "set"; readonly local: number; readonly value: Expression } /** `columns[column][row] = value`, with typed-array conversion. */ | { readonly kind: "store"; readonly column: number; readonly value: Expression } - /** `columns[column][row] = tick` and `columns[chunks][row >> chunkShift] = tick`, for a changed column and its chunk column. */ - | { readonly kind: "stamp"; readonly column: number; readonly chunks: number } + /** + * `columns[column][row] = tick`, `columns[chunks][row >> chunkShift] = tick` and + * `columns[blocks][row >> blockShift] = tick`, for a changed column and its chunk and block columns. + */ + | { readonly kind: "stamp"; readonly column: number; readonly chunks: number; readonly blocks: number } | { readonly kind: "if"; readonly test: Expression; readonly consequent: readonly Statement[]; readonly alternate: readonly Statement[] } /** WASM `block`: a branch to it exits it. */ | { readonly kind: "block"; readonly body: readonly Statement[] } diff --git a/packages/ecs/src/implementation/world/kernels.ts b/packages/ecs/src/implementation/world/kernels.ts index b415c4b5..91a336a8 100644 --- a/packages/ecs/src/implementation/world/kernels.ts +++ b/packages/ecs/src/implementation/world/kernels.ts @@ -105,7 +105,7 @@ export const bindKernel = (state: State, leaf: Leaf, description: KernelDescript return column; } const id = idOf(state, column.component); - if (column.kind === "changed" || column.kind === "chunks") { + if (column.kind === "changed" || column.kind === "chunks" || column.kind === "blocks") { dense(); return { kind: column.kind, id, leaf: 0 }; } @@ -187,6 +187,7 @@ export const bindKernel = (state: State, leaf: Leaf, description: KernelDescript if (storage === undefined) return fail(`column of "${labelOf(state.keys[source.id])}" isn't in its components.`); if (source.kind === "changed") return storage.changed; if (source.kind === "chunks") return storage.chunkChanged; + if (source.kind === "blocks") return storage.blockChanged; const layout = state.layouts[source.id]; return layout.kind === "linear" ? storage.views[layout.leaves[source.leaf].view] : fail("a field column isn't linear."); }); @@ -219,7 +220,7 @@ export const bindKernel = (state: State, leaf: Leaf, description: KernelDescript upCache.set(archetype, { version: archetype.version, columns }); return columns; }; - const argsLayout = leaf.declaration.input === undefined ? null : layoutOf(leaf.declaration.input); + const argsLayout = leaf.input === undefined ? null : layoutOf(leaf.input); const scalarType = (scalar: WasmScalar): Type | null => scalar.source === "args" ? argsLayout === null ? null : leafType(argsLayout, scalar.path) : scalar.index < description.resources.length ? leafType(state.layouts[idOf(state, description.resources[scalar.index])], scalar.path) : null; @@ -238,12 +239,12 @@ export const bindKernel = (state: State, leaf: Leaf, description: KernelDescript }, })); /** - * Changed and chunk columns of components no registered system filters on are muted: nothing reads those ticks, - * so the WASM drops their stamps and the JS kernel stamps one dummy element. Chunk columns of components held on - * the GPU stay, since uploads read them. + * Changed, chunk and block columns of components no registered system filters on are muted: nothing reads those + * ticks, so the WASM drops their stamps and the JS kernel stamps one dummy element. Chunk columns of components + * held on the GPU stay, since uploads read them; block columns, which only filters read, go with the row ticks. */ const stamped = (source: Extract): boolean => state.observed.has(source.id) || (source.kind === "chunks" && state.resident.has(source.id)); - const mutedNow = (): boolean[] => sources.map(source => (source.kind === "changed" || source.kind === "chunks") && !stamped(source)); + const mutedNow = (): boolean[] => sources.map(source => (source.kind === "changed" || source.kind === "chunks" || source.kind === "blocks") && !stamped(source)); const upWasmColumns: WasmColumn[] = upSources.map(source => { const layout = state.layouts[source.id]; return { view: source.leaf.view, stride: layout.kind === "linear" ? layout.stride : 0, offset: source.leaf.offset, boolean: source.leaf.scalar === "boolean", component: source.id }; diff --git a/packages/ecs/src/implementation/world/layout-directory.browser.test.ts b/packages/ecs/src/implementation/world/layout-directory.browser.test.ts index 9dc793c0..a2777a7a 100644 --- a/packages/ecs/src/implementation/world/layout-directory.browser.test.ts +++ b/packages/ecs/src/implementation/world/layout-directory.browser.test.ts @@ -53,7 +53,7 @@ Component.register(tag6, { type: { const: true } }); const lists = Array.from({ length: 100 }, (_, i): (typeof value | (typeof tags)[number])[] => [value, ...tags.filter((_, bit) => (i & (1 << bit)) !== 0)]); /** Spawns `count` entities into each of the 100 archetypes. */ -const fill = System.create({ name: "layoutDirectoryTest/fill", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const fill = System.create({ name: "layoutDirectoryTest/fill" }, ({ commands }, input: { readonly count: number }) => { for (const list of lists) for (let i = 0; i < input.count; i++) commands.spawn(list)({ [value]: 0 }); }); @@ -74,7 +74,7 @@ const make = System.create({ name: "layoutDirectoryTest/make" }, ({ commands }) for (let i = 0; i < 64; i++) commands.spawn([target, seen])({ [target]: Entity.none, [seen]: 0 }); commands.spawn([value])({ [value]: -1 }); }); -const spawnMany = System.create({ name: "layoutDirectoryTest/spawnMany", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const spawnMany = System.create({ name: "layoutDirectoryTest/spawnMany" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) spawned.push(commands.spawn([value])({ [value]: i })); }); /** Aims each looker at one of the last spawned entities. */ diff --git a/packages/ecs/src/implementation/world/layout-directory.fixture.ts b/packages/ecs/src/implementation/world/layout-directory.fixture.ts index a1fbd2fe..18cdc1e6 100644 --- a/packages/ecs/src/implementation/world/layout-directory.fixture.ts +++ b/packages/ecs/src/implementation/world/layout-directory.fixture.ts @@ -55,7 +55,7 @@ Component.register(tag6, { type: { const: true } }); const lists = Array.from({ length: 100 }, (_, i): (typeof value | (typeof tags)[number])[] => [value, ...tags.filter((_, bit) => (i & (1 << bit)) !== 0)]); /** Spawns `count` entities into each of the 100 archetypes. */ -const fill = System.create({ name: "layoutDirectoryTest/fill", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const fill = System.create({ name: "layoutDirectoryTest/fill" }, ({ commands }, input: { readonly count: number }) => { for (const list of lists) for (let i = 0; i < input.count; i++) commands.spawn(list)({ [value]: 0 }); }); @@ -76,7 +76,7 @@ const make = System.create({ name: "layoutDirectoryTest/make" }, ({ commands }) for (let i = 0; i < 64; i++) commands.spawn([target, seen])({ [target]: Entity.none, [seen]: 0 }); commands.spawn([value])({ [value]: -1 }); }); -const spawnMany = System.create({ name: "layoutDirectoryTest/spawnMany", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const spawnMany = System.create({ name: "layoutDirectoryTest/spawnMany" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) spawned.push(commands.spawn([value])({ [value]: i })); }); /** Aims each looker at one of the last spawned entities. */ diff --git a/packages/ecs/src/implementation/world/layout-directory.ts b/packages/ecs/src/implementation/world/layout-directory.ts index 47a53f2c..c4beb2ad 100644 --- a/packages/ecs/src/implementation/world/layout-directory.ts +++ b/packages/ecs/src/implementation/world/layout-directory.ts @@ -12,8 +12,9 @@ import type { Memory } from './memory.js'; * * Header words: the memory's length in pages (`memory.ts`), the table's offset and length, the structure epoch, then * the entity-table record: its `cursor` offset, which never moves, `free` offset and capacity, `archetype`, `row` and - * `generation` offsets and their capacity, and its version; then the resource entity's index plus one, 0 for none. The - * table's length is `cursor[2]`. + * `generation` offsets and their capacity, and its version; then the resource entity's index plus one, 0 for none; the + * structure epoch's waiters; and how many flushes replaced an existing archetype's arrays. The table's length is + * `cursor[2]`. * * The structure epoch is odd from the start of each sync point that applies anything until its directory writes at the * end, and even otherwise. A thread reading entity locations outside a task reads while it's even and checks it again @@ -35,12 +36,18 @@ export const entitiesVersionField = 7; export const resourceEntityWord = 12; /** Readers waiting on the structure epoch, so the world notifies only when one is. */ export const structureWaitersWord = 13; +/** + * Flushes that wrote an existing archetype's record again, its arrays replaced, so a joined thread checks this word + * once a run rather than each record's version for every row it reads. + */ +export const replacedWord = 14; /** * An archetype record: its capacity, `entities` offset, storage count, version, flags and row count, then per storage - * its component number, flags, column, `added`, `changed`, `chunkAdded` and `chunkChanged` offsets, and chunk count. - * Offsets a storage doesn't have are 0. A record is allocated once, since an archetype's storages are fixed, and never - * moves. Its count is written at flushes while a joined thread may read it, for archetypes whose count changed. + * its component number, flags, column, `added`, `changed`, `chunkAdded` and `chunkChanged` offsets, chunk count, and + * `blockAdded` and `blockChanged` offsets, of `chunks << (chunkShift - blockShift)` blocks. Offsets a storage doesn't + * have are 0. A record is allocated once, since an archetype's storages are fixed, and never moves. Its count is + * written at flushes while a joined thread may read it, for archetypes whose count changed. */ export const recordCapacity = 0; export const recordEntities = 1; @@ -53,7 +60,7 @@ export const recordWords = 6; export const resourceFlag = 1; /** Record flags: it holds reports, which only queries that require a component see. */ export const metaFlag = 2; -export const storageWords = 8; +export const storageWords = 10; export const storageComponent = 0; export const storageFlags = 1; export const storageColumn = 2; @@ -62,6 +69,8 @@ export const storageChanged = 4; export const storageChunkAdded = 5; export const storageChunkChanged = 6; export const storageChunks = 7; +export const storageBlockAdded = 8; +export const storageBlockChanged = 9; /** * Storage flags: it keeps ticks; it has a column in linear memory, a string's of handles included; its objects are a joined thread's, whose number is * the flags from `threadShift` on, and its column holds each row's tag (`move-log.ts`). @@ -148,6 +157,8 @@ const writeArchetype = (directory: LayoutDirectory, archetype: Archetype): void record[base + storageChunkAdded] = storage.ticked ? storage.chunkAdded.byteOffset : 0; record[base + storageChunkChanged] = storage.ticked ? storage.chunkChanged.byteOffset : 0; record[base + storageChunks] = storage.chunkAdded.length; + record[base + storageBlockAdded] = storage.ticked ? storage.blockAdded.byteOffset : 0; + record[base + storageBlockChanged] = storage.ticked ? storage.blockChanged.byteOffset : 0; } Atomics.store(record, recordVersion, archetype.version); // Entered once complete, so a reader that finds it finds a whole record. @@ -170,13 +181,16 @@ export const flushDirectory = (directory: LayoutDirectory | null, bumped: Archet directory.entities = entities.version; } if (bumped.length === 0) return; + let replaced = false; for (let k = 0; k < bumped.length; k++) { const archetype = bumped[k]; archetype.listed = false; + if (directory.records[archetype.id] !== undefined) replaced = true; writeArchetype(directory, archetype); } bumped.length = 0; Atomics.store(directory.header, tableLengthWord, archetypes.length); + if (replaced) Atomics.add(directory.header, replacedWord, 1); }; /** diff --git a/packages/ecs/src/implementation/world/lifetime.test-helper.ts b/packages/ecs/src/implementation/world/lifetime.test-helper.ts index 76d12135..eae89cf8 100644 --- a/packages/ecs/src/implementation/world/lifetime.test-helper.ts +++ b/packages/ecs/src/implementation/world/lifetime.test-helper.ts @@ -73,7 +73,7 @@ export const freshKernel = (system: System): System => { const description = kernelOf(system); if (leaf === undefined || leaf.kind === "composite" || description === undefined) throw new Error("Not a kernel system."); // What `System.create` makes of the same declaration and body, without the id a top-level binding takes. - const copy: Leaf & System = { kind: leaf.kind, name: leaf.name, declaration: leaf.declaration, body: leaf.body }; + const copy: Leaf & System = { kind: leaf.kind, name: leaf.name, declaration: leaf.declaration, input: leaf.input, body: leaf.body }; claimSystem(copy, undefined); return attachKernel(copy, description); }; diff --git a/packages/ecs/src/implementation/world/loop.test.ts b/packages/ecs/src/implementation/world/loop.test.ts index e078cc02..11532205 100644 --- a/packages/ecs/src/implementation/world/loop.test.ts +++ b/packages/ecs/src/implementation/world/loop.test.ts @@ -20,7 +20,7 @@ Component.register(count, { type: { type: "number" } }); const increment = requireDefinition(System.create({ name: "loopTest/increment", components: { write: [count] } }, row => { row[count] += 1; })); const reader = requireDefinition(System.create({ name: "loopTest/reader", components: { read: [count] } }, row => { void row[count]; })); const ordered = requireDefinition(System.create({ name: "loopTest/ordered", components: { read: [count] }, after: [System.create({ name: "loopTest/phase" }, () => {})] }, row => { void row[count]; })); -const takesInput = requireDefinition(System.create({ name: "loopTest/input", input: { type: "object", properties: { by: { type: "number" } } } }, () => {})); +const takesInput = requireDefinition(System.create({ name: "loopTest/input" }, (_context, input: { readonly by: number }) => {})); /** A host whose frames and timers run only when the test says, at the times it gives. */ const createHost = (settings: { cadence: Cadence; always: boolean; wake: number }) => { diff --git a/packages/ecs/src/implementation/world/loop.ts b/packages/ecs/src/implementation/world/loop.ts index 7e1ea679..3ab89871 100644 --- a/packages/ecs/src/implementation/world/loop.ts +++ b/packages/ecs/src/implementation/world/loop.ts @@ -105,7 +105,7 @@ const checkNoInput = (system: Definition, seen: Set = new Set()): vo if (seen.has(system)) return; seen.add(system); if (isLeaf(system)) { - if (system.declaration.input !== undefined) throw new Error(`System "${labelOfItem(system)}" takes input, which ticks don't pass; run it instead.`); + if (system.input !== undefined) throw new Error(`System "${labelOfItem(system)}" takes input, which ticks don't pass; run it instead.`); return; } for (const member of system.declaration.systems) checkNoInput(requireDefinition(member), seen); diff --git a/packages/ecs/src/implementation/world/mirror.ts b/packages/ecs/src/implementation/world/mirror.ts index 3f7f6c26..266055f2 100644 --- a/packages/ecs/src/implementation/world/mirror.ts +++ b/packages/ecs/src/implementation/world/mirror.ts @@ -2,7 +2,7 @@ import { indexOf } from './entities.js'; import { cursorField, entitiesWord, recordCount, recordEntities, recordStorages, recordWords, - remoteFlag, storageAdded, storageChanged, storageColumn, storageComponent, storageFlags, storageWords, structureWaitersWord, + remoteFlag, resourceEntityWord, storageAdded, storageChanged, storageColumn, storageComponent, storageFlags, storageWords, structureWaitersWord, structureWord, tableLengthWord, tableOffsetWord, threadShift, tickedFlag, } from './layout-directory.js'; import { @@ -39,6 +39,12 @@ type Staged = { readonly slot: number; readonly entity: number; readonly compone /** Whether replaying is done, waits for a component's key to be named, or found the log lapped or replaced. */ export type Replayed = "current" | "held" | "lapped"; +/** + * Whether rebuilding is done; found a sync point applying before or during it, so it can rebuild once the structure + * epoch is even; or waits for a component's key to be named or a finalizer to be set up. + */ +export type Rebuilt = "rebuilt" | "moved" | "held"; + /** A joined thread's mirror of the world's rows holding its objects. */ export type Mirror = { readonly memory: WebAssembly.Memory; @@ -74,6 +80,8 @@ export type Mirror = { nextSlot: number; /** Times it rebuilt, for tests. */ rebuilds: number; + /** The archetype last found holding the resource entity, or -1. */ + resourceArchetype: number; /** * How many values of components declared `transfer` are staged and not yet settled, so the thread replays to * release those whose commands dropped once it's woken. @@ -97,7 +105,7 @@ export const createMirror = (memory: WebAssembly.Memory, block: number, thread: memory, thread, block, buffer: memory.buffer, words: new Uint32Array(0), signed: new Int32Array(0), header: new Int32Array(0), ring: new Uint32Array(0), ringEntities: new Float64Array(0), version: -1, mask: 0, position: 0, through: 0, tick: 0, keys: new Map(), starts: new Map(), archetypes: [], scratch: new Uint32Array(1024), scratchEntities: new Float64Array(0), - staged: [], head: 0, nextSlot: 0, rebuilds: 0, stagedTransfers: 0, finalize: null, + staged: [], head: 0, nextSlot: 0, rebuilds: 0, resourceArchetype: -1, stagedTransfers: 0, finalize: null, }; refresh(mirror); return mirror; @@ -478,14 +486,14 @@ const previousValue = (old: readonly (Mirrored | null | undefined)[], entity: nu /** * Rebuilds every column from the layout directory, as of the last published sync, under the structure epoch: rows * from each archetype's `entities` and count, values by the tag each row's storage holds, a staged slot's taken, any - * other the thread's own kept by entity, and ticks from the world's. Returns false while a sync point applies, or a - * component it holds has no key named yet, having changed nothing. + * other the thread's own kept by entity, and ticks from the world's. Short of `"rebuilt"`, its columns are as they + * were, though the finalizers of stops through the sync it read may have run first. */ -export const rebuild = (mirror: Mirror): boolean => { +export const rebuild = (mirror: Mirror): Rebuilt => { refresh(mirror); const { words, signed, header, buffer } = mirror; const epoch = Atomics.load(signed, structureWord); - if ((epoch & 1) === 1) return false; + if ((epoch & 1) === 1) return "moved"; const version = Atomics.load(header, logVersionWord); const sync = Atomics.load(header, logSyncWord) >>> 0; const published = Atomics.load(header, logPublishedWord) >>> 0; @@ -500,7 +508,7 @@ export const rebuild = (mirror: Mirror): boolean => { indexed = true; } return previousValue(old, entity, component, start); - })) return false; + })) return "held"; const archetypes: (Mirrored | null)[] = []; /** Staged slots the rebuild takes, settled only once it's complete. */ const taken: number[] = []; @@ -512,7 +520,7 @@ export const rebuild = (mirror: Mirror): boolean => { continue; } const columns = columnsOf(mirror, record); - if (columns === undefined) return false; + if (columns === undefined) return "held"; if (columns === null) { archetypes.push(null); continue; @@ -551,7 +559,7 @@ export const rebuild = (mirror: Mirror): boolean => { archetypes.push(archetype); } // An exchange, so every read above completes before it. - if (Atomics.add(signed, structureWord, 0) !== epoch) return false; + if (Atomics.add(signed, structureWord, 0) !== epoch) return "moved"; for (const slot of taken) { const staged = stagedOf(mirror, slot); if (staged !== undefined) staged.live = false; @@ -564,7 +572,7 @@ export const rebuild = (mirror: Mirror): boolean => { viewRing(mirror); settle(mirror); mirror.rebuilds++; - return true; + return "rebuilt"; }; /** A storage's first word in a record, by its component. */ @@ -576,6 +584,32 @@ const storageOf = (words: Uint32Array, record: number, component: number): numbe return -1; }; +/** + * The value this thread holds for its object resource `component`, as replayed so far, which may be behind the entity + * table; its start while no row holds it. The resource entity is alone in its archetype, so at row 0, and never + * despawned, so its index, which the directory's header holds, names no other entity. + */ +export const resourceValueOf = (mirror: Mirror, component: number, start: unknown): unknown => { + const word = Atomics.load(mirror.words, resourceEntityWord); + if (word === 0) return start; + const index = word - 1; + const { archetypes } = mirror; + let archetype = mirror.resourceArchetype < 0 ? undefined : archetypes[mirror.resourceArchetype]; + if (archetype === null || archetype === undefined || archetype.count === 0 || indexOf(archetype.entities[0]) !== index) { + archetype = undefined; + mirror.resourceArchetype = -1; + for (let id = 0; id < archetypes.length; id++) { + const candidate = archetypes[id]; + if (candidate === null || candidate === undefined || candidate.count === 0 || indexOf(candidate.entities[0]) !== index) continue; + archetype = candidate; + mirror.resourceArchetype = id; + break; + } + } + const column = archetype?.byComponent[component]; + return column === undefined ? start : column.values[0]; +}; + /** A staged slot's value while no sync applied it, else `settled`. */ export const stagedValue = (mirror: Mirror, slot: number, settled: unknown): unknown => { const staged = stagedOf(mirror, slot); diff --git a/packages/ecs/src/implementation/world/move-log.test.ts b/packages/ecs/src/implementation/world/move-log.test.ts index c3131eb7..54c1372d 100644 --- a/packages/ecs/src/implementation/world/move-log.test.ts +++ b/packages/ecs/src/implementation/world/move-log.test.ts @@ -7,10 +7,16 @@ import { createCommands } from "./commands.js"; import { asEntity, indexOf, locate as locateHere } from "./entities.js"; import { recordCount, structureWord, tableOffsetWord } from "./layout-directory.js"; import { applying, cellOf, createMirror, nameComponent, rebuild, replay, stage, waitStructure, type Mirror } from "./mirror.js"; -import { createMoveLog, ownTag, slotValue, type MoveLog } from "./move-log.js"; +import { createMoveLog, logSyncWord, ownTag, slotValue, type MoveLog } from "./move-log.js"; import { claimThreads } from "./remote.js"; import { createState, defaultOf, idOf, startTicking, type State } from "./state.js"; import { applySync, prepareSync, synchronize } from "./synchronize.js"; +import { createThreadDirectory } from "./thread-directory.js"; +import { createThreadSystems } from "./thread-systems.js"; +import { + completionCapacityWord, completionOffsetWord, payloadCapacityWord, payloadCount, payloadInput, payloadOffsetWord, payloadSequence, payloadSync, ringBaseWord, + ringCapacityWord, taskBlockWords, taskDoneWord, taskPostedWord, +} from "./thread-tasks.js"; // The move log on one thread: a world whose objects of thread "moveLogTest" a mirror fills, against an inline world given // the same commands, compared after every sync point by each entity's id. @@ -81,7 +87,7 @@ const inline = (): State => { /** Brings the mirror up to the last sync point, rebuilding when its log lapped it. */ const follow = (mirror: Mirror): void => { const replayed = replay(mirror); - if (replayed === "lapped") expect(rebuild(mirror)).toBe(true); + if (replayed === "lapped") expect(rebuild(mirror)).toBe("rebuilt"); else expect(replayed).toBe("current"); }; @@ -225,6 +231,49 @@ const property = (seed: number, minimum: number, steps: number): { readonly comp return { compared, rebuilds: mirror.rebuilds }; }; +/** + * A thread's systems over a mirror the log lapped, so its next task rebuilds it: `post` posts a task of no systems, + * which runs once the mirror is current, its input by message when `input`; `refreshes` counts its tries. + */ +const lapped = () => { + const { state, mirror, log } = threaded(4); + const { commands, target } = worldCommands(state); + const queue = (): void => { state.queued.push({ buffer: target.buffer, guarded: true }); }; + const entity = commands.spawn([uid, label])({ [uid]: 1 }); + queue(); + synchronize(state); + follow(mirror); + const { memory } = state; + const control = new Int32Array(memory.memory.buffer, memory.allocate(taskBlockWords * 4), taskBlockWords); + const payload = memory.allocate(256); + Atomics.store(control, payloadOffsetWord, payload); + Atomics.store(control, payloadCapacityWord, 256); + Atomics.store(control, completionOffsetWord, memory.allocate(256)); + Atomics.store(control, completionCapacityWord, 256); + Atomics.store(control, ringBaseWord, memory.allocate(1 << 16)); + Atomics.store(control, ringCapacityWord, 1 << 16); + const directory = createThreadDirectory(memory.memory, mirror); + let refreshed = 0; + const counted = { ...directory, refresh: () => { refreshed++; return directory.refresh(); } }; + const systems = createThreadSystems({ + memory: memory.memory, mirror, directory: counted, control, claims: new Set(["moveLogTest"]), + worldWaits: true, post: () => {}, keyNumber: () => 0, numbered: () => 0, + }); + // More entries than the log holds. + for (let k = 0; k < 8; k++) commands.add(entity, amount, k); + queue(); + synchronize(state); + const post = (input: boolean): void => { + const words = new Float64Array(memory.memory.buffer, payload, 32); + words[payloadSequence] = 1; + words[payloadSync] = Atomics.load(log.header, logSyncWord) >>> 0; + words[payloadCount] = 0; + words[payloadInput] = input ? 1 : 0; + Atomics.store(control, taskPostedWord, 1); + }; + return { state, mirror, entity, commands, queue, systems, post, done: () => Atomics.load(control, taskDoneWord) === 1, refreshes: () => refreshed }; +}; + describe("the move log", () => { it("gives a joined thread's columns the inline world's values and ticks after every sync point, replaying its log", () => { for (const seed of [1, 2, 3, 4]) { @@ -324,4 +373,61 @@ describe("the move log", () => { synchronize(state); expect(words[record + recordCount]).toBe(4); }); + it("runs a thread's task once its rebuild failed for a sync point applied during it, with no wake after", async () => { + const { state, mirror, entity, commands, queue, systems, post, done } = lapped(); + // A sync point applies while the thread rebuilds, after it read the structure epoch, and none follows it. + const through = mirror.finalize; + let applied = false; + mirror.finalize = (sync, old) => { + if (!applied) { + applied = true; + commands.add(entity, amount, 9); + queue(); + synchronize(state); + } + return through === null || through(sync, old); + }; + post(false); + expect(systems.process()).toBe(false); + expect(applied).toBe(true); + await new Promise(resolve => setTimeout(resolve, 5)); + expect(done()).toBe(true); + }); + it("runs a thread's task once the sync point its rebuild found applying is applied", async () => { + const { state, mirror, entity, commands, queue, systems, post, done, refreshes } = lapped(); + commands.add(entity, amount, 9); + queue(); + expect(prepareSync(state)).toBe(true); + expect(rebuild(mirror)).toBe("moved"); + post(false); + expect(systems.process()).toBe(false); + const tried = refreshes(); + await new Promise(resolve => setTimeout(resolve, 5)); + expect(done()).toBe(false); + // It waits on the structure epoch, trying nothing until the sync point is applied. + expect(refreshes()).toBe(tried); + applySync(state); + await new Promise(resolve => setTimeout(resolve, 5)); + expect(done()).toBe(true); + }); + it("keeps a thread's event loop while its task waits for its input, after a rebuild outside a task found a sync point applying", async () => { + const { state, mirror, entity, commands, queue, systems, post, done, refreshes } = lapped(); + commands.add(entity, amount, 9); + queue(); + expect(prepareSync(state)).toBe(true); + // Its finalizers replay, and the rebuild finds the sync point applying. + mirror.stagedTransfers = 1; + systems.finalize(); + applySync(state); + // A task whose input is posted after it: nothing runs it until the input arrives. + post(true); + expect(systems.process()).toBe(false); + const before = refreshes(); + for (let k = 0; k < 200; k++) await Promise.resolve(); + const tries = refreshes() - before; + systems.input(1, { given: true }); + expect(done()).toBe(true); + // None while the thread's event loop goes on, so the input's message can arrive. + expect(tries).toBe(0); + }); }); diff --git a/packages/ecs/src/implementation/world/observe-cases.test.ts b/packages/ecs/src/implementation/world/observe-cases.test.ts index 0bf0806f..2edd18f2 100644 --- a/packages/ecs/src/implementation/world/observe-cases.test.ts +++ b/packages/ecs/src/implementation/world/observe-cases.test.ts @@ -36,7 +36,7 @@ Resource.register(other, { type: { type: "number", default: 0 } }); Resource.register(total, { type: { type: "number", default: 0 }, reduce: "+" }); Component.register(weight, { type: { type: "number" } }); -const raise = System.create({ name: "observeCasesTest/raise", input: { type: "object", properties: { by: { type: "number" } } }, resources: { write: [level] } }, ({ input, resources }) => { +const raise = System.create({ name: "observeCasesTest/raise", resources: { write: [level] } }, ({ resources }, input: { readonly by: number }) => { resources[level] += input.by; resources[level] += input.by; }); diff --git a/packages/ecs/src/implementation/world/observe-factories.test.ts b/packages/ecs/src/implementation/world/observe-factories.test.ts index 4c8c1b70..42381980 100644 --- a/packages/ecs/src/implementation/world/observe-factories.test.ts +++ b/packages/ecs/src/implementation/world/observe-factories.test.ts @@ -32,7 +32,7 @@ Component.register(other, { type: { type: "number" } }); Resource.register(level, { type: { type: "number", default: 0 } }); let spawned: Entity[] = []; -const seed = System.create({ name: "observeFactoriesTest/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const seed = System.create({ name: "observeFactoriesTest/seed" }, ({ commands }, input: { readonly count: number }) => { spawned = []; for (let i = 0; i < input.count; i++) spawned.push(commands.spawn([amount])({ [amount]: i })); }); diff --git a/packages/ecs/src/implementation/world/observe-joined-cases.test.ts b/packages/ecs/src/implementation/world/observe-joined-cases.test.ts index f790f2e4..be0c1629 100644 --- a/packages/ecs/src/implementation/world/observe-joined-cases.test.ts +++ b/packages/ecs/src/implementation/world/observe-joined-cases.test.ts @@ -52,13 +52,13 @@ Component.register(kept, { type: { type: "string" } }); Resource.register(total, { type: { type: "number", default: 0 } }); Resource.register(names, { type: { type: "array", items: { type: "string" }, default: [] } }); -const add = System.create({ name: "observeJoinedTest/add", input: { type: "object", properties: { amount: { type: "number" } } }, resources: { write: [total] } }, ({ input, resources }) => { +const add = System.create({ name: "observeJoinedTest/add", resources: { write: [total] } }, ({ resources }, input: { readonly amount: number }) => { resources[total] = resources[total] * 10 + input.amount; }); -const spawn = System.create({ name: "observeJoinedTest/spawn", input: { type: "object", properties: { x: { type: "number" } } } }, ({ input, commands }) => { +const spawn = System.create({ name: "observeJoinedTest/spawn" }, ({ commands }, input: { readonly x: number }) => { commands.spawn([point, label, marked])({ [point]: [input.x, -input.x], [label]: `p${input.x}` }); }); -const spawnPlaced = System.create({ name: "observeJoinedTest/spawnPlaced", input: { type: "object", properties: { x: { type: "number" } } } }, ({ input, commands }) => { +const spawnPlaced = System.create({ name: "observeJoinedTest/spawnPlaced" }, ({ commands }, input: { readonly x: number }) => { commands.spawn([place, caption, widget, kept])({ [place]: input.x, [widget]: null, [kept]: `k${input.x}` }); }); const nullCaptions = System.create({ name: "observeJoinedTest/nullCaptions", components: { has: [caption, place] } }, (row, { commands }) => { commands.add(row.entity, caption, null); }); diff --git a/packages/ecs/src/implementation/world/observer-contained.test-helper.ts b/packages/ecs/src/implementation/world/observer-contained.test-helper.ts index 2456e1fa..092123f4 100644 --- a/packages/ecs/src/implementation/world/observer-contained.test-helper.ts +++ b/packages/ecs/src/implementation/world/observer-contained.test-helper.ts @@ -29,7 +29,7 @@ export const tallies: number[] = []; export const bump = System.create({ name: "observerContainedTest/bump", resources: { write: [tally] } }, ({ resources }) => { resources[tally] += 1; }); export const nothing = System.create({ name: "observerContainedTest/nothing" }, () => {}); -export const setHolding = System.create({ name: "observerContainedTest/setHolding", input: { type: "object", properties: { holding: { type: "number" } } }, resources: { write: [holding] } }, ({ input, resources }) => { +export const setHolding = System.create({ name: "observerContainedTest/setHolding", resources: { write: [holding] } }, ({ resources }, input: { readonly holding: number }) => { resources[holding] = input.holding; }); diff --git a/packages/ecs/src/implementation/world/observer-contained.test.ts b/packages/ecs/src/implementation/world/observer-contained.test.ts index 082edbd4..a2d69767 100644 --- a/packages/ecs/src/implementation/world/observer-contained.test.ts +++ b/packages/ecs/src/implementation/world/observer-contained.test.ts @@ -27,7 +27,7 @@ const widen = System.create({ name: "observerContainedTest/widen", resources: { resources[wide] = [base, base + 1, base + 2, base + 3, base + 4, base + 5, base + 6, base + 7, base + 8, base + 9, base + 10, base + 11]; }); const step = System.create({ name: "observerContainedTest/step", resources: { write: [steps] } }, ({ resources }) => { resources[steps] += 1; }); -const pause = System.create({ name: "observerContainedTest/pause", input: { type: "object", properties: { paused: { type: "boolean" } } }, resources: { write: [paused] } }, ({ input, resources }) => { +const pause = System.create({ name: "observerContainedTest/pause", resources: { write: [paused] } }, ({ resources }, input: { readonly paused: boolean }) => { resources[paused] = input.paused; }); diff --git a/packages/ecs/src/implementation/world/observer-joined.test-helper.ts b/packages/ecs/src/implementation/world/observer-joined.test-helper.ts index 2929cd36..04e00468 100644 --- a/packages/ecs/src/implementation/world/observer-joined.test-helper.ts +++ b/packages/ecs/src/implementation/world/observer-joined.test-helper.ts @@ -101,10 +101,10 @@ export const advance = System.create({ name: "observerJoinedWorkerTest/advance", row[word] = wordOf(value); }); export const ticked = System.create({ name: "observerJoinedWorkerTest/ticked", systems: [count, advance] }); -export const seed = System.create({ name: "observerJoinedWorkerTest/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +export const seed = System.create({ name: "observerJoinedWorkerTest/seed" }, ({ commands }, input: { readonly count: number }) => { for (let k = 0; k < input.count; k++) commands.spawn([stamp, spot, word])({ [stamp]: 0, [spot]: [0, 0, 0], [word]: wordOf(0) }); }); -export const setStall = System.create({ name: "observerJoinedWorkerTest/setStall", input: { type: "object", properties: { pause: { type: "number" } } }, resources: { write: [stall] } }, ({ input, resources }) => { +export const setStall = System.create({ name: "observerJoinedWorkerTest/setStall", resources: { write: [stall] } }, ({ resources }, input: { readonly pause: number }) => { resources[stall] = input.pause; }); @@ -162,7 +162,7 @@ export const countHot = System.create({ name: "observerJoinedWorkerTest/countHot /** On the world's thread: the motto of a round, and the round, every motto of one size. */ export const mottoOf = (round: number): string => `motto ${round % 10}`; -export const setMotto = System.create({ name: "observerJoinedWorkerTest/setMotto", input: { type: "object", properties: { round: { type: "integer" } } }, resources: { write: [motto, mottoRound] } }, ({ input, resources }) => { +export const setMotto = System.create({ name: "observerJoinedWorkerTest/setMotto", resources: { write: [motto, mottoRound] } }, ({ resources }, input: { readonly round: number }) => { resources[motto] = mottoOf(input.round); resources[mottoRound] = input.round; }); diff --git a/packages/ecs/src/implementation/world/observer-joined.test.ts b/packages/ecs/src/implementation/world/observer-joined.test.ts index 14b26e49..8d3dfd98 100644 --- a/packages/ecs/src/implementation/world/observer-joined.test.ts +++ b/packages/ecs/src/implementation/world/observer-joined.test.ts @@ -45,7 +45,7 @@ const allTagged = Index.create({ name: "observerJoinedTest/allTagged", key: [], const raise = System.create({ name: "observerJoinedTest/raise", resources: { write: [level] } }, ({ resources }) => { resources[level] += 1; }); const nothing = System.create({ name: "observerJoinedTest/nothing" }, () => {}); -const spawn = System.create({ name: "observerJoinedTest/spawn", input: { type: "object", properties: { at: { type: "number" } } } }, ({ input, commands }) => { +const spawn = System.create({ name: "observerJoinedTest/spawn" }, ({ commands }, input: { readonly at: number }) => { commands.spawn([amount, cell, tagged])({ [amount]: input.at, [cell]: input.at }); }); const spawnLabelled = System.create({ name: "observerJoinedTest/spawnLabelled" }, ({ commands }) => { commands.spawn([amount, label])({ [amount]: 4, [label]: blank }); }); diff --git a/packages/ecs/src/implementation/world/perf.md b/packages/ecs/src/implementation/world/perf.md index ac28b9a1..92b3489b 100644 --- a/packages/ecs/src/implementation/world/perf.md +++ b/packages/ecs/src/implementation/world/perf.md @@ -124,6 +124,7 @@ A report for every run and tick, `Report.run` written in place and `Report.error - **About 125 ns a run:** two `performance.now()` calls for `elapsed`, 31 ns each, then locating the report, writing its struct and stamping it. - **`Report.run` is written field by field** through its layout's leaves, not the general struct writer: `run idle` 583 → 500 ns, and `run observer` 375 → 291. - **What a tier ran is worked out once per list of runners:** counting each runner every run took `run 50 idle` to 4.08 µs a run. +- **A world's first 256 runs each spawn their report,** so each has a sync point applying a spawn: 3.2–5.1 µs a run in node against 1.1–1.5 µs once reports are reused, about 1 ms over a world's life. One command buffer spawns every report, 3.8–5.7 → 3.2–5.1 µs. Benchmarks timing a run after another (`after` in `perftest/benchmarks.ts`) warm up past the 256, so they time a world's steady state: the filtered `up` benchmarks timed the spawn's sync point instead, `up one root changed` 0.042 against 0.024 ms, `up filtered after a reparent` 0.028 against 0.018, `up small tree in a big world` 0.020 against 0.011. - **A reused report keeps its errors and takes new ones in place,** so a run with a blocking tier every tick changes no archetype, and its sync point isn't one tails wait for. Strings in shared memory, alternating against `c6bb3c83`, node, five and six rounds, medians: every touched benchmark within noise (`run idle` a mean over 20,000 runs 626–644 against 622–660 ns, `run 50 idle`, `run observer`, `loop tick idle`, `sync point one spawn`, `move 1K`, `main thread 1K`, `spawn bulk`, `despawn bulk`, `despawn one`, `add tag`, `add value`, `remote run`, `joined observer resource`, `joined observer 10K`, `handoff`, `tail prompt`, `tail late snapshot 10K`, `despawn objects 10K` 97.5 against 97.7 ns). @@ -133,6 +134,12 @@ Strings in shared memory, alternating against `c6bb3c83`, node, five and six rou - **Records up to 512 bytes are carved from slabs** by power-of-two size class: `string write 10K joined` 142 → 127 ns against the arena's blocks, whose size class costs a `Math.log2` per record. - **A removed row lets go only of its archetype's string storages,** listed once per archetype: `despawn objects 10K` 104 → 97.7 ns against a check of every storage. +A joined thread's finalizers reading its object resources from its mirror, alternating against `998738a3`, node, three rounds: `despawn objects 10K` 98.2–102 against 99.6–103 ns, within noise. The mirror's archetype holding the resource entity is found once per group of stops that reads one, kept and checked before it's searched for again. + +A joined thread's finalizers reading string resources from copies in the stop list, once per group, alternating against `13eb6d6f`, node, six rounds, medians, on a machine busy elsewhere: all within noise. New `despawn objects 10K string resource`, the despawn benchmark's finalizer reading a string resource too, 95.5 ns a despawn against 99.5 reading the live record (the benchmark run on the old build); `despawn objects 10K` 98.0 against 97.2 ns, `string write 10K joined` 129 against 129 ns, `joined observer strings 10K` 478 against 500 µs a tick. Finalizers reading no string resource copy nothing and the group holds no extra bytes. + +Their linear resources copied into the stop list too, once per group, alternating against `92b823c9`, node, six rounds, medians, on a machine busy elsewhere: all within noise. New `despawn objects 10K linear resource`, the finalizer reading a number and a vec3 resource too, 99.5 ns a despawn against 99.1 reading them where they are (the benchmark run on the old build); `despawn objects 10K` 100.5 against 101.0 ns, `despawn objects 10K string resource` 98.4 against 99.3 ns, `joined observer 10K` 584 against 567 µs a tick, a path this doesn't touch, its rounds spread 500–754 µs. + Sync points waiting only for tails that read the world's memory, against `c21e8e1e`, node: `tail late snapshot 10K` 474–484 → 9.0–9.5 ns a row copied, against 8.9 for `e3c30aa1` measured beside it (its record 8.2), and in chromium 450 → 6.0 ns (its record 5.0, one or two 5 µs timer steps); within noise in both: `tail prompt`, `tail overlap`, `tail overlap waiting`. Since every run spawns its report, until 256 are, almost every sync point has something to apply, and each one settled every tail, so the run waited out the whole body of the benchmark's 4 ms tail ahead of the late one, whose head it had claimed. - **A tail reading only its own thread's objects isn't settled at a sync point,** having nothing there a sync point writes: 474 → 12.8 ns. - **The benchmark times the run alone,** reading its report after: 12.8 → 9.0 ns. Reading it through `runReported` timed a second run as well. @@ -142,6 +149,49 @@ GPU copies uploading only the chunks CPU writes changed, against `a4eef10e`, chr - **An upload writes each run of adjacent chunks stamped since its last upload** or holding rows a sync point appended or disturbed, clamped to the row count; a column with GPU writes the CPU hasn't read back writes only those rows. A whole-column stamp (`columnChanged`) uploads nothing: only GPU kernels make one, and their writes are on the GPU already. - **A WASM kernel's column addresses are kept per load as well as per archetype version:** a kernel loaded again to stamp a component that already kept ticks went on passing the muted column's address, 0, and wrote its stamps over the entity table there. +Render systems with `order`, against `9daaa3f8`, chromium, three runs each: new `gpu ordered render unchanged 50K`, 50K shapes and images, two archetypes interleaved in depth, drawn as one table in a sorted index's order with nothing changed, 0.030–0.036 ms a run, against 0.045–0.049 ms for the same 50K drawn without `order` as two tables, which writes both tables' records each run. New `gpu ordered render one key 50K`, a target system moving one item to another depth and then the ordered draw, 0.268–0.294 ms a run, against 0.042–0.050 ms unordered: listing the 50K entries in order costs 0.167–0.175 ms (3.4 ns an entry) and writing the 200 KB block 0.067–0.070 ms, timed inside the run. +- **The order block is listed and written only when the index's entries changed or moved rows,** by the index's `changed` tick and a `moved` tick its refresh stamps when it relocates an entry with its key unchanged, or when the matches grew; the tables block only when a column's address changed; the record only when one of its four words did. An unchanged run writes nothing to the queue and allocates nothing: the table, its `bind`, the tables list and the context are reused. +- **An entry is one u32:** the matching archetype's slot in the high bits, as few as the matches need and at least one, and its row in the rest, so the shader masks by the record's shift. + +A 2D editor's first tick after loading a document, against `826c45ad`, node, alternating, three rounds: new `editor load tick 50K` 31.4–33.2 → 20.0–20.4 ms, and `up load` 221–223 → 182–186 ms; within noise: `up depth 10`, `up depth 1000`, `up after a spawn`, `up filtered after churn`, `up filtered after a reparent`, `up small tree in a big world`, `index rebuild 1M main`, `order run 100K`, `order filtered 100K 1%`, `children lookup`, `sorted range 1M`, `sorted page 100K`. The benchmark spawns 50,201 nodes depth first, a document, 200 groups, two subgroups in each and 124 shapes and images in each subgroup, after a first tick over the empty world, then times the next tick: two `up` row systems with `changed` filters writing a position and an opacity, a layout looking up each group's children through a hash index on the parent, and a paint system run in a sorted index's paint order. Before, each system's part of the tick, run alone after the load: the position `up` 13.1 ms, the opacity `up` 10.4, the layout 2.45 (its index's build), the paint 5.43 (its index's build and 50K ordered rows). Of the opacity `up`'s profile, building the tree 22%, marking every row as passing the filters (`changedRows`, `mark`, clearing the marks) 25%, building the order 19%, listing the marked rows from it 10%, and the body 20%. After: 6.4, 4.4, 2.15 and 4.4 ms. +- **A filtered run after its tree was built again runs the whole order,** as an unfiltered one does, since every row passes then: the tick 32 → 25.1 ms, the position `up` 13.1 → 8.5 and the opacity `up` 10.4 → 7.1. Marking 50K rows, each in a bucket of its depth, and scanning the order for the marks cost as much as the run. A later sparse run is cold for it: `up one root changed` 28 → 33 µs a run over the benchmark's 100 rounds, the marking path no longer warmed by a first full pass, and 14 µs either way after 1,000 rounds. +- **The tree's build reads each archetype's parent column once,** rather than finding the archetype, column and layout for every entity: the opacity `up` 5.35 → 4.5 ms. +- **The build sets every level in one walk per entity,** up to a root or an entity already set and back down, rather than settling the entities as journaled changes do, which walk each one's children again: the opacity `up` 4.77 → 4.49 ms. +- **An order build's scratch is shared by every `up` of a thread:** the second system reuses the first one's arrays rather than allocating its own, 13 of them 50K entries each: the tick 20.6 → 20.0 ms, medians of five rounds. +- **A sorted index's rebuild skips the sort when its entries are in key order already,** as a load in document order lists them, checked in one pass that stops at the first pair out of order: the paint 5.6–5.8 → 5.1–5.3 ms. +- **A main-thread system run in an index's order binds its row in place when the archetype changes,** rather than looking the shared row up again, which happens at every row when archetypes interleave in the order: the paint 4.85 → 4.5 ms. + +Long hash buckets left unsorted until read by position, against `1ff6c206`, node, alternating, medians of five: new `bucket move 1K/10K/100K length`, one entity moved between the two buckets of an index on a boolean and a run reading both lengths, as toggling a todo is, 9.0 → 5.7, 63.9 → 5.5 and 418 → 5.6 µs a move (two runs' fixed cost included); new `bucket move … in order`, the run reading each bucket's first ten rows, 9.4 → 6.8, 65.2 → 12.7 and 416 → 68.9 µs. Within noise: `index refresh 1M 1%` 159 → 152 ns, `unique lookup 1M`, `children lookup`, `grid neighbors 100K main` and `workers`, `sorted range 1M`, `sorted page 100K`, `order run 100K`, `order filtered 100K 1%`, `index idle`, the keyless and `lookup 1K entities` benchmarks, `index rebuild 1M main` and `workers` (7.74 → 8.19 ms, runs spread 7.2–8.4 both); `rekey 1M` 13.6 → 14.4 ms (+6%), its 10K buckets of 100 taking about six moves each a frame, which a batched merge did in one pass. +- **A long bucket's change leaves a hole or joins a tail,** found by binary search over the part in entity order or by the tail position kept per entity, so moving an entity costs no shift of the bucket; its record's length counts entries, so `length` and `changed` read no order. +- **Sorting is lazy where it can be, eager where it must be:** a list's `at` on the world's thread sorts its bucket; `writeIndexParameters` sorts every unsorted bucket before kernels, workers and joined threads read the arrays, since they read entries by position and can't write them. +- **A sort moves 8-byte words:** shared memory copies an element at a time, a 150K-word `copyWithin` 193 µs as `Int32Array` against 25 as `Float64Array`; a shift by an odd number of words goes through a scratch, 58 µs, once it's at least 1,024 words, the two views of it it makes costing tens of nanoseconds against a microsecond's copy; a shorter one copies a word at a time and allocates nothing. `bucket move 100K in order` 222 → 68 µs; every odd shift a word at a time, 180 µs, and `bucket move 10K in order` 12.7 → 23 µs. +- **A full unsorted bucket with holes sorts in place rather than moving its segment, and one that grows gets half its length more spare slots:** `rekey 1M` 15.5 → 14.4 ms, from relocated segments, buffer growth and compactions. + +Block ticks, alternating against `998738a3`, node, medians of five rounds for the filter benchmarks and three for the rest; chromium, medians of three. New benchmarks (`perftest/filters.ts`), 100K rows of one archetype after a write of one row, ten rows 10,000 apart, or every row, then eight main-thread systems or four WASM kernels filtering on `changed`, timed a run of them: + +| Benchmark | Before | Now | +|---|---|---| +| `changed one 100K`, eight systems | 34.9 µs | 5.04 µs | +| `changed few 100K` | 421 µs | 11.5 µs | +| `changed all 100K` | 8.49 ms | 4.69 ms | +| `changed one 100K two filters`, the other component unwritten | 37.6 µs | 8.42 µs | +| `changed one 100K kernels`, four WASM kernels | 5.21 µs | 4.42 µs | +| `changed few 100K kernels` | 15.2 µs | 7.33 µs | +| `changed all 100K kernels` | 504 µs | 420 µs | +| `sparse changed`, 1M, 1% scattered | 354 µs | 317 µs | +| `sparse changed clustered` | 29.3 µs | 28.0 µs | + +In chromium: `sparse changed main` 295 → 260 µs, `sparse changed workers` 70 → 60 µs, `changed few 100K` 115 → 5 µs, `changed one 100K two filters` 40 → 5 µs (timers step by 5 µs), `changed all 100K kernels` 485 → 395 µs. Readers of key and parent ticks: `up one root changed` 27.8 → 26.4 µs, `up filtered after churn` 54.5 → 51.4 µs, `rekey 1M` 13.6 → 12.4 ms, `read run changed` 14.5 → 13.3 µs. Writes within noise in node (`create`, `create flat`, `move`, `churn`, `retag`, `emit`, `emit main`, `reduce`, `chain`, `follow`, `spawn bulk`, `spawn bulk kernel`, `add tag`, `add value`, `add replace`, `despawn bulk`, `index refresh 1M 1%`, `sync point one spawn`, `move 1K`) and chromium (`move`, `chain`, `emit` and `spawn bulk kernel`, main and workers), and so are `run idle`, `run observer`, `index idle`, `observer idle 100`, `joined observer 10K` and `observe gpu entity factory`. Memory: a ticked storage has two block arrays, one for a constant, each 128 bytes per chunk of its capacity, so at least 128 bytes: 1.6% of its row ticks once its capacity is 1024 rows or more, and as much as them at the initial capacity of 16 rows. Each storage's layout record takes two more words. + +- **Every row tick's writer raises its block's tick,** one store beside the chunk's: row writes on any thread, kernels (once per call for a block every row of the call reaches, in JS and WASM), the sync point's writes, spawns and moves, swaps into removed rows, a GPU kernel's column stamp lifted onto a joined thread's rows, and joined threads' own ticks. The arrays live in the arena beside the chunk ticks, and the layout directory gives workers and joined threads their offsets. +- **Main-thread, worker JS and joined-thread filters skip chunks, then blocks, then rows, each in a loop over one array,** and a passing range runs on by row ticks alone until a row doesn't pass; the scan was about 5 ns a row, with a branch per row on the filter count, and is now under 1 ns: `changed all 100K` 8.49 → 4.69 ms. +- **WASM kernels find runs of passing blocks in an outer loop and run their rows in an inner one,** a chunk's passing blocks read into a mask once and each run found with `ctz`, a chunk with more than half its blocks passing taken as one run; every row pays one comparison against the run's end, where it paid a test of whether it started a chunk: `changed all 100K kernels` 504 → 420 µs. +- **Index refreshes and `up` find changed key-part and filter rows by blocks too.** + +Joined threads' index lookups checking archetypes once a run, alternating against `13eb6d6f`, node, on a machine loaded by other work, medians of 9 and 11 rounds: +- **A row a lookup gave stays bound while one count stays the same,** as on the world's thread: each flush that writes an existing archetype's record again adds one to a header word, and a joined thread's directory moves its count on once a run when the word, its mirror's archetypes (a rebuild) or its views (memory grown, keys named) changed. Before, every row read loaded its archetype's view, the mirror's entry and the record's version. `joined observer 10K` 485 → 438 µs and 480 → 417 µs a tick; `joined select 10K factory` 524 → 498 µs. Within noise: `joined observer numbers 10K`, `joined observer strings 10K` (their rows bind once per archetype a run either way), `joined observer resource` 4.7 against 4.6 µs, `unique lookup 1M` 79 against 71 ns (its generated site now checks the count, not the archetype's version), `grid neighbors 100K main` 5.94 against 5.99 µs a row. +- **A rebuilt mirror's rows are bound again:** rows compare the view they were bound to, not only its version, so a row a lookup gave before the thread's log lapped no longer reads the objects of the mirror's old columns. + ## Where the gains came from Step figures marked (vitest) are from the node vitest lane: read them as relative. @@ -154,6 +204,7 @@ Step figures marked (vitest) are from the node vitest lane: read them as relativ **Change filters.** - Chunk ticks, the newest tick per 1024 rows, so filters skip unchanged chunks: clustered sparse to ~0.09 ms. +- Block ticks, the newest tick per 64 rows, so filters skip a changed chunk's unchanged blocks: eight systems finding one changed row of 100K 34.9 → 5.04 µs (above). - Filters inside WASM kernels, one call per archetype instead of one per passing range: uniform sparse 3.5 → 1.15 ms, clustered to 0.02 ms. **Lookups.** @@ -192,7 +243,7 @@ Step figures marked (vitest) are from the node vitest lane: read them as relativ **Tails.** A tail's snapshot is laid out for the rows it found passing and copied a run of consecutive rows at a time, its entities and each linear read one `set` each, its positions, archetypes and rows written in place, and tags marked only for a tail that writes objects: `tail late snapshot 10K` 29.8 → 6.7 ns a row in node, from a row at a time through a scratch list, whose tag pass alone cost 3.3 ns a row for a tail that writes nothing. The copy is what lets the world's work after its writer overlap a late tail: `tail overlap` 2.53 ms a tick against 4.70 for `tail overlap waiting` in node (below). **Indexes.** Node numbers, `pnpm perf`. -- Keeping one current costs nothing while its entities rest: a refresh checks its journal position and its key parts' last writes, which systems' writes and sync points' values mark per component, `index idle` 9 ns an index. Moving 1% of 1M entities to other keys, `index refresh 1M 1%`, 4.3 → 1.56 ms: rows listed with their archetype and row, without marks when one key part's scan is all a refresh reads, short buckets' operations applied as they come and long ones merged once, ordered by a counting sort by bucket; a refresh whose changed chunks' rows pass an eighth counts the rows' own ticks before building again, so a few writes spread over every chunk patch: `rekey 1M`, 1M particles 6% of which cross into another cell a frame, 46.9 → 14.1 ms, against 4.8 ms for the same frame without the index. +- Keeping one current costs nothing while its entities rest: a refresh checks its journal position and its key parts' last writes, which systems' writes and sync points' values mark per component, `index idle` 9 ns an index. Moving 1% of 1M entities to other keys, `index refresh 1M 1%`, 4.3 → 1.56 ms: rows listed with their archetype and row, without marks when one key part's scan is all a refresh reads, short buckets' operations applied as they come and long ones left unsorted; a refresh whose changed chunks' rows pass an eighth counts the rows' own ticks before building again, so a few writes spread over every chunk patch: `rekey 1M`, 1M particles 6% of which cross into another cell a frame, 46.9 → 14.1 ms, against 4.8 ms for the same frame without the index. - Lookups: a call site's encoding, hash and probe generated for its shape and the index's layout, buckets as records of hash, segment and key beside two-word slots of hash and bucket, and a row bound only when its archetype changed: `unique lookup 1M` 86 → 45–50 ns. A microbenchmark of the same probes over the same table reads 29 ns sequential and 85 random a lookup; an identity hash for integer keys is under "Tried and not kept". `children lookup` 32 ns a child, `grid neighbors 100K main` 10.5 µs a row of 27 probes. - Sorted ranges: a search over a compact array of each block's last key and entity, comparing words only for keys of words, and `at` walking blocks through a cursor: `sorted range 1M` 1.08 → 0.76 µs a range of ten; `sorted page 100K` 22 µs a change against 1.12 ms for `select page 100K`. - Kernels: probes in straight-line WASM over the index's arrays, `grid neighbors 100K workers` 179 ns a row across nine workers. @@ -212,7 +263,7 @@ Step figures marked (vitest) are from the node vitest lane: read them as relativ ## GPU particles -The sample (`pnpm sample`, which opens `/particles/`) draws the visible and enabled quarter of the perf particles as points, wrapping positions in the vertex shader, and moves them with the perf `move` kernel: on the GPU (`?mode=gpu`, positions never come back), on workers with the positions uploaded for drawing each frame (`?mode=cpu`), or not at all (`?mode=static`). Headless chromium on Apple silicon at 60 fps, 10 s per sample after warmup, CPU time per frame by process from `ps`, main-thread task time from the DevTools protocol. Four rounds at 1M (250K drawn), two at 4M (1M drawn): +The sample (`pnpm dev`, the GPU Particles tab, or `/particles/` alone) draws the visible and enabled quarter of the perf particles as points, wrapping positions in the vertex shader, and moves them with the perf `move` kernel: on the GPU (`?mode=gpu`, positions never come back), on workers with the positions uploaded for drawing each frame (`?mode=cpu`), or not at all (`?mode=static`). Headless chromium on Apple silicon at 60 fps, 10 s per sample after warmup, CPU time per frame by process from `ps`, main-thread task time from the DevTools protocol. Four rounds at 1M (250K drawn), two at 4M (1M drawn): | Per frame, ms | draw only | CPU move + upload | GPU move | |---|---|---|---| @@ -285,6 +336,10 @@ move, chromium, by worker count and entity count (a quarter of the entities move - **Copying drained columns into fresh, exactly sized arrays:** planning 2.60 → 2.45 ms in chromium, but the copies cost more than that. - **The main thread claiming pieces too:** no gain; workers that wake late are slow at first, so the main thread does about what it would alone. - **A loop per main-thread system** (eval or emitted by the plugin): 5–10% on top of fast-property rows; a micro-benchmark of 50 systems with one loop literal each measured 16%. The tuple each `row[component]` read allocates stays, and it costs more than the shared call site. Worth revisiting with compiled main-thread bodies, where reads become column loads. +- **A WASM kernel testing block starts in its one row loop,** a block's tick checked at each row that starts one, inside which a chunk's start checks the chunk's: `changed all 100K kernels` 504 → 576 µs, from a branch taken every 64 rows that the predictor misses, where the chunk test it replaced was taken every 1024. Checking blocks only at the eight-row SIMD checks, every eighth of them, still cost 7% in a microbenchmark of one kernel over 100K rows that all changed. The outer loop over runs of blocks takes it to 420 µs. +- **Every run of passing blocks as its own run in WASM,** however many of a chunk's blocks pass: 1% of 100K rows changed, scattered, one filter, in a microbenchmark, 31 µs against 24 µs for chunks alone, since a run's end costs more than the eight-row checks spend on a block between two runs. Taking a chunk with more than half its blocks passing as one run measured 23.9 µs, more than a quarter 23.9 µs and more than three quarters 32 µs; one changed row 0.21–0.25 µs against 0.33 for chunks alone at each. +- **Blocks of 32 rows:** against 64, eight systems finding one row of 100K 4.75 against 4.92 µs and ten rows 10.2 against 11.5 µs, but `sparse changed` 421 against 294 µs, a WASM kernel's chunk of 32 blocks with 1% of rows changed having fewer than half passing, so its many runs each pay their end; and twice the memory. Worth revisiting if one-row changes dominate real filters on main-thread systems and WASM finds runs another way. +- **Several filters scanned filter by filter,** each array's next newer tick found in a loop of its own and the earliest taken, in a microbenchmark of three filters: one change of 10K rows 0.40 against 0.39 µs, 1% of 10K 32.3 against 28.8 µs, 100K rows with nothing changed 0.18 against 0.38 µs. Mixed, so the filters are tested together at each index. Worth revisiting if systems filtering on several components over large archetypes that rarely change show in a profile. - **Removing a sync point's rows after all its entries, each archetype's from the highest,** so every row that stays moves at most once: churn 1M +13%, retag +6%. Sorting, the map of rows and a second pass over scattered rows cost more than the copies saved, and copying cells is only about 12% of retag; planning, the system's body and per-entry work make up the rest. Worth revisiting only with column-major moves per pair of archetypes, where removals are batched too. - **Unrolled f32 lane writers for staged spawns:** create 1M −5% for 30 more lines. Profiles attribute the caller's tuple allocations to the writer, which inflates it. Worth revisiting if spawns stop allocating tuples. - **Faster value-object spawns** (`create flat dynamic`): `Object.keys` and the caller's own spreads dominate, on a path the plugin already avoids for literal keys. Worth revisiting only if spawns from data objects matter to real workloads. @@ -337,11 +392,18 @@ move, chromium, by worker count and entity count (a quarter of the entities move - **Reading a patch's keys, hashes and buckets ahead in batches of 64,** so their misses overlap: `index refresh 1M 1%` 165 → 173 ns and `rekey 1M` 14.7 → 23.7 ms. A move in cache costs the same, so there was little latency to hide. - **A list's pool in three banks, its first, last and middle positions,** for hash lists too: `grid neighbors 100K main` 3.63 → 3.84 µs a row from choosing the bank in `at`. Only sorted lists keep a second pool for their last positions. - **Making a pool's rows for every position a list holds:** a 100K-row sorted list made 100K rows for a page of 20; rows are made ahead for the first and last 256 positions only. +- **The tree's build linking parents over locals,** rather than through `link`: the opacity `up` 4.44–4.65 against 4.55–4.76 ms, about 2%, within noise. +- **An index rebuild listing its entities in one loop over locals,** rather than a closure per entity: `index rebuild 1M main` 28.3–30 against 28.7–34.7 ms, and the editor's layout 2.33–2.84 against 2.35–2.56 ms, no gain. +- **Binding a row without making a slot object per component:** the editor's paint 5.02–5.06 against 4.99–5.07 ms, no gain; V8 removes the allocation. +- **Unsorted buckets sorted once their holes and tail passed a quarter of their entries,** rather than when full: `rekey 1M` 15.5 ms against 14.4, sorting buckets of 100 every few frames that nothing reads by position. A bucket at least 256 long before it's left unsorted, shorter ones shifted: 14.8 ms, no better. Merging a long tail an entry at a time rather than by a binary search and a move per tail entry: no measurable difference at `rekey 1M`, 15.4–16.7 ms against 15.4–15.8 then; worth revisiting where tails of thousands are sorted. +- **Moves of 4-byte words over shared memory,** `copyWithin` on the entries' `Int32Array`: `bucket move 100K in order` 222 µs against 68 with 8-byte words; closing holes by scanning the whole bucket rather than stopping at the last, 245 µs. +- **Presizing a hash index's table for a rebuild,** measured rather than tried: the editor's 601 parents rehash and place their buckets in 0.2 of 34 ms of profile, since a table is sized for the buckets it last held. Worth revisiting for a first build of an index with as many keys as entities. ## Next - `grid neighbors 100K main`, 3.66 µs a row against about 1.6 µs for the kernel on one worker: each neighbor costs about 10 ns in loads of the list's state, the pool and the row's slots; rows whose values carry their index and views as their own fields would save a few of them. - Index refreshes, 141 ns a moved entity: two ordered bucket edits and a probe, in cache. +- `editor load tick 50K`, 20 ms: each `up` system builds its own tree and order, about 1.3 ms each over 50K rows, though systems with the same `through` and matches build the same ones; sharing them would need each system's own journal position, touched rows and order patches. Of the rest, the `up` bodies through accessor rows take about 2 ms a system, the 50K ordered paint rows about 2 ms, and each index's rebuild about 1.5 ms, 28 ns an entity listing and encoding its key. - `loop tick add remove`, a new system each tick: about 2 µs more a tick since indexes (above), in `System.create` and `add`, as more garbage; the paths a run of an existing system takes measure as before. - Index rebuilds across the pool: `index rebuild 1M workers` 6.7 ms, nine workers 4.3× one, every phase scattering or gathering by partition; an estimated 2–3 ms if the scatters reached memory bandwidth. Rebuilds are rare, a refresh moving more than an eighth of an index. @@ -351,6 +413,6 @@ move, chromium, by worker count and entity count (a quarter of the entities move - Kernels taking `%` of an f64: a conditional on `resources[clock].tick % 2` costs about 33 ns a row in node, 10K rows 333 µs against 8.5 µs without it. - Replaying the move log: `log replay` takes 24 ns a move in node and 36.5 ns in chromium, two entries each, against about 1 ns to move the bytes; the cost is the per-entry dispatch and the scattered rows a swap reads. - Ordered selections recomputed every tick: `joined observer 10K` spends most of its 0.4–0.5 ms a tick sorting 10K entities. -- `joined observer 10K`, back to back: lookups read the rows in place, so the world's writer waits for each run the thread claimed, about 1.07 ms, in 1,451 of 3,000 ticks; the thread's run is about 60% the benchmark's own sort, and of the rest `at` through the index without a key (14%) and a load of the archetype's version per row (12%), which a check of versions once a post would save. +- `joined observer 10K`, back to back: lookups read the rows in place, so the world's writer waits for each run the thread claimed, about 1.07 ms, in 1,451 of 3,000 ticks; the thread's run is mostly the benchmark's own sort, and of the rest `at` through the index without a key. - A tail's body reading from a snapshot ran slower in one case: a loop counted down from an unbounded integer resource (an f64) took 13.6 ms against 11.2 in place over 10,000 rows of 200 steps, in node, while counted as an i32 it took 10.8 against 11.2; deopts didn't account for it. - Main-thread spawns reserving in blocks too: `spawn bulk` still takes one atomic add per spawn. Finalizers spawn during the sync point, after it materializes the run's reservations, so a world's block would have to end before each materialization. diff --git a/packages/ecs/src/implementation/world/pool-size.fixture.ts b/packages/ecs/src/implementation/world/pool-size.fixture.ts index efa93469..0cc42c53 100644 --- a/packages/ecs/src/implementation/world/pool-size.fixture.ts +++ b/packages/ecs/src/implementation/world/pool-size.fixture.ts @@ -21,7 +21,7 @@ const seed = System.create({ name: "poolSizeTest/seed" }, ({ commands }) => { for (let i = 0; i < 4096; i++) commands.spawn([value])({ [value]: i }); }); const double = System.create({ name: "poolSizeTest/double", kernel: "cpu", components: { write: [value] } }, row => { row[value] = row[value] * 2; }); -const resize = System.create({ name: "poolSizeTest/resize", input: { type: "object", properties: { size: { anyOf: [{ type: "integer" }, { type: "null" }] } } }, resources: { write: [workers] } }, ({ input, resources }) => { +const resize = System.create({ name: "poolSizeTest/resize", resources: { write: [workers] } }, ({ resources }, input: { readonly size: number | null }) => { resources[workers] = input.size; }); diff --git a/packages/ecs/src/implementation/world/port.test.ts b/packages/ecs/src/implementation/world/port.test.ts new file mode 100644 index 00000000..2127c544 --- /dev/null +++ b/packages/ecs/src/implementation/world/port.test.ts @@ -0,0 +1,45 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, it } from "vitest"; +import { lentPort, serveLent } from "./port.js"; + +describe("a lent worker's owner", () => { + it("reports a worker it ends as terminated by its owner, though the worker's own end is heard of first", async () => { + let died: ((error: Error) => void) | null = null; + // As a node worker's `exit` can be, before its `terminate` resolves. + const worker = { + onDeath: (handler: (error: Error) => void) => { died = handler; }, + terminate: async () => { died?.(new Error("A world worker exited with code 1.")); }, + }; + const { port1, port2 } = new MessageChannel(); + const reported = new Promise(resolve => { + port2.addEventListener("message", event => resolve(event.data)); + port2.start(); + }); + await serveLent(worker, port1).terminate(); + expect(await reported).toEqual({ kind: "death", message: "A world worker was terminated by its owner." }); + port1.close(); + port2.close(); + }); +}); + +describe("a lent worker", () => { + it("is reported terminated by its owner when its owner ends it, though its port's close is heard of first", async () => { + const data = new MessageChannel(); + const control = new MessageChannel(); + const lent = lentPort({ port: data.port2, owner: control.port2 }); + const reported = new Promise(resolve => lent.onDeath(error => resolve(error.message))); + const portClosed = new Promise(resolve => data.port2.addEventListener("close", () => resolve())); + // Ending the worker closes its port, and the thread it's lent to hears of that before `terminate` resolves. + const worker = { + onDeath: () => {}, + terminate: async () => { + data.port1.close(); + await portClosed; + }, + }; + await serveLent(worker, control.port1).terminate(); + expect(await reported).toBe("A world worker was terminated by its owner."); + control.port1.close(); + control.port2.close(); + }); +}); diff --git a/packages/ecs/src/implementation/world/port.ts b/packages/ecs/src/implementation/world/port.ts index ce5acf70..05c9fde1 100644 --- a/packages/ecs/src/implementation/world/port.ts +++ b/packages/ecs/src/implementation/world/port.ts @@ -35,9 +35,12 @@ export type Endpoint = { */ export type Lent

= { readonly port: P; readonly owner: P }; -/** Between a lent worker's owner and the thread it's lent to. */ -type ToOwner = { readonly kind: "terminate" }; -type FromOwner = { readonly kind: "death"; readonly message: string } | { readonly kind: "terminated" }; +/** + * Between a lent worker's owner and the thread it's lent to. `closed` tells the owner the worker's port closed; the + * owner answers it in order with what it posts, so a death it reported first arrives first. + */ +type ToOwner = { readonly kind: "terminate" } | { readonly kind: "closed" }; +type FromOwner = { readonly kind: "death"; readonly message: string } | { readonly kind: "terminated" } | { readonly kind: "closed" }; /** A death to report once: the first error wins, and a handler set after it still hears it. */ export const createDeath = (): { readonly die: (error: Error) => void; readonly onDeath: (handler: (error: Error) => void) => void } => { @@ -81,8 +84,9 @@ export const scopeChannel = (): Channel => { /** * The thread a lent worker runs on, as the thread it's lent to sees it: its death is what its owner reports, or either - * port's `close` where the platform has it; `terminate` asks the owner, and closes both ports once it has, or once the - * owner's port closed. + * port's `close` where the platform has it. The worker's port closing is reported once its owner answers that it + * reported nothing before, so an owner's report wins over the close its ending causes. `terminate` asks the owner, and + * closes both ports once it has, or once the owner's port closed. */ export const lentPort = ({ port, owner }: Lent): Port => { const channel = endpointChannel(port); @@ -91,9 +95,10 @@ export const lentPort = ({ port, owner }: Lent): Port void) | null = null; control.onMessage(message => { if (message.kind === "death") death.die(new Error(message.message)); + else if (message.kind === "closed") death.die(new Error("A world worker's port closed.")); else terminated?.(); }); - port.addEventListener("close", () => death.die(new Error("A world worker's port closed."))); + port.addEventListener("close", () => control.post({ kind: "closed" })); // An owner whose thread is gone can't answer; where the platform reports that, its worker is gone with it. owner.addEventListener("close", () => { death.die(new Error("A world worker's owner is gone.")); @@ -116,7 +121,7 @@ export const lentPort = ({ port, owner }: Lent): Port, "onDeath" | "terminate">, owner: Endpoint): { readonly terminate: () => Promise } => { const control = endpointChannel(owner); @@ -127,12 +132,13 @@ export const serveLent = (worker: Pick, "onDeath" | "term control.post({ kind: "death", message: error.message }); }); const end = async (): Promise => { - await worker.terminate(); - if (gone) return; + // Reported before it's ended, so it comes before the worker's end it causes, here and as its port's close there. + if (!gone) control.post({ kind: "death", message: "A world worker was terminated by its owner." }); gone = true; - control.post({ kind: "death", message: "A world worker was terminated by its owner." }); + await worker.terminate(); }; - control.onMessage(async () => { + control.onMessage(async message => { + if (message.kind === "closed") return control.post({ kind: "closed" }); gone = true; await worker.terminate(); control.post({ kind: "terminated" }); diff --git a/packages/ecs/src/implementation/world/ports.cases.ts b/packages/ecs/src/implementation/world/ports.cases.ts index 4a5ecc22..2c7d8e3e 100644 --- a/packages/ecs/src/implementation/world/ports.cases.ts +++ b/packages/ecs/src/implementation/world/ports.cases.ts @@ -10,8 +10,7 @@ export const portsCases = (load: () => Promise) => describe("thr it(`fail the run whose ${wasm ? "WASM" : "JS"} task's worker is killed from outside mid-task, by ${by === "exit" ? "ending it" : "its owner"}, and every later task on it`, async () => { const fixture = await load(); const { first, second, disposed } = await fixture.killedMidTask(by, wasm); - // A lent worker's death is what its owner reports, or its port's close where the platform has it, whichever comes first. - const death = by === "exit" ? /exited/ : /terminated by its owner|port closed/; + const death = by === "exit" ? /exited/ : /terminated by its owner/; expect(first.length).toBeGreaterThan(0); for (const message of first) expect(message).toMatch(death); expect(second.length).toBeGreaterThan(0); diff --git a/packages/ecs/src/implementation/world/protocol.ts b/packages/ecs/src/implementation/world/protocol.ts index f2013070..fcf38496 100644 --- a/packages/ecs/src/implementation/world/protocol.ts +++ b/packages/ecs/src/implementation/world/protocol.ts @@ -5,7 +5,7 @@ import type { WasmScalar } from './wasm.js'; import type { IndexReaderSetup } from './index-readers.js'; /** Where a kernel column comes from in any archetype it matches. */ -export type Source = { readonly kind: "entity" } | { readonly kind: "field" | "changed" | "chunks"; readonly id: number; readonly leaf: number }; +export type Source = { readonly kind: "entity" } | { readonly kind: "field" | "changed" | "chunks" | "blocks"; readonly id: number; readonly leaf: number }; /** Component ids of one lookup, and for a lookup by entity with `changed`, those it watches. */ export type LookupSetup = { readonly read: readonly number[]; readonly has: readonly number[]; readonly not: readonly number[]; readonly changed: readonly number[] | null }; diff --git a/packages/ecs/src/implementation/world/ranges.test.ts b/packages/ecs/src/implementation/world/ranges.test.ts new file mode 100644 index 00000000..03cd3402 --- /dev/null +++ b/packages/ecs/src/implementation/world/ranges.test.ts @@ -0,0 +1,66 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, it } from "vitest"; +import { blockShift, chunkShift } from "./kernel.js"; +import { forEachNewerRow, forEachPassingRange, type Filter } from "./ranges.js"; + +const rows = 5000; + +/** A filter over `rows` rows at tick 1, with `changed` rows at tick 9, their chunks and blocks raised. */ +const filterOf = (changed: readonly number[]): { readonly ticks: Float64Array; readonly chunks: Float64Array; readonly blocks: Float64Array } => { + const ticks = new Float64Array(rows).fill(1); + const chunks = new Float64Array((rows >> chunkShift) + 1).fill(1); + const blocks = new Float64Array(chunks.length << (chunkShift - blockShift)).fill(1); + for (const row of changed) { + ticks[row] = chunks[row >> chunkShift] = blocks[row >> blockShift] = 9; + } + return { ticks, chunks, blocks }; +}; + +const ranges = (filters: readonly Filter[], lastRun: number, start = 0, end = rows): (readonly [number, number])[] => { + const found: (readonly [number, number])[] = []; + forEachPassingRange(filters, lastRun, start, end, (first, last) => found.push([first, last])); + return found; +}; + +describe("forEachPassingRange", () => { + it("gives each maximal run of rows with a newer tick, across blocks and chunks", () => { + const filter = filterOf([3, 4, 5, 63, 64, 1023, 1024, 4999]); + expect(ranges([filter], 5)).toEqual([[3, 6], [63, 65], [1023, 1025], [4999, 5000]]); + expect(ranges([filter], 5, 4, 1024)).toEqual([[4, 6], [63, 65], [1023, 1024]]); + expect(ranges([filter], 9)).toEqual([]); + }); + + it("skips a chunk or a block whose tick isn't newer without reading its rows", () => { + const filter = filterOf([2000]); + filter.ticks[2100] = 9; + filter.ticks[3100] = 9; + // Row 2100's block and row 3100's chunk say nothing in them changed. + expect(ranges([filter], 5)).toEqual([[2000, 2001]]); + }); + + it("passes a row any of several filters passes", () => { + const first = filterOf([10, 3000]); + const second = filterOf([11, 4000]); + expect(ranges([first, second], 5)).toEqual([[10, 12], [3000, 3001], [4000, 4001]]); + expect(ranges([first, second], 5, 11, 3001)).toEqual([[11, 12], [3000, 3001]]); + }); + + it("passes every row with no filters or a lastRun below 0", () => { + const filter = filterOf([]); + expect(ranges([], 5, 7, 70)).toEqual([[7, 70]]); + expect(ranges([filter], -1, 7, 70)).toEqual([[7, 70]]); + }); +}); + +describe("forEachNewerRow", () => { + it("gives each row with a newer tick, skipping chunks and blocks whose tick isn't", () => { + const { ticks, chunks, blocks } = filterOf([0, 64, 1500, 4999]); + ticks[1600] = 9; + const found: number[] = []; + forEachNewerRow(ticks, chunks, blocks, rows, 5, row => found.push(row)); + expect(found).toEqual([0, 64, 1500, 4999]); + found.length = 0; + forEachNewerRow(ticks, chunks, blocks, 1501, 5, row => found.push(row)); + expect(found).toEqual([0, 64, 1500]); + }); +}); diff --git a/packages/ecs/src/implementation/world/ranges.ts b/packages/ecs/src/implementation/world/ranges.ts index b1f2dd2d..4653599e 100644 --- a/packages/ecs/src/implementation/world/ranges.ts +++ b/packages/ecs/src/implementation/world/ranges.ts @@ -1,45 +1,110 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. -import { chunkShift } from './kernel.js'; +import { blockShift, chunkShift } from './kernel.js'; -/** One `added` or `changed` filter: per-row ticks, and the newest tick of each chunk of rows. */ -export type Filter = { readonly ticks: Float64Array; readonly chunks: Float64Array }; +/** + * One `added` or `changed` filter: per-row ticks, the newest tick of each chunk of rows, and the newest tick of each + * block of rows within a chunk. + */ +export type Filter = { readonly ticks: Float64Array; readonly chunks: Float64Array; readonly blocks: Float64Array }; + +/** Whether any filter's chunk tick of `chunk` is newer than `lastRun`. */ +const chunkNewer = (filters: readonly Filter[], chunk: number, lastRun: number): boolean => { + for (let i = 0; i < filters.length; i++) if (filters[i].chunks[chunk] > lastRun) return true; + return false; +}; + +const blockNewer = (filters: readonly Filter[], block: number, lastRun: number): boolean => { + for (let i = 0; i < filters.length; i++) if (filters[i].blocks[block] > lastRun) return true; + return false; +}; + +const rowNewer = (filters: readonly Filter[], row: number, lastRun: number): boolean => { + for (let i = 0; i < filters.length; i++) if (filters[i].ticks[row] > lastRun) return true; + return false; +}; + +/** + * Calls `each` with every row below `count` whose tick in `ticks` is newer than `since`, skipping chunks, then blocks, + * whose ticks aren't. + */ +export const forEachNewerRow = (ticks: Float64Array, chunks: Float64Array, blocks: Float64Array, count: number, since: number, each: (row: number) => void): void => { + for (let chunk = 0, last = (count - 1) >> chunkShift; chunk <= last; chunk++) { + if (!(chunks[chunk] > since)) continue; + for (let block = chunk << (chunkShift - blockShift), end = Math.min((chunk + 1) << (chunkShift - blockShift), ((count - 1) >> blockShift) + 1); block < end; block++) { + if (!(blocks[block] > since)) continue; + for (let row = block << blockShift, rowEnd = Math.min(count, (block + 1) << blockShift); row < rowEnd; row++) if (ticks[row] > since) each(row); + } + } +}; + +/** `forEachPassingRange` for one filter, whose loops each read one array. */ +const forEachPassingRangeOf = (filter: Filter, lastRun: number, start: number, end: number, call: (start: number, end: number) => void): void => { + const { ticks, chunks, blocks } = filter; + const lastChunk = (end - 1) >> chunkShift; + let row = start; + while (row < end) { + let chunk = row >> chunkShift; + if (!(chunks[chunk] > lastRun)) { + do chunk++; while (chunk <= lastChunk && !(chunks[chunk] > lastRun)); + if (chunk > lastChunk) return; + row = chunk << chunkShift; + } + const chunkEnd = Math.min(end, (chunk + 1) << chunkShift); + const lastBlock = (chunkEnd - 1) >> blockShift; + let block = row >> blockShift; + if (!(blocks[block] > lastRun)) { + do block++; while (block <= lastBlock && !(blocks[block] > lastRun)); + if (block > lastBlock) { + row = chunkEnd; + continue; + } + row = block << blockShift; + } + const blockEnd = Math.min(chunkEnd, (block + 1) << blockShift); + while (row < blockEnd && !(ticks[row] > lastRun)) row++; + if (row === blockEnd) continue; + // A passing range runs on by row ticks alone, past blocks and chunks, until a row doesn't pass. + const first = row; + do row++; while (row < end && ticks[row] > lastRun); + call(first, row); + } +}; /** * Calls `call` for each maximal range of rows in `start` to `end` that passes the change filters: any tick newer - * than `lastRun`. Chunks with no newer tick are skipped without reading their rows. With no filters, or `lastRun` - * below 0, every row passes. + * than `lastRun`. Chunks, then blocks, with no newer tick are skipped without reading their rows, so finding the rows + * costs a check per chunk, one per block of a chunk that changed, and one per row of a block that changed. With no + * filters, or `lastRun` below 0, every row passes. */ export const forEachPassingRange = (filters: readonly Filter[], lastRun: number, start: number, end: number, call: (start: number, end: number) => void): void => { if (lastRun < 0 || filters.length === 0) return call(start, end); - let first = -1; - const single = filters.length === 1 ? filters[0].ticks : null; - for (let chunk = start >> chunkShift, last = (end - 1) >> chunkShift; chunk <= last; chunk++) { - const chunkStart = Math.max(start, chunk << chunkShift); - const chunkEnd = Math.min(end, (chunk + 1) << chunkShift); - let candidate = false; - for (let i = 0; i < filters.length && !candidate; i++) candidate = filters[i].chunks[chunk] > lastRun; - if (!candidate) { - if (first >= 0) { - call(first, chunkStart); - first = -1; - } - continue; + if (start >= end) return; + if (filters.length === 1) return forEachPassingRangeOf(filters[0], lastRun, start, end, call); + const lastChunk = (end - 1) >> chunkShift; + let row = start; + while (row < end) { + let chunk = row >> chunkShift; + if (!chunkNewer(filters, chunk, lastRun)) { + do chunk++; while (chunk <= lastChunk && !chunkNewer(filters, chunk, lastRun)); + if (chunk > lastChunk) return; + row = chunk << chunkShift; } - for (let row = chunkStart; row < chunkEnd; row++) { - let passes: boolean; - if (single !== null) passes = single[row] > lastRun; - else { - passes = false; - for (let i = 0; i < filters.length && !passes; i++) passes = filters[i].ticks[row] > lastRun; - } - if (passes) { - if (first < 0) first = row; - } - else if (first >= 0) { - call(first, row); - first = -1; + const chunkEnd = Math.min(end, (chunk + 1) << chunkShift); + const lastBlock = (chunkEnd - 1) >> blockShift; + let block = row >> blockShift; + if (!blockNewer(filters, block, lastRun)) { + do block++; while (block <= lastBlock && !blockNewer(filters, block, lastRun)); + if (block > lastBlock) { + row = chunkEnd; + continue; } + row = block << blockShift; } + const blockEnd = Math.min(chunkEnd, (block + 1) << blockShift); + while (row < blockEnd && !rowNewer(filters, row, lastRun)) row++; + if (row === blockEnd) continue; + const first = row; + do row++; while (row < end && rowNewer(filters, row, lastRun)); + call(first, row); } - if (first >= 0) call(first, end); }; diff --git a/packages/ecs/src/implementation/world/remote-finalizers-worker.fixture.ts b/packages/ecs/src/implementation/world/remote-finalizers-worker.fixture.ts index c5b7c057..2e42569c 100644 --- a/packages/ecs/src/implementation/world/remote-finalizers-worker.fixture.ts +++ b/packages/ecs/src/implementation/world/remote-finalizers-worker.fixture.ts @@ -2,11 +2,11 @@ import type { MessagePort as NodeMessagePort } from "node:worker_threads"; import { join } from "./create.js"; import { scopeChannel } from "./port.js"; -import { entitled, released, type Released } from "./remote-finalizers.test-helper.js"; +import { entitled, gauged, headlined, headlinedBoth, released, surveyed, type Released, type Surveyed } from "./remote-finalizers.test-helper.js"; /** * A thread joining a world through the port it's sent, claiming "remoteFinalizersTest", whose finalizer it imports: it - * tells what its finalizers saw when asked, or spins, without yielding, for a while or for good. + * tells what its finalizers saw when asked, blocks until a shared flag is set, or spins, without yielding, for good. */ type Port = MessagePort | NodeMessagePort; @@ -14,12 +14,25 @@ export type ToRemoteFinalizersWorker = | { readonly kind: "join"; readonly port: Port } | { readonly kind: "released" } | { readonly kind: "entitled" } - | { readonly kind: "pause"; readonly milliseconds: number } + | { readonly kind: "headlined" } + | { readonly kind: "gauged" } + | { readonly kind: "hold"; readonly flag: Int32Array } | { readonly kind: "spin" }; export type FromRemoteFinalizersWorker = | { readonly kind: "joined" } | { readonly kind: "released"; readonly released: readonly Released[] } | { readonly kind: "entitled"; readonly entitled: readonly { readonly serial: number; readonly title: string }[] } + | { + readonly kind: "headlined"; + readonly headlined: readonly { readonly serial: number; readonly heading: string }[]; + readonly both: readonly { readonly serial: number; readonly heading: string; readonly subheading: string }[]; + } + | { + readonly kind: "gauged"; + readonly gauged: readonly { readonly serial: number; readonly level: number; readonly origin: readonly number[] }[]; + readonly surveyed: readonly Surveyed[]; + } + | { readonly kind: "held" } | { readonly kind: "spinning" }; const channel = scopeChannel(); @@ -31,11 +44,12 @@ channel.onMessage(message => { } else if (message.kind === "released") channel.post({ kind: "released", released }); else if (message.kind === "entitled") channel.post({ kind: "entitled", entitled }); - else if (message.kind === "pause") { - channel.post({ kind: "spinning" }); - // Runs nothing meanwhile, finalizers included, then goes on. - const until = performance.now() + message.milliseconds; - while (performance.now() < until) { /* busy */ } + else if (message.kind === "gauged") channel.post({ kind: "gauged", gauged, surveyed }); + else if (message.kind === "headlined") channel.post({ kind: "headlined", headlined, both: headlinedBoth }); + else if (message.kind === "hold") { + channel.post({ kind: "held" }); + // Runs nothing, finalizers included, until the flag over shared memory is set. + Atomics.wait(message.flag, 0, 0); } else { channel.post({ kind: "spinning" }); diff --git a/packages/ecs/src/implementation/world/remote-finalizers.cases.ts b/packages/ecs/src/implementation/world/remote-finalizers.cases.ts index dbeb61ea..dd050ba8 100644 --- a/packages/ecs/src/implementation/world/remote-finalizers.cases.ts +++ b/packages/ecs/src/implementation/world/remote-finalizers.cases.ts @@ -1,6 +1,7 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. import { describe, expect, it } from "vitest"; import type * as Fixture from "./remote-finalizers.fixture.js"; +import { surveyOf } from "./remote-finalizers.test-helper.js"; /** Finalizers on a joined worker: the cases, over the fixture built for node or from source. */ export const remoteFinalizersCases = (load: () => Promise) => describe("finalizers on a joined worker", () => { @@ -22,4 +23,20 @@ export const remoteFinalizersCases = (load: () => Promise) => de const fixture = await load(); expect(await fixture.titledStops()).toEqual(Array.from({ length: 50 }, (_, k) => ({ serial: k, title: `t${k}` }))); }, 30000); + it("read one string resource, or two, as they were before their stops' sync point applied, after the world wrote them again and freed their records", async () => { + const fixture = await load(); + const heading = (k: number): string => k < 10 ? "h0" : "h6"; + expect(await fixture.headedStops()).toEqual({ + one: Array.from({ length: 20 }, (_, k) => ({ serial: k, heading: heading(k) })), + two: Array.from({ length: 20 }, (_, k) => ({ serial: k, heading: heading(k), subheading: `s${heading(k)}` })), + }); + }, 30000); + it("read linear resources of every width, and strings beside them, as they were before their stops' sync point applied, after the world wrote them again", async () => { + const fixture = await load(); + const level = (k: number): number => k < 10 ? 0 : 6; + expect(await fixture.gaugedStops()).toEqual({ + linear: Array.from({ length: 20 }, (_, k) => ({ serial: k, level: level(k), origin: [level(k), level(k) + 10] })), + mixed: Array.from({ length: 20 }, (_, k) => ({ serial: k, ...surveyOf(level(k)) })), + }); + }, 30000); }); diff --git a/packages/ecs/src/implementation/world/remote-finalizers.fixture.ts b/packages/ecs/src/implementation/world/remote-finalizers.fixture.ts index 6f22b5a2..bc9c2773 100644 --- a/packages/ecs/src/implementation/world/remote-finalizers.fixture.ts +++ b/packages/ecs/src/implementation/world/remote-finalizers.fixture.ts @@ -6,7 +6,7 @@ import { workers } from "../../interface/resources/workers.js"; import { createWorld, type CreatedWorld } from "./create.js"; import { indexOf } from "./entities.js"; import type { FromRemoteFinalizersWorker, ToRemoteFinalizersWorker } from "./remote-finalizers-worker.fixture.js"; -import { despawn, released, retitle, scenario, serial, sorted, spawn, spawnTitled, type Released } from "./remote-finalizers.test-helper.js"; +import { despawn, regauge, rehead, released, retitle, scenario, serial, sorted, spawn, spawnHeaded, spawnTitled, type Released, type Surveyed } from "./remote-finalizers.test-helper.js"; import { fillReports, runReported } from "./reports.test-helper.js"; /** @@ -66,6 +66,16 @@ const joined = async (world: CreatedWorld): Promise<{ readonly worker: Worker }> return { worker }; }; +/** Holds the worker, which runs nothing until the returned function lets it go on. */ +const hold = async (worker: Worker): Promise<() => void> => { + const flag = new Int32Array(new SharedArrayBuffer(4)); + await worker.ask({ kind: "hold", flag }); + return () => { + Atomics.store(flag, 0, 1); + Atomics.notify(flag, 0); + }; +}; + /** What the worker's finalizer saw. */ const releasedThere = async (worker: Worker): Promise => { const reply = await worker.ask({ kind: "released" }); @@ -118,7 +128,7 @@ export const diesHolding = async () => { }; /** - * Titled rows despawned while the worker runs nothing, then the world's thread rewriting the titles of others six + * Titled rows despawned while the worker is held, then the world's thread rewriting the titles of others six * times, so the records the despawned rows' titles were in are freed and reused before the worker finalizes them: * the titles its finalizer read. */ @@ -126,10 +136,15 @@ export const titledStops = async () => { const world = createWorld({ [workers]: 0 }); const { worker } = await joined(world); await world.run(spawnTitled, { from: 0, count: 50 }); - await worker.ask({ kind: "pause", milliseconds: 300 }); - await world.run(despawn, { serials: Array.from({ length: 50 }, (_, k) => k) }); - await world.run(spawnTitled, { from: 100, count: 50 }); - for (let round = 0; round < 6; round++) await world.run(retitle, { round }); + const release = await hold(worker); + try { + await world.run(despawn, { serials: Array.from({ length: 50 }, (_, k) => k) }); + await world.run(spawnTitled, { from: 100, count: 50 }); + for (let round = 0; round < 6; round++) await world.run(retitle, { round }); + } + finally { + release(); + } let seen: readonly { readonly serial: number; readonly title: string }[] = []; for (let k = 0; k < 200 && seen.length < 50; k++) { const reply = await worker.ask({ kind: "entitled" }); @@ -139,3 +154,64 @@ export const titledStops = async () => { await world.dispose(); return [...seen].sort((a, b) => a.serial - b.serial); }; + +/** + * Headed rows despawned with the heading "h0" while the worker is held, then the world's thread writing the heading + * and subheading six times, so the records of "h0" and "sh0" are freed and reused before the worker finalizes them, + * and the rest despawned once it goes on: what its finalizers of one and two string resources read. + */ +export const headedStops = async () => { + const world = createWorld({ [workers]: 0 }); + const { worker } = await joined(world); + await world.run(rehead, { heading: "h0" }); + await world.run(spawnHeaded, { from: 0, count: 20 }); + const release = await hold(worker); + try { + await world.run(despawn, { serials: Array.from({ length: 10 }, (_, k) => k) }); + for (let round = 1; round <= 6; round++) await world.run(rehead, { heading: `h${round}` }); + } + finally { + release(); + } + await world.run(despawn, { serials: Array.from({ length: 10 }, (_, k) => 10 + k) }); + let seen: readonly { readonly serial: number; readonly heading: string }[] = []; + let both: readonly { readonly serial: number; readonly heading: string; readonly subheading: string }[] = []; + for (let k = 0; k < 200 && (seen.length < 20 || both.length < 20); k++) { + const reply = await worker.ask({ kind: "headlined" }); + seen = reply.kind === "headlined" ? reply.headlined : []; + both = reply.kind === "headlined" ? reply.both : []; + } + await worker.terminate(); + await world.dispose(); + return { one: [...seen].sort((a, b) => a.serial - b.serial), two: [...both].sort((a, b) => a.serial - b.serial) }; +}; + +/** + * Headed rows despawned at level 0 while the worker is held, then the world's thread writing the level and origin six + * times, with resources of mixed widths, and the rest despawned once it goes on: what its finalizers read. + */ +export const gaugedStops = async () => { + const world = createWorld({ [workers]: 0 }); + const { worker } = await joined(world); + await world.run(regauge, { level: 0 }); + await world.run(spawnHeaded, { from: 0, count: 20 }); + const release = await hold(worker); + try { + await world.run(despawn, { serials: Array.from({ length: 10 }, (_, k) => k) }); + for (let round = 1; round <= 6; round++) await world.run(regauge, { level: round }); + } + finally { + release(); + } + await world.run(despawn, { serials: Array.from({ length: 10 }, (_, k) => 10 + k) }); + let seen: readonly { readonly serial: number; readonly level: number; readonly origin: readonly number[] }[] = []; + let mixed: readonly Surveyed[] = []; + for (let k = 0; k < 200 && (seen.length < 20 || mixed.length < 20); k++) { + const reply = await worker.ask({ kind: "gauged" }); + seen = reply.kind === "gauged" ? reply.gauged : []; + mixed = reply.kind === "gauged" ? reply.surveyed : []; + } + await worker.terminate(); + await world.dispose(); + return { linear: [...seen].sort((a, b) => a.serial - b.serial), mixed: [...mixed].sort((a, b) => a.serial - b.serial) }; +}; diff --git a/packages/ecs/src/implementation/world/remote-finalizers.test-helper.ts b/packages/ecs/src/implementation/world/remote-finalizers.test-helper.ts index 7ebe0494..6d473b38 100644 --- a/packages/ecs/src/implementation/world/remote-finalizers.test-helper.ts +++ b/packages/ecs/src/implementation/world/remote-finalizers.test-helper.ts @@ -18,6 +18,16 @@ export const widget = Symbol("remoteFinalizersTest/widget"); export const label = Symbol("remoteFinalizersTest/label"); export const tone = Symbol("remoteFinalizersTest/tone"); export const title = Symbol("remoteFinalizersTest/title"); +export const headed = Symbol("remoteFinalizersTest/headed"); +export const heading = Symbol("remoteFinalizersTest/heading"); +export const subheading = Symbol("remoteFinalizersTest/subheading"); +export const level = Symbol("remoteFinalizersTest/level"); +export const origin = Symbol("remoteFinalizersTest/origin"); +export const caption = Symbol("remoteFinalizersTest/caption"); +export const lit = Symbol("remoteFinalizersTest/lit"); +export const tint = Symbol("remoteFinalizersTest/tint"); +export const sample = Symbol("remoteFinalizersTest/sample"); +export const material = Symbol("remoteFinalizersTest/material"); declare module "@adobe/ecs" { export interface Threads { remoteFinalizersTest: true; @@ -30,9 +40,19 @@ declare module "@adobe/ecs" { [widget]: { type: Widget | null; thread: "remoteFinalizersTest" }; [label]: { type: { readonly text: string } | null; thread: "remoteFinalizersTest" }; [title]: { type: string }; + [headed]: { type: true }; } export interface Resources { [tone]: { type: { readonly text: string }; thread: "remoteFinalizersTest" }; + [heading]: { type: string }; + [subheading]: { type: string }; + [level]: { type: number }; + [origin]: { type: readonly [number, number] }; + [caption]: { type: string }; + [lit]: { type: boolean }; + [tint]: { type: readonly [number, number, number] }; + [sample]: { type: Sample }; + [material]: { type: Material }; } } /** A value that isn't data, which only its thread holds. */ @@ -47,6 +67,38 @@ Component.register(label, { type: { anyOf: [text, { type: "null" }], default: { Resource.register(tone, { type: { ...text, default: { text: "quiet" } }, thread: "remoteFinalizersTest" }); /** A string, in shared memory, which the world's thread writes and the finalizer on the thread reads. */ Component.register(title, { type: { type: "string" } }); +Component.register(headed, { type: { const: true } }); +/** A string resource, which the world's thread writes and the finalizer on the thread reads. */ +Resource.register(heading, { type: { type: "string", default: "h" } }); +Resource.register(subheading, { type: { type: "string", default: "s" } }); +/** Linear resources, a number and a struct, which the world's thread writes and the finalizer on the thread reads. */ +Resource.register(level, { type: { type: "number", default: -1 } }); +Resource.register(origin, { type: { type: "array", items: { type: "number" }, minItems: 2, maxItems: 2, default: [-1, -1] } }); +/** Resources of mixed widths: a string, a boolean (1 byte), an f32 vec3 (12), a padded CPU struct (24) and a storage struct (32). */ +type Sample = { readonly on: boolean; readonly count: number; readonly weight: number; readonly ratio: number }; +type Material = { readonly alpha: number; readonly color: readonly [number, number, number]; readonly glossy: boolean }; +const f32 = { type: "number", precision: 1 } as const; +const vec3 = { type: "array", items: f32, minItems: 3, maxItems: 3 } as const; +Resource.register(caption, { type: { type: "string", default: "c" } }); +Resource.register(lit, { type: { type: "boolean", default: false } }); +Resource.register(tint, { type: { ...vec3, default: [-1, -1, -1] } }); +Resource.register(sample, { + type: { + type: "object", + properties: { on: { type: "boolean" }, count: { type: "integer", minimum: 0, maximum: 0xFFFFFFFF }, weight: { type: "number" }, ratio: f32 }, + required: ["on", "count", "weight", "ratio"], + default: { on: false, count: 0, weight: -1, ratio: -1 }, + }, +}); +Resource.register(material, { + type: { + type: "object", + properties: { alpha: f32, color: vec3, glossy: { type: "boolean" } }, + required: ["alpha", "color", "glossy"], + layout: "storage", + default: { alpha: -1, color: [-1, -1, -1], glossy: false }, + }, +}); /** A stop as the finalizer saw it, on the thread it ran on. */ export type Released = { readonly serial: number; readonly place: readonly number[]; readonly marked: boolean; readonly widget: string | null; readonly label: string | null; readonly tone: string }; @@ -60,7 +112,7 @@ export const throwing = new Set(); /** On the finalizers' thread: what each stop held there, its structs before the sync point applied. */ export const release = Finalizer.create({ name: "remoteFinalizersTest/release", components: { read: [serial, place, marked, widget, label] }, resources: { read: [tone] } }, (row, { resources }) => { finalized.push(row.entity); - released.push({ serial: row[serial], place: [...row[place]], marked: row[marked], widget: row[widget]?.tag ?? null, label: row[label]?.text ?? null, tone: resources[tone]?.text }); + released.push({ serial: row[serial], place: [...row[place]], marked: row[marked], widget: row[widget]?.tag ?? null, label: row[label]?.text ?? null, tone: resources[tone].text }); if (throwing.has(row[serial])) throw new Error(`remoteFinalizersTest: ${row[serial]} throws`); }); @@ -72,17 +124,83 @@ export const entitle = Finalizer.create({ name: "remoteFinalizersTest/entitle", entitled.push({ serial: row[serial], title: row[title] }); }); +/** Every headed stop finalized on this thread, in order, with the heading it read. */ +export const headlined: { readonly serial: number; readonly heading: string }[] = []; + +/** On the finalizers' thread: the heading as it was before each headed stop's sync point applied. */ +export const headline = Finalizer.create({ name: "remoteFinalizersTest/headline", components: { read: [serial, widget], has: [headed] }, resources: { read: [heading] } }, (row, { resources }) => { + headlined.push({ serial: row[serial], heading: resources[heading] }); +}); + +/** Every headed stop finalized on this thread, in order, with the heading and subheading it read. */ +export const headlinedBoth: { readonly serial: number; readonly heading: string; readonly subheading: string }[] = []; + +/** On the finalizers' thread: two string resources as they were before each headed stop's sync point applied. */ +export const headlines = Finalizer.create({ name: "remoteFinalizersTest/headlines", components: { read: [serial, widget], has: [headed] }, resources: { read: [subheading, heading] } }, (row, { resources }) => { + headlinedBoth.push({ serial: row[serial], heading: resources[heading], subheading: resources[subheading] }); +}); + +/** Every headed stop finalized on this thread, in order, with the linear resources it read. */ +export const gauged: { readonly serial: number; readonly level: number; readonly origin: readonly number[] }[] = []; + +/** On the finalizers' thread: a number and a struct resource as they were before each headed stop's sync point applied. */ +export const gauge = Finalizer.create({ name: "remoteFinalizersTest/gauge", components: { read: [serial, widget], has: [headed] }, resources: { read: [origin, level] } }, (row, { resources }) => { + gauged.push({ serial: row[serial], level: resources[level], origin: [...resources[origin]] }); +}); + +/** Every headed stop finalized on this thread, in order, with the resources of mixed widths it read. */ +export type Surveyed = { readonly serial: number; readonly caption: string; readonly lit: boolean; readonly tint: readonly number[]; readonly sample: Sample; readonly material: Material }; +export const surveyed: Surveyed[] = []; + +/** On the finalizers' thread: resources of mixed widths as they were before each headed stop's sync point applied. */ +export const survey = Finalizer.create({ name: "remoteFinalizersTest/survey", components: { read: [serial, widget], has: [headed] }, resources: { read: [material, caption, sample, lit, tint] } }, (row, { resources }) => { + const { alpha, color, glossy } = resources[material]; + surveyed.push({ + serial: row[serial], caption: resources[caption], lit: resources[lit], tint: [...resources[tint]], + sample: { ...resources[sample] }, material: { alpha, color: [...color], glossy }, + }); +}); + +/** What `regauge` writes to the resources of mixed widths at a level. */ +export const surveyOf = (at: number): Omit => ({ + caption: `c${at}`, lit: at > 3, tint: [at, at + 0.5, at * 2], + sample: { on: at > 3, count: at * 1000 + 1, weight: at + 0.25, ratio: at * 0.5 + 1 }, + material: { alpha: at * 0.25 + 1, color: [at + 1, at + 2, at + 3], glossy: at > 3 }, +}); + +/** From the world's thread: the level, the origin at it, and the resources of mixed widths at it. */ +export const regauge = System.create({ name: "remoteFinalizersTest/regauge", resources: { write: [level, origin, caption, lit, tint, sample, material] } }, ({ resources }, input: { readonly level: number }) => { + resources[level] = input.level; + resources[origin] = [input.level, input.level + 10]; + const values = surveyOf(input.level); + resources[caption] = values.caption; + resources[lit] = values.lit; + resources[tint] = [values.tint[0], values.tint[1], values.tint[2]]; + resources[sample] = values.sample; + resources[material] = values.material; +}); + +/** Spawns headed rows from the world's thread, with the thread's widget at null. */ +export const spawnHeaded = System.create({ name: "remoteFinalizersTest/spawnHeaded" }, ({ commands }, input: { readonly from: number; readonly count: number }) => { + for (let k = input.from; k < input.from + input.count; k++) commands.spawn([serial, headed, widget])({ [serial]: k, [widget]: null }); +}); +/** From the world's thread: the heading and subheading. */ +export const rehead = System.create({ name: "remoteFinalizersTest/rehead", resources: { write: [heading, subheading] } }, ({ resources }, input: { readonly heading: string }) => { + resources[heading] = input.heading; + resources[subheading] = `s${input.heading}`; +}); + /** Spawns titled rows from the world's thread, with the thread's widget at null. */ -export const spawnTitled = System.create({ name: "remoteFinalizersTest/spawnTitled", input: { type: "object", properties: { from: { type: "integer" }, count: { type: "integer" } } } }, ({ input, commands }) => { +export const spawnTitled = System.create({ name: "remoteFinalizersTest/spawnTitled" }, ({ commands }, input: { readonly from: number; readonly count: number }) => { for (let k = input.from; k < input.from + input.count; k++) commands.spawn([serial, title, widget])({ [serial]: k, [title]: `t${k}`, [widget]: null }); }); /** From the world's thread: every title as another of the same size, so the records of titles let go are reused. */ -export const retitle = System.create({ name: "remoteFinalizersTest/retitle", input: { type: "object", properties: { round: { type: "integer" } } }, components: { write: [title] } }, (row, { input }) => { +export const retitle = System.create({ name: "remoteFinalizersTest/retitle", components: { write: [title] } }, (row, _context, input: { readonly round: number }) => { row[title] = `r${input.round}`; }); /** Spawns from the world's thread: structs, the thread's objects at null and their start. */ -export const spawn = System.create({ name: "remoteFinalizersTest/spawn", input: { type: "object", properties: { from: { type: "integer" }, count: { type: "integer" } } } }, ({ input, commands }) => { +export const spawn = System.create({ name: "remoteFinalizersTest/spawn" }, ({ commands }, input: { readonly from: number; readonly count: number }) => { for (let k = input.from; k < input.from + input.count; k++) commands.spawn([serial, place, marked, widget, label])({ [serial]: k, [place]: [k, -k], [widget]: null }); }); /** On the thread: widgets written in place. */ @@ -96,7 +214,7 @@ export const relabel = System.create({ name: "remoteFinalizersTest/relabel", com if (row[serial] % 2 === 1) commands.add(row.entity, label, relabelled); }); /** On the thread: its object resource. */ -export const retone = System.create({ name: "remoteFinalizersTest/retone", input: { type: "object", properties: { tone: { type: "string" } } }, resources: { write: [tone] } }, ({ input, resources }) => { +export const retone = System.create({ name: "remoteFinalizersTest/retone", resources: { write: [tone] } }, ({ resources }, input: { readonly tone: string }) => { resources[tone] = { text: input.tone }; }); /** From the world's thread: places move, so a stop's values are those before the sync point applies. */ @@ -104,11 +222,11 @@ export const shift = System.create({ name: "remoteFinalizersTest/shift", compone row[place] = [row[place][0] + 100, row[place][1]]; }); /** From the world's thread: despawns these serials. */ -export const despawn = System.create({ name: "remoteFinalizersTest/despawn", input: { type: "object", properties: { serials: { type: "array", items: { type: "integer" } } } }, components: { read: [serial] } }, (row, { input, commands }) => { +export const despawn = System.create({ name: "remoteFinalizersTest/despawn", components: { read: [serial] } }, (row, { commands }, input: { readonly serials: readonly number[] }) => { if (input.serials.includes(row[serial])) commands.despawn(row.entity); }); /** From the world's thread: removes widgets of these serials, a stop that despawns nothing. */ -export const strip = System.create({ name: "remoteFinalizersTest/strip", input: { type: "object", properties: { serials: { type: "array", items: { type: "integer" } } } }, components: { read: [serial], has: [widget] } }, (row, { input, commands }) => { +export const strip = System.create({ name: "remoteFinalizersTest/strip", components: { read: [serial], has: [widget] } }, (row, { commands }, input: { readonly serials: readonly number[] }) => { if (input.serials.includes(row[serial])) commands.remove(row.entity, widget); }); /** From the world's thread: every row moves archetype, flagged on one run and not on the next. */ diff --git a/packages/ecs/src/implementation/world/remote-finalizers.test.ts b/packages/ecs/src/implementation/world/remote-finalizers.test.ts index 95aff7c8..b605f7e9 100644 --- a/packages/ecs/src/implementation/world/remote-finalizers.test.ts +++ b/packages/ecs/src/implementation/world/remote-finalizers.test.ts @@ -4,6 +4,7 @@ import type { Entity } from "../../interface/entity/entity.js"; import { Finalizer } from "../../interface/finalizer/finalizer.js"; import { Observe } from "../../observe/index.js"; import { identify } from "../../interface/registry/ids.test-helper.js"; +import { Resource } from "../../interface/resource/resource.js"; import { workers } from "../../interface/resources/workers.js"; import { System } from "../../interface/system/system.js"; import type { World } from "../../interface/world/world.js"; @@ -12,6 +13,30 @@ import { indexOf } from "./entities.js"; import { attach, despawn, eventually, finalized, marked, place, released, retone, scenario, serial, sorted, spawn, throwing, tone, widget } from "./remote-finalizers.test-helper.js"; import { runReported } from "./reports.test-helper.js"; +/** Resources of the world's thread, each added once the thread joined, so the resource entity moves to another archetype. */ +const later = Symbol("remoteFinalizersTest/later"); +const second = Symbol("remoteFinalizersTest/second"); +const third = Symbol("remoteFinalizersTest/third"); +declare module "@adobe/ecs" { + export interface Resources { + [later]: { type: number }; + [second]: { type: number }; + [third]: { type: number }; + } +} +Resource.register(later, { type: { type: "number", default: 0 } }); +Resource.register(second, { type: { type: "number", default: 0 } }); +Resource.register(third, { type: { type: "number", default: 0 } }); +const bump = System.create({ name: "remoteFinalizersTest/bump", resources: { write: [later] } }, ({ resources }) => { + resources[later] += 1; +}); +const bumpSecond = System.create({ name: "remoteFinalizersTest/bumpSecond", resources: { write: [second] } }, ({ resources }) => { + resources[second] += 1; +}); +const bumpThird = System.create({ name: "remoteFinalizersTest/bumpThird", resources: { write: [third] } }, ({ resources }) => { + resources[third] += 1; +}); + // Finalizers on a joined thread, with the world and the thread on this one, joined through a MessageChannel. /** The first value an observer gives that `accept` takes. */ @@ -89,7 +114,7 @@ describe("finalizers on a joined thread", () => { let reusedEarly = 0; let reused = 0; // Spawns check each entity they reserve: an index whose last entity stopped is reused only once it was finalized. - const check = System.create(identify({ name: "remoteFinalizersTest/check", input: { type: "object", properties: { serial: { type: "integer" } } } }, "remoteFinalizersTest#check"), ({ input, commands }) => { + const check = System.create(identify({ name: "remoteFinalizersTest/check" }, "remoteFinalizersTest#check"), ({ commands }, input: { readonly serial: number }) => { const entity = commands.spawn([serial, place, marked, widget])({ [serial]: input.serial, [place]: [0, 0], [widget]: null }); const before = stopped.get(indexOf(entity)); if (before === undefined) return; @@ -156,6 +181,52 @@ describe("finalizers on a joined thread", () => { await world.dispose(); expect(released.map(stop => stop.serial)).toEqual([0, 1, 2]); }); + it("read their thread's object resource as the thread holds it, from the first sync point after joining and once the resource entity moved before they ran", async () => { + reset(); + const { world, remote } = await joined(); + // Stops the thread runs as it first rebuilds, before it holds any row. + await world.run(spawn, { from: 0, count: 3 }); + await world.run(despawn, { serials: [0] }); + expect(await eventually(() => released.length === 1)).toBe(true); + await world.run(retone, { tone: "soft" }); + // Not awaited: the resource entity moves to another archetype before the thread replays the stop's sync. + await Promise.all([world.run(despawn, { serials: [1] }), world.run(bump)]); + expect(await eventually(() => released.length === 2)).toBe(true); + expect(released.map(stop => [stop.serial, stop.tone])).toEqual([[0, "quiet"], [1, "soft"]]); + await remote.dispose(); + await world.dispose(); + }); + it("read their thread's object resource where the resource entity moved since it was last found, before a later write applies", async () => { + reset(); + const { world, remote } = await joined(); + await world.run(spawn, { from: 0, count: 3 }); + await world.run(despawn, { serials: [0] }); + expect(await eventually(() => released.length === 1)).toBe(true); + // Replayed, not rebuilt: the resource entity is found in the mirror. + await world.run(despawn, { serials: [1] }); + expect(await eventually(() => released.length === 2)).toBe(true); + await world.run(bump); + await world.run(retone, { tone: "soft" }); + // The write after the stop's sync is a task the thread runs once it replayed that sync. + await Promise.all([world.run(despawn, { serials: [2] }), world.run(retone, { tone: "loud" })]); + expect(await eventually(() => released.length === 3)).toBe(true); + expect(released.map(stop => [stop.serial, stop.tone])).toEqual([[0, "quiet"], [1, "quiet"], [2, "soft"]]); + await remote.dispose(); + await world.dispose(); + }); + it("read their thread's object resource once the resource entity moved twice before the thread replayed", async () => { + reset(); + const { world, remote } = await joined(); + await world.run(spawn, { from: 0, count: 2 }); + await world.run(despawn, { serials: [0] }); + expect(await eventually(() => released.length === 1)).toBe(true); + await world.run(retone, { tone: "soft" }); + await Promise.all([world.run(despawn, { serials: [1] }), world.run(bumpSecond), world.run(bumpThird)]); + expect(await eventually(() => released.length === 2)).toBe(true); + expect(released.map(stop => [stop.serial, stop.tone])).toEqual([[0, "quiet"], [1, "soft"]]); + await remote.dispose(); + await world.dispose(); + }); // Last: the finalizers it creates stay in the program. it("created after the claim run there; one the thread can't name doesn't activate, reported once", async () => { reset(); diff --git a/packages/ecs/src/implementation/world/remote-systems.ts b/packages/ecs/src/implementation/world/remote-systems.ts index eddcd11c..98a51a3f 100644 --- a/packages/ecs/src/implementation/world/remote-systems.ts +++ b/packages/ecs/src/implementation/world/remote-systems.ts @@ -6,8 +6,7 @@ import type { Warning } from '../../interface/report/failure.js'; import { declarationOf, idOfItem, idOfKey, indexOfLookup, labelOf, labelOfItem, threadOf, watchedBy, watchesEntries } from '../../interface/registry/index.js'; import { unsharedKeyOf } from './index-keys.js'; import { layoutOf } from '../../interface/registry/layout.js'; -import type { Archetype } from './archetype.js'; -import { chunkShift } from './kernel.js'; +import { raiseRange, type Archetype } from './archetype.js'; import type { LocalIds, Segment } from './command-ring.js'; import type { ToJoined } from './joining.js'; import { lineWords, type WorkerError } from './protocol.js'; @@ -575,7 +574,7 @@ export const createRemoteRunner = (state: State, leaf: Leaf, channel: TaskChanne watched: entityWatchesOf(declaration.lookup).map(key => idOf(state, key)), archetypes: state.archetypes, writes, - declaresInput: declaration.input !== undefined, + declaresInput: leaf.input !== undefined, lastRun: since, result: { flags: 0, segment: undefined, error: undefined }, tail: null, @@ -623,6 +622,6 @@ export const liftColumns = (archetype: Archetype, ids: readonly number[], since: if (storage === undefined || !storage.ticked || storage.columnChanged <= since) continue; const stamp = storage.columnChanged; for (let row = 0; row < archetype.count; row++) if (storage.changed[row] < stamp) storage.changed[row] = stamp; - for (let chunk = 0; chunk <= (archetype.count - 1) >> chunkShift; chunk++) if (storage.chunkChanged[chunk] < stamp) storage.chunkChanged[chunk] = stamp; + raiseRange(storage.chunkChanged, storage.blockChanged, 0, archetype.count, stamp); } }; diff --git a/packages/ecs/src/implementation/world/remote.test.ts b/packages/ecs/src/implementation/world/remote.test.ts index 71553f4a..e7aae3be 100644 --- a/packages/ecs/src/implementation/world/remote.test.ts +++ b/packages/ecs/src/implementation/world/remote.test.ts @@ -44,7 +44,7 @@ Component.register(kept, { type: { type: "string" } }); Component.register(handle, { thread: "remoteTest" }); Component.register(guarded, { thread: "remoteTestFinalized" }); -const spawn = System.create({ name: "remoteTest/spawn", input: { type: "object", properties: { x: { type: "number" } } } }, ({ input, commands }) => { +const spawn = System.create({ name: "remoteTest/spawn" }, ({ commands }, input: { readonly x: number }) => { commands.spawn([point, caption, widget, kept])({ [point]: input.x, [kept]: `k${input.x}`, [widget]: null }); }); const nullCaptions = System.create({ name: "remoteTest/nullCaptions", components: { has: [caption, point] } }, (row, { commands }) => { diff --git a/packages/ecs/src/implementation/world/remove.test-helper.ts b/packages/ecs/src/implementation/world/remove.test-helper.ts index 8185405f..e6f3e519 100644 --- a/packages/ecs/src/implementation/world/remove.test-helper.ts +++ b/packages/ecs/src/implementation/world/remove.test-helper.ts @@ -80,6 +80,6 @@ export const secondTail = System.create({ name: "removeTest/secondTail", resourc export const ticking = System.create({ name: "removeTest/ticking", after: [second] }, () => {}); /** Sets how long `first` and `firstTail` spin. */ -export const setPause = System.create({ name: "removeTest/setPause", input: { type: "object", properties: { milliseconds: { type: "number" } } }, resources: { write: [pause] } }, ({ input, resources }) => { +export const setPause = System.create({ name: "removeTest/setPause", resources: { write: [pause] } }, ({ resources }, input: { readonly milliseconds: number }) => { resources[pause] = input.milliseconds; }); diff --git a/packages/ecs/src/implementation/world/render-order.browser.test.ts b/packages/ecs/src/implementation/world/render-order.browser.test.ts new file mode 100644 index 00000000..3bed86a2 --- /dev/null +++ b/packages/ecs/src/implementation/world/render-order.browser.test.ts @@ -0,0 +1,216 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, it } from "vitest"; +import { Component } from "../../interface/component/component.js"; +import type { Entity } from "../../interface/entity/entity.js"; +import { Gpu } from "../../interface/gpu/gpu.js"; +import { Index } from "../../interface/indexes/index-type.js"; +import { gpuDevice } from "../../interface/resources/gpu-device.js"; +import { System } from "../../interface/system/system.js"; +import { createWorld } from "./create.js"; +import { runReported } from "./reports.test-helper.js"; + +const depth = Symbol("renderOrderTest/depth"); +const code = Symbol("renderOrderTest/code"); +const shape = Symbol("renderOrderTest/shape"); +const image = Symbol("renderOrderTest/image"); +const size = Symbol("renderOrderTest/size"); +const visible = Symbol("renderOrderTest/visible"); +const hidden = Symbol("renderOrderTest/hidden"); + +declare module "@adobe/ecs" { + export interface Components { + [depth]: { type: number }; + [code]: { type: number }; + [shape]: { type: true }; + [image]: { type: true }; + [size]: { type: readonly [number, number, number] }; + [visible]: { type: true }; + [hidden]: { type: true }; + } +} + +Component.register(depth, { type: { type: "number", precision: 1 } }); +Component.register(code, { type: { type: "integer", minimum: 0, maximum: 0xFFFFFFFF } }); +Component.register(shape, { type: { const: true } }); +Component.register(image, { type: { const: true } }); +Component.register(size, { type: { type: "array", items: { type: "number", precision: 1 }, minItems: 3, maxItems: 3 } }); +Component.register(visible, { type: { const: true } }); +Component.register(hidden, { type: { const: true } }); + +const byDepth = Index.create({ name: "renderOrderTest/byDepth", key: [depth], sorted: true }); + +const device = async (): Promise => { + const adapter = await navigator.gpu.requestAdapter(); + if (adapter === null) throw new Error("No WebGPU adapter."); + return adapter.requestDevice(); +}; + +const count = 2000; +const spawned: Entity[] = []; + +/** + * Shapes and images interleaved in depth, spawned in reverse depth order, so neither spawn nor row order is paint + * order. Every tenth is hidden or not visible, which the render system filters out, and plain depths match nothing. + */ +const spawn = System.create({ name: "renderOrderTest/spawn" }, ({ commands }) => { + for (let i = count - 1; i >= 0; i--) { + if (i % 10 === 3) spawned[i] = commands.spawn([depth, code, shape, visible, hidden])({ [depth]: i, [code]: i, [shape]: true, [visible]: true, [hidden]: true }); + else if (i % 10 === 7) spawned[i] = commands.spawn([depth, code, image])({ [depth]: i, [code]: i, [image]: true }); + else if (i % 2 === 0) spawned[i] = commands.spawn([depth, code, shape, visible])({ [depth]: i, [code]: i, [shape]: true, [visible]: true }); + else spawned[i] = commands.spawn([depth, code, image, size, visible])({ [depth]: i, [code]: i, [image]: true, [size]: [i, i, i], [visible]: true }); + } + commands.spawn([depth])({ [depth]: -1 }); +}); + +/** The codes a frame should draw, in paint order. */ +const drawn = (codes: Iterable): number[] => [...codes].filter(i => i % 10 !== 3 && i % 10 !== 7).sort((a, b) => a - b); + +describe("render systems with an order", () => { + it("draw every matching archetype as one table, in the index's order, through the same accessors", async () => { + const own = await device(); + const output = own.createBuffer({ size: (count + 16) * 8, usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_SRC | GPUBufferUsage.COPY_DST }); + const tableCounts: number[][] = []; + const draw = System.render({ + name: "renderOrderTest/draw", + components: { has: [visible], not: [hidden] }, + order: byDepth, + shader: { accessors: { paintDepth: depth, paintCode: code } }, + }, ({ encoder, tables }) => { + tableCounts.push(tables.map(table => table.count)); + encoder.clearBuffer(output); + const pass = encoder.beginComputePass(); + pass.setPipeline(pipeline); + pass.setBindGroup(1, outputGroup); + for (const table of tables) { + table.bind(pass); + pass.dispatchWorkgroups(Math.ceil(table.count / 64)); + } + pass.end(); + }); + const outputLayout = own.createBindGroupLayout({ entries: [{ binding: 0, visibility: GPUShaderStage.COMPUTE, buffer: { type: "storage" } }] }); + const outputGroup = own.createBindGroup({ layout: outputLayout, entries: [{ binding: 0, resource: { buffer: output } }] }); + const module = own.createShaderModule({ + code: `${Gpu.wgsl(draw)} +@group(1) @binding(0) var output: array; +@compute @workgroup_size(64) fn main(@builtin(global_invocation_id) id: vec3u) { + if (id.x >= ecs_count()) { return; } + output[id.x] = vec2u(u32(paintDepth(id.x)), paintCode(id.x)); +}`, + }); + expect((await module.getCompilationInfo()).messages.filter(message => message.type === "error")).toEqual([]); + const pipeline = own.createComputePipeline({ layout: own.createPipelineLayout({ bindGroupLayouts: [Gpu.layout(own), outputLayout] }), compute: { module, entryPoint: "main" } }); + + /** What the last frame drew: depth and code per instance. */ + const read = async (): Promise => { + const staging = own.createBuffer({ size: output.size, usage: GPUBufferUsage.MAP_READ | GPUBufferUsage.COPY_DST }); + const encoder = own.createCommandEncoder(); + encoder.copyBufferToBuffer(output, 0, staging, 0, output.size); + own.queue.submit([encoder.finish()]); + await staging.mapAsync(GPUMapMode.READ); + const words = new Uint32Array(staging.getMappedRange().slice(0)); + staging.destroy(); + const instances = tableCounts.at(-1)?.[0] ?? 0; + return Array.from({ length: instances }, (_, i) => [words[2 * i], words[2 * i + 1]]); + }; + const frame = async (): Promise => expect((await runReported(world, draw)).failures).toEqual([]); + + const world = createWorld({ [gpuDevice]: own }); + await world.run(spawn); + await frame(); + const codes = new Set(Array.from({ length: count }, (_, i) => i)); + let expected = drawn(codes); + expect(tableCounts.at(-1)).toEqual([expected.length]); + expect(await read()).toEqual(expected.map(i => [i, i])); + + // A despawn moves another shape into its row, a spawn adds an image between two others, and a key moves a shape + // to the back; a frame after each draws the new order. + const despawn = System.create({ name: "renderOrderTest/despawn" }, ({ commands }) => commands.despawn(spawned[count - 2])); + await world.run(despawn); + codes.delete(count - 2); + await frame(); + expected = drawn(codes); + expect(await read()).toEqual(expected.map(i => [i, i])); + + const addImage = System.create({ name: "renderOrderTest/addImage" }, ({ commands }) => { + commands.spawn([depth, code, image, size, visible])({ [depth]: 500.5, [code]: 5001, [image]: true, [size]: [0, 0, 0], [visible]: true }); + }); + await world.run(addImage); + await frame(); + const withImage = expected.flatMap(i => i === 500 ? [[i, i], [500, 5001]] : [[i, i]]); + expect(await read()).toEqual(withImage); + + const toBack = System.create({ name: "renderOrderTest/toBack", components: { read: [code], write: [depth] } }, row => { + if (row[code] === 10) row[depth] = count + 5; + }); + await world.run(toBack); + await frame(); + const atBack = [...withImage.filter(([, c]) => c !== 10), [count + 5, 10]]; + expect(await read()).toEqual(atBack); + + // Showing a hidden shape adds it to the one table, in its place. + const show = System.create({ name: "renderOrderTest/show" }, ({ commands }) => commands.remove(spawned[13], hidden)); + await world.run(show); + await frame(); + const shown = atBack.flatMap(([d, c]) => d === 12 ? [[d, c], [13, 13]] : [[d, c]]); + expect(await read()).toEqual(shown); + expect(tableCounts.every(counts => counts.length === 1)).toBe(true); + await world.dispose(); + }); + + it("write their order only when the index's entries change or move", async () => { + const own = await device(); + const draw = System.render({ name: "renderOrderTest/writes", components: { has: [visible], not: [hidden] }, order: byDepth, shader: { accessors: { paintDepth: depth } } }, () => {}); + const world = createWorld({ [gpuDevice]: own }); + await world.run(spawn); + await world.run(draw); + const instances = drawn(Array.from({ length: count }, (_, i) => i)).length; + // Row uploads and block writes pass a size; record writes don't. + const sizes: number[] = []; + const { queue } = own; + const write = queue.writeBuffer.bind(queue); + let records = 0; + queue.writeBuffer = (buffer, offset, data, dataOffset, size) => { + if (size === undefined) records++; + else sizes.push(size); + write(buffer, offset, data, dataOffset, size); + }; + await world.run(draw); + expect([sizes, records]).toEqual([[], 0]); + // A new key writes its chunk of the column and the order, in elements; the record stays. + const toBack = System.create({ name: "renderOrderTest/toBackOnce", components: { read: [code], write: [depth] } }, row => { + if (row[code] === 10) row[depth] = count + 5; + }); + await world.run(toBack); + await world.run(draw); + expect(sizes.slice(1)).toEqual([instances]); + expect(records).toBe(0); + sizes.length = 0; + await world.run(draw); + expect(sizes).toEqual([]); + await world.dispose(); + }); + + it("give a shader one table when nothing matches", async () => { + const own = await device(); + let tables: readonly System.RenderTable[] = []; + const draw = System.render({ name: "renderOrderTest/empty", components: { has: [hidden, image] }, order: byDepth, shader: { accessors: { paintDepth: depth } } }, context => { tables = context.tables; }); + const world = createWorld({ [gpuDevice]: own }); + await world.run(spawn); + expect((await runReported(world, draw)).failures).toEqual([]); + expect(tables.map(table => table.count)).toEqual([0]); + await world.dispose(); + }); + + it("declare their ordered accessors, which compile", async () => { + const own = await device(); + const draw = System.render({ name: "renderOrderTest/declared", order: byDepth, shader: { group: 2, accessors: { paintDepth: depth, paintSize: size } } }, () => {}); + const code = Gpu.wgsl(draw); + expect(code).toContain("fn paintDepth(row: u32) -> f32"); + expect(code).toContain("fn paintSize(row: u32) -> vec3"); + expect(code).toContain("@group(2) @binding(1) var ecs_table: EcsTable;"); + // An instance is its position in the order, which the order block maps to a table and a row. + expect(code).toContain("struct EcsTable { ecs_rows: u32, ecs_order: u32, ecs_tables: u32, ecs_shift: u32 }"); + const module = own.createShaderModule({ code: `${code}\n@compute @workgroup_size(1) fn main() { _ = paintDepth(0u); _ = paintSize(0u); _ = ecs_count(); }` }); + expect((await module.getCompilationInfo()).messages.filter(message => message.type === "error")).toEqual([]); + }); +}); diff --git a/packages/ecs/src/implementation/world/render-order.ts b/packages/ecs/src/implementation/world/render-order.ts new file mode 100644 index 00000000..607755d9 --- /dev/null +++ b/packages/ecs/src/implementation/world/render-order.ts @@ -0,0 +1,150 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import type { RenderContext, RenderTable } from '../../interface/system/render.js'; +import type { Archetype } from './archetype.js'; +import type { GpuEngine } from './gpu.js'; +import type { Ordered } from './index-order.js'; +import { blockBase } from './index-sorted.js'; +import type { WorldIndex } from './indexes.js'; + +// A render system with `order` draws one table: instance `i` is the i-th matching entity in its sorted index's order. +// Its record is the instance count, the word addresses of its order block and tables block, and the bits a slot is +// shifted by. The order block holds, by position, a matched archetype's slot above `shift` bits and its row below +// them; the tables block holds, by slot, the word address of each accessor's column. The order block is built and +// written only when the index's entries changed or moved, or the matches grew; the tables block only when a column's +// address changed; the record only when one of its words did. + +/** The table of a render system's run and its context, both reused. */ +type MutableTable = { count: number; bind: (pass: GPURenderPassEncoder | GPUComputePassEncoder) => void }; +type MutableContext = { device: GPUDevice; encoder: GPUCommandEncoder; tables: readonly RenderTable[] }; + +export type OrderedRender = { + /** + * Uploads what the run's one table reads; returns null, or the reason it can't be drawn: more matched archetypes + * and rows than an entry's 32 bits hold. + */ + readonly prepare: (gpu: GpuEngine, matches: readonly Archetype[], tick: number) => string | null; + /** The body's context, once the work recorded before it is submitted. */ + readonly context: (gpu: GpuEngine) => RenderContext; +}; + +/** + * `columns` are the accessors' component ids, in accessor order; `group` the bind group the shader declares; `key` + * the prefix of the runner's GPU records. + */ +export const createOrderedRender = (index: WorldIndex, ordered: Ordered, columns: readonly number[], group: number, key: string): OrderedRender => { + const width = columns.length; + const orderKey = `${key}order`; + const tablesKey = `${key}tables`; + const recordKey = `${key}ordered`; + let entries = new Int32Array(64); + let instances = 0; + let tables = new Int32Array(Math.max(1, width)); + /** What the order block was last built from: the index's ticks, the matches, and the GPU it was written to. */ + let builtChanged = -1; + let builtMoved = -1; + let builtMatches = -1; + let builtGpu: GpuEngine | null = null; + let shift = 31; + const words = new Uint32Array(4); + let recorded = false; + const offsets = [0]; + let bound: GpuEngine | null = null; + const table: MutableTable = { + count: 0, + bind: pass => { + if (bound !== null) pass.setBindGroup(group, bound.renderGroup(), offsets); + }, + }; + const list: readonly RenderTable[] = [table]; + let context: MutableContext | null = null; + + /** Lists the index's entries of matched archetypes, in its order, as slot and row. */ + const build = (matches: readonly Archetype[]): void => { + const sorted = index.sorted; + instances = 0; + if (sorted === null) return; + if (entries.length < sorted.size) entries = new Int32Array(Math.max(sorted.size, 2 * entries.length)); + const owners = ordered.owners(matches); + const known = owners.length; + const slotShift = shift; + const { order, lengths, stride, words: keyWords, blocks } = sorted; + const sortedEntries = sorted.entries; + for (let k = 0; k < blocks; k++) { + const block = order[k]; + const base = blockBase(sorted, block) + keyWords; + for (let at = base, end = base + lengths[block] * stride; at < end; at += stride) { + const archetype = sortedEntries[at + 1]; + const owner = archetype < known ? owners[archetype] : -1; + if (owner < 0) continue; + entries[instances++] = (owner << slotShift) | sortedEntries[at + 2]; + } + } + }; + + const prepare = (gpu: GpuEngine, matches: readonly Archetype[], tick: number): string | null => { + const slots = matches.length; + if (gpu !== builtGpu) { + builtGpu = gpu; + builtChanged = -1; + recorded = false; + } + // The tables block: every column's address, uploading rows the CPU changed first. + if (tables.length < slots * width) { + const grown = new Int32Array(Math.max(slots * width, 2 * tables.length)); + grown.set(tables); + tables = grown; + } + let addressed = false; + let most = 0; + for (let slot = 0; slot < slots; slot++) { + const archetype = matches[slot]; + const { count } = archetype; + if (count > most) most = count; + for (let k = 0; k < width; k++) { + const storage = count === 0 ? undefined : archetype.storageById[columns[k]]; + const address = storage === undefined ? 0 : gpu.upload(archetype, storage, tick); + const at = slot * width + k; + if (tables[at] !== address) { + tables[at] = address; + addressed = true; + } + } + } + if (slots !== builtMatches) { + // At least one bit of slot, so a row's mask is never 32 bits. + const slotBits = Math.max(1, Math.ceil(Math.log2(slots))); + shift = 32 - slotBits; + builtMatches = slots; + builtChanged = -1; + } + if (most > 2 ** shift) return `with ${slots} matching archetypes an ordered table names at most ${2 ** shift} rows of each, and one holds ${most}`; + const rebuilt = index.changed !== builtChanged || index.moved !== builtMoved; + if (rebuilt) { + build(matches); + builtChanged = index.changed; + builtMoved = index.moved; + } + const orderAddress = gpu.uploadIndexes(orderKey, entries, instances, rebuilt); + const tablesAddress = gpu.uploadIndexes(tablesKey, tables, slots * width, addressed); + if (!recorded || words[0] !== instances || words[1] !== orderAddress || words[2] !== tablesAddress || words[3] !== shift) { + words[0] = instances; + words[1] = orderAddress; + words[2] = tablesAddress; + words[3] = shift; + offsets[0] = gpu.record(recordKey, words); + recorded = true; + } + bound = gpu; + table.count = instances; + return null; + }; + + const contextOf = (gpu: GpuEngine): RenderContext => { + if (context === null) return context = { device: gpu.device, encoder: gpu.encoder(), tables: list }; + context.device = gpu.device; + context.encoder = gpu.encoder(); + return context; + }; + + return { prepare, context: contextOf }; +}; diff --git a/packages/ecs/src/implementation/world/reports.test.ts b/packages/ecs/src/implementation/world/reports.test.ts index db328c33..ab80b52e 100644 --- a/packages/ecs/src/implementation/world/reports.test.ts +++ b/packages/ecs/src/implementation/world/reports.test.ts @@ -133,7 +133,7 @@ describe("reports", () => { await using world = inline(); logged.length = 0; world.add(logger); - const failWith = System.create({ name: "reportsTest/failWith", input: { type: "object", properties: { k: { type: "integer" } } } }, ({ input }) => { throw new Error(`run ${input.k}`); }); + const failWith = System.create({ name: "reportsTest/failWith" }, (_context, input: { readonly k: number }) => { throw new Error(`run ${input.k}`); }); for (let k = 0; k < retainedRuns; k++) await world.run(failWith, { k }); logged.length = 0; const reused = await world.run(failWith, { k: retainedRuns }); diff --git a/packages/ecs/src/implementation/world/reports.ts b/packages/ecs/src/implementation/world/reports.ts index 8613ad5c..66c7bc50 100644 --- a/packages/ecs/src/implementation/world/reports.ts +++ b/packages/ecs/src/implementation/world/reports.ts @@ -2,7 +2,7 @@ import { Report } from '../../interface/report/report.js'; import type { Failure, Warning } from '../../interface/report/failure.js'; import { stampChanged } from './archetype.js'; -import { addCommand, createBuffer, pushCommand, pushComponent, removeCommand, spawnCommand, type CommandBuffer } from './buffer.js'; +import { addCommand, clearBuffer, createBuffer, pushCommand, pushComponent, removeCommand, spawnCommand, type CommandBuffer } from './buffer.js'; import { writeCell } from './cells.js'; import type { Views } from '../../interface/registry/layout.js'; import { pushValues } from './commands.js'; @@ -79,6 +79,8 @@ export const createReports = (state: State, applyAlone: (buffer: CommandBuffer) let oldest = 0; /** The slot the last run took. */ let last: Slot | null = null; + /** The buffer that spawns reports. */ + const spawns = createBuffer(1); /** The storage and row of a slot's component `id`, while its entity holds it. */ const cellOf = (slot: Slot, id: number) => { @@ -130,7 +132,10 @@ export const createReports = (state: State, applyAlone: (buffer: CommandBuffer) slot.errors = null; } slot.entity = reserve(state.entities); - const buffer = createBuffer(); + // One buffer spawns every report, as each sync point is done with it before the next run takes one; a new one + // only while it's still queued. + const buffer = state.queued.some(queued => queued.buffer === spawns) ? createBuffer(1) : spawns; + clearBuffer(buffer); pushValues(state, buffer, spawnCommand, slot.entity, { [Report.run]: { number: run, elapsed: 0, systems: 0, workers: 0, joined: 0, main: false, gpu: false, dropped: 0 } satisfies RunValues, }); diff --git a/packages/ecs/src/implementation/world/rows.ts b/packages/ecs/src/implementation/world/rows.ts index 4d9f194e..be16c3a5 100644 --- a/packages/ecs/src/implementation/world/rows.ts +++ b/packages/ecs/src/implementation/world/rows.ts @@ -19,7 +19,7 @@ import type { WorldIndex } from './indexes.js'; * What an accessor row reads and writes: a storage, or a joined thread's view of one. A joined thread's own objects * have `tags`, the world's tags of their rows, which a write marks as the thread's own value. */ -export type RowStorage = Pick; +export type RowStorage = Pick; /** What a row binds to: an archetype, or a joined thread's view of one. */ export type RowArchetype = { readonly id: number; readonly entities: Float64Array; readonly storageById: readonly (RowStorage | undefined)[] }; @@ -412,11 +412,11 @@ const createPlainArrayLookup = (state: State, lookup: SystemLookup) => { }; /** - * A body's context: its resources, lookups, reductions, input and commands, each under its own key and present only + * A body's context: its resources, lookups, reductions and commands, each under its own key and present only * when declared. `bind` resolves the resources' cells for a run; `reduced[t]` holds the leaves of the fold into its * `t`th reduction target so far this run. */ -export type Context = { readonly value: { input?: unknown; up?: unknown }; readonly bind: () => void; readonly reduced: readonly Float64Array[] }; +export type Context = { readonly value: { up?: unknown }; readonly bind: () => void; readonly reduced: readonly Float64Array[] }; /** * What a context holds beyond the declaration's resources and lookups, and for lookups through indexes, the index of @@ -425,7 +425,6 @@ export type Context = { readonly value: { input?: unknown; up?: unknown }; reado type Extras = { readonly targets?: readonly ReductionTarget[]; readonly commands?: Commands; - readonly input?: boolean; readonly indexes?: ReadonlyMap; readonly since?: Since; }; @@ -437,8 +436,8 @@ const never: Since = { value: -1 }; const own = (): object => Object.create({}); /** Resource cells are resolved once per run: they move only at sync points. An unresolved resource throws on access. */ -export const createContext = (state: State, declaration: Pick, { targets = [], commands, input = false, indexes, since = never }: Extras = {}): Context => { - const value: { input?: unknown; up?: unknown } = own(); +export const createContext = (state: State, declaration: Pick, { targets = [], commands, indexes, since = never }: Extras = {}): Context => { + const value: { up?: unknown } = own(); const { lookup, resources } = declaration; const names = Object.keys(lookup ?? {}); if (names.length > 0) { @@ -501,7 +500,6 @@ export const createContext = (state: State, declaration: Pick { if (keys.length > 0) { diff --git a/packages/ecs/src/implementation/world/rules.test.ts b/packages/ecs/src/implementation/world/rules.test.ts index 8bd5f49f..c36f5e4b 100644 --- a/packages/ecs/src/implementation/world/rules.test.ts +++ b/packages/ecs/src/implementation/world/rules.test.ts @@ -67,7 +67,7 @@ Rule.create({ name: "rulesTest/sprites", components: { has: [sprite] }, derive: Rule.create({ name: "rulesTest/tag", components: { has: [tagged] }, derive: [tag] }); /** Runs `edit` with commands once, as its own run. */ -const edit = System.create({ name: "rulesTest/edit", input: { type: "object", properties: { step: { type: "integer" } } } }, ({ input, commands }) => { +const edit = System.create({ name: "rulesTest/edit" }, ({ commands }, input: { readonly step: number }) => { for (const step of steps[input.step] ?? []) step(commands); }); let steps: ((commands: System.Commands) => void)[][] = []; diff --git a/packages/ecs/src/implementation/world/runner.ts b/packages/ecs/src/implementation/world/runner.ts index 4dc78bc2..1ed82036 100644 --- a/packages/ecs/src/implementation/world/runner.ts +++ b/packages/ecs/src/implementation/world/runner.ts @@ -33,6 +33,7 @@ import { acquireLookups, type IndexLookups, type IndexUse } from './indexes.js'; import { createOrdered, type Ordered, type Passing } from './index-order.js'; import { blockBase, type SortedTable } from './index-sorted.js'; import type { Since } from './index-lookups.js'; +import { createOrderedRender } from './render-order.js'; /** * One unit of a system's work for a run: the commands it issued, or for a worker's chunk its segment of the worker's @@ -185,7 +186,7 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { const leafNumber = numberOf(leaf); const label = labelOfItem(leaf); // Method parameters are bivariant, so the runtime can see any leaf's body with erased parameter types. - const system: { body(subject: unknown, context?: unknown): void } = leaf; + const system: { body(subject: unknown, context?: unknown, input?: unknown): void } = leaf; const { components: query, resources } = declaration; const required = [...new Set([...query?.read ?? [], ...query?.write ?? [], ...query?.has ?? [], ...query?.added ?? [], ...query?.changed ?? []])].map(key => idOf(state, key)); const excluded = (query?.not ?? []).map(key => idOf(state, key)); @@ -232,6 +233,9 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { }); const accessors = declaration.shader === undefined ? [] : accessorsOf(declaration.shader); if (typeof accessors === "string") return failPlacement(`can't render: ${accessors}`); + /** A render system's one table in its `order` index's order. */ + const orderedRender = leaf.kind !== "render" || indexLookups.ordered === null || ordered === null ? null + : createOrderedRender(indexLookups.ordered, ordered, accessors.map(accessor => idOf(state, accessor.component)), declaration.shader?.group ?? 0, `${leafNumber}:`); const touches = [...new Set([...query?.read ?? [], ...query?.write ?? []])].map(key => idOf(state, key)); const writes = (query?.write ?? []).map(key => idOf(state, key)); const looks = [...new Set([...Object.values(declaration.lookup ?? {}).flatMap(lookup => lookup.read ?? []), ...declaration.up?.read ?? []])].map(key => idOf(state, key)); @@ -309,10 +313,11 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { // A kernel's reductions fold in its partials, so only a body that runs as written gets reducer functions. Render // bodies, GPU kernels and observer systems issue no commands. const ownInput = inputFor(leaf); + /** The current run's input, which the body takes after its context; kept until the next run, as its context is. */ + let runInput: unknown; const context = createContext(state, declaration, { targets: kernel === null ? targets : [], commands: leaf.kind === "render" || declaration.kernel === "gpu" || declaration.observer === true ? undefined : commands, - input: declaration.input !== undefined, indexes: indexLookups.byName, since: lookupSince, }); @@ -374,7 +379,7 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { }; /** - * By archetype: its change filters, `added` then `changed`, rows and chunks, kept until its arrays are replaced; + * By archetype: its change filters, `added` then `changed`, rows, chunks and blocks, kept until its arrays are replaced; * and the WASM addresses of its columns and filters, for the WASM kernel `addressed` they were taken for, since a * kernel loaded again may stamp a tick column the last one muted. */ @@ -387,11 +392,11 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { const filters = addedIds.length + changedIds.length === 0 ? noFilters : [ ...addedIds.flatMap(id => { const storage = archetype.storageById[id]; - return storage === undefined ? [] : [{ ticks: storage.added, chunks: storage.chunkAdded }]; + return storage === undefined ? [] : [{ ticks: storage.added, chunks: storage.chunkAdded, blocks: storage.blockAdded }]; }), ...changedIds.flatMap(id => { const storage = archetype.storageById[id]; - return storage === undefined ? [] : [{ ticks: storage.changed, chunks: storage.chunkChanged }]; + return storage === undefined ? [] : [{ ticks: storage.changed, chunks: storage.chunkChanged, blocks: storage.blockChanged }]; }), ]; const entry: Filtered = { version: archetype.version, filters, addresses: noAddresses, addressed: null }; @@ -411,27 +416,28 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { if (archetype === undefined || !matches.includes(archetype)) return; const row = bindRow(archetype); row.index = state.entities.row[index]; - system.body(row.value, context.value); + system.body(row.value, context.value, runInput); }; const runBody = (archetype: Archetype, since: number): void => { const row = bindRow(archetype); const { value } = row; const values = context.value; + const input = runInput; const filters = filtersOf(archetype); // Every row passes, so nothing is filtered. // A body may remove its own system, after which no row of it runs. if (since < 0 || filters.length === 0) { for (let index = 0, count = archetype.count; index < count && !holdings.removed; index++) { row.index = index; - system.body(value, values); + system.body(value, values, input); } return; } forEachPassingRange(filters, since, 0, archetype.count, (start, end) => { for (let index = start; index < end && !holdings.removed; index++) { row.index = index; - system.body(value, values); + system.body(value, values, input); } }); }; @@ -450,6 +456,7 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { ...bound.columnsOf(archetype).map(column => column.byteOffset), ...entry.filters.map(filter => filter.ticks.byteOffset), ...entry.filters.map(filter => filter.chunks.byteOffset), + ...entry.filters.map(filter => filter.blocks.byteOffset), ]; entry.addressed = bound.wasm; } @@ -511,11 +518,12 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { // The parent's row is bound after the system's own, for the archetype both may share. const parentRow = parentArchetype < 0 ? null : bindParent(state.archetypes[parentArchetype]); const upContext = context.value; + const input = runInput; upContext.up = parentRow === null ? null : parentRow.value; for (let n = start; n < end && !holdings.removed; n++) { row.index = rows[n]; if (parentRow !== null) parentRow.index = parents[n]; - system.body(row.value, upContext); + system.body(row.value, upContext, input); } }; @@ -535,6 +543,7 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { const runs = order.runs(sorted, matches, since < 0 ? null : orderPassing); const slot = partialCount === 0 ? 0 : takeSlots(1); const contextValue = context.value; + const input = runInput; for (let k = 0; k < runs.length && !holdings.removed; k++) { const run = runs[k]; const archetype = matches[run.owner]; @@ -546,7 +555,7 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { const { value } = row; for (let n = run.start; n < run.end && !holdings.removed; n++) { row.index = run.rows[n]; - system.body(value, contextValue); + system.body(value, contextValue, input); } } }; @@ -556,6 +565,7 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { const owners = order.owners(matches); const { entries, stride, words, lengths, blocks } = sorted; const contextValue = context.value; + const input = runInput; let current = -1; let row: ReturnType | null = null; let value: object | null = null; @@ -568,12 +578,14 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { const owner = archetype < owners.length ? owners[archetype] : -1; if (owner < 0) continue; if (owner !== current || row === null || value === null) { - row = bindRow(matches[owner]); + // The row is found once; after, it's bound in place, as often as every row when archetypes interleave. + if (row === null) row = bindRow(matches[owner]); + else row.bind(matches[owner]); value = row.value; current = owner; } row.index = entries[at + 2]; - system.body(value, contextValue); + system.body(value, contextValue, input); } } }; @@ -607,28 +619,9 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { const runUp = (since: number, values: readonly unknown[], args: unknown, tick: number, workers: Workers | null, index: number): Promise | null => { if (up === null) return null; const slot = partialCount === 0 ? 0 : takeSlots(1); - let next: () => readonly Run[] | null; - let clear = (): void => {}; - if (since < 0) { - const order = up.levels(matches); - let depth = 0; - next = () => { - if (depth >= order.levels) return null; - const runs: Run[] = []; - for (let group = order.depthGroups[depth]; group < order.depthGroups[depth + 1]; group++) { - if (order.groupEnds[group] === order.groupStarts[group]) continue; - runs.push({ owner: order.groupOwners[group], parent: order.groupParents[group], rows: order.rows, parents: order.parents, start: order.groupStarts[group], end: order.groupEnds[group] }); - } - depth++; - return runs; - }; - } - else { - const pass = up.filtered(matches, addedIds, changedIds, since); - if (pass === null) return null; - next = pass.next; - clear = pass.clear; - } + const pass = since < 0 ? up.wholePass(up.levels(matches)) : up.filtered(matches, addedIds, changedIds, since); + if (pass === null) return null; + const { next, clear } = pass; const runLocally = (run: Run): void => runUpRows(matches[run.owner], run.parent, run.rows, run.parents, run.start, run.end, values, args, tick, slot); const parallel = workers !== null && kernel !== null && kernel.wasm !== null && kernel.compiled() !== null && partialCount === 0 && kernel.lookupParameters().length + indexParameters.length + 3 + (kernel.description.upColumns?.length ?? 0) + kernel.wasm.scalars.length <= maximumScalars; @@ -831,6 +824,7 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { if (gpu === null) return failPlacement("needs a world with a gpu device"); if (typeof accessors === "string") return; holdings.gpu = gpu; + if (orderedRender !== null) return runOrderedRender(gpu, tick, orderedRender); const group = declaration.shader?.group ?? 0; const tables = matches.filter(archetype => archetype.count > 0).map(archetype => { words[0] = archetype.count; @@ -852,6 +846,20 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { } }; + /** A render system with `order`: one table, nothing allocated while nothing it reads changed. */ + const runOrderedRender = (gpu: GpuEngine, tick: number, render: NonNullable): void => { + const failure = render.prepare(gpu, matches, tick); + if (failure !== null) return failPlacement(`can't render: ${failure}`); + gpu.submit(); + try { + system.body(render.context(gpu), context.value); + } + catch (error) { + gpu.discard(); + throw error; + } + }; + const holdings: Holdings = { removed: false, released: false, partials: null, up: up ?? ordered, directories: kernel?.directories ?? [], indexes: indexLookups, gpu: null, prefix: `${leafNumber}:`, afterSync: partialCount > 0 ? new WeakRef(sizeSlots) : null, remote: null, @@ -924,7 +932,7 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { /** -1 passes every row. */ const since = !filtered || resourcesChanged ? -1 : lastRun; context.bind(); - if (declaration.input !== undefined) context.value.input = args; + runInput = args; if (indexed && kernel !== null) indexParameters = kernel.refreshIndexes(lastRun); if (placement === "render") { refreshMatches(); @@ -933,7 +941,7 @@ export const createRunner = (state: State, leaf: Leaf, since = 0): Runner => { } if (query === undefined) { ran = since < 0; - if (ran) system.body(context.value); + if (ran) system.body(context.value, args); return chunks; } // Only resource and index filters, none of which changed: no row passes, unless its parent changed. diff --git a/packages/ecs/src/implementation/world/schedule.test.ts b/packages/ecs/src/implementation/world/schedule.test.ts index 1251fa95..a7cfdfdd 100644 --- a/packages/ecs/src/implementation/world/schedule.test.ts +++ b/packages/ecs/src/implementation/world/schedule.test.ts @@ -21,7 +21,7 @@ Component.register(value, { type: { type: "number" } }); Component.register(other, { type: { type: "number" } }); /** Systems built as values, not registered, so each test's are its own. */ -const leaf = (declaration: Declaration & { readonly name: string }): Leaf => ({ kind: "system", name: declaration.name, declaration, body: () => {} }); +const leaf = (declaration: Declaration & { readonly name: string }): Leaf => ({ kind: "system", name: declaration.name, declaration, input: undefined, body: () => {} }); const composite = (declaration: Composite["declaration"] & { readonly name: string }): Composite => ({ kind: "composite", name: declaration.name, declaration }); const names = (system: Definition): string[][] => scheduleOf(system).tiers.map(tier => tier.map(leaf => leaf.name)); diff --git a/packages/ecs/src/implementation/world/sharing.ts b/packages/ecs/src/implementation/world/sharing.ts index 59fb1ea8..192e9871 100644 --- a/packages/ecs/src/implementation/world/sharing.ts +++ b/packages/ecs/src/implementation/world/sharing.ts @@ -240,7 +240,7 @@ export const createSharing = (host: SharingHost) => { released(holder, thread, sequence); continue; } - const layout = bytes >= 0 && definition.kind !== "composite" ? inputLayoutOf(definition, definition.declaration.input) : null; + const layout = bytes >= 0 && definition.kind !== "composite" ? inputLayoutOf(definition, definition.input) : null; const held = holder !== null && holder.tasks !== null ? heldInput(thread.number, sequence, holder.tasks) : null; const value = own ? joinedRun(sequence) : held ?? (layout !== null && layout.kind === "linear" ? layout.read(thread.views[slot], 0) : input); let ran: Promise; diff --git a/packages/ecs/src/implementation/world/stop-lists.ts b/packages/ecs/src/implementation/world/stop-lists.ts index 8a5390f6..2f03faac 100644 --- a/packages/ecs/src/implementation/world/stop-lists.ts +++ b/packages/ecs/src/implementation/world/stop-lists.ts @@ -1,5 +1,6 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. import type { Finalizer } from '../../interface/finalizer/created.js'; +import { viewsOf, type Layout } from '../../interface/registry/layout.js'; import type { Memory } from './memory.js'; import type { StringStore } from './strings.js'; import { logFinalizedWord, logStopsWord, type MoveLog } from './move-log.js'; @@ -12,7 +13,9 @@ import { logFinalizedWord, logStopsWord, type MoveLog } from './move-log.js'; // its bytes. Then a group per finalizer with stops, in the world's finalizer order: its index on the thread, its stops, // its bytes; then as columns, each on 8 bytes, every stop's entity (f64), its archetype and row before apply (u32), the // tag of each object it reads (u32), and the value of each linear struct it reads in that component's layout, or for a -// string, the handle of a copy of its record, freed with the block, since the thread may read it sync points later. +// string, the handle of a copy of its record; then its resources as they were before apply: the handle of a copy of +// each string resource it reads (u32), then each linear resource it reads in its layout, each on 8 bytes. Copies of +// records are freed with the block, since the thread may read them sync points later. export const blockSyncWord = 0; export const blockNextWord = 1; @@ -37,11 +40,29 @@ export const valuesAt = (group: number, count: number, objects: number): number /** Bytes of one linear read's column of `count` values of `stride` bytes. */ export const columnBytes = (count: number, stride: number): number => align(count * stride); -/** Bytes of a group of `count` stops of a finalizer reading `objects` objects and linear values of `strides` bytes. */ -const groupBytes = (count: number, objects: number, strides: readonly number[]): number => { +/** + * Where each linear resource's copy starts past a group's columns, after the handles of `strings` string resources, + * for linear resources of `strides` bytes; then the bytes of them all. + */ +export const resourceOffsetsOf = (strings: number, strides: readonly number[]): number[] => { + const offsets: number[] = []; + let at = align(4 * strings); + for (const stride of strides) { + offsets.push(at); + at += align(stride); + } + offsets.push(at); + return offsets; +}; + +/** + * Bytes of a group of `count` stops of a finalizer reading `objects` objects, linear values of `strides` bytes and + * `resources` bytes of resources. + */ +const groupBytes = (count: number, objects: number, strides: readonly number[], resources: number): number => { let bytes = valuesAt(0, count, objects); for (const stride of strides) bytes += columnBytes(count, stride); - return bytes; + return bytes + resources; }; /** One of a joined thread's finalizers, as the world writes its stops. */ @@ -55,6 +76,18 @@ export type StopsOf = { readonly strides: readonly number[]; /** By linear read: whether it's a string's handles, whose records the block holds copies of. */ readonly copies: readonly boolean[]; + /** + * The string resources it reads, then its linear resources with their strides, in its context's order, which each + * group holds copies of; and where each linear one's copy starts past the group's columns, then their bytes. + */ + readonly stringResources: readonly number[]; + readonly linearResources: readonly number[]; + readonly resourceStrides: readonly number[]; + readonly resourceOffsets: readonly number[]; + /** What its string and linear resources start at, with the linear ones' layouts, for one the world doesn't hold. */ + readonly stringStarts: readonly unknown[]; + readonly linearStarts: readonly unknown[]; + readonly resourceLayouts: readonly Layout[]; /** This sync point's stops, as plan entries, reused by every sync point. */ readonly entries: number[]; }; @@ -128,7 +161,10 @@ const copyStrings = (strings: StringStore, copied: number[], u32: Uint32Array, a /** A storage as stops are copied from it: its tags, or its linear values. */ type Copied = { readonly tags: Uint32Array; readonly views: { readonly u8: Uint8Array; readonly u32: Uint32Array } }; -/** What a sync point's stops come from: entries' entities and archetypes before apply, and the entity table's rows. */ +/** + * What a sync point's stops come from: entries' entities and archetypes before apply, the entity table's rows, and the + * resource entity's archetype and row, -1 without one. + */ export type StopSource = { readonly entity: Float64Array; readonly from: Int32Array; @@ -136,6 +172,8 @@ export type StopSource = { readonly archetypes: readonly { readonly storageById: readonly (Copied | undefined)[] }[]; readonly row: Uint32Array; readonly holds: Holds; + readonly resourceArchetype: number; + readonly resourceRow: number; }; /** @@ -147,7 +185,7 @@ export const writeStops = (writer: StopWriter, sync: number, source: StopSource) const { staged, memory } = writer; if (staged.length === 0) return; let bytes = blockHeaderBytes; - for (const stops of staged) bytes += groupBytes(stops.entries.length, stops.objects.length, stops.strides); + for (const stops of staged) bytes += groupBytes(stops.entries.length, stops.objects.length, stops.strides, stops.resourceOffsets[stops.linearResources.length]); const offset = memory.allocate(bytes); const buffer = memory.memory.buffer; const u8 = new Uint8Array(buffer); @@ -163,9 +201,9 @@ export const writeStops = (writer: StopWriter, sync: number, source: StopSource) let group = offset + blockHeaderBytes; const { entity, from, despawned, archetypes, row: rows, holds } = source; for (const stops of staged) { - const { entries, objects, linear, strides, copies } = stops; + const { entries, objects, linear, strides, copies, stringResources, linearResources, resourceStrides, resourceOffsets } = stops; const count = entries.length; - const size = groupBytes(count, objects.length, strides); + const size = groupBytes(count, objects.length, strides, resourceOffsets[linearResources.length]); u32[(group >> 2) + groupIndexWord] = stops.index; u32[(group >> 2) + groupCountWord] = count; u32[(group >> 2) + groupBytesWord] = size; @@ -229,6 +267,36 @@ export const writeStops = (writer: StopWriter, sync: number, source: StopSource) } column += columnBytes(count, stride); } + // As before apply, as its components are: the thread may read them after the resources are written again. + if (stringResources.length > 0 || linearResources.length > 0) { + const { strings } = writer; + const { resourceRow } = source; + const storageById = source.resourceArchetype < 0 ? undefined : archetypes[source.resourceArchetype].storageById; + for (let k = 0; k < stringResources.length; k++) { + const handles = storageById?.[stringResources[k]]?.views.u32; + // Without the resource, unreachable while a finalizer activates only once its resources exist: its start. + const start = stops.stringStarts[k]; + const copy = strings === null ? 0 : handles !== undefined ? strings.copy(handles[resourceRow]) : typeof start === "string" ? strings.write(start) : 0; + if (copy !== 0) copied.push(copy); + u32[(column >> 2) + k] = copy; + } + for (let k = 0; k < linearResources.length; k++) { + const values = storageById?.[linearResources[k]]?.views; + const stride = resourceStrides[k]; + const target = column + resourceOffsets[k]; + if (values === undefined) { + // Unreachable while a finalizer activates only once its resources exist: its start. + const layout = stops.resourceLayouts[k]; + if (layout.kind === "linear") layout.write(viewsOf(buffer, target, stride), 0, stops.linearStarts[k]); + continue; + } + if ((stride & 3) === 0) { + const words = stride >> 2; + for (let w = 0; w < words; w++) u32[(target >> 2) + w] = values.u32[resourceRow * words + w]; + } + else for (let b = 0; b < stride; b++) u8[target + b] = values.u8[resourceRow * stride + b]; + } + } stopped.push([stops.finalizer, count]); group += size; entries.length = 0; diff --git a/packages/ecs/src/implementation/world/strings.test.ts b/packages/ecs/src/implementation/world/strings.test.ts index 13608233..923f1019 100644 --- a/packages/ecs/src/implementation/world/strings.test.ts +++ b/packages/ecs/src/implementation/world/strings.test.ts @@ -39,23 +39,23 @@ Resource.register(title, { type: { type: "string", default: "" } }); /** Text of every kind a string holds: none, letters, astral characters as surrogate pairs, and a long run. */ const textOf = (n: number): string => n % 4 === 0 ? "" : n % 4 === 1 ? `item ${n}` : n % 4 === 2 ? `✓ ${n} 😀 naïve` : `${"long ".repeat(40)}${n}`; -const spawn = System.create({ name: "stringsTest/spawn", input: { type: "object", properties: { count: { type: "integer" }, from: { type: "integer" } } } }, ({ input, commands }) => { +const spawn = System.create({ name: "stringsTest/spawn" }, ({ commands }, input: { readonly count: number; readonly from: number }) => { for (let i = 0; i < input.count; i++) commands.spawn([label, order])({ [label]: textOf(input.from + i), [order]: input.from + i }); }); /** `count` rows with labels of 400 code units, whose records are 1024-byte blocks of the arena. */ -const spawnLong = System.create({ name: "stringsTest/spawnLong", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const spawnLong = System.create({ name: "stringsTest/spawnLong" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) commands.spawn([label, order])({ [label]: `${i}`.padStart(400, "·"), [order]: i }); }); /** A run over every row that changes nothing, for its sync point. */ const pass = System.create({ name: "stringsTest/pass", components: { read: [order] } }, row => { void row[order]; }); -const rename = System.create({ name: "stringsTest/rename", input: { type: "object", properties: { suffix: { type: "string" } } }, components: { read: [order], write: [label] } }, (row, { input }) => { +const rename = System.create({ name: "stringsTest/rename", components: { read: [order], write: [label] } }, (row, _context, input: { readonly suffix: string }) => { row[label] = `${textOf(row[order])}${input.suffix}`; }); -const relabel = System.create({ name: "stringsTest/relabel", input: { type: "object", properties: { suffix: { type: "string" } } }, components: { read: [order], has: [label] } }, (row, { input, commands }) => { +const relabel = System.create({ name: "stringsTest/relabel", components: { read: [order], has: [label] } }, (row, { commands }, input: { readonly suffix: string }) => { commands.add(row.entity, label, `${textOf(row[order])}${input.suffix}`); }); @@ -73,11 +73,11 @@ const noted = System.create({ name: "stringsTest/noted", components: { has: [ord const despawnEven = System.create({ name: "stringsTest/despawnEven", components: { read: [order] } }, (row, { commands }) => { if (row[order] % 2 === 0) commands.despawn(row.entity); }); -const labelAgain = System.create({ name: "stringsTest/labelAgain", input: { type: "object", properties: { suffix: { type: "string" } } }, components: { read: [order], not: [label] } }, (row, { input, commands }) => { +const labelAgain = System.create({ name: "stringsTest/labelAgain", components: { read: [order], not: [label] } }, (row, { commands }, input: { readonly suffix: string }) => { commands.add(row.entity, label, `${textOf(row[order])}${input.suffix}`); }); -const edit = System.create({ name: "stringsTest/edit", input: { type: "object", properties: { title: { type: "string" } } }, resources: { write: [mode, title] } }, ({ input, resources }) => { +const edit = System.create({ name: "stringsTest/edit", resources: { write: [mode, title] } }, ({ resources }, input: { readonly title: string }) => { resources[mode] = "edit"; resources[title] = input.title; }); diff --git a/packages/ecs/src/implementation/world/synchronize.ts b/packages/ecs/src/implementation/world/synchronize.ts index 6777bc46..fe40b98f 100644 --- a/packages/ecs/src/implementation/world/synchronize.ts +++ b/packages/ecs/src/implementation/world/synchronize.ts @@ -1,7 +1,7 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. import type { Views } from '../../interface/registry/layout.js'; import { copyValue, ensureCapacity, swapRemove, writeF32, writeF32Lanes, writeF64, writeI32, writeU32, type Archetype, type Storage } from './archetype.js'; -import { chunkShift } from './kernel.js'; +import { blockShift, chunkShift } from './kernel.js'; import { addCommand, clearBuffer, despawnCommand, endOf, referenceAt, removeCommand, spawnCommand, type CommandBuffer } from './buffer.js'; import { readCell, writeCell } from './cells.js'; import { commitBatches, commitStaged } from './batches.js'; @@ -281,10 +281,12 @@ const writeRange = (buffer: CommandBuffer, start: number, end: number, to: Arche // A constant's `changed` ticks are its `added` ticks, and its value can't change: only a new start stamps it. if (storage.changed !== storage.added || starts) { storage.changed[toRow] = tick; + storage.blockChanged[toRow >> blockShift] = tick; storage.chunkChanged[toRow >> chunkShift] = tick; } if (starts) { storage.added[toRow] = tick; + storage.blockAdded[toRow >> blockShift] = tick; storage.chunkAdded[toRow >> chunkShift] = tick; } } @@ -297,6 +299,7 @@ export const writeDefault = (state: State, storage: Storage, row: number, tick: else storage.tags[row] = startTag; if (!storage.ticked) return; storage.added[row] = storage.changed[row] = tick; + storage.blockAdded[row >> blockShift] = storage.blockChanged[row >> blockShift] = tick; storage.chunkAdded[row >> chunkShift] = storage.chunkChanged[row >> chunkShift] = tick; }; @@ -416,7 +419,10 @@ const copyColumn = (from: Storage, to: Storage, plan: Plan, order: Int32Array, f const changed = from.changed[fromRow]; to.added[toRow] = added; to.changed[toRow] = changed; - // A chunk's tick is the newest of its rows, so a row moving in can raise it. + // A block's and a chunk's tick are the newest of their rows, so a row moving in can raise them. + const block = toRow >> blockShift; + if (added > to.blockAdded[block]) to.blockAdded[block] = added; + if (changed > to.blockChanged[block]) to.blockChanged[block] = changed; const chunk = toRow >> chunkShift; if (added > to.chunkAdded[chunk]) to.chunkAdded[chunk] = added; if (changed > to.chunkChanged[chunk]) to.chunkChanged[chunk] = changed; diff --git a/packages/ecs/src/implementation/world/tail-blocks.ts b/packages/ecs/src/implementation/world/tail-blocks.ts index 08b80c8b..867e3ffc 100644 --- a/packages/ecs/src/implementation/world/tail-blocks.ts +++ b/packages/ecs/src/implementation/world/tail-blocks.ts @@ -6,10 +6,11 @@ import { lineWords } from './protocol.js'; // // A tail's work is numbered: position 0 is its head, which checks its conditions and resolves its resources, and // positions from 1 are the rows of its matched archetypes in order. The tail claims positions in chunks by adding to -// its progress word; the world exchanges that word for `snapshotted` once it copied the passing rows from the position -// it read, so a claim that returns `snapshotted` or more reads the rest from the copy. +// its progress word; the world adds `snapshotted` to that word once it copied the passing rows from the position it +// read, so a claim that returns `snapshotted` or more reads the rest from the copy, and the word still says whether a +// claim came before the copy. -/** The positions the tail claimed, which it adds to and the world exchanges; and the end of its work, 1 plus its rows. */ +/** The positions the tail claimed, which it and the world add to; and the end of its work, 1 plus its rows. */ export const progressWord = 0; export const endWord = 1; /** The position the tail finished reading through, which it stores after each chunk; whether the world waits for it. */ @@ -28,7 +29,10 @@ export const snapshotCapacityWord = 2 * lineWords + 6; export const tailBlockWords = 3 * lineWords; export const tailBlockBytes = tailBlockWords * 4; -/** What the progress word holds once the world copied the rest: claims add to it and stay past it. */ +/** + * What the world adds to the progress word once it copied the rest, `snapshotted` itself when no claim came first: + * claims add to it and stay past it. + */ export const snapshotted = 0x40000000; /** Positions a claim takes: the most the world waits for once a writer is ready. */ export const tailChunk = 64; diff --git a/packages/ecs/src/implementation/world/tails.test-helper.ts b/packages/ecs/src/implementation/world/tails.test-helper.ts index 80736502..4f387eab 100644 --- a/packages/ecs/src/implementation/world/tails.test-helper.ts +++ b/packages/ecs/src/implementation/world/tails.test-helper.ts @@ -77,7 +77,7 @@ export const advance = System.create({ name: "tailsTest/advance", components: { row[word] = wordOf(value); }); /** On the world's thread: every row's word as another, of the size of the first ticks', so their records are reused. */ -export const reword = System.create({ name: "tailsTest/reword", input: { type: "object", properties: { round: { type: "integer" } } }, components: { write: [word] } }, (row, { input }) => { +export const reword = System.create({ name: "tailsTest/reword", components: { write: [word] } }, (row, _context, input: { readonly round: number }) => { row[word] = `r${input.round}`; }); /** On the world's thread: spawns rows stamped with this tick's counter, `spawning` of them, then no more. */ @@ -87,16 +87,16 @@ export const grow = System.create({ name: "tailsTest/grow", resources: { read: [ resources[spawning] = 0; }); /** Spawns `count` rows. */ -export const seed = System.create({ name: "tailsTest/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +export const seed = System.create({ name: "tailsTest/seed" }, ({ commands }, input: { readonly count: number }) => { for (let k = 0; k < input.count; k++) commands.spawn([stamp, spot, label, word])({ [stamp]: 0, [spot]: [0, 0, 0], [word]: wordOf(0) }); }); -export const setStall = System.create({ name: "tailsTest/setStall", input: { type: "object", properties: { mode: { type: "integer" }, pause: { type: "number" } } }, resources: { write: [stall] } }, ({ input, resources }) => { +export const setStall = System.create({ name: "tailsTest/setStall", resources: { write: [stall] } }, ({ resources }, input: { readonly mode: number; readonly pause: number }) => { resources[stall] = { mode: input.mode, pause: input.pause }; }); -export const setSpawning = System.create({ name: "tailsTest/setSpawning", input: { type: "object", properties: { count: { type: "integer" } } }, resources: { write: [spawning] } }, ({ input, resources }) => { +export const setSpawning = System.create({ name: "tailsTest/setSpawning", resources: { write: [spawning] } }, ({ resources }, input: { readonly count: number }) => { resources[spawning] = input.count; }); -export const setFailAt = System.create({ name: "tailsTest/setFailAt", input: { type: "object", properties: { at: { type: "integer" } } }, resources: { write: [failAt] } }, ({ input, resources }) => { +export const setFailAt = System.create({ name: "tailsTest/setFailAt", resources: { write: [failAt] } }, ({ resources }, input: { readonly at: number }) => { resources[failAt] = input.at; }); diff --git a/packages/ecs/src/implementation/world/tails.ts b/packages/ecs/src/implementation/world/tails.ts index 012667d8..8b28ba92 100644 --- a/packages/ecs/src/implementation/world/tails.ts +++ b/packages/ecs/src/implementation/world/tails.ts @@ -287,11 +287,11 @@ const filtersOf = (tail: Copied, archetype: Archetype): Filter[] => { const filters: Filter[] = []; for (const id of tail.added) { const storage = archetype.storageById[id]; - if (storage !== undefined) filters.push({ ticks: storage.added, chunks: storage.chunkAdded }); + if (storage !== undefined) filters.push({ ticks: storage.added, chunks: storage.chunkAdded, blocks: storage.blockAdded }); } for (const id of tail.changed) { const storage = archetype.storageById[id]; - if (storage !== undefined) filters.push({ ticks: storage.changed, chunks: storage.chunkChanged }); + if (storage !== undefined) filters.push({ ticks: storage.changed, chunks: storage.chunkChanged, blocks: storage.blockChanged }); } return filters; }; @@ -399,7 +399,7 @@ const markOwn = (state: State, tail: Copied, claimed: number): void => { /** * Before a writer of what a tail reads: unless it read everything, copies the rest from the position it reached, - * exchanges its progress for `snapshotted`, and waits for the chunk it claimed. Null when nothing is left to wait for. + * adds `snapshotted` to its progress, and waits for the chunk it claimed. Null when nothing is left to wait for. */ const resolve = (state: State, tail: Tail): Promise | null => { settledTail(state, tail); @@ -407,8 +407,8 @@ const resolve = (state: State, tail: Tail): Promise | null => { }; /** - * Unless the reader read everything, copies the rest of what it reads from the position it reached, exchanges its - * progress for `snapshotted`, and waits for the chunk it claimed. Null when nothing is left to wait for. + * Unless the reader read everything, copies the rest of what it reads from the position it reached, adds `snapshotted` + * to its progress, and waits for the chunk it claimed. Null when nothing is left to wait for. */ export const copyRest = (state: State, tail: Copied): Promise | null => { const { control, end } = tail; @@ -417,7 +417,10 @@ export const copyRest = (state: State, tail: Copied): Promise | null => { const from = Atomics.load(control, progressWord); runs.length = 0; if (from < end) writeSnapshot(state, tail, from); - const claimed = Atomics.exchange(control, progressWord, snapshotted); + // Added rather than exchanged, so a claim before the copy stays in the word: a post a claim reached is never taken + // for one none did (`cancel` in joined-observer.ts). Claims past it, a chunk at a time, keep the word below + // `cancelled` there, `snapshotted` plus at most the end and a chunk, as ends are far below their difference. + const claimed = Atomics.add(control, progressWord, snapshotted); tailWriters.copies++; // Rows the tail claimed while this copied it reads where they are. if (claimed < end) markOwn(state, tail, claimed); diff --git a/packages/ecs/src/implementation/world/target.test.ts b/packages/ecs/src/implementation/world/target.test.ts index a8364fe8..1a9a1c95 100644 --- a/packages/ecs/src/implementation/world/target.test.ts +++ b/packages/ecs/src/implementation/world/target.test.ts @@ -14,16 +14,14 @@ declare module "@adobe/ecs" { Component.register(amount, { type: { type: "number" } }); Component.register(tagged, { type: { const: true } }); -const one = { type: "object", properties: { entity: Entity.schema } } as const; - let spawned: Entity[] = []; const spawn = System.create({ name: "targetTest/spawn" }, ({ commands }) => { spawned = [commands.spawn([amount, tagged])({ [amount]: 1 }), commands.spawn([amount, tagged])({ [amount]: 2 }), commands.spawn([amount])({ [amount]: 3 })]; }); -const bump = System.create({ name: "targetTest/bump", input: one, target: "entity", components: { write: [amount], has: [tagged] } }, row => { +const bump = System.create({ name: "targetTest/bump", target: "entity", components: { write: [amount], has: [tagged] } }, (row, _context, input: { readonly entity: Entity }) => { row[amount] += 10; }); -const remove = System.create({ name: "targetTest/remove", input: one, target: "entity", components: { has: [tagged] } }, (row, { commands }) => { +const remove = System.create({ name: "targetTest/remove", target: "entity", components: { has: [tagged] } }, (row, { commands }, _input: { readonly entity: Entity }) => { commands.despawn(row.entity); }); let amounts: number[] = []; @@ -66,9 +64,9 @@ describe("systems on one entity", () => { it("take their entity from an entity field of their input, and have components, no kernel, up, order or reduce", () => { // @ts-expect-error its target isn't an entity field - expect(() => System.create({ name: "targetTest/notEntity", input: { type: "object", properties: { count: { type: "number" } } }, target: "count", components: { has: [amount] } }, () => {})) + expect(() => System.create({ name: "targetTest/notEntity", target: "count", components: { has: [amount] } }, (_row, _context, input: { readonly count: number }) => {})) .toThrow('System "targetTest/notEntity" runs on one entity, so it takes it from an input field "count" holding an entity.'); // @ts-expect-error it needs components - expect(() => System.create({ name: "targetTest/noComponents", input: one, target: "entity" }, () => {})).toThrow('System "targetTest/noComponents" runs on one entity, so it needs components.'); + expect(() => System.create({ name: "targetTest/noComponents", target: "entity" }, (_context, input: { readonly entity: Entity }) => {})).toThrow('System "targetTest/noComponents" runs on one entity, so it needs components.'); }); }); diff --git a/packages/ecs/src/implementation/world/thread-directory.ts b/packages/ecs/src/implementation/world/thread-directory.ts index 220fc0a1..95008772 100644 --- a/packages/ecs/src/implementation/world/thread-directory.ts +++ b/packages/ecs/src/implementation/world/thread-directory.ts @@ -6,10 +6,11 @@ import { emptyViews } from './arrays.js'; import { tableLength } from './entities.js'; import { archetypeField, capacityField, cursorField, entitiesVersionField, entitiesWord, freeCapacityField, freeField, generationField, linearFlag, - recordCapacity, recordCount, recordEntities, recordFlags, recordStorages, recordVersion, recordWords, metaFlag, remoteFlag, resourceEntityWord, resourceFlag, - rowField, storageAdded, storageChanged, storageChunkAdded, storageChunkChanged, storageChunks, storageColumn, storageComponent, storageFlags, - storageWords, tableLengthWord, tableOffsetWord, threadShift, tickedFlag, + recordCapacity, recordCount, recordEntities, recordFlags, recordStorages, recordVersion, recordWords, metaFlag, remoteFlag, replacedWord, resourceEntityWord, resourceFlag, + rowField, storageAdded, storageBlockAdded, storageBlockChanged, storageChanged, storageChunkAdded, storageChunkChanged, storageChunks, storageColumn, + storageComponent, storageFlags, storageWords, tableLengthWord, tableOffsetWord, threadShift, tickedFlag, } from './layout-directory.js'; +import { blockShift, chunkShift } from './kernel.js'; import { coveringBuffer, pageBytes, pagesWord } from './memory.js'; import { createStringReader } from './strings.js'; import type { Mirror, Mirrored } from './mirror.js'; @@ -66,12 +67,38 @@ export const createThreadDirectory = (memory: WebAssembly.Memory, mirror: Mirror const thread = mirror?.thread ?? -1; /** The records strings' columns name, which this thread reads. */ const strings = createStringReader(memory); + /** + * Moved on whenever a view `archetypeAt` gave a row may have been made again: the world replaced an archetype's + * arrays, the views were dropped, or the mirror rebuilt. Index lookups' rows stay bound while it stays the same, so + * they check it rather than each archetype for every row. A mirror that begins holding an archetype needs no move: + * it does from the first entry naming it, replayed before any run reads a row there. + */ + const replaced = { count: 0 }; + /** The directory's `replacedWord`, and the mirror's archetypes, as this thread last saw them. */ + let replacedSeen = 0; + let mirroredSeen: readonly (Mirrored | null | undefined)[] = noMirrored; + + /** + * Moves `replaced` on if any archetype's view may have changed since it last looked: once a run, before it reads + * rows, since the world replaces arrays only at a sync point or a move outside one, which wait for a run in progress. + */ + const settle = (): void => { + const word = Atomics.load(viewed.words, replacedWord); + const mirrored = mirror?.archetypes ?? noMirrored; + if (word === replacedSeen && mirrored === mirroredSeen) return; + replacedSeen = word; + mirroredSeen = mirrored; + replaced.count++; + }; - /** Views of the memory again, once it grew; every archetype's views with them. Returns its buffer. */ + /** Views of the memory again, once it grew, every archetype's with them, and `replaced` settled. Returns its buffer. */ const refresh = (): ArrayBufferLike => { if (viewed.words.length === 0) viewed.words = new Uint32Array(memory.buffer); const current = Atomics.load(viewed.words, pagesWord); - if (current === pages) return viewed.buffer; + if (current === pages) { + settle(); + return viewed.buffer; + } const latest = coveringBuffer(memory, current * pageBytes); if (latest.byteLength < current * pageBytes) throw new Error(`This thread's memory is ${latest.byteLength} bytes, short of the ${current * pageBytes} the world stored.`); viewed.buffer = latest; @@ -79,7 +106,10 @@ export const createThreadDirectory = (memory: WebAssembly.Memory, mirror: Mirror viewed.views = viewsOf(latest, 0, latest.byteLength); pages = current; archetypes.length = 0; + // Defensive: rows bound to the old views still read the same shared memory, and arrays that moved bumped `replacedWord`. + replaced.count++; entitiesVersion = -1; + settle(); return latest; }; @@ -133,6 +163,8 @@ export const createThreadDirectory = (memory: WebAssembly.Memory, mirror: Mirror changed: layout.kind === "constant" ? added : changed, chunkAdded: ticked ? new Float64Array(buffer, words[base + storageChunkAdded], chunks) : noTicks, chunkChanged: ticked ? new Float64Array(buffer, words[layout.kind === "constant" ? base + storageChunkAdded : base + storageChunkChanged], chunks) : noTicks, + blockAdded: ticked ? new Float64Array(buffer, words[base + storageBlockAdded], chunks << (chunkShift - blockShift)) : noTicks, + blockChanged: ticked ? new Float64Array(buffer, words[layout.kind === "constant" ? base + storageBlockAdded : base + storageBlockChanged], chunks << (chunkShift - blockShift)) : noTicks, }; }; @@ -191,6 +223,8 @@ export const createThreadDirectory = (memory: WebAssembly.Memory, mirror: Mirror layouts[number] = layoutOf(declarationOf(key)?.type); // Views hold storages only of components with layouts here, so they're made again. archetypes.length = 0; + // Defensive: a lookup's read keys are named before its runner and pool exist, so no bound row lacks a storage. + replaced.count++; } } }; @@ -202,7 +236,7 @@ export const createThreadDirectory = (memory: WebAssembly.Memory, mirror: Mirror }; return { - viewed, rows, keys, layouts, numbers, mirror, strings, + viewed, rows, keys, layouts, numbers, mirror, strings, replaced, refresh, refreshEntities, archetypeAt, countOf, resourceRow, tableCount, name, numberOf, layoutOf: (key: Key): Layout | undefined => layouts[numbers.get(firstKeyOf(key)) ?? -1], }; diff --git a/packages/ecs/src/implementation/world/thread-finalizers.ts b/packages/ecs/src/implementation/world/thread-finalizers.ts index 0c42a704..94ff200c 100644 --- a/packages/ecs/src/implementation/world/thread-finalizers.ts +++ b/packages/ecs/src/implementation/world/thread-finalizers.ts @@ -6,12 +6,12 @@ import { writeOf } from './archetype.js'; import { emptyViews } from './arrays.js'; import { readCell } from './cells.js'; import { logFinalizedWord, logStopsWord, nullTag, ownTag, slotTags, startTag } from './move-log.js'; -import { stagedValue, type Mirror, type Previously } from './mirror.js'; +import { resourceValueOf, stagedValue, type Mirror, type Previously } from './mirror.js'; import type { WorkerError } from './protocol.js'; import { rowBinderOf, type Row, type RowArchetype, type RowHost, type RowStorage } from './rows.js'; import { archetypesAt, blockGroupsWord, blockHeaderBytes, blockNextWord, blockSyncWord, columnBytes, entitiesAt, groupBytesWord, groupCountWord, - groupIndexWord, rowsAt, tagsAt, valuesAt, + groupIndexWord, resourceOffsetsOf, rowsAt, tagsAt, valuesAt, } from './stop-lists.js'; import type { Key } from './state.js'; import type { StringStore } from './strings.js'; @@ -63,14 +63,17 @@ type ThreadFinalizer = { readonly constantLayouts: readonly Layout[]; readonly bind: (archetype: RowArchetype) => Row; readonly context: object; - /** Resolves its resources' cells, as of now. */ - readonly cells: () => void; + /** + * Resolves its resources: its thread's objects as it holds them, and the rest from the copies a group holds from + * byte `copies`, as before its stops' sync point applied, or with -1, once the world is disposed, where they are. + */ + readonly cells: (copies: number) => void; readonly body: (row: object, context: object) => void; }; /** A storage a stop list's column holds, for a row over its stops. */ const columnStorage = (component: number, layout: Layout, views: RowStorage["views"], values: unknown[], strings: StringStore | null): FinalizedStorage => ({ - component, layout, views, strings: layout.kind === "string" ? strings : null, values, ...writeOf(layout), ticked: false, added: noTicks, changed: noTicks, chunkAdded: noTicks, chunkChanged: noTicks, tags: noTags, + component, layout, views, strings: layout.kind === "string" ? strings : null, values, ...writeOf(layout), ticked: false, added: noTicks, changed: noTicks, chunkAdded: noTicks, chunkChanged: noTicks, blockAdded: noTicks, blockChanged: noTicks, tags: noTags, }); /** @@ -128,11 +131,44 @@ export const createThreadFinalizers = (host: Threa const keys = [...new Set(resources?.read ?? [])]; const numbers = keys.map(host.numberOf); const resolved: { storage: FinalizedStorage | undefined; row: number }[] = keys.map(() => ({ storage: undefined, row: 0 })); + // Its thread's object resources are read from the mirror, which may be behind the entity table as it replays. + const own = keys.map(key => host.layoutOf(key)?.kind === "object"); + const resourceStarts = numbers.map(number => mirror.starts.get(number) ?? null); + const ownValues: unknown[] = keys.map(() => undefined); + const shared = own.includes(false); + // The world copies the rest for each group, in this order: by slot, its strings' handles, then its linear + // values, each with its key's index and layout. A constant needs no copy. + const stringKeys: number[] = []; + const stringLayouts: Layout[] = []; + const linearKeys: number[] = []; + const linearLayouts: Layout[] = []; + const linearStrides: number[] = []; + const constantKeys: number[] = []; + const constantStorages: FinalizedStorage[] = []; + keys.forEach((key, k) => { + const layout = host.layoutOf(key); + if (layout === undefined || layout.kind === "object") return; + if (layout.kind === "string") { + stringKeys.push(k); + stringLayouts.push(layout); + } + else if (layout.kind === "linear") { + linearKeys.push(k); + linearLayouts.push(layout); + linearStrides.push(layout.stride); + } + else { + constantKeys.push(k); + constantStorages.push(columnStorage(numbers[k], layout, emptyViews, [], null)); + } + }); + const resourceOffsets = resourceOffsetsOf(stringKeys.length, linearStrides); if (keys.length > 0) { const values: object = Object.create({}); keys.forEach((key, k) => { Object.defineProperty(values, key, { get: () => { + if (own[k]) return ownValues[k]; const { storage, row } = resolved[k]; if (storage === undefined) throw new Error(`Resource "${labelOf(key)}" isn't in this world.`); return readCell(storage, row); @@ -150,11 +186,36 @@ export const createThreadFinalizers = (host: Threa excluded: (components.not ?? []).map(host.numberOf), bind: rowBinderOf(host.rows, host.numberOf, reads, []), context: value, - cells: () => { - const found = keys.length === 0 ? null : host.resourceRow(); - for (let k = 0; k < keys.length; k++) { - resolved[k].storage = found?.archetype.storageById[numbers[k]]; - resolved[k].row = found?.row ?? 0; + cells: copies => { + // Before a rebuild too, its old rows are the mirror's, which hold what the rebuild keeps. + for (let k = 0; k < keys.length; k++) if (own[k]) ownValues[k] = resourceValueOf(mirror, numbers[k], resourceStarts[k]); + if (!shared) return; + if (copies < 0) { + const found = host.resourceRow(); + for (let k = 0; k < keys.length; k++) { + if (own[k]) continue; + resolved[k].storage = found?.archetype.storageById[numbers[k]]; + resolved[k].row = found?.row ?? 0; + } + return; + } + if (stringKeys.length > 0) { + // One view over every handle, a slot each. + const views = viewsOf(buffer, copies, 4 * stringKeys.length); + for (let slot = 0; slot < stringKeys.length; slot++) { + const k = stringKeys[slot]; + resolved[k].storage = columnStorage(numbers[k], stringLayouts[slot], views, [], host.strings); + resolved[k].row = slot; + } + } + for (let slot = 0; slot < linearKeys.length; slot++) { + const k = linearKeys[slot]; + resolved[k].storage = columnStorage(numbers[k], linearLayouts[slot], viewsOf(buffer, copies + resourceOffsets[slot], linearStrides[slot]), [], null); + resolved[k].row = 0; + } + for (let slot = 0; slot < constantKeys.length; slot++) { + resolved[constantKeys[slot]].storage = constantStorages[slot]; + resolved[constantKeys[slot]].row = 0; } }, body: (row, context) => untyped.body(row, context), @@ -236,7 +297,7 @@ export const createThreadFinalizers = (host: Threa } for (let k = 0; k < finalizer.constants.length; k++) storageById[finalizer.constants[k]] = columnStorage(finalizer.constants[k], finalizer.constantLayouts[k], emptyViews, [], null); const row = finalizer.bind({ id: -1, entities: new Float64Array(buffer, entitiesAt(group), count), storageById }); - finalizer.cells(); + finalizer.cells(column); for (let i = 0; i < count; i++) { row.index = i; guarded(index, finalizer, row); @@ -297,7 +358,7 @@ export const createThreadFinalizers = (host: Threa for (let index = 0; index < finalizers.length; index++) { const finalizer = finalizers[index]; if (finalizer === null || finalizer === undefined) continue; - finalizer.cells(); + finalizer.cells(-1); for (let id = 0; id < host.tableCount(); id++) { const archetype = host.archetypeAt(id); if (archetype === undefined || archetype.resource || (archetype.meta && finalizer.required.length === 0)) continue; diff --git a/packages/ecs/src/implementation/world/thread-indexes.ts b/packages/ecs/src/implementation/world/thread-indexes.ts index 27c224a4..96dc2a05 100644 --- a/packages/ecs/src/implementation/world/thread-indexes.ts +++ b/packages/ecs/src/implementation/world/thread-indexes.ts @@ -39,19 +39,20 @@ const u32From = (buffer: ArrayBufferLike, at: number): Uint32Array => new Uint32 const i32From = (buffer: ArrayBufferLike, at: number): Int32Array => new Int32Array(buffer, at); const f64From = (buffer: ArrayBufferLike, at: number): Float64Array => new Float64Array(buffer, at); -export const createThreadIndex = (rows: RowHost, archetypeAt: (id: number) => LookupArchetype | undefined, setup: ThreadIndexSetup): ThreadIndex => { +export const createThreadIndex = (rows: RowHost, archetypeAt: (id: number) => LookupArchetype | undefined, replaced: { readonly count: number }, setup: ThreadIndexSetup): ThreadIndex => { const { parts, words } = keyPartsOf(setup.layouts); const watchedPositions = Array.from({ length: setup.watched }, (_, k) => k); const hashWatched = new Map(); const sortedWatched = new Map(); - const hash: HashView | null = setup.sorted ? null : { words, stride: recordKey + words, slots: none32, records: none32, ticks: none64, cleared: 0, entries: noneI32, watched: hashWatched }; + // The world sorts a hash index's buckets before writing its record, so this view has none unsorted. + const hash: HashView | null = setup.sorted ? null : { words, stride: recordKey + words, slots: none32, records: none32, ticks: none64, cleared: 0, entries: noneI32, unsorted: 0, watched: hashWatched }; const sorted: SortedView | null = !setup.sorted ? null : { words, stride: sortedStride(words), parts, objects: false, entries: none32, lengths: noneI32, order: noneI32, blocks: 0, counts: noneI32, tree: none64, leaves: 1, fences: none32, size: 0, cleared: 0, watched: sortedWatched, }; const index: IndexView = { parts, words, unique: setup.unique, hash, sorted }; const since: Since = { value: 0 }; - const lookup = lookupFunction({ rows, archetypeAt, index, unique: setup.unique, reads: setup.reads, readIds: setup.readIds, watched: watchedPositions, since, replaced: null }); + const lookup = lookupFunction({ rows, archetypeAt, index, unique: setup.unique, reads: setup.reads, readIds: setup.readIds, watched: watchedPositions, since, replaced }); const count = recordWordsOf(setup.sorted, setup.watched); let record: Float64Array = none64; const refresh = (buffer: ArrayBufferLike): void => { diff --git a/packages/ecs/src/implementation/world/thread-observer.ts b/packages/ecs/src/implementation/world/thread-observer.ts index ea6ecb2f..572864e3 100644 --- a/packages/ecs/src/implementation/world/thread-observer.ts +++ b/packages/ecs/src/implementation/world/thread-observer.ts @@ -73,7 +73,7 @@ export const createThreadObserver = (host: ThreadObserverHost) => { const runnerHost: RunnerHost = { rows: directory.rows, archetypeAt: directory.archetypeAt, countOf: directory.countOf, tableCount: directory.tableCount, resourceRow: directory.resourceRow, refreshEntities: directory.refreshEntities, commands: noCommands, - numberOf: directory.numberOf, layoutOf: directory.layoutOf, mirror, refresh: directory.refresh, strings: directory.strings, + numberOf: directory.numberOf, layoutOf: directory.layoutOf, mirror, refresh: directory.refresh, replaced: directory.replaced, strings: directory.strings, }; const placed: (Placed | undefined)[] = []; diff --git a/packages/ecs/src/implementation/world/thread-systems.test-helper.ts b/packages/ecs/src/implementation/world/thread-systems.test-helper.ts index 93451c1a..bd995cdd 100644 --- a/packages/ecs/src/implementation/world/thread-systems.test-helper.ts +++ b/packages/ecs/src/implementation/world/thread-systems.test-helper.ts @@ -67,7 +67,7 @@ Resource.register(nextUid, { type: { type: "integer", default: 1 } }); const makeNote = (text: string): Note => ({ text, read: () => text }); /** Spawns from the world's thread: structs and its own object, the thread's objects at their starts. */ -export const seed = System.create({ name: "threadSystemsTest/seed", input: { type: "object", properties: { count: { type: "integer" } } }, resources: { write: [nextUid] } }, ({ input, resources, commands }) => { +export const seed = System.create({ name: "threadSystemsTest/seed", resources: { write: [nextUid] } }, ({ resources, commands }, input: { readonly count: number }) => { for (let k = 0; k < input.count; k++) { const id = resources[nextUid]++; commands.spawn([uid, position, velocity, caption, kept])({ [uid]: id, [position]: [id, -id], [velocity]: id % 3, [kept]: `k${id}` }); @@ -145,7 +145,7 @@ export const failing = System.create({ name: "threadSystemsTest/failing", system export const tick = System.create({ name: "threadSystemsTest/tick", systems: [move, describe, relabel, breed, tally, keep, watchCaptions, repaint, peek] }); /** On the thread, with input, which comes to it by message. */ -export const stamp = System.create({ name: "threadSystemsTest/stamp", input: { type: "object", properties: { text: { type: "string" } } }, components: { read: [uid], write: [caption] } }, (row, { input }) => { +export const stamp = System.create({ name: "threadSystemsTest/stamp", components: { read: [uid], write: [caption] } }, (row, _context, input: { readonly text: string }) => { if (row[uid] === 1) row[caption] = { text: input.text }; }); @@ -256,7 +256,7 @@ export const scenario = async (world: World, objects: Objects = objectsThere(wor }; /** Spawns entities with captions, from the world's thread. */ -export const crowd = System.create({ name: "threadSystemsTest/crowd", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +export const crowd = System.create({ name: "threadSystemsTest/crowd" }, ({ commands }, input: { readonly count: number }) => { for (let k = 0; k < input.count; k++) commands.spawn([uid, caption])({ [uid]: 1000 + k }); }); /** On the world's thread: every crowd entity but uid 1000 moves archetype, flagged on one run and not on the next. */ diff --git a/packages/ecs/src/implementation/world/thread-systems.ts b/packages/ecs/src/implementation/world/thread-systems.ts index bcb38cce..110857a9 100644 --- a/packages/ecs/src/implementation/world/thread-systems.ts +++ b/packages/ecs/src/implementation/world/thread-systems.ts @@ -30,7 +30,7 @@ import { createThreadFinalizers } from './thread-finalizers.js'; import { createThreadIndex, type ThreadIndex } from './thread-indexes.js'; import type { EntityViews, ThreadArchetype, ThreadDirectory, ThreadStorage } from './thread-directory.js'; import { keptSorted } from './index-keys.js'; -import { chunkShift } from './kernel.js'; +import { blockShift, chunkShift } from './kernel.js'; import { doneWord, endWord, headExecuted, headRan, progressWord, snapshotFlagsWord, snapshotLayout, snapshotOffsetWord, snapshotResourceArchetypeWord, snapshotResourceRowWord, snapshotRowsWord, snapshotCapacityWord, snapshotted, tailBlockWords, tailChunk, waitingWord, @@ -99,6 +99,9 @@ type CommandKey = { /** The system that ran a task's commands, for their checks' errors. */ type Running = { error: unknown }; +/** Whether a task's mirror is current, or why not (`bringCurrent`). */ +type Readiness = "current" | "moved" | "held"; + /** * Runs the systems the world places on this thread, as tasks it posts in the task block: each task's systems in order, * after replaying the mirror through the task's sync, writing each system's commands and spawns as a segment of the @@ -283,7 +286,7 @@ export const createThreadSystems = (host: ThreadSystemsHost) => { const runnerHost: RunnerHost = { rows: rowHost, archetypeAt, countOf, tableCount, resourceRow, refreshEntities, commands, mirror, - refresh: directory.refresh, + refresh: directory.refresh, replaced: directory.replaced, numberOf: directory.numberOf, layoutOf: directory.layoutOf, strings: directory.strings, @@ -359,18 +362,25 @@ export const createThreadSystems = (host: ThreadSystemsHost) => { })); }; - /** Brings the mirror up to the last published sync, which runs the finalizers of its stops; false while it can't. */ - const current = (): boolean => { + /** + * Brings the mirror up to the last published sync, which runs the finalizers of its stops: `"moved"` when its + * rebuild found a sync point applying before or during it, so it can rebuild once that's applied, and `"held"` + * while it waits for something else. + */ + const bringCurrent = (): Readiness => { const replayed = replay(mirror); - if (replayed === "held") return false; - return replayed !== "lapped" || rebuild(mirror); + if (replayed === "held") return "held"; + if (replayed !== "lapped") return "current"; + const rebuilt = rebuild(mirror); + return rebuilt === "rebuilt" ? "current" : rebuilt; }; + const current = (): boolean => bringCurrent() === "current"; /** Whether a task can run: its systems known here, its input arrived, and the mirror through its sync. */ - const ready = (payload: Float64Array, count: number): boolean => { - for (let k = 0; k < count; k++) if (runners[payload[payloadSystems + k * systemFields + systemIndex]] === undefined) return false; - if (payload[payloadInput] !== 0 && !inputs.has(payload[payloadSequence])) return false; - return current(); + const ready = (payload: Float64Array, count: number): Readiness => { + for (let k = 0; k < count; k++) if (runners[payload[payloadSystems + k * systemFields + systemIndex]] === undefined) return "held"; + if (payload[payloadInput] !== 0 && !inputs.has(payload[payloadSequence])) return "held"; + return bringCurrent(); }; /** Views of the payload and completion, kept while the world keeps them where they are. */ @@ -428,15 +438,26 @@ export const createThreadSystems = (host: ThreadSystemsHost) => { return control; }; - /** A task waiting for a sync point to finish applying, to rebuild its mirror, runs once it has. */ + /** + * A task waiting for a sync point to finish applying, to rebuild its mirror, runs once it has. One whose rebuild + * found a sync point applied during it runs again after a turn of the event loop: nothing else may wake this thread + * before the task is done, and sync points overlapping every rebuild mustn't keep it from its event loop. + */ let structureWaited = false; - const retryAfterStructure = (): void => { - if (structureWaited || !applying(mirror)) return; - structureWaited = true; - void waitStructure(mirror).then(() => { - structureWaited = false; - process(); - }); + const retried = (): void => { + structureWaited = false; + process(); + }; + const retryAfterStructure = (moved: boolean): void => { + if (structureWaited) return; + if (applying(mirror)) { + structureWaited = true; + void waitStructure(mirror).then(retried); + } + else if (moved) { + structureWaited = true; + setTimeout(retried, 0); + } }; const runTask = (posted: number): void => { @@ -445,8 +466,9 @@ export const createThreadSystems = (host: ThreadSystemsHost) => { const count = payload[payloadCount]; const sequence = payload[payloadSequence]; pending = sequence; - if (!ready(payload, count)) { - retryAfterStructure(); + const readiness = ready(payload, count); + if (readiness !== "current") { + retryAfterStructure(readiness === "moved"); return; } pending = -1; @@ -574,8 +596,10 @@ export type RunnerHost = { readonly layoutOf: (key: Key) => Layout | undefined; /** This thread's objects, null where it fills none. */ readonly mirror: Mirror | null; - /** Views of the memory again, once it grew: its buffer. */ + /** Views of the memory again, once it grew, and `replaced` moved on if a view may have: its buffer. */ readonly refresh: () => ArrayBufferLike; + /** Moved on by `refresh` once any archetype's view may have changed, which index lookups' rows check (`thread-directory.ts`). */ + readonly replaced: { readonly count: number }; /** The records strings' columns name. */ readonly strings: StringStore; }; @@ -607,7 +631,7 @@ type ThreadCell = { storage: ThreadStorage | undefined; row: number }; export const createThreadRunner = (host: RunnerHost, leaf: Leaf, records: ReadonlyMap = noRecords): ThreadRunner => { const { declaration } = leaf; // Method parameters are bivariant, so the runtime can see any leaf's body with erased parameter types. - const system: { body(subject: unknown, context?: unknown): void } = leaf; + const system: { body(subject: unknown, context?: unknown, input?: unknown): void } = leaf; const { components: query, resources } = declaration; const numberOf = host.numberOf; const required = [...new Set([...query?.read ?? [], ...query?.write ?? [], ...query?.has ?? [], ...query?.added ?? [], ...query?.changed ?? []])].map(numberOf); @@ -678,7 +702,7 @@ export const createThreadRunner = (host: RunnerHost, leaf: Leaf, records: Readon if (index === undefined || record === undefined) found = entityLookupOf(host, declared, lookupSince); else { const reads = declared.read ?? []; - const threadIndex = createThreadIndex(host.rows, host.archetypeAt, { + const threadIndex = createThreadIndex(host.rows, host.archetypeAt, host.replaced, { layouts: index.declaration.key.map(key => { const layout = host.layoutOf(key); if (layout === undefined) throw new Error(`"${labelOf(key)}" has no registered schema; import the module that registers it.`); @@ -712,14 +736,14 @@ export const createThreadRunner = (host: RunnerHost, leaf: Leaf, records: Readon }); Object.defineProperty(value, "reduce", { value: reducers, enumerable: true }); } - if (declaration.input !== undefined) value.input = undefined; // A tail is terminal, so its context has no commands. - const tailValue: { input?: unknown } = own(); + const tailValue: object = own(); for (const name of Object.getOwnPropertyNames(value)) { const descriptor = Object.getOwnPropertyDescriptor(value, name); - if (descriptor !== undefined && name !== "input") Object.defineProperty(tailValue, name, descriptor); + if (descriptor !== undefined) Object.defineProperty(tailValue, name, descriptor); } - if (declaration.input !== undefined) tailValue.input = undefined; + /** The current run's input, which the body takes after its context; kept until the next run, as its context is. */ + let runInput: unknown; Object.defineProperty(value, "commands", { value: host.commands, enumerable: true }); /** Resolves resource cells for a run: they move only at sync points. */ @@ -764,11 +788,11 @@ export const createThreadRunner = (host: RunnerHost, leaf: Leaf, records: Readon const filters: Filter[] = []; for (const id of addedIds) { const storage = archetype.storageById[id]; - if (storage !== undefined) filters.push({ ticks: storage.added, chunks: storage.chunkAdded }); + if (storage !== undefined) filters.push({ ticks: storage.added, chunks: storage.chunkAdded, blocks: storage.blockAdded }); } for (const id of changedIds) { const storage = archetype.storageById[id]; - if (storage !== undefined) filters.push({ ticks: storage.changed, chunks: storage.chunkChanged }); + if (storage !== undefined) filters.push({ ticks: storage.changed, chunks: storage.chunkChanged, blocks: storage.blockChanged }); } filterCache.set(archetype.id, { archetype, filters }); return filters; @@ -777,18 +801,19 @@ export const createThreadRunner = (host: RunnerHost, leaf: Leaf, records: Readon const runBody = (archetype: ThreadArchetype, count: number, since: number): void => { const row = bindRow(archetype); const subject = row.value; + const input = runInput; const filters = filtersOf(archetype); if (since < 0 || filters.length === 0) { for (let index = 0; index < count; index++) { row.index = index; - system.body(subject, value); + system.body(subject, value, input); } return; } forEachPassingRange(filters, since, 0, count, (start, end) => { for (let index = start; index < end; index++) { row.index = index; - system.body(subject, value); + system.body(subject, value, input); } }); }; @@ -809,13 +834,14 @@ export const createThreadRunner = (host: RunnerHost, leaf: Leaf, records: Readon const since = !filtered || resourcesChanged ? -1 : lastRun; bind(); if (threadIndexes.length !== 0) { + // Settles the count after `ready()` or `current()` may have rebuilt the mirror, since `runTask` settles before them. const buffer = host.refresh(); for (let k = 0; k < threadIndexes.length; k++) threadIndexes[k].refresh(buffer); } - if (declaration.input !== undefined) value.input = ownInput(input); + runInput = ownInput(input); if (query === undefined) { outcome.ran = since < 0; - if (outcome.ran) system.body(value); + if (outcome.ran) system.body(value, runInput); return outcome; } if (since >= 0 && addedIds.length + changedIds.length === 0) return outcome; @@ -867,18 +893,19 @@ export const createThreadRunner = (host: RunnerHost, leaf: Leaf, records: Readon const runRows = (archetype: ThreadArchetype, start: number, end: number, since: number): void => { const row = bindRow(archetype); const subject = row.value; + const input = runInput; const filters = filtersOf(archetype); if (since < 0 || filters.length === 0) { for (let index = start; index < end && !halted; index++) { row.index = index; - system.body(subject, tailValue); + system.body(subject, tailValue, input); } return; } forEachPassingRange(filters, since, start, end, (first, last) => { for (let index = first; index < last && !halted; index++) { row.index = index; - system.body(subject, tailValue); + system.body(subject, tailValue, input); } }); }; @@ -954,10 +981,10 @@ export const createThreadRunner = (host: RunnerHost, leaf: Leaf, records: Readon if ((flags & headExecuted) === 0) return; outcome.executed = true; outcome.ran = (flags & headRan) !== 0; - if (declaration.input !== undefined) tailValue.input = ownInput(input); + runInput = ownInput(input); if (!outcome.ran) return; if (query === undefined) { - system.body(tailValue); + system.body(tailValue, runInput); writeBackResources(ownResources, resourceArchetype, resourceRow, tick, at, stamps); return; } @@ -990,10 +1017,11 @@ export const createThreadRunner = (host: RunnerHost, leaf: Leaf, records: Readon while (first < rows && positions[first] < position) first++; const row = bindRow({ id: -1, entities, storageById, record: -1, version: -1, mirrored: null, components: [], resource: false, meta: false }); const subject = row.value; + const ownedInput = runInput; try { for (let index = first; index < rows && !halted; index++) { row.index = index; - system.body(subject, tailValue); + system.body(subject, tailValue, ownedInput); } } finally { @@ -1052,10 +1080,10 @@ export const createThreadRunner = (host: RunnerHost, leaf: Leaf, records: Readon const resourcesChanged = resourceChanged.length > 0 && resourcesChangedSince(lastRun); const since = !filtered || resourcesChanged ? -1 : lastRun; bind(); - if (declaration.input !== undefined) tailValue.input = ownInput(input); + runInput = ownInput(input); if (query === undefined) { outcome.ran = since < 0; - if (outcome.ran) system.body(tailValue); + if (outcome.ran) system.body(tailValue, runInput); return outcome; } if (since >= 0 && addedIds.length + changedIds.length === 0) return outcome; @@ -1106,10 +1134,12 @@ type Columned = Extract; /** A storage over a snapshot's column, or of values only this thread holds, with ticks of its own for `ticked` rows. */ const localStorage = (component: number, layout: Layout, views: Views, values: unknown[], ticked: number, strings: StringStore): ThreadStorage => { const chunks = ticked === 0 ? 0 : (ticked >> chunkShift) + 1; + const blocks = ticked === 0 ? 0 : (ticked >> blockShift) + 1; return { component, layout, views, strings: layout.kind === "string" ? strings : null, values, ...writeOf(layout), tags: noTags, ticked: ticked > 0, added: ticked === 0 ? noTicks : new Float64Array(ticked), changed: ticked === 0 ? noTicks : new Float64Array(ticked), chunkAdded: chunks === 0 ? noTicks : new Float64Array(chunks), chunkChanged: chunks === 0 ? noTicks : new Float64Array(chunks), + blockAdded: blocks === 0 ? noTicks : new Float64Array(blocks), blockChanged: blocks === 0 ? noTicks : new Float64Array(blocks), }; }; diff --git a/packages/ecs/src/implementation/world/threads.browser.test.ts b/packages/ecs/src/implementation/world/threads.browser.test.ts index a7c89510..04e8181c 100644 --- a/packages/ecs/src/implementation/world/threads.browser.test.ts +++ b/packages/ecs/src/implementation/world/threads.browser.test.ts @@ -6,6 +6,7 @@ import { poolSizeCases } from "./pool-size.cases.js"; import { dispatchCases } from "./dispatch.cases.js"; import { reductionsCases } from "./reductions.cases.js"; import { observedWorkersCases } from "./observed-workers.cases.js"; +import { changeBlocksCases } from "./change-blocks.cases.js"; import { upWorkersCases } from "./up-workers.cases.js"; import { portsCases } from "./ports.cases.js"; import { commandRingsCases } from "./command-rings.cases.js"; @@ -30,6 +31,7 @@ poolSizeCases(() => import("./pool-size.fixture.js")); dispatchCases(() => import("./dispatch.fixture.js")); reductionsCases(() => import("./reductions.fixture.js")); observedWorkersCases(() => import("./observed-workers.fixture.js")); +changeBlocksCases(() => import("./change-blocks.fixture.js")); upWorkersCases(() => import("./up-workers.fixture.js")); portsCases(() => import("./ports.fixture.js")); commandRingsCases(() => import("./command-rings.fixture.js")); diff --git a/packages/ecs/src/implementation/world/threads.test.ts b/packages/ecs/src/implementation/world/threads.test.ts index c1ccb4fa..3581f207 100644 --- a/packages/ecs/src/implementation/world/threads.test.ts +++ b/packages/ecs/src/implementation/world/threads.test.ts @@ -14,6 +14,8 @@ import type * as Reductions from "./reductions.fixture.js"; import { reductionsCases } from "./reductions.cases.js"; import type * as ObservedWorkers from "./observed-workers.fixture.js"; import { observedWorkersCases } from "./observed-workers.cases.js"; +import type * as ChangeBlocks from "./change-blocks.fixture.js"; +import { changeBlocksCases } from "./change-blocks.cases.js"; import type * as UpWorkers from "./up-workers.fixture.js"; import { upWorkersCases } from "./up-workers.cases.js"; import type * as Ports from "./ports.fixture.js"; @@ -56,6 +58,7 @@ poolSizeCases((): Promise => builtFixture("pool-size")); dispatchCases((): Promise => builtFixture("dispatch")); reductionsCases((): Promise => builtFixture("reductions")); observedWorkersCases((): Promise => builtFixture("observed-workers")); +changeBlocksCases((): Promise => builtFixture("change-blocks")); upWorkersCases((): Promise => builtFixture("up-workers")); portsCases((): Promise => builtFixture("ports")); commandRingsCases((): Promise => builtFixture("command-rings")); diff --git a/packages/ecs/src/implementation/world/ticks.test.ts b/packages/ecs/src/implementation/world/ticks.test.ts index 7b2cad3a..7ef52049 100644 --- a/packages/ecs/src/implementation/world/ticks.test.ts +++ b/packages/ecs/src/implementation/world/ticks.test.ts @@ -18,7 +18,7 @@ const count = System.create({ name: "ticksTest/count", resources: { write: [fram describe("ticks", () => { it("reject a system that conflicts with one added with no order between them, and one with input", async () => { const other = System.create({ name: "ticksTest/other", resources: { write: [frames] } }, ({ resources }) => { resources[frames] = 0; }); - const withInput = System.create({ name: "ticksTest/withInput", input: { type: "object", properties: { by: { type: "number" } } } }, () => {}); + const withInput = System.create({ name: "ticksTest/withInput" }, (_context, input: { readonly by: number }) => {}); await using world = createWorld({ [workers]: 0 }); world.add(count); expect(() => world.add(other)).toThrow('both access "ticksTest/frames"'); diff --git a/packages/ecs/src/implementation/world/transfers.test-helper.ts b/packages/ecs/src/implementation/world/transfers.test-helper.ts index ba9fad9b..7614da63 100644 --- a/packages/ecs/src/implementation/world/transfers.test-helper.ts +++ b/packages/ecs/src/implementation/world/transfers.test-helper.ts @@ -58,40 +58,40 @@ const store = (value: unknown, resources: { [order]: number }): void => { }; /** On "transfersTest": records the buffer it's given. */ -export const storeThere = System.create({ name: "transfersTest/storeThere", input: { type: "object", properties: { buffer: any } }, resources: { read: [anchor], write: [order] } }, ({ input, resources }) => { +export const storeThere = System.create({ name: "transfersTest/storeThere", resources: { read: [anchor], write: [order] } }, ({ resources }, input: { readonly buffer: unknown }) => { if (resources[anchor].text === "anchor") store(input.buffer, resources); }); /** On the world's thread: records the buffer it's given. */ -export const storeHere = System.create({ name: "transfersTest/storeHere", input: { type: "object", properties: { buffer: any } }, resources: { write: [order] } }, ({ input, resources }) => { +export const storeHere = System.create({ name: "transfersTest/storeHere", resources: { write: [order] } }, ({ resources }, input: { readonly buffer: unknown }) => { store(input.buffer, resources); }); /** On the world's thread: what a run from a joined thread saw of the order, by its value. */ export const marks: { readonly value: number; readonly order: number }[] = []; -export const mark = System.create({ name: "transfersTest/mark", input: { type: "object", properties: { value: { type: "number" } } }, resources: { write: [order] } }, ({ input, resources }) => { +export const mark = System.create({ name: "transfersTest/mark", resources: { write: [order] } }, ({ resources }, input: { readonly value: number }) => { marks.push({ value: input.value, order: resources[order]++ }); }); /** On "transfersTest": gives an entity the port or bitmap it's given. */ export const attachThere = System.create({ - name: "transfersTest/attachThere", input: { type: "object", properties: { entity: Entity.schema, value: any } }, resources: { read: [anchor] }, -}, ({ input, resources, commands }) => { + name: "transfersTest/attachThere", resources: { read: [anchor] }, +}, ({ resources, commands }, input: { readonly entity: Entity; readonly value: unknown }) => { if (resources[anchor].text !== "anchor") return; kept.push(input.value); if (input.value instanceof MessagePort) commands.add(input.entity, port, input.value); else if (typeof ImageBitmap === "function" && input.value instanceof ImageBitmap) commands.add(input.entity, bitmap, input.value); }); /** On the world's thread: gives an entity the port or bitmap it's given. */ -export const attachHere = System.create({ name: "transfersTest/attachHere", input: { type: "object", properties: { entity: Entity.schema, value: any } } }, ({ input, commands }) => { +export const attachHere = System.create({ name: "transfersTest/attachHere" }, ({ commands }, input: { readonly entity: Entity; readonly value: unknown }) => { kept.push(input.value); if (input.value instanceof MessagePort) commands.add(input.entity, portHere, input.value); else if (typeof ImageBitmap === "function" && input.value instanceof ImageBitmap) commands.add(input.entity, bitmapHere, input.value); }); /** On "transfersTest": says hello through the port it's given, which only moving it there, not copying it, gives. */ -export const greetThere = System.create({ name: "transfersTest/greetThere", input: { type: "object", properties: { port: any } }, resources: { read: [anchor] } }, ({ input, resources }) => { +export const greetThere = System.create({ name: "transfersTest/greetThere", resources: { read: [anchor] } }, ({ resources }, input: { readonly port: unknown }) => { if (input.port instanceof MessagePort) input.port.postMessage(resources[anchor].text); }); /** On the world's thread: spawns an entity holding the port it's given. */ -export const spawnHere = System.create({ name: "transfersTest/spawnHere", input: { type: "object", properties: { value: any } } }, ({ input, commands }) => { +export const spawnHere = System.create({ name: "transfersTest/spawnHere" }, ({ commands }, input: { readonly value: unknown }) => { if (input.value instanceof MessagePort) commands.spawn([portHere])({ [portHere]: input.value }); }); @@ -100,8 +100,8 @@ export const spawnHere = System.create({ name: "transfersTest/spawnHere", input: * "transfersTest", it fails to register, so a run of it never runs. */ export const unregistered = System.create({ - name: "transfersTest/unregistered", input: { type: "object", properties: { value: any } }, components: { read: [port], write: [depth] }, up: { through: parent, read: [depth] }, -}, (row, { input, up }) => { + name: "transfersTest/unregistered", components: { read: [port], write: [depth] }, up: { through: parent, read: [depth] }, +}, (row, { up }, input: { readonly value: unknown }) => { row[depth] = (up === null ? 0 : up[depth]) + (row[port] === input.value ? 1 : 0); }); diff --git a/packages/ecs/src/implementation/world/up.test.ts b/packages/ecs/src/implementation/world/up.test.ts index e021ed25..27ff139d 100644 --- a/packages/ecs/src/implementation/world/up.test.ts +++ b/packages/ecs/src/implementation/world/up.test.ts @@ -90,7 +90,7 @@ const spawn = System.create({ name: "upTest/spawn" }, ({ commands }) => { entities.other = commands.spawn([label, local, world])({ [label]: "other", [local]: 1000, [world]: 0 }); }); const nest = System.create({ name: "upTest/nest" }, ({ commands }) => { commands.add(entities.grandchild, parent, entities.child); }); -const bump = System.create({ name: "upTest/bump", input: { type: "object", properties: { entity: Entity.schema } } }, ({ input, commands }) => { +const bump = System.create({ name: "upTest/bump" }, ({ commands }, input: { readonly entity: Entity }) => { commands.add(input.entity, local, 2); }); const loop = System.create({ name: "upTest/loop" }, ({ commands }) => { @@ -101,6 +101,12 @@ const loop = System.create({ name: "upTest/loop" }, ({ commands }) => { const close = System.create({ name: "upTest/close" }, ({ commands }) => { commands.add(entities.root, parent, entities.grandchild); }); const open = System.create({ name: "upTest/open" }, ({ commands }) => { commands.add(entities.root, parent, entities.other); }); const orphan = System.create({ name: "upTest/orphan" }, ({ commands }) => { commands.remove(entities.grandchild, parent); }); +/** Spawns `count` leaves under the grandchild, more than the journal keeps for a world this small. */ +const bulk: Entity[] = []; +const bulkUnder = System.create({ name: "upTest/bulkUnder" }, ({ commands }, input: { readonly count: number }) => { + bulk.length = 0; + for (let i = 0; i < input.count; i++) bulk.push(commands.spawn([label, local, world, parent])({ [label]: "bulk", [local]: 1, [world]: 0, [parent]: entities.grandchild })); +}); const worlds = async (value: (entity: Entity) => Promise): Promise<{ [name: string]: number | null }> => Object.fromEntries(await Promise.all(Object.entries(entities).map(async ([name, entity]) => [name, await value(entity)]))); @@ -198,6 +204,27 @@ describe("up", () => { expect(await read(ecs, entities.grandchild, world)).toBe(100); }); + it("runs every row after a spawn outruns the journal, with change filters, then only the rows that changed", async () => { + await using ecs = createWorld({ [workers]: 0 }); + await ecs.run(spawn); + await ecs.run(nest); + await ecs.run(propagateChanged); + await ecs.run(bulkUnder, { count: 5000 }); + ran.length = 0; + await ecs.run(propagateChanged); + expect(ran.length).toBe(4 + 5000); + expect(ran.slice(0, 4).sort()).toEqual(["child", "grandchild", "other", "root"]); + expect(await readEach(ecs, bulk.slice(0, 3), world)).toEqual([112, 112, 112]); + ran.length = 0; + await ecs.run(bump, { entity: entities.grandchild }); + await ecs.run(propagateChanged); + expect(ran.length).toBe(1 + 5000); + expect(await readEach(ecs, [entities.grandchild, bulk[4999]], world)).toEqual([13, 14]); + ran.length = 0; + await ecs.run(propagateChanged); + expect(ran).toEqual([]); + }); + it("finds a cycle a reparent closes after the tree was built, and runs its rows again once it opens", async () => { await using ecs = createWorld({ [workers]: 0 }); await ecs.run(spawn); diff --git a/packages/ecs/src/implementation/world/up.ts b/packages/ecs/src/implementation/world/up.ts index 369a0221..eff4e4c4 100644 --- a/packages/ecs/src/implementation/world/up.ts +++ b/packages/ecs/src/implementation/world/up.ts @@ -3,7 +3,7 @@ import type { Up } from '../../interface/system/access.js'; import type { Archetype, Storage } from './archetype.js'; import { generationOf, indexOf } from './entities.js'; import { journalEnd } from './journal.js'; -import { chunkShift } from './kernel.js'; +import { forEachNewerRow } from './ranges.js'; import { idOf, type Key, type State } from './state.js'; import { allocateArray, freeArray } from './arrays.js'; import type { Memory } from './memory.js'; @@ -44,6 +44,29 @@ const extend = (array: Int32Array, length: number, value: number): return grown; }; +// Scratch of an order build, shared by every `up` of a thread, since a build runs start to end without yielding: a +// second system reuses the first one's arrays rather than allocating its own. + +/** By index among the matched rows: its entity index, depth, archetype, parent row, and parent's archetype id when readable or -1. */ +let rowEntities = new Int32Array(0); +let depths = new Int32Array(0); +let rowParents = new Int32Array(0); +let keys = new Int32Array(0); +let owners = new Int32Array(0); +/** The matched rows' indexes, by depth, and where each depth ends. */ +let sortedRows = new Int32Array(0); +let counts = new Int32Array(0); +/** By index in `matches`: the index of its first row among the matched rows. */ +let bases = new Int32Array(0); +/** The groups of a build, laid out one after another, before the order lays them out with spare slots. */ +let builtRows = new Int32Array(0); +let builtParents = new Int32Array(0); +let builtEntities = new Int32Array(0); +let groupEnds = new Int32Array(0); +let groupDepths = new Int32Array(0); +let groupOwners = new Int32Array(0); +let groupParents = new Int32Array(0); + /** * Orders an `up` system's rows by depth through its `through` reference. A row's depth is one more than its parent's * when its parent is among the matched rows, and 0 otherwise, so rows of one depth never read each other's writes. @@ -254,15 +277,67 @@ export const createUp = (state: State, up: Up, isFiltered: boolean) => { for (const archetype of matches) count += archetype.count; const all = new Int32Array(count); let at = 0; + const levels = levelOf; for (const archetype of matches) { - for (let row = 0; row < archetype.count; row++) { - const index = archetype.entities[row] >>> 0; - levelOf[index] = 1; + const rows = archetype.entities; + for (let row = 0, end = archetype.count; row < end; row++) { + const index = rows[row] >>> 0; + levels[index] = 1; all[at++] = index; } } - for (let k = 0; k < count; k++) link(all[k], parentAt(all[k])); - settle(all, count); + // Each archetype's parent column is found once, as `parentAt` finds it per entity. + const { generation, archetype: archetypeOf, length: entityCount } = state.entities; + const parents = parentOf; + at = 0; + for (const archetype of matches) { + const storage: Storage | undefined = archetype.storageById[through]; + const layout = storage?.layout; + const leaf = layout?.kind === "linear" ? layout.leaves[0] : undefined; + if (storage === undefined || leaf === undefined) { + for (let row = 0; row < archetype.count; row++) parents[all[at++]] = -1; + continue; + } + const view = storage.views[leaf.view]; + const stride = leaf.unitStride, offset = leaf.unitOffset; + for (let row = 0, end = archetype.count; row < end; row++) { + const index = all[at++]; + const parent = view[row * stride + offset]; + const target = parent >>> 0; + link(index, parent > 0 && target < entityCount && archetypeOf[target] >= 0 && generation[target] === generationOf(parent) ? target : -1); + } + } + settleAll(all, count); + }; + + /** + * Sets the level of every matched entity in `all`, the whole tree: each walks up to a root or to an entity already + * set, then sets the levels on its way back down, so each entity is walked once. A walk meeting itself is a cycle, + * and it and everything below it on the walk is cyclic. + */ + const settleAll = (all: Int32Array, count: number): void => { + const walking = ++pass; + const done = ++pass; + stack = extend(stack, count, 0); + const levels = levelOf, parents = parentOf, seen = stamps, path = stack; + for (let k = 0; k < count; k++) { + let current = all[k]; + if (seen[current] === done) continue; + let height = 0; + for (; current >= 0 && levels[current] !== unmatched; current = parents[current]) { + const stamp = seen[current]; + if (stamp === done || stamp === walking) break; + seen[current] = walking; + path[height++] = current; + } + let level = current < 0 || levels[current] === unmatched ? 0 : seen[current] === done ? levels[current] : cyclic; + for (let h = height - 1; h >= 0; h--) { + const node = path[h]; + if (level !== cyclic) level++; + levels[node] = level; + seen[node] = done; + } + } }; /** Brings the tree up to date with the journal. */ @@ -326,26 +401,6 @@ export const createUp = (state: State, up: Up, isFiltered: boolean) => { settle(pending, pending.length); }; - /** By index among the matched rows: its entity index, depth, archetype, parent row, and parent's archetype id when readable or -1. */ - let rowEntities = new Int32Array(0); - let depths = new Int32Array(0); - let rowParents = new Int32Array(0); - let keys = new Int32Array(0); - let owners = new Int32Array(0); - /** The matched rows' indexes, by depth, and where each depth ends. */ - let sortedRows = new Int32Array(0); - let counts = new Int32Array(0); - /** By index in `matches`: the index of its first row among the matched rows. */ - let bases = new Int32Array(0); - /** The groups of a build, laid out one after another, before the order lays them out with spare slots. */ - let builtRows = new Int32Array(0); - let builtParents = new Int32Array(0); - let builtEntities = new Int32Array(0); - let groupEnds = new Int32Array(0); - let groupDepths = new Int32Array(0); - let groupOwners = new Int32Array(0); - let groupParents = new Int32Array(0); - /** The matched rows in depth order from the roots. Rows on or under a parent cycle are in none. */ const levels = (matches: readonly Archetype[]): Order => { update(matches); @@ -548,15 +603,12 @@ export const createUp = (state: State, up: Up, isFiltered: boolean) => { const changedRows = (archetype: Archetype, id: number, added: boolean, since: number, each: (row: number) => void): void => { const storage = archetype.storageById[id]; if (storage === undefined) return; - const whole = storage.columnChanged > since; - const ticks = added ? storage.added : storage.changed; - const chunks = added ? storage.chunkAdded : storage.chunkChanged; - for (let chunk = 0, count = Math.ceil(archetype.count / 2 ** chunkShift); chunk < count; chunk++) { - if (!whole && !(chunks[chunk] > since)) continue; - for (let row = chunk << chunkShift, end = Math.min(archetype.count, (chunk + 1) << chunkShift); row < end; row++) { - if (whole || ticks[row] > since) each(row); - } + if (storage.columnChanged > since) { + for (let row = 0; row < archetype.count; row++) each(row); + return; } + if (added) forEachNewerRow(storage.added, storage.chunkAdded, storage.blockAdded, archetype.count, since, each); + else forEachNewerRow(storage.changed, storage.chunkChanged, storage.blockChanged, archetype.count, since, each); }; /** @@ -566,6 +618,13 @@ export const createUp = (state: State, up: Up, isFiltered: boolean) => { */ const filtered = (matches: readonly Archetype[], added: readonly number[], changed: readonly number[], since: number): Pass | null => { update(matches); + // Every row passes after a rebuild, so the pass is the whole order, with nothing to mark. + if (rebuilt) { + touched = []; + rebuilt = false; + const ordered = levels(matches); + return ordered.levels === 0 ? null : wholePass(ordered); + } const length = state.entities.length; if (dirty.length < length) dirty = new Uint8Array(Math.max(length, 2 * dirty.length)); for (const bucket of buckets) if (bucket !== undefined) bucket.length = 0; @@ -672,6 +731,22 @@ export const createUp = (state: State, up: Up, isFiltered: boolean) => { return { next, clear }; }; + /** Every row of `ordered`, a depth at a time. */ + const wholePass = (ordered: Order): Pass => { + let depth = 0; + const next = (): readonly Run[] | null => { + if (depth >= ordered.levels) return null; + const runs: Run[] = []; + for (let group = ordered.depthGroups[depth]; group < ordered.depthGroups[depth + 1]; group++) { + if (ordered.groupEnds[group] === ordered.groupStarts[group]) continue; + runs.push({ owner: ordered.groupOwners[group], parent: ordered.groupParents[group], rows: ordered.rows, parents: ordered.parents, start: ordered.groupStarts[group], end: ordered.groupEnds[group] }); + } + depth++; + return runs; + }; + return { next, clear: noClear }; + }; + /** Frees what it holds in the arena and stops its reading of the journal, once nothing runs it again. */ const release = (): void => { state.journal.readers--; @@ -681,5 +756,7 @@ export const createUp = (state: State, up: Up, isFiltered: boolean) => { list = parentList = new Int32Array(0); }; - return { levels, filtered, release }; + return { levels, filtered, wholePass, release }; }; + +const noClear = (): void => {}; diff --git a/packages/ecs/src/implementation/world/wasm-kernels.test.ts b/packages/ecs/src/implementation/world/wasm-kernels.test.ts index 6216563b..2837c543 100644 --- a/packages/ecs/src/implementation/world/wasm-kernels.test.ts +++ b/packages/ecs/src/implementation/world/wasm-kernels.test.ts @@ -50,7 +50,7 @@ Component.register(target, { type: Entity.schema }); Resource.register(seconds, { type: { type: "number", default: 1 / 60 } }); Resource.register(floor, { type: { type: "number", precision: 1, default: -2.5 } }); -const seed = System.create({ name: "wasmTest/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const seed = System.create({ name: "wasmTest/seed" }, ({ commands }, input: { readonly count: number }) => { let previous = Entity.none; for (let i = 0; i < input.count; i++) { previous = commands.spawn([position, velocity, bounces, flipped, score, target])({ [position]: [i * 0.37, (i % 13) * 0.71], [velocity]: [Math.sin(i) * 3, Math.cos(i) * 5], [bounces]: 0, [flipped]: false, [score]: 0, [target]: previous }); @@ -58,7 +58,7 @@ const seed = System.create({ name: "wasmTest/seed", input: { type: "object", pro }); /** Despawns every seventh entity and spawns replacements, so some targets go stale and their indexes are reused. */ -const cull = System.create({ name: "wasmTest/cull", input: { type: "object", properties: { frame: { type: "integer" } } }, components: { read: [position], has: [bounces] } }, (row, { input, commands }) => { +const cull = System.create({ name: "wasmTest/cull", components: { read: [position], has: [bounces] } }, (row, { commands }, input: { readonly frame: number }) => { if ((row.entity >>> 0) % 7 !== input.frame % 7) return; commands.despawn(row.entity); commands.spawn([position, velocity, bounces, flipped, score, target])({ [position]: row[position], [velocity]: [1, 1], [bounces]: 0, [flipped]: true, [score]: 0, [target]: row.entity }); @@ -118,9 +118,8 @@ const tally = System.create({ const scale = System.create({ name: "wasmTest/scale", kernel: "cpu", - input: { type: "object", properties: { factor: { type: "number" } } }, components: { write: [score] }, -}, (row, { input }) => { +}, (row, _context, input: { readonly factor: number }) => { row[score] = row[score] * input.factor; }); diff --git a/packages/ecs/src/implementation/world/wasm.test.ts b/packages/ecs/src/implementation/world/wasm.test.ts index d2b72a00..dcf80767 100644 --- a/packages/ecs/src/implementation/world/wasm.test.ts +++ b/packages/ecs/src/implementation/world/wasm.test.ts @@ -138,12 +138,14 @@ describe("generateWasm with SIMD", () => { const floats = columns.map((_, k) => new Float32Array(memory.buffer, bases[k], 512)); const ticks = new Float64Array(memory.buffer, 4096 * 8, 64); const chunks = new Float64Array(memory.buffer, 4096 * 9, 4); + const blocks = new Float64Array(memory.buffer, 4096 * 10, 4); fill(floats, ticks); const runKernel: unknown = new WebAssembly.Instance(new WebAssembly.Module(kernel.bytes), { env: { memory }, math: importsOf(kernel.imports) }).exports.run; if (typeof runKernel !== "function") throw new Error("no run export"); - // f64 columns are ticks, then their chunks. - runKernel(0, rows, tick, -1, ...bases.map((base, k) => columns[k].view !== "f64" ? base : columns[k].component === -2 ? 4096 * 9 : 4096 * 8)); - return { kernel, floats, ticks, chunks }; + // f64 columns are ticks, then their chunks, then their blocks. + const tickBase = (component: number): number => component === -3 ? 4096 * 10 : component === -2 ? 4096 * 9 : 4096 * 8; + runKernel(0, rows, tick, -1, ...bases.map((base, k) => columns[k].view !== "f64" ? base : tickBase(columns[k].component))); + return { kernel, floats, ticks, chunks, blocks }; }; const column = (component: number, stride: number, offset: number): WasmColumn => ({ view: "f32", stride, offset, boolean: false, component }); const read = (k: number): Expression => ({ kind: "column", column: k }); @@ -201,7 +203,7 @@ describe("generateWasm with SIMD", () => { column: leaf, value: add(add(local(leaf), { kind: "binary", operator: "*", left: local(leaves + leaf), right: { kind: "args", path: [] } }), { kind: "number", value: leaf + 1 }), })), - { kind: "stamp", column: 2 * leaves, chunks: 2 * leaves + 1 }, + { kind: "stamp", column: 2 * leaves, chunks: 2 * leaves + 1, blocks: 2 * leaves + 2 }, ], }; const columns: WasmColumn[] = [ @@ -209,6 +211,7 @@ describe("generateWasm with SIMD", () => { ...Array.from({ length: leaves }, (_, leaf) => column(1, stride, 4 * leaf)), { view: "f64", stride: 8, offset: 0, boolean: false, component: -1 }, { view: "f64", stride: 8, offset: 0, boolean: false, component: -2 }, + { view: "f64", stride: 8, offset: 0, boolean: false, component: -3 }, ]; const kernel = generateWasm(program, columns, () => "number"); if (kernel === null) throw new Error("expected a kernel"); @@ -221,7 +224,7 @@ describe("generateWasm with SIMD", () => { const runKernel: unknown = new WebAssembly.Instance(new WebAssembly.Module(kernel.bytes), { env: { memory }, math: importsOf(kernel.imports) }).exports.run; if (typeof runKernel !== "function") throw new Error("no run export"); const [start, end] = [3, 22]; - runKernel(start, end, 7, -1, ...columns.slice(0, leaves).map(() => 4096), ...columns.slice(0, leaves).map(() => 8192), 16384, 20480, 0.5); + runKernel(start, end, 7, -1, ...columns.slice(0, leaves).map(() => 4096), ...columns.slice(0, leaves).map(() => 8192), 16384, 20480, 24576, 0.5); const row = (index: number): number => Math.floor(index / leaves); expect([...p]).toEqual([...p].map((_, i) => row(i) >= start && row(i) < end ? i + (2 * i + 100) * 0.5 + (i % leaves) + 1 : i)); expect([...ticks]).toEqual([...ticks].map((_, i) => i >= start && i < end ? 7 : 1)); @@ -230,11 +233,12 @@ describe("generateWasm with SIMD", () => { it("blends stores and tick stamps under an if, per lane", () => { const positive: Expression = { kind: "binary", operator: ">", left: read(0), right: { kind: "number", value: 0 } }; - const program: Program = { locals: 0, body: [{ kind: "if", test: positive, consequent: [{ kind: "store", column: 1, value: read(0) }, { kind: "stamp", column: 2, chunks: 3 }], alternate: [] }] }; + const program: Program = { locals: 0, body: [{ kind: "if", test: positive, consequent: [{ kind: "store", column: 1, value: read(0) }, { kind: "stamp", column: 2, chunks: 3, blocks: 4 }], alternate: [] }] }; const ticksColumn: WasmColumn = { view: "f64", stride: 8, offset: 0, boolean: false, component: -1 }; const chunksColumn: WasmColumn = { view: "f64", stride: 8, offset: 0, boolean: false, component: -2 }; + const blocksColumn: WasmColumn = { view: "f64", stride: 8, offset: 0, boolean: false, component: -3 }; const inputs = [1, -1, 2, -2, 0, 3, NaN, 4, -5, 6]; - const { kernel, floats, ticks, chunks } = runVector(program, [column(0, 4, 0), column(1, 4, 0), ticksColumn, chunksColumn], ([input, out], tickValues) => { + const { kernel, floats, ticks, chunks, blocks } = runVector(program, [column(0, 4, 0), column(1, 4, 0), ticksColumn, chunksColumn, blocksColumn], ([input, out], tickValues) => { input.set(inputs); out.fill(-7); tickValues.fill(1); @@ -243,6 +247,32 @@ describe("generateWasm with SIMD", () => { expect([...floats[1].subarray(0, 10)]).toEqual(inputs.map(x => x > 0 ? x : -7)); expect([...ticks.subarray(0, 10)]).toEqual(inputs.map(x => x > 0 ? 7 : 1)); expect(chunks[0]).toBe(7); + expect(blocks[0]).toBe(7); + }); + + /** Runs `out = in` with its stamps where `in > 0`, rows `start` to `end` of 2048, four at a time; the blocks then stamped. */ + const stampedFourAtATime = (start: number, end: number, positive: readonly number[]): number[] => { + const test: Expression = { kind: "binary", operator: ">", left: read(0), right: { kind: "number", value: 0 } }; + const program: Program = { locals: 0, body: [{ kind: "if", test, consequent: [{ kind: "store", column: 1, value: read(0) }, { kind: "stamp", column: 2, chunks: 3, blocks: 4 }], alternate: [] }] }; + const dense = (component: number): WasmColumn => ({ view: "f64", stride: 8, offset: 0, boolean: false, component }); + const kernel = generateWasm(program, [column(0, 4, 0), column(1, 4, 0), dense(-1), dense(-2), dense(-3)], () => "number"); + if (kernel === null) throw new Error("expected a kernel"); + expect(kernel.vectorized).toBe(true); + const memory = new WebAssembly.Memory({ initial: 2, maximum: 65536, shared: true }); + const input = new Float32Array(memory.buffer, 0, 2048).fill(-1); + for (const row of positive) input[row] = 1; + const blocks = new Float64Array(memory.buffer, 65536 + 1024, 32); + const runKernel = new WebAssembly.Instance(new WebAssembly.Module(kernel.bytes), { env: { memory }, math: importsOf(kernel.imports) }).exports.run; + if (typeof runKernel !== "function") throw new Error("no run export"); + runKernel(start, end, 7, -1, 0, 8192, 16384, 65536, 65536 + 1024); + return [...blocks].flatMap((tick, block) => tick === 7 ? [block] : []); + }; + + it("stamps the blocks of rows a conditional write reaches four rows at a time", () => { + // 300 rows are 75 whole groups, so no row runs one at a time. + expect(stampedFourAtATime(0, 300, [200])).toContain(200 >> 6); + // From row 1 the last group, rows 61 to 64, straddles blocks 0 and 1, and only its last row writes. + expect(stampedFourAtATime(1, 65, [64])).toEqual([0, 1]); }); it("stays scalar for operations without an exact f32x4 equivalent", () => { @@ -253,7 +283,7 @@ describe("generateWasm with SIMD", () => { }); describe("generateWasm with change filters", () => { - it("runs only rows with a newer tick, skipping chunks whose tick isn't newer", () => { + it("runs only rows with a newer tick, skipping chunks and blocks whose tick isn't newer", () => { for (const simd of [false, true]) { // out = in, for rows whose filter tick is newer than lastRun 5. const columns: WasmColumn[] = [{ view: "f32", stride: 4, offset: 0, boolean: false, component: 0 }, { view: "f32", stride: 4, offset: 0, boolean: false, component: 1 }]; @@ -266,46 +296,107 @@ describe("generateWasm with change filters", () => { const output = new Float32Array(memory.buffer, 16384, rows).fill(-1); const ticks = new Float64Array(memory.buffer, 32768, rows); const chunks = new Float64Array(memory.buffer, 65536, 3); - // Chunk 0's chunk tick is stale-new (8) but its rows are old; chunk 1 has three new rows; chunk 2 is old throughout. + const blocks = new Float64Array(memory.buffer, 66560, 48); + // Chunk 0's chunk and block ticks are stale-new (8) but its rows are old; chunk 1 has three new rows in + // blocks marked new; chunk 2 is old throughout. chunks.set([8, 9, 3]); - for (const row of [1030, 1500, 2047]) ticks[row] = 9; + blocks.fill(8, 0, 16).fill(3, 16); + for (const row of [1030, 1500, 2047]) { + ticks[row] = 9; + blocks[row >> 6] = 9; + } + ticks[1100] = 9; ticks[2050] = 9; const run = new WebAssembly.Instance(new WebAssembly.Module(kernel.bytes), { env: { memory }, math: importsOf(kernel.imports) }).exports.run; if (typeof run !== "function") throw new Error("no run export"); - run(0, rows, 10, 5, 0, 16384, 32768, 65536); + run(0, rows, 10, 5, 0, 16384, 32768, 65536, 66560); const written = [...output].flatMap((value, row) => value === -1 ? [] : [row]); - // Row 2050 has a new tick, but its chunk says nothing in it changed, so it's skipped by design. + // Rows 1100 and 2050 have new ticks, but their block or chunk says nothing in it changed, so they're skipped by design. expect([simd, kernel.vectorized, written]).toEqual([simd, false, [1030, 1500, 2047]]); + // From a start inside a block, that block's tick is still checked. + output.fill(-1); + run(1029, 1040, 10, 5, 0, 16384, 32768, 65536, 66560); + expect([...output].flatMap((value, row) => value === -1 ? [] : [row])).toEqual([1030]); // lastRun below 0 passes everything. output.fill(-1); - run(0, rows, 10, -1, 0, 16384, 32768, 65536); + run(0, rows, 10, -1, 0, 16384, 32768, 65536, 66560); expect(output.every((value, row) => value === row)).toBe(true); } }); }); -describe("generateWasm chunk stamps", () => { +describe("generateWasm with up", () => { + it("refuses change filters, which its runner applies by listing only passing rows", () => { + const columns: WasmColumn[] = [{ view: "f32", stride: 4, offset: 0, boolean: false, component: 0 }]; + const program: Program = { locals: 0, up: true, body: [{ kind: "store", column: 0, value: { kind: "number", value: 1 } }] }; + expect(() => generateWasm(program, columns, () => "number", true, 1)).toThrow("takes no change filters"); + expect(generateWasm(program, columns, () => "number", true, 0)).not.toBeNull(); + }); +}); + +describe("generateWasm with several change filters", () => { + it("runs rows either filter passes, a chunk with most blocks passing as one run", () => { + for (const simd of [false, true]) { + const columns: WasmColumn[] = [{ view: "f32", stride: 4, offset: 0, boolean: false, component: 0 }, { view: "f32", stride: 4, offset: 0, boolean: false, component: 1 }]; + const kernel = generateWasm({ locals: 0, body: [{ kind: "store", column: 1, value: { kind: "column", column: 0 } }] }, columns, () => "number", simd, 2); + if (kernel === null) throw new Error("expected a kernel"); + const rows = 2100; + const memory = new WebAssembly.Memory({ initial: 4, maximum: 65536, shared: true }); + const input = new Float32Array(memory.buffer, 0, rows); + for (let row = 0; row < rows; row++) input[row] = row; + const output = new Float32Array(memory.buffer, 16384, rows).fill(-1); + // Each filter's rows, chunks and blocks. + const ticks = [new Float64Array(memory.buffer, 32768, rows).fill(1), new Float64Array(memory.buffer, 65536 + 4096, rows).fill(1)]; + const chunks = [new Float64Array(memory.buffer, 65536, 3).fill(1), new Float64Array(memory.buffer, 65536 + 64, 3).fill(1)]; + const blocks = [new Float64Array(memory.buffer, 65536 + 1024, 48).fill(1), new Float64Array(memory.buffer, 65536 + 2048, 48).fill(1)]; + const mark = (filter: number, row: number): void => { + ticks[filter][row] = chunks[filter][row >> 10] = blocks[filter][row >> 6] = 9; + }; + // Chunk 0: the first filter marks ten blocks of sixteen, so the chunk runs whole, and a row in an + // unmarked block with a new tick runs too. Chunk 1: each filter marks a row; a new tick in an unmarked + // block is skipped. + for (let block = 0; block < 10; block++) mark(0, block * 64 + 1); + ticks[1][900] = 9; + chunks[1][0] = 9; + mark(0, 1030); + mark(1, 1500); + ticks[1][1700] = 9; + const run = new WebAssembly.Instance(new WebAssembly.Module(kernel.bytes), { env: { memory }, math: importsOf(kernel.imports) }).exports.run; + if (typeof run !== "function") throw new Error("no run export"); + run(0, rows, 10, 5, 0, 16384, 32768, 65536 + 4096, 65536, 65536 + 64, 65536 + 1024, 65536 + 2048); + const written = [...output].flatMap((value, row) => value === -1 ? [] : [row]); + expect([simd, written]).toEqual([simd, [...Array.from({ length: 10 }, (_, block) => block * 64 + 1), 900, 1030, 1500]]); + } + }); +}); + +describe("generateWasm chunk and block stamps", () => { const columns: WasmColumn[] = [ { view: "f32", stride: 4, offset: 0, boolean: false, component: 0 }, { view: "f64", stride: 8, offset: 0, boolean: false, component: -1 }, { view: "f64", stride: 8, offset: 0, boolean: false, component: -2 }, + { view: "f64", stride: 8, offset: 0, boolean: false, component: -3 }, ]; - const run = (program: Program, start: number, end: number): number[] => { + /** The blocks whose tick is 7. */ + const stampedBlocks = (blocks: Float64Array): number[] => [...blocks].flatMap((tick, block) => tick === 7 ? [block] : []); + const run = (program: Program, start: number, end: number): { readonly chunks: number[]; readonly blocks: number[] } => { const kernel = generateWasm(program, columns, () => "number"); if (kernel === null) throw new Error("expected a kernel"); const memory = new WebAssembly.Memory({ initial: 4, maximum: 65536, shared: true }); new Float32Array(memory.buffer, 0, 4096).fill(1); const chunks = new Float64Array(memory.buffer, 65536, 4); + const blocks = new Float64Array(memory.buffer, 66560, 64); const runKernel = new WebAssembly.Instance(new WebAssembly.Module(kernel.bytes), { env: { memory }, math: importsOf(kernel.imports) }).exports.run; if (typeof runKernel !== "function") throw new Error("no run export"); - runKernel(start, end, 7, -1, 0, 32768, 65536); - return [...chunks]; + runKernel(start, end, 7, -1, 0, 32768, 65536, 66560); + return { chunks: [...chunks], blocks: stampedBlocks(blocks) }; }; - const stamp: Program.Statement = { kind: "stamp", column: 1, chunks: 2 }; + const stamp: Program.Statement = { kind: "stamp", column: 1, chunks: 2, blocks: 3 }; + const range = (first: number, last: number): number[] => Array.from({ length: last - first + 1 }, (_, k) => first + k); - it("stamps every chunk of the range once when every row stamps", () => { - expect(run({ locals: 0, body: [stamp] }, 1000, 2100)).toEqual([7, 7, 7, 0]); - expect(run({ locals: 0, body: [stamp] }, 5, 5)).toEqual([0, 0, 0, 0]); + it("stamps every chunk and block of the range once when every row stamps", () => { + expect(run({ locals: 0, body: [stamp] }, 1000, 2100)).toEqual({ chunks: [7, 7, 7, 0], blocks: range(1000 >> 6, 2099 >> 6) }); + expect(run({ locals: 0, body: [stamp] }, 5, 5)).toEqual({ chunks: [0, 0, 0, 0], blocks: [] }); }); it("stamps per row when a row can end before its stamp", () => { @@ -323,10 +414,12 @@ describe("generateWasm chunk stamps", () => { const values = new Float32Array(memory.buffer, 0, 4096).fill(1); values.fill(2, 1024, 2048); const chunks = new Float64Array(memory.buffer, 65536, 4); + const blocks = new Float64Array(memory.buffer, 66560, 64); const runKernel = new WebAssembly.Instance(new WebAssembly.Module(kernel.bytes), { env: { memory }, math: importsOf(kernel.imports) }).exports.run; if (typeof runKernel !== "function") throw new Error("no run export"); - runKernel(0, 3000, 7, -1, 0, 32768, 65536); + runKernel(0, 3000, 7, -1, 0, 32768, 65536, 66560); expect([...chunks]).toEqual([7, 0, 7, 0]); + expect(stampedBlocks(blocks)).toEqual([...range(0, 15), ...range(32, 2999 >> 6)]); }); }); diff --git a/packages/ecs/src/implementation/world/wasm.ts b/packages/ecs/src/implementation/world/wasm.ts index fe6846f8..167ce43c 100644 --- a/packages/ecs/src/implementation/world/wasm.ts +++ b/packages/ecs/src/implementation/world/wasm.ts @@ -1,5 +1,5 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. -import { chunkShift, type Program } from './kernel.js'; +import { blockShift, chunkShift, type Program } from './kernel.js'; import type { Reduction } from '../../interface/resource/reduction.js'; import type { View } from '../../interface/registry/layout.js'; import { withoutIntrinsics } from './intrinsics.js'; @@ -133,7 +133,7 @@ const op = { i32Const: 0x41, f64Const: 0x44, i32ReinterpretF32: 0xbc, i64ReinterpretF64: 0xbd, i32LtU: 0x49, i32GtU: 0x4b, i32LeU: 0x4d, i32GtS: 0x4a, f64Load2: 0x2b, i32Eqz: 0x45, i32GeS: 0x4e, f64Eq: 0x61, f64Ne: 0x62, f64Lt: 0x63, f64Gt: 0x64, f64Le: 0x65, f64Ge: 0x66, - i32Eq: 0x46, i32LtS: 0x48, i32LeS: 0x4c, i32Clz: 0x67, i32Add: 0x6a, i32Sub: 0x6b, i32Mul: 0x6c, i32And: 0x71, i32Or: 0x72, i32Xor: 0x73, i32Shl: 0x74, i32ShrS: 0x75, i32ShrU: 0x76, + i32Eq: 0x46, i32LtS: 0x48, i32LeS: 0x4c, i32Clz: 0x67, i32Ctz: 0x68, i32Popcnt: 0x69, i32Add: 0x6a, i32Sub: 0x6b, i32Mul: 0x6c, i32And: 0x71, i32Or: 0x72, i32Xor: 0x73, i32Shl: 0x74, i32ShrS: 0x75, i32ShrU: 0x76, f64Abs: 0x99, f64Neg: 0x9a, f64Ceil: 0x9b, f64Floor: 0x9c, f64Trunc: 0x9d, f64Sqrt: 0x9f, f64Add: 0xa0, f64Sub: 0xa1, f64Mul: 0xa2, f64Div: 0xa3, f64Min: 0xa4, f64Max: 0xa5, i32WrapI64: 0xa7, i32GeU: 0x4f, i32Ne: 0x47, i64Const: 0x42, i64ShrU: 0x88, i64RemS: 0x81, f64ConvertI64S: 0xb9, f64Copysign: 0xa6, f32DemoteF64: 0xb6, f64ConvertI32S: 0xb7, f64ConvertI32U: 0xb8, f64PromoteF32: 0xbb, @@ -352,18 +352,23 @@ export const generateWasm = ( }; while (inferIntegral(program.body)); - // Parameters: start, end, tick, lastRun, column addresses, filter row tick addresses, filter chunk tick addresses, scalars, and with reductions the partials address. + // Parameters: start, end, tick, lastRun, column addresses, filter row, chunk and block tick addresses, scalars, and with reductions the partials address. const columnParameter = (column: number): number => 4 + column; const filterRowParameter = (filter: number): number => 4 + columns.length + filter; const filterChunkParameter = (filter: number): number => 4 + columns.length + filters + filter; + const filterBlockParameter = (filter: number): number => 4 + columns.length + 2 * filters + filter; + /** Parameters before the lookups': the fixed four, the columns' and the filters'. */ + const beforeLookups = 4 + columns.length + 3 * filters; /** * With lookups: the entity table (archetype, row and generation addresses, then its length), then each lookup's * directory address and record count, all as f64 so workers pass them like scalars. */ const lookupParameters = lookups.length === 0 ? 0 : 4 + 2 * lookups.length + (watching ? 1 : 0); - const lookupParameter = (index: number): number => 4 + columns.length + 2 * filters + index; + const lookupParameter = (index: number): number => beforeLookups + index; /** For `up`: the rows' and their parents' rows' addresses, whether there's a parent, then each parent column's address. */ const upMode = program.up === true; + // Its runner passes an `up` kernel only the rows that pass, and the filter loops step the row, not its list slot. + if (upMode && filters > 0) throw new Error("A WASM kernel reading `up` takes no change filters: its runner lists only the rows that pass."); const upParameters = upMode ? 3 + upColumns.length : 0; /** Each lookup through an index's parameters, after the lookup parameters: `indexParameterCount` of them, as f64. */ const indexOffsets: number[] = []; @@ -373,9 +378,9 @@ export const generateWasm = ( const index = indexLookups[lookup]; if (index !== null && index !== undefined) indexParameters += indexParameterCount(index); } - const indexParameter = (lookup: number, index: number): number => 4 + columns.length + 2 * filters + lookupParameters + indexOffsets[lookup] + index; - const upParameter = (index: number): number => 4 + columns.length + 2 * filters + lookupParameters + indexParameters + index; - const scalarParameter = (index: number): number => 4 + columns.length + 2 * filters + lookupParameters + indexParameters + upParameters + index; + const indexParameter = (lookup: number, index: number): number => beforeLookups + lookupParameters + indexOffsets[lookup] + index; + const upParameter = (index: number): number => beforeLookups + lookupParameters + indexParameters + index; + const scalarParameter = (index: number): number => beforeLookups + lookupParameters + indexParameters + upParameters + index; const code: number[] = []; const extraLocals: number[] = []; let localBase = 0; @@ -611,31 +616,34 @@ export const generateWasm = ( }; /** - * Chunk columns stamped by every row: top-level stamps with no `next` before them, in an unfiltered kernel. - * Their chunks are stamped once per call, for the whole range, instead of on each row. + * Chunk and block columns stamped by every row, by the shift from a row to its element: top-level stamps with no + * `next` before them, in an unfiltered kernel. Their chunks and blocks are stamped once per call, for the whole + * range, instead of on each row. */ - const everyRow = new Set(); - // Listed rows needn't be consecutive, so each stamps its own chunk. + const everyRow = new Map(); + // Listed rows needn't be consecutive, so each stamps its own chunk and block. if (filters === 0 && !upMode) { const ends = (statement: Statement): boolean => statement.kind === "next" || (statement.kind === "if" && (statement.consequent.some(ends) || statement.alternate.some(ends))) || ((statement.kind === "block" || statement.kind === "loop") && statement.body.some(ends)); for (const statement of program.body) { if (ends(statement)) break; - if (statement.kind === "stamp" && columns[statement.chunks].muted !== true) everyRow.add(statement.chunks); + if (statement.kind !== "stamp") continue; + if (columns[statement.blocks].muted !== true) everyRow.set(statement.blocks, blockShift); + if (columns[statement.chunks].muted !== true) everyRow.set(statement.chunks, chunkShift); } } - /** Stamps every chunk from `start`'s to `end - 1`'s, for the chunk columns every row stamps. */ + /** Stamps every chunk or block from `start`'s to `end - 1`'s, for the chunk and block columns every row stamps. */ const stampRangeChunks = (): void => { - for (const chunks of everyRow) { + for (const [dense, shift] of everyRow) { const chunk = scratch(i32Type); const last = scratch(i32Type); - // if (start < end) for (chunk = start >> shift; chunk <= (end - 1) >> shift; chunk++) chunks[chunk] = tick + // if (start < end) for (chunk = start >> shift; chunk <= (end - 1) >> shift; chunk++) dense[chunk] = tick code.push(op.localGet, 0, op.localGet, 1, op.i32LtS, op.if, 0x40); - code.push(op.localGet, 0, op.i32Const, ...signed(chunkShift), op.i32ShrU, op.localSet, ...unsigned(chunk)); - code.push(op.localGet, 1, op.i32Const, 1, op.i32Sub, op.i32Const, ...signed(chunkShift), op.i32ShrU, op.localSet, ...unsigned(last)); + code.push(op.localGet, 0, op.i32Const, ...signed(shift), op.i32ShrU, op.localSet, ...unsigned(chunk)); + code.push(op.localGet, 1, op.i32Const, 1, op.i32Sub, op.i32Const, ...signed(shift), op.i32ShrU, op.localSet, ...unsigned(last)); code.push(op.loop, 0x40); - code.push(op.localGet, ...unsigned(columnParameter(chunks)), op.localGet, ...unsigned(chunk), op.i32Const, 3, op.i32Shl, op.i32Add, op.localGet, 2, op.f64Store, 3, 0); + code.push(op.localGet, ...unsigned(columnParameter(dense)), op.localGet, ...unsigned(chunk), op.i32Const, 3, op.i32Shl, op.i32Add, op.localGet, 2, op.f64Store, 3, 0); code.push(op.localGet, ...unsigned(chunk), op.i32Const, 1, op.i32Add, op.localTee, ...unsigned(chunk), op.localGet, ...unsigned(last), op.i32LeS, op.brIf, 0); code.push(op.end, op.end); } @@ -1235,12 +1243,15 @@ export const generateWasm = ( emitRecordLeaf(row, 1, { view: "f64", stride: 8, offset: 0, boolean: false }); }; - /** Sets the chunk tick of the row `rowOffset` rows past the current one, unless every row's chunk is stamped up front. */ - const stampChunk = (chunks: number, rowOffset: number): void => { - if (everyRow.has(chunks)) return; - code.push(op.localGet, ...unsigned(columnParameter(chunks)), op.localGet, ...unsigned(rowLocal)); + /** + * Sets the chunk or block tick, by `shift`, of the row `rowOffset` rows past the current one, unless it's muted + * or every row's is stamped up front. + */ + const stampDense = (dense: number, shift: number, rowOffset: number): void => { + if (columns[dense].muted === true || everyRow.has(dense)) return; + code.push(op.localGet, ...unsigned(columnParameter(dense)), op.localGet, ...unsigned(rowLocal)); if (rowOffset !== 0) code.push(op.i32Const, ...signed(rowOffset), op.i32Add); - code.push(op.i32Const, ...signed(chunkShift), op.i32ShrU, op.i32Const, 3, op.i32Shl, op.i32Add, op.localGet, 2, op.f64Store, 3, 0); + code.push(op.i32Const, ...signed(shift), op.i32ShrU, op.i32Const, 3, op.i32Shl, op.i32Add, op.localGet, 2, op.f64Store, 3, 0); }; const statement = (current: Statement): void => { @@ -1250,8 +1261,9 @@ export const generateWasm = ( // A chunk column stays when its row column is muted, for a component held on the GPU. case "stamp": if (columns[current.column].muted !== true) store(current.column, "tick"); - if (columns[current.chunks].muted === true) return; - return stampChunk(current.chunks, 0); + stampDense(current.blocks, blockShift, 0); + stampDense(current.chunks, chunkShift, 0); + return; case "if": emit(current.test); emitTruthy(typeOf(current.test)); @@ -1469,10 +1481,11 @@ export const generateWasm = ( code.push(...simd(vop.store), 3, ...unsigned(16 * half)); } } - if (columns[current.chunks].muted === true) return; - // The group may straddle two chunks. Under a mask this can stamp a chunk no lane changed, which only costs a scan. - stampChunk(current.chunks, 0); - stampChunk(current.chunks, 3); + // The group may straddle two chunks or blocks. Under a mask this can stamp one no lane changed, which only costs a scan. + stampDense(current.blocks, blockShift, 0); + stampDense(current.blocks, blockShift, 3); + stampDense(current.chunks, chunkShift, 0); + stampDense(current.chunks, chunkShift, 3); return; } case "if": { @@ -1911,7 +1924,7 @@ export const generateWasm = ( }; try { - const parameterCount = 4 + columns.length + 2 * filters + lookupParameters + indexParameters + upParameters; + const parameterCount = beforeLookups + lookupParameters + indexParameters + upParameters; collectFinds(program.body); /** Leaves 1 if any filter's tick at `(row >> shift) * 8` in its `parameter` array is newer than `lastRun`. */ const anyNewer = (parameter: (filter: number) => number, shift: number): void => { @@ -1922,36 +1935,88 @@ export const generateWasm = ( if (filter > 0) code.push(op.i32Or); } }; - /** At a chunk's first row, skips the whole chunk unless a filter's chunk tick is newer. */ - const skipChunks = (): void => { - const next = scratch(i32Type); - code.push(op.localGet, ...unsigned(rowLocal), op.localGet, 0, op.i32Eq, op.localGet, ...unsigned(rowLocal), op.i32Const, ...signed(2 ** chunkShift - 1), op.i32And, op.i32Eqz, op.i32Or); - code.push(op.if, 0x40); + /** + * i32 locals: the row after the current chunk; the row after the current run of passing blocks, both `start` + * before the first; the current chunk's blocks that pass, a bit each; and two temporaries. + */ + let chunkEnd = 0; + let blockEnd = 0; + let passingBlocks = 0; + let ahead = 0; + let skipped = 0; + const startChunksAndBlocks = (): void => { + chunkEnd = scratch(i32Type); + blockEnd = scratch(i32Type); + passingBlocks = scratch(i32Type); + ahead = scratch(i32Type); + skipped = scratch(i32Type); + code.push(op.localGet, 0, op.localTee, ...unsigned(chunkEnd), op.localSet, ...unsigned(blockEnd)); + }; + const get = (local: number): number[] => [op.localGet, ...unsigned(local)]; + const set = (local: number): number[] => [op.localSet, ...unsigned(local)]; + /** Leaves `min(end, ((row >> shift) + count) << shift)`, with `count` on the stack first. */ + const runEnd = (shift: number, target: number): void => { + code.push(...get(rowLocal), op.i32Const, ...signed(shift), op.i32ShrU, op.i32Add, op.i32Const, ...signed(shift), op.i32Shl, ...set(target)); + code.push(...get(target), op.localGet, 1, ...get(target), op.localGet, 1, op.i32LtS, op.select, ...set(target)); + }; + const blocksPerChunk = 2 ** (chunkShift - blockShift); + /** + * In the outer row loop, at `row`, the first row after the last run of passing blocks: past the current chunk, + * skips chunks no filter's chunk tick says changed, and notes which blocks of the next one pass, a bit each; + * then skips the blocks that don't pass, and sets `blockEnd` to the end of the run of blocks that do, which the + * inner loop runs. A chunk with more than half its blocks passing is one run: ending a run costs more than the + * eight-row checks spend on a block between runs. + */ + const findRun = (): void => { + code.push(...get(rowLocal), ...get(chunkEnd), op.i32GeS, op.if, 0x40); + code.push(op.i32Const, 1); + runEnd(chunkShift, chunkEnd); anyNewer(filterChunkParameter, chunkShift); - code.push(op.i32Eqz, op.if, 0x40); - // row = min(end, start of the next chunk) - code.push(op.localGet, ...unsigned(rowLocal), op.i32Const, ...signed(chunkShift), op.i32ShrU, op.i32Const, 1, op.i32Add, op.i32Const, ...signed(chunkShift), op.i32Shl, op.localSet, ...unsigned(next)); - code.push(op.localGet, ...unsigned(next), op.localGet, 1, op.localGet, ...unsigned(next), op.localGet, 1, op.i32LtS, op.select, op.localSet, ...unsigned(rowLocal)); - code.push(op.br, 2, op.end, op.end); + code.push(op.i32Eqz, op.if, 0x40, ...get(chunkEnd), ...set(rowLocal), op.br, 2, op.end); + // The chunk's blocks that pass, from its first block, `(row >> chunkShift) << (chunkShift - blockShift)`. + for (let block = 0; block < blocksPerChunk; block++) { + for (let filter = 0; filter < filters; filter++) { + code.push(...get(filterBlockParameter(filter)), ...get(rowLocal), op.i32Const, ...signed(chunkShift), op.i32ShrU, op.i32Const, ...signed(chunkShift - blockShift), op.i32Shl); + code.push(op.i32Const, ...signed(block), op.i32Add, op.i32Const, 3, op.i32Shl, op.i32Add, op.f64Load, 3, 0, op.localGet, 3, op.f64Gt); + if (filter > 0) code.push(op.i32Or); + } + if (block > 0) code.push(op.i32Const, ...signed(block), op.i32Shl, ...get(passingBlocks), op.i32Or); + code.push(...set(passingBlocks)); + } + code.push(...get(passingBlocks), op.i32Popcnt, op.i32Const, ...signed(blocksPerChunk / 2), op.i32GtS, op.if, 0x40); + code.push(op.i32Const, ...signed((2 ** blocksPerChunk - 1) | 0), ...set(passingBlocks), op.end); + code.push(op.end); + // ahead = passingBlocks >> (row's block in its chunk); none ahead skips the rest of the chunk. + code.push(...get(passingBlocks), ...get(rowLocal), op.i32Const, ...signed(blockShift), op.i32ShrU, op.i32Const, ...signed(blocksPerChunk - 1), op.i32And, op.i32ShrU, op.localTee, ...unsigned(ahead)); + code.push(op.i32Eqz, op.if, 0x40, ...get(chunkEnd), ...set(rowLocal), op.br, 1, op.end); + // Skips to the next passing block, then runs to the end of the passing blocks after it. + code.push(...get(ahead), op.i32Ctz, op.localTee, ...unsigned(skipped), op.if, 0x40); + code.push(...get(skipped), ...get(rowLocal), op.i32Const, ...signed(blockShift), op.i32ShrU, op.i32Add, op.i32Const, ...signed(blockShift), op.i32Shl, ...set(rowLocal)); + code.push(op.end); + code.push(...get(ahead), ...get(skipped), op.i32ShrU, op.i32Const, ...signed(-1), op.i32Xor, op.i32Ctz); + runEnd(blockShift, blockEnd); + code.push(...get(blockEnd), ...get(chunkEnd), ...get(blockEnd), ...get(chunkEnd), op.i32LtS, op.select, ...set(blockEnd)); + // A row moved to a later block goes round the outer loop, which checks it against `end`. + code.push(...get(skipped), op.brIf, 0); }; - /** Rows a block skip checks at once: four f64x2 loads of each filter's row ticks. */ - const blockRows = 8; + /** Rows an eight-row skip checks at once: four f64x2 loads of each filter's row ticks. */ + const eightRows = 8; /** - * With SIMD, at a row that's a multiple of `blockRows`, skips that many rows when none passes a filter, testing - * two ticks per comparison. Only aligned rows check, so rows that do pass pay one check per block. + * With SIMD, at a row that's a multiple of `eightRows`, skips that many rows when none passes a filter, testing + * two ticks per comparison. Only aligned rows check, so rows that do pass pay one check per eight. */ - const skipBlocks = (): void => { - code.push(op.localGet, ...unsigned(rowLocal), op.i32Const, blockRows - 1, op.i32And, op.i32Eqz); - code.push(op.localGet, ...unsigned(rowLocal), op.i32Const, blockRows, op.i32Add, op.localGet, 1, op.i32LeS, op.i32And, op.if, 0x40); + const skipEights = (): void => { + code.push(op.localGet, ...unsigned(rowLocal), op.i32Const, eightRows - 1, op.i32And, op.i32Eqz); + code.push(op.localGet, ...unsigned(rowLocal), op.i32Const, eightRows, op.i32Add, op.localGet, 1, op.i32LeS, op.i32And, op.if, 0x40); for (let filter = 0; filter < filters; filter++) { - for (let pair = 0; pair < blockRows / 2; pair++) { + for (let pair = 0; pair < eightRows / 2; pair++) { code.push(op.localGet, ...unsigned(filterRowParameter(filter)), op.localGet, ...unsigned(rowLocal), op.i32Const, 3, op.i32Shl, op.i32Add); code.push(...simd(vop.load), 3, ...unsigned(16 * pair), op.localGet, 3, ...simd(vop.f64Splat), ...simd(vop.f64Gt)); if (filter > 0 || pair > 0) code.push(...simd(vop.or)); } } code.push(...simd(vop.anyTrue), op.i32Eqz, op.if, 0x40); - code.push(op.localGet, ...unsigned(rowLocal), op.i32Const, blockRows, op.i32Add, op.localSet, ...unsigned(rowLocal)); + code.push(op.localGet, ...unsigned(rowLocal), op.i32Const, eightRows, op.i32Add, op.localSet, ...unsigned(rowLocal)); code.push(op.br, 2, op.end, op.end); }; // Scalars are discovered while generating, so code is generated first with locals placed after a provisional scalar count. @@ -1986,6 +2051,7 @@ export const generateWasm = ( if (lookupParameters > 0) convertLookupParameters(); if (indexParameters > 0) convertIndexParameters(); stampRangeChunks(); + if (filters > 0) startChunksAndBlocks(); if (mode === "rows") rowsLoop(); if (mode === "lanes") { const plan = planLanes(); @@ -1999,12 +2065,16 @@ export const generateWasm = ( } code.push(op.block, 0x40, op.loop, 0x40); code.push(op.localGet, ...unsigned(slot), op.localGet, 1, op.i32GeS, op.brIf, 1); + // With change filters, an outer loop finds each run of passing blocks and an inner one runs its rows. + if (filters > 0) { + findRun(); + code.push(op.block, 0x40, op.loop, 0x40, ...get(rowLocal), ...get(blockEnd), op.i32GeS, op.brIf, 1); + } if (upMode) { code.push(op.localGet, ...unsigned(upParameter(0)), op.localGet, ...unsigned(slot), op.i32Const, 2, op.i32Shl, op.i32Add, op.i32Load, 2, 0, op.localSet, ...unsigned(rowLocal)); code.push(op.localGet, ...unsigned(upParameter(1)), op.localGet, ...unsigned(slot), op.i32Const, 2, op.i32Shl, op.i32Add, op.i32Load, 2, 0, op.localSet, ...unsigned(parentLocal)); } - if (filters > 0) skipChunks(); - if (filters > 0 && simdEnabled && !upMode) skipBlocks(); + if (filters > 0 && simdEnabled && !upMode) skipEights(); code.push(op.block, 0x40); // A row that no filter passes goes to the next. if (filters > 0) { @@ -2028,6 +2098,7 @@ export const generateWasm = ( }); code.push(op.localGet, ...unsigned(slot), op.i32Const, 1, op.i32Add, op.localSet, ...unsigned(slot), op.br, 0); code.push(op.end, op.end); + if (filters > 0) code.push(op.br, 0, op.end, op.end); rowLocal = slot; accumulators.forEach((accumulator, k) => { const extra = lanes[k]; @@ -2064,7 +2135,7 @@ export const generateWasm = ( if (scalars.length !== discovered) throw unsupported; const vectorized = mode !== "scalar"; - const parameterTypes = [i32Type, i32Type, f64Type, f64Type, ...columns.map(() => i32Type), ...new Array(2 * filters).fill(i32Type), ...new Array(lookupParameters).fill(f64Type), ...new Array(indexParameters).fill(f64Type), ...new Array(upParameters).fill(i32Type), ...scalars.map(() => f64Type), ...accumulators.length > 0 ? [i32Type] : []]; + const parameterTypes = [i32Type, i32Type, f64Type, f64Type, ...columns.map(() => i32Type), ...new Array(3 * filters).fill(i32Type), ...new Array(lookupParameters).fill(f64Type), ...new Array(indexParameters).fill(f64Type), ...new Array(upParameters).fill(i32Type), ...scalars.map(() => f64Type), ...accumulators.length > 0 ? [i32Type] : []]; const localTypesList = [...new Array(program.locals).fill(f64Type)]; // Locals in order: program locals, then extras in creation order; the row index was the first extra. const declared = [...localTypesList.map(type => [1, type]), ...extraLocals.map(type => [1, type])]; diff --git a/packages/ecs/src/implementation/world/worker-window.test.ts b/packages/ecs/src/implementation/world/worker-window.test.ts new file mode 100644 index 00000000..d0590739 --- /dev/null +++ b/packages/ecs/src/implementation/world/worker-window.test.ts @@ -0,0 +1,43 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { describe, expect, it } from "vitest"; +import { controlWords, done, epoch, posted, ringSize, taskWords } from "./protocol.js"; +import { runWindow } from "./worker-window.js"; + +describe("a pool worker's window", () => { + it("runs a task the pool posted just before closing it, after the worker last looked for tasks", () => { + const control = new Int32Array(new SharedArrayBuffer(controlWords * 4)); + const statuses = new Int32Array(new SharedArrayBuffer(ringSize * 4)); + const payloads = new Float64Array(new SharedArrayBuffer(ringSize * taskWords * 8)); + const windowEpoch = 2; + control[epoch] = windowEpoch; + // The pool, between the worker's look at `posted` and its look at the epoch: it posts a task of this window, + // then closes it. + const load = Atomics.load; + let lookedForTasks = false; + let played = false; + const playPool = (array: Int32Array, index: number): number => { + const value = load(array, index); + if (array !== control) return value; + if (index === posted) lookedForTasks = true; + else if (index === epoch && lookedForTasks && !played) { + played = true; + payloads[6] = windowEpoch; + Atomics.store(control, posted, 1); + return Atomics.add(control, epoch, 1) + 1; + } + return value; + }; + const original = Object.getOwnPropertyDescriptor(Atomics, "load"); + let ran = 0; + try { + Object.defineProperty(Atomics, "load", { value: playPool, configurable: true, writable: true }); + runWindow(control, statuses, payloads, windowEpoch, () => { ran++; }); + } + finally { + if (original !== undefined) Object.defineProperty(Atomics, "load", original); + } + expect(played).toBe(true); + expect(ran).toBe(1); + expect(Atomics.load(control, done)).toBe(1); + }); +}); diff --git a/packages/ecs/src/implementation/world/worker-window.ts b/packages/ecs/src/implementation/world/worker-window.ts new file mode 100644 index 00000000..5d3755b6 --- /dev/null +++ b/packages/ecs/src/implementation/world/worker-window.ts @@ -0,0 +1,43 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { done, epoch, posted, ringSize, signal, taskWords } from './protocol.js'; + +/** + * A pool worker's window (`pool.ts`): runs the tasks of its ring, each by `runTask` with its payload's base, which + * throws when the task fails, blocking between them, until the pool closes the window or posts a task of a later one; + * then it returns, so the worker handles its messages. `control`, `statuses` and `payloads` are the worker's views of + * the pool's state; `windowEpoch` is the epoch the window opened in. + */ +export const runWindow = (control: Int32Array, statuses: Int32Array, payloads: Float64Array, windowEpoch: number, runTask: (base: number) => void): void => { + let next = Atomics.load(control, done); + for (;;) { + const seen = Atomics.load(control, signal); + if (next < Atomics.load(control, posted)) { + const ring = next % ringSize; + const base = ring * taskWords; + // A task from a later window waits until this worker has handled the messages sent before it. + if (payloads[base + 6] !== windowEpoch) return; + let status = next + 1; + try { + runTask(base); + } + catch { + status = -status; + } + Atomics.store(statuses, ring, status); + next++; + Atomics.store(control, done, next); + Atomics.notify(control, done); + continue; + } + // Closed, it leaves once it ran this window's tasks: the pool posts a task before it closes the window, so a task + // posted since the check above is seen now, and runs above if it's this window's. + if (Atomics.load(control, epoch) !== windowEpoch) { + if (next < Atomics.load(control, posted)) continue; + return; + } + // Tiers follow each other closely, so spin briefly before paying for a blocking wait's wake-up. + const deadline = performance.now() + 0.05; + while (Atomics.load(control, signal) === seen && performance.now() < deadline); + if (Atomics.load(control, signal) === seen) Atomics.wait(control, signal, seen); + } +}; diff --git a/packages/ecs/src/implementation/world/worker.ts b/packages/ecs/src/implementation/world/worker.ts index b0ed1928..944a4afd 100644 --- a/packages/ecs/src/implementation/world/worker.ts +++ b/packages/ecs/src/implementation/world/worker.ts @@ -3,11 +3,12 @@ import { buildKernel, buildViews, phases, readPlan, runBuildStep } from './index import type { Entity } from '../../interface/entity/entity.js'; import { createBlock, endBlock, locate, reserveFrom, tableLength, type Block } from './entities.js'; import { createBatch, positionalSpawner, type Positional, type SpawnBatch } from './staging.js'; -import type { Kernel, KernelCommands, KernelModule } from './kernel.js'; +import { blockShift, chunkShift, type Kernel, type KernelCommands, type KernelModule } from './kernel.js'; import { layoutOf, viewsOf, type Layout, type TypedArray, type Views } from '../../interface/registry/layout.js'; import { emptyViews } from './arrays.js'; import { claimPiece } from './claims.js'; -import { controlWords, done, epoch, idleKernel, lineWords, posted, ringBase, ringCapacity, ringCursor, ringNeeded, ringSize, signal, taskFields, taskWords, type FromWorker, type KernelSetup, type LookupSetup, type ToWorker, type WorkerError } from './protocol.js'; +import { controlWords, idleKernel, lineWords, ringBase, ringCapacity, ringCursor, ringNeeded, ringSize, taskFields, taskWords, type FromWorker, type KernelSetup, type LookupSetup, type ToWorker, type WorkerError } from './protocol.js'; +import { runWindow } from './worker-window.js'; import { beginCommand, createRingWriter, encoderOf, endSegment, noExtent, objectStart, putComponent, putStart, putValue, startSegment, type Encoder, type Extent } from './command-ring.js'; import { forEachPassingRange, type Filter } from './ranges.js'; import { indexParameterCount, instantiateWasm, scalarValues, type WasmRun } from './wasm.js'; @@ -16,7 +17,7 @@ import { coveringBuffer, pageBytes, pagesWord } from './memory.js'; import { endpointChannel, scopeChannel } from './port.js'; import { archetypeField, capacityField, cursorField, entitiesVersionField, entitiesWord, freeCapacityField, freeField, generationField, linearFlag, - recordCapacity, recordEntities, recordStorages, recordVersion, recordWords, rowField, storageAdded, storageChanged, storageChunkAdded, + recordCapacity, recordEntities, recordStorages, recordVersion, recordWords, rowField, storageAdded, storageBlockAdded, storageBlockChanged, storageChanged, storageChunkAdded, storageChunkChanged, storageChunks, storageColumn, storageComponent, storageFlags, storageWords, tableLengthWord, tableOffsetWord, tickedFlag, } from './layout-directory.js'; @@ -27,6 +28,8 @@ type StorageViews = { readonly changed: Float64Array; readonly chunkAdded: Float64Array; readonly chunkChanged: Float64Array; + readonly blockAdded: Float64Array; + readonly blockChanged: Float64Array; }; /** A worker's views of one archetype's columns, as of its record's `version`. */ @@ -161,6 +164,8 @@ const viewArchetype = (record: number, version: number): ArchetypeViews => { changed: ticked ? new Float64Array(memoryBuffer, words[base + storageChanged], capacity) : noTicks, chunkAdded: ticked ? new Float64Array(memoryBuffer, words[base + storageChunkAdded], words[base + storageChunks]) : noTicks, chunkChanged: ticked ? new Float64Array(memoryBuffer, words[base + storageChunkChanged], words[base + storageChunks]) : noTicks, + blockAdded: ticked ? new Float64Array(memoryBuffer, words[base + storageBlockAdded], words[base + storageChunks] << (chunkShift - blockShift)) : noTicks, + blockChanged: ticked ? new Float64Array(memoryBuffer, words[base + storageBlockChanged], words[base + storageChunks] << (chunkShift - blockShift)) : noTicks, }); } return { version, entities: new Float64Array(memoryBuffer, words[record + recordEntities], capacity), storages }; @@ -248,15 +253,15 @@ const readerArchetype = (id: number) => { /** The one element a muted tick column's stores land on. */ const mutedColumn = new Float64Array(1); -/** A kernel's change filters in an archetype: `added` then `changed`, rows and chunks. */ +/** A kernel's change filters in an archetype: `added` then `changed`, rows, chunks and blocks. */ const filtersOf = (loaded: LoadedKernel, archetype: ArchetypeViews): Filter[] => [ ...loaded.setup.added.flatMap(id => { const storage = archetype.storages.get(id); - return storage === undefined ? [] : [{ ticks: storage.added, chunks: storage.chunkAdded }]; + return storage === undefined ? [] : [{ ticks: storage.added, chunks: storage.chunkAdded, blocks: storage.blockAdded }]; }), ...loaded.setup.changed.flatMap(id => { const storage = archetype.storages.get(id); - return storage === undefined ? [] : [{ ticks: storage.changed, chunks: storage.chunkChanged }]; + return storage === undefined ? [] : [{ ticks: storage.changed, chunks: storage.chunkChanged, blocks: storage.blockChanged }]; }), ]; @@ -268,12 +273,13 @@ const bind = (loaded: LoadedKernel, archetypeId: number, archetype: ArchetypeVie if (storage === undefined) throw new Error(`Archetype ${archetypeId} lacks component ${source.id}.`); if (source.kind === "changed") return storage.changed; if (source.kind === "chunks") return storage.chunkChanged; + if (source.kind === "blocks") return storage.blockChanged; const layout = layouts[source.id]; if (layout.kind !== "linear") throw new Error(`Component ${source.id} isn't linear.`); return storage.views[layout.leaves[source.leaf].view]; }); const filters = filtersOf(loaded, archetype); - const addresses = [...columns.map(column => column.byteOffset), ...filters.map(filter => filter.ticks.byteOffset), ...filters.map(filter => filter.chunks.byteOffset)]; + const addresses = [...columns.map(column => column.byteOffset), ...filters.map(filter => filter.ticks.byteOffset), ...filters.map(filter => filter.chunks.byteOffset), ...filters.map(filter => filter.blocks.byteOffset)]; return { views: archetype, columns, filters, addresses }; }; @@ -424,54 +430,30 @@ const runBuildTask = (payloads: Float64Array, base: number, buffer: ArrayBufferL * Runs tasks from this worker's ring in shared memory, blocking between them, until the main thread asks it to leave; * then it returns to handling messages. */ -const runWindow = (message: Extract): void => { +const openWindow = (message: Extract): void => { const buffer = required(memory, "memory").buffer; // The pool's control state never moves, so these views last the window. const control = new Int32Array(buffer, message.control + message.worker * controlWords * 4, controlWords); const statuses = new Int32Array(buffer, message.statuses + message.worker * ringSize * 4, ringSize); const payloads = new Float64Array(buffer, message.payloads + message.worker * ringSize * taskWords * 8, ringSize * taskWords); - let next = Atomics.load(control, done); - for (;;) { - const seen = Atomics.load(control, signal); - if (next < Atomics.load(control, posted)) { - const ring = next % ringSize; - const base = ring * taskWords; - // A task from a later window waits until this worker has handled the messages sent before it. - if (payloads[base + 6] !== message.epoch) return; - let status = next + 1; - try { - refreshMemory(); - const kernel = payloads[base]; - if (kernel === buildKernel) runBuildTask(payloads, base, buffer, message); - else if (kernel !== idleKernel) { - const loaded = kernels.get(kernel); - if (loaded === undefined) throw new Error("Worker can't run this task."); - const line = payloads[base + 8]; - const participants = payloads[base + 11]; - const scalarCount = payloads[base + 7]; - const partials = loaded.setup.reduces ? payloads[base + taskFields + scalarCount] : -1; - const partialBytes = loaded.setup.reduces ? payloads[base + taskFields + scalarCount + 1] : 0; - runWasm(loaded, payloads[base + 1], payloads[base + 2], payloads[base + 3], payloads[base + 4], payloads[base + 5], - Array.from(payloads.subarray(base + taskFields, base + taskFields + scalarCount)), - line < 0 ? null : { words: new Int32Array(buffer, message.claims + line * lineWords * 4, participants * lineWords), self: payloads[base + 10], participants }, - payloads[base + 9], partials ?? -1, partialBytes ?? 0); - } - } - catch { - status = -status; - } - Atomics.store(statuses, ring, status); - next++; - Atomics.store(control, done, next); - Atomics.notify(control, done); - continue; + runWindow(control, statuses, payloads, message.epoch, base => { + refreshMemory(); + const kernel = payloads[base]; + if (kernel === buildKernel) runBuildTask(payloads, base, buffer, message); + else if (kernel !== idleKernel) { + const loaded = kernels.get(kernel); + if (loaded === undefined) throw new Error("Worker can't run this task."); + const line = payloads[base + 8]; + const participants = payloads[base + 11]; + const scalarCount = payloads[base + 7]; + const partials = loaded.setup.reduces ? payloads[base + taskFields + scalarCount] : -1; + const partialBytes = loaded.setup.reduces ? payloads[base + taskFields + scalarCount + 1] : 0; + runWasm(loaded, payloads[base + 1], payloads[base + 2], payloads[base + 3], payloads[base + 4], payloads[base + 5], + Array.from(payloads.subarray(base + taskFields, base + taskFields + scalarCount)), + line < 0 ? null : { words: new Int32Array(buffer, message.claims + line * lineWords * 4, participants * lineWords), self: payloads[base + 10], participants }, + payloads[base + 9], partials ?? -1, partialBytes ?? 0); } - if (Atomics.load(control, epoch) !== message.epoch) return; - // Tiers follow each other closely, so spin briefly before paying for a blocking wait's wake-up. - const deadline = performance.now() + 0.05; - while (Atomics.load(control, signal) === seen && performance.now() < deadline); - if (Atomics.load(control, signal) === seen) Atomics.wait(control, signal, seen); - } + }); }; /** Where messages come from and replies go: this worker's own scope, until a lent worker's owner gives it a port. */ @@ -493,7 +475,7 @@ const handle = async (message: ToWorker): Promise => { return; } if (message.kind === "window") { - runWindow(message); + openWindow(message); return; } if (message.kind === "layout") { diff --git a/packages/ecs/src/implementation/world/workers.fixture.ts b/packages/ecs/src/implementation/world/workers.fixture.ts index 0bceffef..fec596d0 100644 --- a/packages/ecs/src/implementation/world/workers.fixture.ts +++ b/packages/ecs/src/implementation/world/workers.fixture.ts @@ -81,7 +81,7 @@ const scatter = System.create({ name: "workersTest/scatter" }, ({ commands }) => Resource.register(seconds, { type: { type: "number", default: 0.25 } }); /** Spawns `count` moving entities, each following the one before it. */ -const seed = System.create({ name: "workersTest/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const seed = System.create({ name: "workersTest/seed" }, ({ commands }, input: { readonly count: number }) => { let previous = commands.spawn([position, velocity])({ [position]: [0, 0], [velocity]: [1, 0] }); for (let i = 1; i < input.count; i++) { previous = commands.spawn([position, velocity, target])({ [position]: [i, 0], [velocity]: [1, i % 7], [target]: previous }); @@ -132,9 +132,8 @@ const main = System.create({ name: "workersTest/main", systems: [integrate, spli const stamp = System.create({ name: "workersTest/stamp", kernel: "cpu", - input: { type: "object", properties: { entity: Entity.schema } }, components: { read: [position] }, -}, (row, { input, commands }) => { +}, (row, { commands }, input: { readonly entity: Entity }) => { const [x, y] = row[position]; commands.add(input.entity, offset, x * 1000 + y); }); @@ -160,9 +159,8 @@ const sources = System.create({ name: "workersTest/sources" }, ({ commands }) => const emit = System.create({ name: "workersTest/emit", kernel: "cpu", - input: { type: "object", properties: { edit: { type: "boolean" } } }, components: { read: [offset], not: [spawned] }, -}, (row, { input, commands }) => { +}, (row, { commands }, input: { readonly edit: boolean }) => { const entity = commands.spawn([position, spawned, tagC])({ [position]: [row[offset], -row[offset]] }); if (input.edit) { commands.add(entity, velocity, [row[offset], 1]); diff --git a/packages/ecs/src/interface/gpu/accessors.ts b/packages/ecs/src/interface/gpu/accessors.ts index d628fad3..9d40716d 100644 --- a/packages/ecs/src/interface/gpu/accessors.ts +++ b/packages/ecs/src/interface/gpu/accessors.ts @@ -28,14 +28,15 @@ const reserved = new Set(("alias break case const const_assert continue continui /** * Accessors for a render system's shader, in `accessors` order, or the reason one can't be read on the GPU: scalars - * and tuples of two to four scalars of one type, laid out in 4-byte words. + * and tuples of two to four scalars of one type, laid out in 4-byte words. An `ordered` system's accessors take a + * row's position in its order, which `ecs_address` resolves to a table's column. */ -export const accessorsOf = (shader: Shader): readonly Accessor[] | string => { +export const accessorsOf = (shader: Shader, ordered = false): readonly Accessor[] | string => { const accessors: Accessor[] = []; const entries = Object.entries(shader.accessors); // A table record is a row count, then a word address per accessor. if (entries.length > 63) return "it names more than 63 accessors"; - for (const [name, key] of entries) { + for (const [column, [name, key]] of entries.entries()) { const component = labelOf(key); if (!/^[A-Za-z][A-Za-z0-9_]*$/.test(name) || name.startsWith("ecs_") || reserved.has(name)) return `"${name}" isn't usable as a WGSL function name`; const declaration = declarationOf(key); @@ -48,26 +49,41 @@ export const accessorsOf = (shader: Shader): readonly Accessor[] | string => { return `"${component}" isn't laid out in 4-byte words of f32, i32, u32 or booleans; give it a "storage" or "vertex" layout or 4-byte scalars`; } const words = stride / 4; - const word = (leaf: Leaf): string => `ecs_table.ecs_${name} + row * ${words}u + ${leaf.offset / 4}u`; + const word = ordered + ? (leaf: Leaf): string => `ecs_base + ${leaf.offset / 4}u` + : (leaf: Leaf): string => `ecs_table.ecs_${name} + row * ${words}u + ${leaf.offset / 4}u`; + const base = ordered ? `let ecs_base = ecs_address(row, ${column}u, ${words}u); ` : ""; const [first] = leaves; if (leaves.length === 1 && first.path.length === 0) { - accessors.push({ name, component: key, code: `fn ${name}(row: u32) -> ${types[0]} { return ${readOf(first, word(first))}; }` }); + accessors.push({ name, component: key, code: `fn ${name}(row: u32) -> ${types[0]} { ${base}return ${readOf(first, word(first))}; }` }); continue; } const tuple = leaves.length >= 2 && leaves.length <= 4 && leaves.every((leaf, i) => leaf.path.length === 1 && leaf.path[0] === i && types[i] === types[0]); if (!tuple) return `"${component}" isn't a scalar or a tuple of two to four scalars of one type`; const items = leaves.map(leaf => readOf(leaf, word(leaf))).join(", "); - accessors.push({ name, component: key, code: `fn ${name}(row: u32) -> vec${leaves.length}<${types[0]}> { return vec${leaves.length}<${types[0]}>(${items}); }` }); + accessors.push({ name, component: key, code: `fn ${name}(row: u32) -> vec${leaves.length}<${types[0]}> { ${base}return vec${leaves.length}<${types[0]}>(${items}); }` }); } return accessors; }; /** The WGSL declarations of a render system's bind group and accessors. */ -export const declarationsOf = (group: number, accessors: readonly Accessor[]): string => [ +export const declarationsOf = (group: number, accessors: readonly Accessor[], ordered = false): string => [ `@group(${group}) @binding(0) var ecs_heap: array;`, - `struct EcsTable { ecs_rows: u32, ${accessors.map(accessor => `ecs_${accessor.name}: u32`).join(", ")} }`, + ordered + ? "struct EcsTable { ecs_rows: u32, ecs_order: u32, ecs_tables: u32, ecs_shift: u32 }" + : `struct EcsTable { ecs_rows: u32, ${accessors.map(accessor => `ecs_${accessor.name}: u32`).join(", ")} }`, `@group(${group}) @binding(1) var ecs_table: EcsTable;`, `fn ecs_count() -> u32 { return ecs_table.ecs_rows; }`, + ...ordered ? [orderedAddress(accessors.length)] : [], ...accessors.map(accessor => accessor.code), "", ].join("\n"); + +/** + * The word address of a column's value at a position of an ordered system's order. The order block holds, by position, + * a table's slot above `ecs_shift` bits and its row below them; the tables block holds, by slot, a word address per + * accessor. + */ +const orderedAddress = (columns: number): string => + "fn ecs_address(row: u32, column: u32, words: u32) -> u32 { let ecs_entry = ecs_heap[ecs_table.ecs_order + row]; " + + `return ecs_heap[ecs_table.ecs_tables + (ecs_entry >> ecs_table.ecs_shift) * ${columns}u + column] + (ecs_entry & ((1u << ecs_table.ecs_shift) - 1u)) * words; }`; diff --git a/packages/ecs/src/interface/gpu/wgsl.ts b/packages/ecs/src/interface/gpu/wgsl.ts index b9f7a39c..8def4790 100644 --- a/packages/ecs/src/interface/gpu/wgsl.ts +++ b/packages/ecs/src/interface/gpu/wgsl.ts @@ -7,15 +7,18 @@ import { accessorsOf, declarationsOf } from './accessors.js'; /** * The WGSL a render system's shaders include: its bind group at `shader.group` (the heap and its table record), and a * function per accessor returning that component of a row, such as `fn position(row: u32) -> vec3`, plus - * `ecs_count()`. Depends only on registered schemas, so pipelines can be built before any world exists. + * `ecs_count()`. For a system with `order`, `row` is an instance's position in its order and `ecs_count()` the number + * of instances. Depends only on registered schemas, so pipelines can be built before any world exists. * Throws if the system isn't a render system or a component can't be read on the GPU. */ export const wgsl = (system: System): string => { const definition = definitionOf(system); - const shader = definition?.kind === "render" ? definition.declaration.shader : undefined; - if (shader === undefined) throw new Error(`System "${system.name}" isn't a render system.`); + const declaration = definition?.kind === "render" ? definition.declaration : undefined; + const shader = declaration?.shader; + if (declaration === undefined || shader === undefined) throw new Error(`System "${system.name}" isn't a render system.`); const { name } = system; - const accessors = accessorsOf(shader); + const ordered = declaration.order !== undefined; + const accessors = accessorsOf(shader, ordered); if (typeof accessors === "string") throw new Error(`Render system "${name}": ${accessors}.`); - return declarationsOf(shader.group ?? 0, accessors); + return declarationsOf(shader.group ?? 0, accessors, ordered); }; diff --git a/packages/ecs/src/interface/indexes/create.test.ts b/packages/ecs/src/interface/indexes/create.test.ts index c72b27c0..8ab3007d 100644 --- a/packages/ecs/src/interface/indexes/create.test.ts +++ b/packages/ecs/src/interface/indexes/create.test.ts @@ -143,4 +143,9 @@ describe("systems ordering their rows by an index", () => { expect(() => untypedSystem({ name: "bad", components: { write: [amount] }, order: ranked })).toThrow(`its rows are ordered by an index keyed by "indexCreateTest/amount", which the system writes`); expect(() => untypedSystem({ name: "bad", kernel: "cpu", components: { write: [amount] }, order: Index.create({ key: [name], sorted: true }) })).toThrow(`ordered by an index keyed by "indexCreateTest/name", a string, which kernels don't read yet`); }); + it("throw for a render system's order by what isn't a sorted index", () => { + const untypedRender = (order: object): unknown => Reflect.apply(System.render, undefined, [{ name: "bad", order, shader: { accessors: { amount } } }, () => {}]); + expect(() => untypedRender({ name: "fake" })).toThrow(`System "bad" orders its rows by a value Index.create didn't return`); + expect(() => untypedRender(Index.create({ name: "indexCreateTest/hashedForRender", key: [amount] }))).toThrow(`orders its rows by index "indexCreateTest/hashedForRender", which isn't sorted`); + }); }); diff --git a/packages/ecs/src/interface/indexes/index.type-test.ts b/packages/ecs/src/interface/indexes/index.type-test.ts index 7966ce0f..d1c50e39 100644 --- a/packages/ecs/src/interface/indexes/index.type-test.ts +++ b/packages/ecs/src/interface/indexes/index.type-test.ts @@ -107,9 +107,8 @@ export const separate = System.create({ // A unique lookup gives the row or null. export const applyDelta = System.create({ - input: { type: "object", properties: { id: { type: "number" }, hp: { type: "number" } } }, lookup: { known: { index: byServerId, read: [health] } }, -}, ({ input, lookup, commands }) => { +}, ({ lookup, commands }, input: { readonly id: number; readonly hp: number }) => { const known = lookup.known(input.id); check>(); if (known === null) commands.spawn([serverId, health])({ [serverId]: input.id, [health]: input.hp }); diff --git a/packages/ecs/src/interface/registry/schedule.ts b/packages/ecs/src/interface/registry/schedule.ts index 0127bcd1..0b012995 100644 --- a/packages/ecs/src/interface/registry/schedule.ts +++ b/packages/ecs/src/interface/registry/schedule.ts @@ -9,18 +9,14 @@ import { lookupKeysOf, orderKeysOf } from './indexes.js'; type Key = symbol; -/** Throws unless each leaf's input is an object schema of named fields, and a field several declare has one schema. */ +/** Throws unless a field of the input several leaves take has one schema. */ const checkInputs = (leaves: readonly Leaf[]): void => { const fields = new Map(); for (const leaf of leaves) { - const { input } = leaf.declaration; + const { input } = leaf; const name = labelOfItem(leaf); if (input === undefined) continue; - const schema: Schema = input; - if (!("type" in schema) || schema.type !== "object" || typeof schema.properties !== "object" || schema.properties === null) { - throw new Error(`System "${name}": input is an object schema of named fields.`); - } - for (const [field, property] of Object.entries(schema.properties)) { + for (const [field, property] of Object.entries(input.properties)) { const seen = fields.get(field); if (seen === undefined) fields.set(field, { schema: property, system: name }); else if (!sameDeclaration(seen.schema, property)) { diff --git a/packages/ecs/src/interface/system/body.ts b/packages/ecs/src/interface/system/body.ts index 4f89a7e1..135134d2 100644 --- a/packages/ecs/src/interface/system/body.ts +++ b/packages/ecs/src/interface/system/body.ts @@ -3,7 +3,6 @@ import type { Component } from '../component/component.js'; import type { Entity } from '../entity/entity.js'; import type { Resource } from '../resource/resource.js'; import type { Holding } from '../resource/condition.js'; -import type { Schema } from '../schema/schema.js'; import type { Commands } from './commands.js'; import type { Declaration } from './declaration.js'; import type { Listed, Property } from './property.js'; @@ -36,15 +35,18 @@ export type Context = /** Each resource it reduces into, as a function contributing a value to the run's fold. */ readonly reduce: { readonly [K in Reduced]: (value: Resource.Value) => void }; }) - & (Property extends Schema ? { readonly input: Schema.ToType> } : {}) & (Property extends object ? { /** The parent's `up.read` values, or null for a row without a parent. */ readonly up: { readonly [K in Listed, "read"> & Component.Key]: Component.Value } | null; } : {}) & (D extends { readonly kernel: "gpu" } | { readonly shader: object } | { readonly observer: true } ? {} : { readonly commands: Commands }); -/** `(row, context)` for a system with `components`, `(context)` for one without, and `(gpu, context)` for a render system. */ -export type Body = +/** + * `(row, context, input)` for a system with `components`, `(context, input)` for one without, and `(gpu, context)` for + * a render system. `input` is the value each run passes, of the type the body gives its parameter; a body without one + * takes none. + */ +export type Body = D extends { readonly shader: object } ? (gpu: RenderContext, context: Context) => void : - Property extends undefined ? (context: Context) => void : - (row: Row, context: Context) => void; + D extends { readonly components: object } ? (row: Row, context: Context, input: I) => void : + (context: Context, input: I) => void; diff --git a/packages/ecs/src/interface/system/check.ts b/packages/ecs/src/interface/system/check.ts index fb94dca8..2267dbfa 100644 --- a/packages/ecs/src/interface/system/check.ts +++ b/packages/ecs/src/interface/system/check.ts @@ -11,7 +11,7 @@ import { labelOfItem } from '../registry/items.js'; import type { Schema } from '../schema/schema.js'; import type { Composite } from './composite.js'; import type { Definition } from './definition.js'; -import type { Leaf } from './leaf.js'; +import type { InputSchema, Leaf } from './leaf.js'; type Key = symbol; @@ -136,20 +136,19 @@ const checkObserver = (leaf: Leaf): void => { if (declaration.kernel !== undefined) fail("isn't a kernel"); if ((declaration.components?.write?.length ?? 0) + (declaration.resources?.write?.length ?? 0) > 0) fail("writes nothing"); if (reducedBy(leaf).length > 0) fail("reduces into nothing"); - if (declaration.input !== undefined) fail("takes no input"); + if (leaf.input !== undefined) fail("takes no input"); if (declaration.up !== undefined) fail("has no up"); if ((declaration.before?.length ?? 0) + (declaration.after?.length ?? 0) + (declaration.during?.length ?? 0) > 0) fail("is ordered with nothing"); }; /** * Throws for a leaf that names a component or resource that isn't registered as one, lists a key in both `read` and - * `write`, writes a reference, has a lookup reading a component it writes, takes input that isn't an object schema, - * uses object components and resources of different threads, declares what an observer system can't, or whose + * `write`, writes a reference, has a lookup reading a component it writes, uses object components and resources of different threads, declares what an observer system can't, or whose * reductions, conditions, `up` or placement can't hold. */ export const checkLeaf = (leaf: Leaf): void => { const { name, declaration } = leaf; - const { components, resources, lookup, input } = declaration; + const { components, resources, lookup } = declaration; checkLists(name, "components", components); checkLists(name, "resources", resources); checkWorldWritten(`System "${name}"`, resources?.write); @@ -166,12 +165,6 @@ export const checkLeaf = (leaf: Leaf): void => { for (const key of orderKeysOf(declaration)) { if (writes.has(key)) throw new Error(`System "${name}": its rows are ordered by an index keyed by "${labelOf(key)}", which the system writes.`); } - if (input !== undefined) { - const schema: Schema = input; - if (!("type" in schema) || schema.type !== "object" || typeof schema.properties !== "object" || schema.properties === null) { - throw new Error(`System "${name}": input is an object schema of named fields.`); - } - } checkTarget(leaf); threadOf(declaration, "System", name); requireRegistered("System", name, "component", components?.has); @@ -190,13 +183,31 @@ export const checkLeaf = (leaf: Leaf): void => { checkPlacement(leaf); }; +const isInputSchema = (value: unknown): value is InputSchema => + typeof value === "object" && value !== null && Reflect.get(value, "type") === "object" + && typeof Reflect.get(value, "properties") === "object" && Reflect.get(value, "properties") !== null; + +/** + * The schema of a body's input, which the plugin passes from the type of its parameter after the `before` before it. + * Throws for a body taking input without one, as one not built with the plugin does, and for a schema that isn't an + * object of named fields. + */ +export const inputSchemaOf = (name: string, body: (...parameters: never[]) => void, before: number, input: unknown): InputSchema | undefined => { + if (input === undefined) { + if (body.length > before) throw new Error(`System "${name}" takes input, whose schema the plugin gives from the type of its body's parameter: write its body in place and build it with the kernel plugin.`); + return undefined; + } + if (!isInputSchema(input)) throw new Error(`System "${name}": input is an object schema of named fields.`); + return input; +}; + /** * Throws if a system on one entity takes it from what isn't an entity field of its input, has no components, or is a * kernel, has `up` or `order`, or reduces. */ const checkTarget = (leaf: Leaf): void => { - const { name, declaration } = leaf; - const { target, input, components, kernel, up, order, reduce } = declaration; + const { name, declaration, input } = leaf; + const { target, components, kernel, up, order, reduce } = declaration; if (target === undefined) return; const fail = (message: string): never => { throw new Error(`System "${name}" runs on one entity, so it ${message}.`); }; const field: Schema | undefined = input !== undefined && "properties" in input ? Reflect.get(input.properties, target) : undefined; diff --git a/packages/ecs/src/interface/system/composite-input.type-test.ts b/packages/ecs/src/interface/system/composite-input.type-test.ts index edd7a682..a8d1ae94 100644 --- a/packages/ecs/src/interface/system/composite-input.type-test.ts +++ b/packages/ecs/src/interface/system/composite-input.type-test.ts @@ -5,22 +5,10 @@ import { System } from './system.js'; type Equal = (() => T extends X ? 1 : 2) extends (() => T extends Y ? 1 : 2) ? true : false; const check = <_T extends true>() => { }; -const move = System.create({ - name: "compositeInputTest/move", - input: { type: "object", properties: { distance: { type: "number" } } }, -}, ({ input }) => { void input.distance; }); -const step = System.create({ - name: "compositeInputTest/step", - input: { type: "object", properties: { distance: { type: "number" }, dt: { type: "number" } } }, -}, ({ input }) => { void input.dt; }); -const pause = System.create({ - name: "compositeInputTest/pause", - input: { type: "object", properties: { dt: { type: "number" } }, required: [] }, -}, ({ input }) => { void input.dt; }); -const label = System.create({ - name: "compositeInputTest/label", - input: { type: "object", properties: { distance: { type: "string" } } }, -}, ({ input }) => { void input.distance; }); +const move = System.create({ name: "compositeInputTest/move" }, (_context, input: { readonly distance: number }) => { void input.distance; }); +const step = System.create({ name: "compositeInputTest/step" }, (_context, input: { readonly distance: number; readonly dt: number }) => { void input.dt; }); +const pause = System.create({ name: "compositeInputTest/pause" }, (_context, input: { readonly dt?: number }) => { void input.dt; }); +const label = System.create({ name: "compositeInputTest/label" }, (_context, input: { readonly distance: string }) => { void input.distance; }); const idle = System.create({ name: "compositeInputTest/idle" }, () => {}); // A composite takes every field its members' inputs have; a field in several is one value for all of them. @@ -57,8 +45,3 @@ void world.run(frame, { distance: 1, dt: 0.5 }); // @ts-expect-error run takes every field void world.run(frame, { distance: 1 }); -System.create({ - name: "compositeInputTest/scalar", - // @ts-expect-error input is an object of named fields - input: { type: "number" }, -}, () => {}); diff --git a/packages/ecs/src/interface/system/create.test.ts b/packages/ecs/src/interface/system/create.test.ts index 1f78e727..5f2971f9 100644 --- a/packages/ecs/src/interface/system/create.test.ts +++ b/packages/ecs/src/interface/system/create.test.ts @@ -51,6 +51,8 @@ Component.register(height, { type: { type: "number", precision: 1 } }); /** Creates as an untyped caller would, bypassing the types' checks. */ const create = (declaration: object): unknown => Reflect.apply(System.create, undefined, [declaration, () => {}]); +/** As the plugin calls it for a body taking input: with the schema of its input after the body. */ +const createTaking = (declaration: object, input: unknown): unknown => Reflect.apply(System.create, undefined, [declaration, (_context: unknown, _input: unknown) => {}, input]); describe("System.create", () => { it("records the system, its value holding its declaration and body", () => { @@ -106,7 +108,7 @@ describe("System.create", () => { expect(() => create({ name: "typesSystemTest/both", components: { read: [position], write: [position] } })).toThrow("in both components read and write"); expect(() => create({ name: "typesSystemTest/bothResources", resources: { read: [time], write: [time] } })).toThrow("in both resources read and write"); expect(() => create({ name: "typesSystemTest/lookup", components: { write: [position] }, lookup: { others: { read: [position] } } })).toThrow('lookup "others" reads "typesSystemTest/position", which the system writes'); - expect(() => create({ name: "typesSystemTest/input", input: { type: "number" } })).toThrow("input is an object schema of named fields"); + expect(() => createTaking({ name: "typesSystemTest/input" }, { type: "number" })).toThrow("input is an object schema of named fields"); expect(labelOfItem(System.create({ name: "typesSystemTest/both" }, () => {}))).toBe("typesSystemTest/both"); }); it("rejects at creation what the registry rules out, for untyped callers", () => { @@ -132,13 +134,13 @@ describe("System.create", () => { [{ observer: true, components: { write: [position] } }, "is an observer, so it writes nothing"], [{ observer: true, resources: { write: [time] } }, "is an observer, so it writes nothing"], [{ observer: true, components: { has: [position] }, reduce: { [total]: "+" } }, "is an observer, so it reduces into nothing"], - [{ observer: true, input: { type: "object", properties: {} } }, "is an observer, so it takes no input"], [{ observer: true, kernel: "cpu", components: { read: [position] } }, "is an observer, so it isn't a kernel"], [{ observer: true, components: { read: [position] }, up: { through: parent } }, "is an observer, so it has no up"], [{ observer: true, resources: { read: [time] }, after: [System.create(() => {})] }, "is an observer, so it is ordered with nothing"], [{ observer: true, resources: { read: [time] }, before: [System.create(() => {})] }, "is an observer, so it is ordered with nothing"], [{ observer: true, resources: { read: [time] }, during: [System.create(() => {})] }, "is an observer, so it is ordered with nothing"], ]; + expect(() => createTaking({ name: "typesSystemTest/observing", observer: true }, { type: "object", properties: {} })).toThrow("is an observer, so it takes no input"); rejected.forEach(([declaration, message], index) => { expect(() => create({ name: `typesSystemTest/rejected/${index}`, ...declaration })).toThrow(message); // A rejected system records nothing, so its name labels only the next. @@ -203,8 +205,8 @@ describe("System.create of a composite", () => { }); it("rejects a member no constructor returned, and an input field two members declare differently", () => { expect(() => System.create({ name: "typesSystemTest/forged", systems: [{ name: "typesSystemTest/showMember" }] })).toThrow('member "typesSystemTest/showMember" isn\'t a system a System constructor returned'); - const move = System.create({ name: "typesSystemTest/moveBy", input: { type: "object", properties: { distance: { type: "number" } } } }, () => {}); - const tag = System.create({ name: "typesSystemTest/tagBy", input: { type: "object", properties: { distance: { type: "string" } } } }, () => {}); + const move = System.create({ name: "typesSystemTest/moveBy" }, (_context, input: { readonly distance: number }) => {}); + const tag = System.create({ name: "typesSystemTest/tagBy" }, (_context, input: { readonly distance: string }) => {}); const nested = System.create({ name: "typesSystemTest/nested", systems: [tag] }); const untyped: (...parameters: never[]) => unknown = System.create; const composite = (declaration: object): unknown => Reflect.apply(untyped, undefined, [declaration]); diff --git a/packages/ecs/src/interface/system/create.ts b/packages/ecs/src/interface/system/create.ts index 3160273a..61c71e43 100644 --- a/packages/ecs/src/interface/system/create.ts +++ b/packages/ecs/src/interface/system/create.ts @@ -6,10 +6,9 @@ import { takeIdOfDeclaration } from '../registry/ids.js'; import { nameOf } from '../registry/items.js'; import { claimSystem } from '../registry/program.js'; import type { Resource } from '../resource/resource.js'; -import type { Schema } from '../schema/schema.js'; import type { Body } from './body.js'; import type { Commands } from './commands.js'; -import { checkComposite, checkLeaf } from './check.js'; +import { checkComposite, checkLeaf, inputSchemaOf } from './check.js'; import type { Composite } from './composite.js'; import type { CompositeDeclaration } from './composite-declaration.js'; import type { Declaration } from './declaration.js'; @@ -82,27 +81,22 @@ type GpuKernel = { }; /** - * An observer system only reads, after a sync point: it writes and reduces into nothing, takes no input, isn't a - * kernel, has no `up`, and is ordered with nothing. + * An observer system only reads, after a sync point: it writes and reduces into nothing, isn't a kernel, has no `up`, + * and is ordered with nothing. That it takes no input is checked at its body. */ type Observer = { readonly kernel?: never; readonly components?: { readonly write?: never }; readonly resources?: { readonly write?: never }; readonly reduce?: never; - readonly input?: never; readonly up?: never; readonly before?: never; readonly after?: never; readonly during?: never; }; -/** The fields of input `I` that hold an entity. */ -type EntityFields = { [K in keyof I]-?: [Exclude] extends [Entity] ? K : never }[keyof I] & string; - -/** A system on one entity takes it from an entity field of its input, and runs on its row, in no order. */ -type Targeted = { - readonly target?: EntityFields>; +/** A system on one entity runs on its row, in no order; that its target is an entity field of its input is checked at its body. */ +type Targeted = { readonly kernel?: never; readonly up?: never; readonly order?: never; @@ -113,16 +107,34 @@ type Validate = Lists & IndexLookups & Order - & (D extends { readonly input: Schema } ? { readonly components?: { readonly added?: never; readonly changed?: never } } : {}) & (D extends { readonly kernel: "cpu" | "gpu" } ? Kernel : {}) & (D extends { readonly kernel: "gpu" } ? GpuKernel : {}) // `up` and `order` order rows, so a system without them has neither, and one has at most one of them. & (Property extends undefined ? { readonly up?: { readonly [key: string]: never }; readonly order?: { readonly "error: a system without components runs once, in no order": true } } : {}) & (D extends { readonly up: object } ? { readonly order?: { readonly "error: up and order both decide the order of its rows": true } } : {}) & (D extends { readonly observer: true } ? Observer : {}) - & (D extends { readonly target: string } ? Targeted : {}) + & (D extends { readonly target: string } ? Targeted : {}) & (D extends { readonly target: string } ? Property extends undefined ? { readonly target?: { readonly "error: a system on one entity runs on its row, so it needs components": true } } : {} : {}) - & { readonly shader?: never }; + // A body's input is the type of its parameter after its context, not part of its declaration. + & { readonly shader?: never; readonly input?: never }; + +/** The fields of input `I` that hold an entity. */ +type EntityFields = { [K in keyof I]-?: [Exclude] extends [Entity] ? K : never }[keyof I] & string; + +/** + * What a declaration requires of the input its body takes, checked at the body, since the input is known only once the + * body is: a target names an entity field of it, and a system with input doesn't filter on added or changed, as its + * runs pass different input, nor is it an observer. Every check passes for `never`, the input before the body is + * inferred. + */ +type Checks = [I] extends [never] ? unknown : + (D extends { readonly target: infer T extends string } + ? [I] extends [undefined] ? { readonly "error: a system with a target takes its input after its context": T } + : T extends EntityFields ? unknown : { readonly "error: its target is no entity field of its input": T } + : unknown) + & ([I] extends [undefined] ? unknown + : (D extends { readonly components: { readonly added: object } | { readonly changed: object } } ? { readonly "error: a system with input can't filter on added or changed, as its runs pass different input; pass it through a resource": true } : unknown) + & (D extends { readonly observer: true } ? { readonly "error: an observer system takes no input": true } : unknown)); /** The body of a system that declares nothing, which runs once per run with its commands. */ type Alone = (context: { readonly commands: Commands }) => void; @@ -155,9 +167,10 @@ type Checked = C extends Alone ? unknown : C extends CompositeDeclaration ? { * One argument, a body or a composite, is one overload, so a mistake in it is reported where it is rather than as no * overload matching. */ -export function create(declaration: D & Validate & OneThread, ThreadedBy>, body: Body): System>; +export function create>(declaration: D & Validate & OneThread, ThreadedBy>, body: B & Checks>): System>; export function create(only: C & Checked): C extends Alone ? System : System>>; -export function create(first: Declaration | Leaf["body"] | readonly System[] | CompositeDeclaration, second?: Leaf["body"]): System { +// The plugin passes the schema of the body's input as a third argument. +export function create(first: Declaration | Leaf["body"] | readonly System[] | CompositeDeclaration, second?: Leaf["body"], input?: unknown): System { // The plugin gives its id to the call's first argument: the declaration, the members, or the body made into a // system alone. const id = takeIdOfDeclaration(first); @@ -166,7 +179,9 @@ export function create(first: Declaration | Leaf["body"] | readonly System = { kind: "system", name: nameOf(declaration.name, id, "system"), declaration, body }; + const name = nameOf(declaration.name, id, "system"); + const schema = inputSchemaOf(name, body, declaration.components === undefined ? 1 : 2, input); + const system: Leaf & System = { kind: "system", name, declaration, input: schema, body }; checkLeaf(system); claimSystem(system, id); return system; diff --git a/packages/ecs/src/interface/system/declaration.ts b/packages/ecs/src/interface/system/declaration.ts index 1fdb39b2..362f49e2 100644 --- a/packages/ecs/src/interface/system/declaration.ts +++ b/packages/ecs/src/interface/system/declaration.ts @@ -1,7 +1,6 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. import type { Index } from '../indexes/index-type.js'; import type { Resource } from '../resource/resource.js'; -import type { Schema } from '../schema/schema.js'; import type { ComponentAccess, ResourceAccess, SystemLookup, Up } from './access.js'; import type { Name } from './name.js'; import type { Shader } from './render.js'; @@ -42,12 +41,10 @@ export type Declaration = { readonly resources?: ResourceAccess; /** The resources its body contributes to, each with the operator its `reduce` trait declares. */ readonly reduce?: { readonly [K in Reduced]?: OperatorOf }; - /** The object of named fields passed to each run. A system with input can't use `added` or `changed`. */ - readonly input?: Extract; /** - * The input field naming the one entity it runs on: its body runs once, on that entity's row, when the entity is - * alive and matches `components`, and not at all otherwise. Needs `components`; not with `kernel`, `up`, `order` - * or `reduce`. + * The field of its body's input naming the one entity it runs on: its body runs once, on that entity's row, when + * the entity is alive and matches `components`, and not at all otherwise. Needs `components`; not with `kernel`, + * `up`, `order` or `reduce`. */ readonly target?: string; /** Makes this a render system (see `render`): its body records GPU work over the GPU copy of the components it names. */ diff --git a/packages/ecs/src/interface/system/input.ts b/packages/ecs/src/interface/system/input.ts index 9f15c759..c064e3fb 100644 --- a/packages/ecs/src/interface/system/input.ts +++ b/packages/ecs/src/interface/system/input.ts @@ -1,12 +1,11 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. -import type { Schema } from '../schema/schema.js'; import type { System } from './system.js'; -/** - * The input a system declares: a value of its `input` schema, or `undefined` without one; `unknown` for `any`, as an - * overload's declaration erases to, since every schema's value type is too deep to compute. - */ -export type InputOf = 0 extends 1 & D ? unknown : D extends { readonly input: infer A extends Schema } ? Schema.ToType : undefined; +/** How many parameters come before a body's input: the row and the context, or the context alone. */ +type Before = D extends { readonly components: object } ? 2 : 1; + +/** The input body `B` of declaration `D` takes: the type of its parameter after the context, or `undefined` without one. */ +export type InputOf = B extends (...parameters: infer P) => void ? P["length"] extends Before ? undefined : P[Before] : undefined; /** What `run` takes after a system: nothing, or one value. */ export type Input = [I] extends [undefined] ? [] : [input: I]; diff --git a/packages/ecs/src/interface/system/leaf.ts b/packages/ecs/src/interface/system/leaf.ts index c8b7b0f4..627fc38a 100644 --- a/packages/ecs/src/interface/system/leaf.ts +++ b/packages/ecs/src/interface/system/leaf.ts @@ -1,14 +1,20 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. +import type { Schema } from '../schema/schema.js'; import type { Declaration } from './declaration.js'; import type { Name } from './name.js'; +/** The schema of a body's input: an object of named fields. */ +export type InputSchema = Extract; + /** * A system that runs a body: `"render"` for one made by `render`, `"system"` otherwise. Its body's parameters are - * checked where it's created, so a leaf holds any body. + * checked where it's created, so a leaf holds any body. `input` is the schema of its body's input, which the plugin + * gives from the type of the body's parameter after its context, or undefined; every leaf has it, so leaves share a shape. */ export type Leaf = { readonly kind: "system" | "render"; readonly name: Name; readonly declaration: Declaration; + readonly input: InputSchema | undefined; readonly body: (...parameters: never[]) => void; }; diff --git a/packages/ecs/src/interface/system/render.ts b/packages/ecs/src/interface/system/render.ts index f031d58a..d99130ff 100644 --- a/packages/ecs/src/interface/system/render.ts +++ b/packages/ecs/src/interface/system/render.ts @@ -1,5 +1,6 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. import type { Component } from '../component/component.js'; +import type { Index } from '../indexes/index-type.js'; import { takeIdOfDeclaration } from '../registry/ids.js'; import { nameOf } from '../registry/items.js'; import { claimSystem } from '../registry/program.js'; @@ -22,7 +23,10 @@ import type { OneThread, ThreadedBy, ThreadsOf } from './threads.js'; */ export type Shader = { readonly group?: number; readonly accessors: { readonly [name: string]: StructComponent } }; -/** One group of matching rows, `0` to `count`. `bind` sets the world's bind group for it on a pass. */ +/** + * One group of matching rows, `0` to `count`: an archetype's, or with `order` every matching row in its order. `bind` + * sets the world's bind group for it on a pass. + */ export type RenderTable = { readonly count: number; readonly bind: (pass: GPURenderPassEncoder | GPUComputePassEncoder) => void; @@ -44,6 +48,11 @@ type RenderDeclaration = { readonly components?: { readonly has?: readonly Component.Key[]; readonly not?: readonly Component.Key[] }; /** It runs only while `gpuDevice` is set, as if `is` listed it. */ readonly resources?: { readonly read?: readonly Resource.Key[]; readonly is?: readonly Condition[]; readonly not?: readonly Condition[] }; + /** + * Draws every matching row as one table, in this sorted index's order: key order, then entity order. Its shaders' + * accessors then take a row's position in that order. + */ + readonly order?: Index<{ readonly sorted: true; readonly key: readonly Component.Key[] }>; readonly shader: Shader; readonly before?: readonly System[]; readonly after?: readonly System[]; @@ -56,7 +65,12 @@ type Rendered = D & { readonly components: { readon /** * Declares a render system: it runs once per run on `gpuDevice`'s thread, in schedule order, and records GPU work over * the GPU copy of the components its accessors name, which the world uploads first as needed. It runs only while the - * world has a `gpuDevice`. Throws as `create` does, and for an accessor its shader can't read on the GPU. + * world has a `gpuDevice`. Its body's `tables` hold one table per matching archetype, each in row order; with `order`, + * exactly one table of every matching row, across archetypes, in the index's order, so instance `i` is the i-th: a + * shader's accessors take that position as their `row`, with the same names and signatures. Throws as `create` does, + * for an `order` that isn't a sorted index, and for an accessor its shader can't read on the GPU. A run with `order` + * fails, as a failure in its report, when a matching archetype holds more rows than 32 bits name beside its slot: + * 2^31 with up to two matching archetypes, 2^30 with up to four, and so on. */ export const render = ( declaration: D & OneThread | ThreadOfResource, ThreadedBy>, @@ -72,6 +86,7 @@ export const render = ( components: { ...declaration.components, read: [...new Set(Object.values(declaration.shader.accessors))] }, resources: { ...declaration.resources, is: [...new Set([...declaration.resources?.is ?? [], gpuDevice])] }, }, + input: undefined, body, }; checkLeaf(system); diff --git a/packages/ecs/src/interface/system/system.type-test.ts b/packages/ecs/src/interface/system/system.type-test.ts index e000bb66..554d86aa 100644 --- a/packages/ecs/src/interface/system/system.type-test.ts +++ b/packages/ecs/src/interface/system/system.type-test.ts @@ -112,16 +112,25 @@ System.create({ lookup: { targets: { read: [position], changed: [velocity] } }, }, () => {}); -// Input: typed from its schema. +// Input: the type of the body's parameter after its context. export const move = System.create({ name: "test/move", - input: { type: "object", properties: { distance: { type: "number" } } }, components: { has: [player], write: [position] }, -}, (row, { input }) => { +}, (row, _context, input: { readonly distance: number }) => { row[position] += input.distance; - // @ts-expect-error input has its schema's type + // @ts-expect-error input has the type the body gives it void input.missing; }); +// The row and the context are still typed by the declaration. +System.create({ components: { read: [velocity], write: [position] }, resources: { read: [time] } }, (row, { resources }, input: { readonly by: number }) => { + row[position] += row[velocity] * resources[time] * input.by; + // @ts-expect-error a read component is read-only + row[velocity] = input.by; + // @ts-expect-error score isn't a declared resource + void resources[score]; +}); +// @ts-expect-error input is the body's parameter, not part of its declaration +System.create({ input: { type: "object", properties: {} } }, () => {}); // A system without components takes only its context; resource writes are writable, and written ones readable. export const tally = System.create({ @@ -145,13 +154,14 @@ System.create({ name: "test/tally" }, context => { void context.lookup; // @ts-expect-error no reductions declared void context.reduce; - // @ts-expect-error no input declared + // @ts-expect-error input isn't in the context void context.input; void context.commands; }); -// @ts-expect-error a system without components has no row -System.create({ name: "test/tally" }, (row: object, context: object) => { void [row, context]; }); +// A system without components has no row: its second parameter is its input. +check>>(); +const alone = System.create({ name: "test/tally" }, (_context, input: { readonly count: number }) => { void input.count; }); System.create({ name: "test/tally", @@ -165,12 +175,11 @@ System.create({ after: ["test/unregistered"], }, () => {}); -System.create({ - name: "test/move", - input: { type: "object", properties: { distance: { type: "number" } } }, - // @ts-expect-error a system with input can't use change filters - components: { changed: [position] }, -}, () => {}); +// @ts-expect-error a system with input can't use change filters +System.create({ name: "test/move", components: { changed: [position] } }, (_row, _context, input: { readonly distance: number }) => { void input; }); +// @ts-expect-error a system with input can't use change filters +System.create({ name: "test/move", components: { added: [position] } }, (_row, _context, input: { readonly distance: number }) => { void input; }); +System.create({ name: "test/move", components: { changed: [position] } }, row => { void row; }); export const main = System.create({ name: "test/main", systems: [integrate, follow, tally] }); const moving = System.create({ name: "test/moving", systems: [move] }); @@ -386,13 +395,40 @@ System.render({ name: "test/drawShared", shader: { accessors: { place: position drawAll(gpu, pass); pass.end(); }); +// `order` draws every matching archetype as one table, in a sorted index's order. +declare const byPosition: Index<{ readonly key: readonly [typeof position]; readonly sorted: true }>; +declare const hashedByPosition: Index<{ readonly key: readonly [typeof position] }>; +declare const someIndex: Index; +System.render({ name: "test/drawOrdered", components: { has: [player] }, order: byPosition, shader: { accessors: { place: position } } }, gpu => { + const pass = gpu.encoder.beginRenderPass({ colorAttachments: [] }); + drawAll(gpu, pass); + pass.end(); +}); +System.render({ + name: "test/drawOrdered", + // @ts-expect-error a render system is ordered by a sorted index + order: hashedByPosition, + shader: { accessors: { place: position } }, +}, () => {}); +System.render({ + name: "test/drawOrdered", + // @ts-expect-error an index whose declaration isn't known isn't known to be sorted + order: someIndex, + shader: { accessors: { place: position } }, +}, () => {}); +System.render({ + name: "test/drawOrdered", + // @ts-expect-error an order is an index + order: [position], + shader: { accessors: { place: position } }, +}, () => {}); // Names are optional: a declaration may leave it out, and a system that declares nothing is just its body. export const unnamed = System.create({ components: { write: [position] }, resources: { read: [time] } }, (row, { resources }) => { row[position] += resources[time]; }); check>(); -export const unnamedInput = System.create({ input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input }) => { +export const unnamedInput = System.create({}, (_context, input: { readonly count: number }) => { const count: number = input.count; void count; }); @@ -438,7 +474,7 @@ System.create({ observer: true, resources: { write: [score] } }, () => {}); // @ts-expect-error an observer system reduces into nothing System.create({ observer: true, components: { read: [velocity] }, reduce: { [total]: "+" } }, () => {}); // @ts-expect-error an observer system takes no input -System.create({ observer: true, input: { type: "object", properties: { count: { type: "integer" } } } }, () => {}); +System.create({ observer: true, resources: { read: [time] } }, (_context, input: { readonly count: number }) => { void input; }); // @ts-expect-error an observer system isn't a kernel System.create({ observer: true, kernel: "cpu", components: { read: [position] } }, () => {}); // @ts-expect-error an observer system is ordered with nothing @@ -449,13 +485,16 @@ System.create({ observer: true, resources: { read: [time] }, before: [integrate] System.create({ observer: true, resources: { read: [time] }, during: [integrate] }, () => {}); // A system on one entity: its input names the entity, and its body runs on that entity's row. -const onOne = { type: "object", properties: { entity: Entity.schema, by: { type: "number" } } } as const; -export const nudge = System.create({ input: onOne, target: "entity", components: { write: [position] } }, (row, { input }) => { +type OnOne = { readonly entity: Entity; readonly by: number }; +export const nudge = System.create({ target: "entity", components: { write: [position] } }, (row, _context, input: OnOne) => { row[position] += input.by; }); +check>>(); // @ts-expect-error its target is an entity field of its input -System.create({ input: onOne, target: "by", components: { write: [position] } }, () => {}); +System.create({ target: "by", components: { write: [position] } }, (_row, _context, input: OnOne) => { void input; }); +// @ts-expect-error it takes its target from its input, so it takes input +System.create({ target: "entity", components: { write: [position] } }, row => { void row; }); // @ts-expect-error it runs on a row, so it needs components -System.create({ input: onOne, target: "entity" }, () => {}); +System.create({ target: "entity" }, (_context, input: OnOne) => { void input; }); // @ts-expect-error not a kernel -System.create({ input: onOne, target: "entity", kernel: "cpu", components: { write: [position] } }, () => {}); +System.create({ target: "entity", kernel: "cpu", components: { write: [position] } }, (_row, _context, input: OnOne) => { void input; }); diff --git a/packages/ecs/src/perftest/benchmarks.ts b/packages/ecs/src/perftest/benchmarks.ts index e0cea84b..db76d24e 100644 --- a/packages/ecs/src/perftest/benchmarks.ts +++ b/packages/ecs/src/perftest/benchmarks.ts @@ -9,6 +9,7 @@ import * as structural from "./structural.js"; import { chain, churn, emit, failing, fastest, follow, highest, hits, idle, idles, idleObserver, index, indexSum, observe, poke, pokeCluster, reduce, retag, seeded, small, smalls, spawnOne, tag, totalOf } from "./suite.js"; import type { Benchmark } from "./benchmark.js"; +import { retainedRuns } from "../implementation/world/reports.js"; import { loopTickIdle, loopTicks, observerIdles } from "./loop.js"; import { main, placed, pooled } from "./placed.js"; import { wake } from "./wake.js"; @@ -21,8 +22,10 @@ import { readBenchmarks } from "./reads.js"; import { observedBenchmarks } from "./observed.js"; import { systemBenchmarks } from "./systems.js"; import { indexBenchmarks } from "./indexes.js"; +import { editorBenchmarks } from "./editor.js"; import { stringBenchmarks } from "./strings.js"; import { structReplace, valueBenchmarks } from "./values.js"; +import { filterBenchmarks } from "./filters.js"; import { createWorld } from "../implementation/index.js"; import { workers } from "../interface/resources/workers.js"; // Last, so the bundle orders the modules the benchmarks share as it did before these imports. @@ -50,14 +53,19 @@ const observed = async (world: World, writer: typeof poke | typeof pokeCluster): return best; }; -/** The fastest run of `measured` right after `before` runs, over `rounds` rounds after `warmup`. */ +/** + * The fastest run of `measured` right after `before` runs, over `rounds - warmup` rounds after a warmup of at least + * `warmup` rounds, and enough for the world to keep its full count of reports: until then each run spawns its own, + * which its sync point costs. + */ const after = async (before: () => Promise, measured: () => Promise, rounds = 100, warmup = 20): Promise => { let best = Infinity; - for (let i = 0; i < rounds; i++) { + const warm = Math.max(warmup, Math.ceil(retainedRuns / 2)); + for (let i = 0; i < warm + rounds - warmup; i++) { await before(); const start = performance.now(); await measured(); - if (i >= warmup) best = Math.min(best, performance.now() - start); + if (i >= warm) best = Math.min(best, performance.now() - start); } return best; }; @@ -687,7 +695,9 @@ export const benchmarks: readonly Benchmark[] = [ ...observedBenchmarks, ...systemBenchmarks, ...indexBenchmarks, + ...editorBenchmarks, ...stringBenchmarks, ...valueBenchmarks, structReplace("main", 0), + ...filterBenchmarks, ]; diff --git a/packages/ecs/src/perftest/editor.ts b/packages/ecs/src/perftest/editor.ts new file mode 100644 index 00000000..c5821ab8 --- /dev/null +++ b/packages/ecs/src/perftest/editor.ts @@ -0,0 +1,139 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { Archetype } from "../interface/archetype/archetype.js"; +import { Component } from "../interface/component/component.js"; +import { Entity } from "../interface/entity/entity.js"; +import { Index } from "../interface/indexes/index-type.js"; +import { workers } from "../interface/resources/workers.js"; +import { System } from "../interface/system/system.js"; +import { createWorld } from "../implementation/index.js"; +import type { Benchmark } from "./benchmark.js"; + +// A 2D editor loading a document: a tree of groups and leaves, a sorted paint-order index, a hash index of children +// by parent, and `up` systems with change filters propagating transforms and opacity down the tree. + +const localX = Symbol("editor/localX"); +const localY = Symbol("editor/localY"); +const worldX = Symbol("editor/worldX"); +const worldY = Symbol("editor/worldY"); +const opacity = Symbol("editor/opacity"); +const worldOpacity = Symbol("editor/worldOpacity"); +const paintDepth = Symbol("editor/paintDepth"); +const parent = Symbol("editor/parent"); +const group = Symbol("editor/group"); +const shape = Symbol("editor/shape"); +const image = Symbol("editor/image"); +declare module "@adobe/ecs" { + export interface Components { + [localX]: { type: number }; + [localY]: { type: number }; + [worldX]: { type: number }; + [worldY]: { type: number }; + [opacity]: { type: number }; + [worldOpacity]: { type: number }; + [paintDepth]: { type: number }; + [parent]: { type: Entity; reference: "owner" }; + [group]: { type: true }; + [shape]: { type: true }; + [image]: { type: true }; + } +} +Component.register(localX, { type: { type: "number" } }); +Component.register(localY, { type: { type: "number" } }); +Component.register(worldX, { type: { type: "number" } }); +Component.register(worldY, { type: { type: "number" } }); +Component.register(opacity, { type: { type: "number" } }); +Component.register(worldOpacity, { type: { type: "number" } }); +Component.register(paintDepth, { type: { type: "number" } }); +Component.register(parent, { type: Entity.schema, reference: "owner" }); +Component.register(group, { type: { const: true } }); +Component.register(shape, { type: { const: true } }); +Component.register(image, { type: { const: true } }); + +const node = [localX, localY, worldX, worldY, opacity, worldOpacity, paintDepth] as const; +const Document = Archetype.create([...node, group]); +const Group = Archetype.create([...node, parent, group]); +const Shape = Archetype.create([...node, parent, shape]); +const Image = Archetype.create([...node, parent, image]); + +const byPaintDepth = Index.create({ name: "editor/byPaintDepth", key: [paintDepth], sorted: true }); +const childrenOf = Index.create({ name: "editor/childrenOf", key: [parent] }); + +/** Groups under the document, subgroups in each, and leaves in each subgroup: 50,201 nodes. */ +const groups = 200; +const subgroups = 2; +const leaves = 124; +const nodes = 1 + groups * (1 + subgroups * (1 + leaves)); + +/** Spawns the document depth first, as a file lists it, its paint depth its place in that order. */ +const load = System.create({ name: "editor/load" }, ({ commands }) => { + let depth = 0; + const values = (x: number) => ({ [localX]: x, [localY]: 1, [worldX]: 0, [worldY]: 0, [opacity]: 0.99, [worldOpacity]: 1, [paintDepth]: depth++ }); + const document = commands.spawn(Document)(values(0)); + for (let g = 0; g < groups; g++) { + const outer = commands.spawn(Group)({ ...values(g), [parent]: document, [group]: true }); + for (let s = 0; s < subgroups; s++) { + const inner = commands.spawn(Group)({ ...values(s), [parent]: outer, [group]: true }); + for (let l = 0; l < leaves; l++) { + if (l % 2 === 0) commands.spawn(Shape)({ ...values(l), [parent]: inner, [shape]: true }); + else commands.spawn(Image)({ ...values(l), [parent]: inner, [image]: true }); + } + } + } +}); + +/** Each node's world position is its parent's plus its own. */ +const place = System.create({ + name: "editor/place", + components: { read: [localX, localY], write: [worldX, worldY], changed: [localX, localY] }, + up: { through: parent, read: [worldX, worldY] }, +}, (row, { up }) => { + row[worldX] = (up === null ? 0 : up[worldX]) + row[localX]; + row[worldY] = (up === null ? 0 : up[worldY]) + row[localY]; +}); +/** Each node's world opacity is its parent's times its own. */ +const fade = System.create({ + name: "editor/fade", + components: { read: [opacity], write: [worldOpacity], changed: [opacity] }, + up: { through: parent, read: [worldOpacity] }, +}, (row, { up }) => { + row[worldOpacity] = (up === null ? 1 : up[worldOpacity]) * row[opacity]; +}); +let checksum = 0; +/** Each group counts its children, as a layout does. */ +const layout = System.create({ name: "editor/layout", components: { has: [group] }, lookup: { children: { index: childrenOf } } }, (row, { lookup }) => { + checksum += lookup.children(row.entity).length; +}); +/** Visits every node in paint order, as drawing does. */ +const paint = System.create({ name: "editor/paint", components: { read: [worldX, worldOpacity] }, order: byPaintDepth, after: [place, fade] }, row => { + checksum = (checksum * 31 + row[worldX] * row[worldOpacity]) % 1000003; +}); +const frame = System.create({ name: "editor/frame", systems: [place, fade, layout, paint] }); + +const check = (condition: boolean, message: string): void => { + if (!condition) throw new Error(message); +}; + +export const editorBenchmarks: readonly Benchmark[] = [{ + name: "editor load tick 50K", + operation: "A 2D editor's first tick after loading a 50K-node document of groups and leaves: two `up` systems with change filters propagate transforms and opacity down the tree, a layout looks up children through a hash index on the parent, and painting runs in a sorted index's order, so every `up` order and index is built from scratch.", + per: "nodes", + n: nodes, + typical: [1_000, 100_000], + best: { bytes: 100, basis: "Each node reads its parent link, its 7 values and its parent's 3, writes 3, and enters two indexes and two orders." }, + run: async () => { + let best = Infinity; + for (let i = 0; i < 8; i++) { + const world = createWorld({ [workers]: 0 }); + await world.run(frame); + await world.run(load); + checksum = 0; + const start = performance.now(); + await world.run(frame); + best = Math.min(best, performance.now() - start); + check(checksum !== 0, "editor load tick: nothing ran"); + await world.dispose(); + } + return best; + }, +}]; + diff --git a/packages/ecs/src/perftest/filters.ts b/packages/ecs/src/perftest/filters.ts new file mode 100644 index 00000000..4ace0afd --- /dev/null +++ b/packages/ecs/src/perftest/filters.ts @@ -0,0 +1,137 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { Archetype } from "../interface/archetype/archetype.js"; +import { Component } from "../interface/component/component.js"; +import type { I32 } from "../interface/math/i32/index.js"; +import { workers } from "../interface/resources/workers.js"; +import { System } from "../interface/system/system.js"; +import type { World } from "../interface/world/world.js"; +import { createWorld } from "../implementation/index.js"; +import { retainedRuns } from "../implementation/world/reports.js"; +import type { Benchmark } from "./benchmark.js"; + +// Change filters over one archetype of 100K rows after a writer changed one row, a few, or every one: what finding the +// rows that pass costs each system that filters, apart from running them. + +const value = Symbol("filters/value"); +const other = Symbol("filters/other"); +const at = Symbol("filters/at"); +const out0 = Symbol("filters/out0"); +const out1 = Symbol("filters/out1"); +const out2 = Symbol("filters/out2"); +const out3 = Symbol("filters/out3"); +declare module "@adobe/ecs" { + export interface Components { + [value]: { type: number }; + [other]: { type: number }; + [at]: { type: number }; + [out0]: { type: number }; + [out1]: { type: number }; + [out2]: { type: number }; + [out3]: { type: number }; + } +} +Component.register(value, { type: { type: "number" } }); +Component.register(other, { type: { type: "number" } }); +Component.register(at, { type: { type: "integer", minimum: 0, maximum: 0x7FFFFFFF } }); +Component.register(out0, { type: { type: "number" } }); +Component.register(out1, { type: { type: "number" } }); +Component.register(out2, { type: { type: "number" } }); +Component.register(out3, { type: { type: "number" } }); + +const rows = 100_000; +const filtered = Archetype.create([value, other, at, out0, out1, out2, out3]); + +const seed = System.create({ name: "filters/seed" }, ({ commands }, input: { readonly count: number }) => { + for (let i = 0; i < input.count; i++) commands.spawn(filtered)({ [value]: 0, [other]: 0, [at]: i, [out0]: 0, [out1]: 0, [out2]: 0, [out3]: 0 }); +}); + +/** Writes the value of the rows whose `at` is `first` plus a multiple of `stride`. */ +const write = System.create({ + name: "filters/write", + kernel: "cpu", + components: { read: [at], write: [value] }, +}, (row, _context, input: { readonly first: I32; readonly stride: I32 }) => { + const offset = row[at] - input.first; + if (offset >= 0 && offset % input.stride === 0) row[value] += 1; +}); + +let seen = 0; +/** Main-thread systems each filtering on `value`'s changes. */ +const readers = System.create({ + name: "filters/readers", + systems: Array.from({ length: 8 }, (_, i) => System.create({ name: `filters/reader${i}`, components: { read: [value], changed: [value] } }, row => { + seen += row[value]; + })), +}); +/** The same filtering on `value`'s or `other`'s changes, of which only `value` is written. */ +const readersOfTwo = System.create({ + name: "filters/readersOfTwo", + systems: Array.from({ length: 8 }, (_, i) => System.create({ name: `filters/readerOfTwo${i}`, components: { read: [value], changed: [value, other] } }, row => { + seen += row[value]; + })), +}); +/** Kernels each filtering on `value`'s changes, writing an output of its own. */ +const kernels = System.create({ + name: "filters/kernels", + systems: [ + System.create({ name: "filters/kernel0", kernel: "cpu", components: { read: [value], write: [out0], changed: [value] } }, row => { row[out0] = row[value] * 2; }), + System.create({ name: "filters/kernel1", kernel: "cpu", components: { read: [value], write: [out1], changed: [value] } }, row => { row[out1] = row[value] * 3; }), + System.create({ name: "filters/kernel2", kernel: "cpu", components: { read: [value], write: [out2], changed: [value] } }, row => { row[out2] = row[value] * 4; }), + System.create({ name: "filters/kernel3", kernel: "cpu", components: { read: [value], write: [out3], changed: [value] } }, row => { row[out3] = row[value] * 5; }), + ], +}); + +/** The fastest run of `measured` right after a write of every `stride`th row from a moving first row. */ +const afterWrites = async (measured: System, stride: number): Promise => { + const world: World = createWorld({ [workers]: 0 }); + await world.run(seed, { count: rows }); + await world.run(measured); + let best = Infinity; + const warm = Math.max(20, Math.ceil(retainedRuns / 2)); + for (let i = 0; i < warm + 200; i++) { + await world.run(write, { first: (i * 7919) % Math.min(stride, rows), stride }); + const start = performance.now(); + await world.run(measured); + if (i >= warm) best = Math.min(best, performance.now() - start); + } + await world.dispose(); + return best; +}; + +const cases = [ + { name: "one", stride: rows * 2, said: "one row changed" }, + { name: "few", stride: 10_000, said: "ten rows changed, 10,000 apart" }, + { name: "all", stride: 1, said: "every row changed" }, +] as const; + +export const filterBenchmarks: readonly Benchmark[] = [ + ...cases.map(({ name, stride, said }): Benchmark => ({ + name: `changed ${name} 100K`, + operation: `Eight main-thread systems filtering on \`changed\` over 100K rows after ${said}: views and derived values reacting to edits.`, + per: "systems", + n: 8, + typical: [1, 100], + best: { bytes: 0, basis: "Finding the changed rows could cost nothing beyond running them." }, + run: () => afterWrites(readers, stride), + })), + { + name: "changed one 100K two filters", + operation: "Eight main-thread systems filtering on `changed` of two components over 100K rows after one row's first component changed.", + per: "systems", + n: 8, + typical: [1, 100], + best: { bytes: 0, basis: "Finding the changed row could cost nothing beyond running it." }, + run: () => afterWrites(readersOfTwo, rows * 2), + }, + ...cases.map(({ name, stride, said }): Benchmark => ({ + name: `changed ${name} 100K kernels`, + operation: `Four kernels filtering on \`changed\` over 100K rows after ${said}, on the world's thread: derived columns kept current.`, + per: "systems", + n: 4, + typical: [1, 100], + best: { bytes: 0, basis: "Finding the changed rows could cost nothing beyond running them." }, + run: () => afterWrites(kernels, stride), + })), +]; + +export const seenTotal = (): number => seen; diff --git a/packages/ecs/src/perftest/finalized.ts b/packages/ecs/src/perftest/finalized.ts index 6039cd49..503a83b5 100644 --- a/packages/ecs/src/perftest/finalized.ts +++ b/packages/ecs/src/perftest/finalized.ts @@ -2,6 +2,7 @@ /// import { Component } from "../interface/component/component.js"; import { Finalizer } from "../interface/finalizer/finalizer.js"; +import { Resource } from "../interface/resource/resource.js"; import { workers } from "../interface/resources/workers.js"; import { System } from "../interface/system/system.js"; import { createWorld, type CreatedWorld } from "../implementation/index.js"; @@ -10,10 +11,16 @@ import { runReported } from "../implementation/world/reports.test-helper.js"; // Despawning entities whose objects a joined thread fills, with a finalizer on that thread: the world's thread writes // each stop's declared struct reads into the thread's stop list and doesn't wait for it. The joined thread is -// `joined.ts`, which imports this module, so it runs the finalizer. +// `joined.ts`, which imports this module, so it runs the finalizers. A second finalizer, over other entities, reads a +// string resource too, and a third two linear resources, which each sync point's stop list holds copies of. const spot = Symbol("finalized/spot"); const handle = Symbol("finalized/handle"); +const noted = Symbol("finalized/noted"); +const note = Symbol("finalized/note"); +const measured = Symbol("finalized/measured"); +const scale = Symbol("finalized/scale"); +const offset = Symbol("finalized/offset"); declare module "@adobe/ecs" { export interface Threads { finalizedBenchmark: true; @@ -21,11 +28,23 @@ declare module "@adobe/ecs" { export interface Components { [spot]: { type: readonly [number, number, number] }; [handle]: { type: { readonly release: () => void } | null; thread: "finalizedBenchmark" }; + [noted]: { type: { readonly release: () => void } | null; thread: "finalizedBenchmark" }; + [measured]: { type: { readonly release: () => void } | null; thread: "finalizedBenchmark" }; + } + export interface Resources { + [note]: { type: string }; + [scale]: { type: number }; + [offset]: { type: readonly [number, number, number] }; } } Component.register(spot, { type: { type: "array", items: { type: "number", precision: 1 }, minItems: 3, maxItems: 3 } }); /** An object of the joined thread, as a DOM node or a GPU handle is. */ Component.register(handle, { thread: "finalizedBenchmark" }); +Component.register(noted, { thread: "finalizedBenchmark" }); +Resource.register(note, { type: { type: "string", default: "released" } }); +Component.register(measured, { thread: "finalizedBenchmark" }); +Resource.register(scale, { type: { type: "number", default: 1 } }); +Resource.register(offset, { type: { type: "array", items: { type: "number", precision: 1 }, minItems: 3, maxItems: 3, default: [0, 0, 0] } }); /** Entities each timed sync point despawns. */ export const despawnedObjects = 10_000; @@ -37,12 +56,36 @@ export const release = Finalizer.create({ name: "finalized/release", components: if (row[spot][0] >= 0) releases.count++; }); +/** On the joined thread: the same, reading the world's string resource too. */ +export const releaseNoted = Finalizer.create({ name: "finalized/releaseNoted", components: { read: [noted, spot] }, resources: { read: [note] } }, (row, { resources }) => { + row[noted]?.release(); + if (row[spot][0] >= 0 && resources[note].length > 0) releases.count++; +}); + +/** On the joined thread: the same, reading two linear resources of the world's too. */ +export const releaseMeasured = Finalizer.create({ name: "finalized/releaseMeasured", components: { read: [measured, spot] }, resources: { read: [scale, offset] } }, (row, { resources }) => { + row[measured]?.release(); + if (row[spot][0] * resources[scale] + resources[offset][0] >= 0) releases.count++; +}); + const spawn = System.create({ name: "finalized/spawn" }, ({ commands }) => { for (let k = 0; k < despawnedObjects; k++) commands.spawn([spot, handle])({ [spot]: [k, 0, 0], [handle]: null }); }); const despawn = System.create({ name: "finalized/despawn", components: { has: [handle, spot] } }, (row, { commands }) => { commands.despawn(row.entity); }); +const spawnNoted = System.create({ name: "finalized/spawnNoted" }, ({ commands }) => { + for (let k = 0; k < despawnedObjects; k++) commands.spawn([spot, noted])({ [spot]: [k, 0, 0], [noted]: null }); +}); +const spawnMeasured = System.create({ name: "finalized/spawnMeasured" }, ({ commands }) => { + for (let k = 0; k < despawnedObjects; k++) commands.spawn([spot, measured])({ [spot]: [k, 0, 0], [measured]: null }); +}); +const despawnMeasured = System.create({ name: "finalized/despawnMeasured", components: { has: [measured, spot] } }, (row, { commands }) => { + commands.despawn(row.entity); +}); +const despawnNoted = System.create({ name: "finalized/despawnNoted", components: { has: [noted, spot] } }, (row, { commands }) => { + commands.despawn(row.entity); +}); const joinedFile = "./joined.js"; @@ -87,15 +130,15 @@ const startClaiming = async (world: CreatedWorld): Promise<{ readonly terminate: * The fastest of `rounds` sync points despawning `despawnedObjects` entities holding the joined thread's objects, * timed on the world's thread, each after a spawn and a pause in which the thread finalizes the last. */ -const despawnObjects = async (rounds = 30): Promise => { +const despawnObjects = async (spawning: typeof spawn = spawn, despawning: typeof despawn = despawn, rounds = 30): Promise => { const world = createWorld({ [workers]: 0 }); const joined = await startClaiming(world); let best = Infinity; try { for (let round = 0; round < rounds; round++) { - await world.run(spawn); + await world.run(spawning); const start = performance.now(); - const report = await runReported(world, despawn); + const report = await runReported(world, despawning); const time = performance.now() - start; if (report.failures.length > 0) throw new Error(`despawn objects: ${report.failures[0].message}`); if (round > 1) best = Math.min(best, time); @@ -120,4 +163,22 @@ export const finalizedBenchmarks: readonly Benchmark[] = [ best: { bytes: 56, basis: "Each despawn frees its 8-byte location and entity, and writes its stop: its entity, archetype, row and tag, 20 bytes, and its 12-byte vec3, read and written." }, run: () => despawnObjects(), }, + { + name: "despawn objects 10K string resource", + operation: "`despawn objects 10K` with the finalizer reading a string resource too, whose record the sync point's stop list holds a copy of, so the thread reads it as it was before apply.", + per: "despawns", + n: despawnedObjects, + typical: [1, 10_000], + best: { bytes: 56, basis: "As `despawn objects 10K`: the resource's copy is once per sync point." }, + run: () => despawnObjects(spawnNoted, despawnNoted), + }, + { + name: "despawn objects 10K linear resource", + operation: "`despawn objects 10K` with the finalizer reading a number and a vec3 resource too, which the sync point's stop list holds copies of, so the thread reads them as they were before apply.", + per: "despawns", + n: despawnedObjects, + typical: [1, 10_000], + best: { bytes: 56, basis: "As `despawn objects 10K`: the resources' copies are once per sync point." }, + run: () => despawnObjects(spawnMeasured, despawnMeasured), + }, ]; diff --git a/packages/ecs/src/perftest/flat.ts b/packages/ecs/src/perftest/flat.ts index 26382f7c..d9e09768 100644 --- a/packages/ecs/src/perftest/flat.ts +++ b/packages/ecs/src/perftest/flat.ts @@ -53,14 +53,14 @@ const archetypes = [ ] as const; /** Spawns with values passed inline. */ -export const spawnFlat = System.create({ name: "flat/spawn", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +export const spawnFlat = System.create({ name: "flat/spawn" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) { commands.spawn(archetypes[i % 4])({ [color]: 0xff00ff, [mass]: 10, [positionX]: i + 1, [positionY]: i + 1, [positionZ]: i + 1, [velocityX]: -i, [velocityY]: -i, [velocityZ]: -i }); } }); /** The same spawns with each value object built first, as values from data or from another system's results are. */ -export const spawnFlatDynamic = System.create({ name: "flat/dynamic", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +export const spawnFlatDynamic = System.create({ name: "flat/dynamic" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) { const values = { [color]: 0xff00ff, [mass]: 10, [positionX]: i + 1, [positionY]: i + 1, [positionZ]: i + 1, [velocityX]: -i, [velocityY]: -i, [velocityZ]: -i }; commands.spawn(archetypes[i % 4])(values); diff --git a/packages/ecs/src/perftest/growth.ts b/packages/ecs/src/perftest/growth.ts index 27b64400..2db886f4 100644 --- a/packages/ecs/src/perftest/growth.ts +++ b/packages/ecs/src/perftest/growth.ts @@ -41,7 +41,7 @@ const tags = [g0, g1, g2, g3, g4, g5, g6] as const; const hundred = Array.from({ length: 100 }, (_, bits) => Archetype.create([value, ...tags.filter((_, bit) => (bits & (1 << bit)) !== 0)])); /** Spawns `count` entities into each of the hundred archetypes. */ -export const fillEach = System.create({ name: "growth/fillEach", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +export const fillEach = System.create({ name: "growth/fillEach" }, ({ commands }, input: { readonly count: number }) => { for (const archetype of hundred) for (let i = 0; i < input.count; i++) commands.spawn(archetype)({ [value]: i }); }); diff --git a/packages/ecs/src/perftest/hierarchy.ts b/packages/ecs/src/perftest/hierarchy.ts index 60d97a84..68dffdda 100644 --- a/packages/ecs/src/perftest/hierarchy.ts +++ b/packages/ecs/src/perftest/hierarchy.ts @@ -50,8 +50,7 @@ export const propagateChanged = System.create({ /** `roots` chains of `depth` rows each. */ export const forest = System.create({ name: "hierarchy/forest", - input: { type: "object", properties: { roots: { type: "integer" }, depth: { type: "integer" } } }, -}, ({ input, commands }) => { +}, ({ commands }, input: { readonly roots: number; readonly depth: number }) => { for (let r = 0; r < input.roots; r++) { let previous = commands.spawn(root)({ [local]: 1, [world]: 0 }); for (let d = 1; d < input.depth; d++) previous = commands.spawn(child)({ [local]: 1, [world]: 0, [parent]: previous }); @@ -61,12 +60,12 @@ export const forest = System.create({ const target = { type: "object", properties: { entity: Entity.schema } } as const; /** Changes one row's own value. */ -export const nudge = System.create({ name: "hierarchy/nudge", input: target, components: { write: [local] } }, (row, { input }) => { +export const nudge = System.create({ name: "hierarchy/nudge", components: { write: [local] } }, (row, _context, input: { readonly entity: Entity }) => { if (row.entity === input.entity) row[local] += 1; }); /** Adds one leaf under a row. */ -export const sprout = System.create({ name: "hierarchy/sprout", input: target }, ({ input, commands }) => { +export const sprout = System.create({ name: "hierarchy/sprout" }, ({ commands }, input: { readonly entity: Entity }) => { commands.spawn(child)({ [local]: 1, [world]: 0, [parent]: input.entity }); }); @@ -77,8 +76,7 @@ const leaves: Entity[] = []; /** Despawns the leaves it spawned last time and spawns `count` new ones, one under each of `under`'s first rows; `tagged` puts them in an archetype of their own. */ export const turnover = System.create({ name: "hierarchy/turnover", - input: { type: "object", properties: { count: { type: "integer" }, tagged: { type: "boolean" } } }, -}, ({ input, commands }) => { +}, ({ commands }, input: { readonly count: number; readonly tagged: boolean }) => { for (const entity of leaves.splice(0)) commands.despawn(entity); for (let i = 0; i < input.count; i++) { leaves.push(input.tagged @@ -90,8 +88,7 @@ export const turnover = System.create({ /** Moves a row, and the subtree under it, to another parent. */ export const reparent = System.create({ name: "hierarchy/reparent", - input: { type: "object", properties: { entity: Entity.schema, to: Entity.schema } }, -}, ({ input, commands }) => { +}, ({ commands }, input: { readonly entity: Entity; readonly to: Entity }) => { commands.add(input.entity, parent, input.to); }); @@ -124,8 +121,7 @@ export const place = System.create({ /** `forest` in 4-byte values. */ export const gpuForest = System.create({ name: "hierarchy/gpuForest", - input: { type: "object", properties: { roots: { type: "integer" }, depth: { type: "integer" } } }, -}, ({ input, commands }) => { +}, ({ commands }, input: { readonly roots: number; readonly depth: number }) => { for (let r = 0; r < input.roots; r++) { let previous = commands.spawn(gpuRoot)({ [offset]: 1, [placed]: 0 }); for (let d = 1; d < input.depth; d++) previous = commands.spawn(gpuChild)({ [offset]: 1, [placed]: 0, [parent]: previous }); diff --git a/packages/ecs/src/perftest/indexes.ts b/packages/ecs/src/perftest/indexes.ts index 20f38444..a4df81f3 100644 --- a/packages/ecs/src/perftest/indexes.ts +++ b/packages/ecs/src/perftest/indexes.ts @@ -43,7 +43,7 @@ const million = 1_000_000; const keys = million / 10; const byKey = Index.create({ name: "indexes/byKey", key: [key] }); -const seedKeys = System.create({ name: "indexes/seedKeys", input: { type: "object", properties: { count: { type: "integer" }, marked: { type: "integer" } } } }, ({ input, commands }) => { +const seedKeys = System.create({ name: "indexes/seedKeys" }, ({ commands }, input: { readonly count: number; readonly marked: number }) => { for (let i = 0; i < input.count; i++) { if (i < input.marked) commands.spawn([key, amount, marked])({ [key]: (i * 7919) % keys, [amount]: i, [marked]: true }); else commands.spawn([key, amount])({ [key]: (i * 7919) % keys, [amount]: i }); @@ -343,6 +343,82 @@ const restingCounts = async (filtered: boolean): Promise => { return best; }; +/** Entities toggled between `byFlag`'s two buckets, one a run, each a different one spread through entity order. */ +let flaggedEntities: Entity[] = []; +const seedFlagged = System.create({ name: "indexes/seedFlagged" }, ({ commands }, input: { readonly count: number }) => { + flaggedEntities = []; + for (let i = 0; i < input.count; i++) flaggedEntities.push(commands.spawn([flag])({ [flag]: i % 2 === 0 })); +}); +const toggleFlag = System.create({ name: "indexes/toggleFlag", target: "entity", components: { write: [flag] } }, (row, _context, _input: { readonly entity: Entity }) => { + row[flag] = !row[flag]; +}); +/** Both buckets' lengths, as a count of done and not done todos. */ +const readFlagLengths = System.create({ name: "indexes/readFlagLengths", lookup: { byFlag: { index: byFlag } } }, ({ lookup }) => { + found = lookup.byFlag(false).length + lookup.byFlag(true).length; +}); +/** Both buckets' first ten rows, in entity order. */ +const readFlagRows = System.create({ name: "indexes/readFlagRows", lookup: { byFlag: { index: byFlag } } }, ({ lookup }) => { + let sum = 0; + for (const value of [false, true]) { + const rows = lookup.byFlag(value); + let last = 0; + for (let i = 0; i < 10 && i < rows.length; i++) { + const entity = rows.at(i).entity; + check(entity > last, "bucket move: rows out of entity order"); + last = entity; + sum += entity; + } + } + found = sum; +}); +/** Moves a round of `bucketMove` takes. */ +const movesPerRound = 200; +/** + * The fastest round's mean time of moving one entity between `byFlag`'s two buckets of `count` entities between them, + * then running `reader`, which refreshes the index and reads it. + */ +const bucketMove = async (count: number, reader: System): Promise => { + const world = inline(); + await world.run(seedFlagged, { count }); + await world.run(reader); + let best = Infinity; + let next = 0; + for (let round = 0; round < 8; round++) { + const start = performance.now(); + for (let k = 0; k < movesPerRound; k++) { + await world.run(toggleFlag, { entity: flaggedEntities[(next++ * 7919) % count] }); + await world.run(reader); + } + if (round >= 2) best = Math.min(best, (performance.now() - start) / movesPerRound); + } + check(found > 0, "bucket move: nothing found"); + await world.dispose(); + return best; +}; +const bucketMoveBenchmarks = [1_000, 10_000, 100_000].flatMap((count): Benchmark[] => { + const label = `${count / 1000}K`; + return [ + { + name: `bucket move ${label} length`, + operation: `A run moving one entity between the two buckets of a hash index on a boolean over ${label} entities, then a system reading both buckets' lengths: toggling a todo while counts of done and not done ones are shown.`, + per: "moves", + n: 1, + typical: [1, 100], + best: { bytes: 48, basis: "The move reads its 8-byte key and tick, and writes one 12-byte entry out of one bucket and one into the other; each length reads 4 bytes." }, + run: () => bucketMove(count, readFlagLengths), + }, + { + name: `bucket move ${label} in order`, + operation: `As \`bucket move ${label} length\`, the system reading each bucket's first ten rows in entity order: a list of done todos shown after a toggle.`, + per: "moves", + n: 1, + typical: [1, 100], + best: { bytes: 288, basis: "As the length's, and twenty 12-byte entries read." }, + run: () => bucketMove(count, readFlagRows), + }, + ]; +}); + export const indexBenchmarks: readonly Benchmark[] = [ { name: "observer index unchanged 100", @@ -689,4 +765,5 @@ export const indexBenchmarks: readonly Benchmark[] = [ return best; }, }, + ...bucketMoveBenchmarks, ]; diff --git a/packages/ecs/src/perftest/observed.ts b/packages/ecs/src/perftest/observed.ts index 8f02842f..84d48620 100644 --- a/packages/ecs/src/perftest/observed.ts +++ b/packages/ecs/src/perftest/observed.ts @@ -35,7 +35,7 @@ Component.register(marked, { type: { const: true } }); Component.register(unmatched, { type: { const: true } }); /** `count` entities with a value and a rank, the first of them marked. */ -const seed = System.create({ name: "observed/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const seed = System.create({ name: "observed/seed" }, ({ commands }, input: { readonly count: number }) => { commands.spawn([value, rank, marked])({ [value]: 0, [rank]: 0, [marked]: true }); for (let i = 1; i < input.count; i++) commands.spawn([value, rank])({ [value]: (i * 7919) % input.count, [rank]: i }); }); diff --git a/packages/ecs/src/perftest/particles.ts b/packages/ecs/src/perftest/particles.ts index 3eb23ad2..7c15f501 100644 --- a/packages/ecs/src/perftest/particles.ts +++ b/packages/ecs/src/perftest/particles.ts @@ -43,7 +43,7 @@ const archetypes = [ Archetype.create([color, mass, position, velocity]), ] as const; -export const spawn = System.create({ name: "perf/spawn", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +export const spawn = System.create({ name: "perf/spawn" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) { const p = [i + 1, i + 1, i + 1] as const; const v = [-i, -i, -i] as const; diff --git a/packages/ecs/src/perftest/reads.ts b/packages/ecs/src/perftest/reads.ts index 94653341..feb05839 100644 --- a/packages/ecs/src/perftest/reads.ts +++ b/packages/ecs/src/perftest/reads.ts @@ -32,7 +32,7 @@ Component.register(rank, { type: { type: "number" } }); Component.register(marked, { type: { const: true } }); /** `count` entities with a value and a rank, the first of them marked. */ -const seed = System.create({ name: "reads/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const seed = System.create({ name: "reads/seed" }, ({ commands }, input: { readonly count: number }) => { commands.spawn([value, rank, marked])({ [value]: 0, [rank]: 0, [marked]: true }); for (let i = 1; i < input.count; i++) commands.spawn([value, rank])({ [value]: (i * 7919) % input.count, [rank]: i }); }); diff --git a/packages/ecs/src/perftest/remote.ts b/packages/ecs/src/perftest/remote.ts index e3b64549..004837a2 100644 --- a/packages/ecs/src/perftest/remote.ts +++ b/packages/ecs/src/perftest/remote.ts @@ -43,7 +43,7 @@ Component.register(amount, { type: { type: "number" } }); export const empty = System.create({ name: "remote/empty" }, () => {}); /** Changes the observed resource every tick. */ export const count = System.create({ name: "remote/count", resources: { write: [counter] } }, ({ resources }) => { resources[counter] += 1; }); -export const seedRanks = System.create({ name: "remote/seedRanks", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +export const seedRanks = System.create({ name: "remote/seedRanks" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) commands.spawn([rank, order])({ [rank]: i, [order]: i }); }); const flip = System.create({ name: "remote/flip", resources: { write: [sign] } }, ({ resources }) => { resources[sign] = -resources[sign]; }); @@ -92,7 +92,7 @@ export const observerSelect = System.create({ name: "remote/observerSelect", obs }); /** Rows with a caption and an amount, a todo's label and a number, which `retext` and `reamount` rewrite 1% of a tick. */ -export const seedCaptions = System.create({ name: "remote/seedCaptions", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +export const seedCaptions = System.create({ name: "remote/seedCaptions" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) commands.spawn([rank, caption, amount])({ [rank]: i, [caption]: `Buy milk and bread #${i}`, [amount]: i }); }); const recaption = System.create({ name: "remote/recaption", components: { read: [rank], write: [caption] }, resources: { read: [counter] }, after: [count] }, (row, { resources }) => { diff --git a/packages/ecs/src/perftest/render-order.ts b/packages/ecs/src/perftest/render-order.ts new file mode 100644 index 00000000..8e311f7f --- /dev/null +++ b/packages/ecs/src/perftest/render-order.ts @@ -0,0 +1,90 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +import { Component } from "../interface/component/component.js"; +import { Entity } from "../interface/entity/entity.js"; +import { Index } from "../interface/indexes/index-type.js"; +import { gpuDevice } from "../interface/resources/gpu-device.js"; +import { workers } from "../interface/resources/workers.js"; +import { System } from "../interface/system/system.js"; +import { createWorld } from "../implementation/world/create.js"; +import type { Benchmark } from "./benchmark.js"; + +// A 2D editor's paint order: shapes and images, two archetypes, interleaved in depth, drawn as one table in a sorted +// index's order. + +const paintDepth = Symbol("renderOrder/paintDepth"); +const shape = Symbol("renderOrder/shape"); +const image = Symbol("renderOrder/image"); +declare module "@adobe/ecs" { + export interface Components { + [paintDepth]: { type: number }; + [shape]: { type: true }; + [image]: { type: true }; + } +} +Component.register(paintDepth, { type: { type: "number", precision: 1 } }); +Component.register(shape, { type: { const: true } }); +Component.register(image, { type: { const: true } }); + +const byDepth = Index.create({ name: "renderOrder/byDepth", key: [paintDepth], sorted: true }); + +const items = 50_000; +let spawned: Entity[] = []; +const spawn = System.create({ name: "renderOrder/spawn" }, ({ commands }) => { + spawned = []; + for (let i = 0; i < items; i++) { + spawned.push(i % 2 === 0 ? commands.spawn([paintDepth, shape])({ [paintDepth]: i, [shape]: true }) : commands.spawn([paintDepth, image])({ [paintDepth]: i, [image]: true })); + } +}); +/** Moves one item to a new depth, as reordering it does. */ +const restack = System.create({ + name: "renderOrder/restack", + components: { write: [paintDepth] }, + target: "item", +}, (row, _context, input: { readonly item: Entity; readonly depth: number }) => { + row[paintDepth] = input.depth; +}); +/** Draws nothing: its run uploads what it reads and its order. */ +const draw = System.render({ name: "renderOrder/draw", order: byDepth, shader: { accessors: { paintDepth } }, after: [restack] }, () => {}); +const frame = System.create({ name: "renderOrder/frame", systems: [restack, draw] }); + +/** Timed per run by `timed`, which waits for the GPU. */ +export const renderOrderBenchmarks = (device: GPUDevice, timed: (run: () => Promise) => Promise): readonly Benchmark[] => [{ + name: "gpu ordered render unchanged 50K", + operation: "A render system with an order over 50K shapes and images, two archetypes interleaved in depth, run with nothing changed: it uploads nothing and writes neither its order nor its record.", + per: "runs", + n: 1, + typical: [1, 1], + best: { bytes: 64, basis: "A run reads at least one cache line of its system's state." }, + run: async () => { + const world = createWorld({ [workers]: 0, [gpuDevice]: device }); + try { + await world.run(spawn); + await world.run(draw); + return await timed(() => world.run(draw)); + } + finally { + await world.dispose(); + } + }, +}, { + name: "gpu ordered render one key 50K", + operation: "A target system moving one of 50K shapes and images to another depth, then a render system ordered by depth, which refreshes the index, lists the 50K in order and writes them, 200 KB.", + per: "runs", + n: 1, + typical: [1, 1], + best: { bytes: 400_000, basis: "The order's 50K 4-byte entries are written on the CPU and again by the upload.", on: "gpu" }, + run: async () => { + const world = createWorld({ [workers]: 0, [gpuDevice]: device }); + try { + await world.run(spawn); + const item = spawned[items / 2]; + let depth = 0.5; + await world.run(frame, { item, depth }); + // Between two others each time, so its place in the order moves. + return await timed(() => world.run(frame, { item, depth: depth = (depth + 1009) % items })); + } + finally { + await world.dispose(); + } + }, +}]; diff --git a/packages/ecs/src/perftest/strings.ts b/packages/ecs/src/perftest/strings.ts index 0873fcbd..46f73686 100644 --- a/packages/ecs/src/perftest/strings.ts +++ b/packages/ecs/src/perftest/strings.ts @@ -26,7 +26,7 @@ const rows = 10_000; /** A todo's label, of a typical length, different per row and round. */ const labelOf = (row: number, round: number): string => `Buy milk and bread #${row}.${round}`; -const seed = System.create({ name: "strings/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const seed = System.create({ name: "strings/seed" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) commands.spawn([text, boxed])({ [text]: labelOf(i, 0), [boxed]: { text: labelOf(i, 0) } }); }); diff --git a/packages/ecs/src/perftest/structural.ts b/packages/ecs/src/perftest/structural.ts index ae187419..1c2d423e 100644 --- a/packages/ecs/src/perftest/structural.ts +++ b/packages/ecs/src/perftest/structural.ts @@ -67,15 +67,14 @@ const counted = { type: "object", properties: { count: { type: "integer" } } } a /** `count` entities of one archetype, or with `sources: 4`, interleaved over four by two source tags. */ export const seed = System.create({ name: "structural/seed", - input: { type: "object", properties: { count: { type: "integer" }, sources: { type: "integer" } } }, -}, ({ input, commands }) => { +}, ({ commands }, input: { readonly count: number; readonly sources: number }) => { for (let i = 0; i < input.count; i++) { commands.spawn(seeded[input.sources === 1 ? 0 : i % 4])({ [index]: i, [value]: i }); } }); /** `count` spawns of one archetype. */ -export const spawnBulk = System.create({ name: "structural/spawnBulk", input: counted }, ({ input, commands }) => { +export const spawnBulk = System.create({ name: "structural/spawnBulk" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) commands.spawn(made)({ [index]: i, [value]: i }); }); @@ -88,7 +87,7 @@ export const spawnSingles = System.create({ name: "structural/spawnSingles" }, ( }); /** `count` spawns over sixteen archetypes, entity i into archetype i % 16: more component sets than spawns cache. */ -export const spawnSixteen = System.create({ name: "structural/spawnSixteen", input: counted }, ({ input, commands }) => { +export const spawnSixteen = System.create({ name: "structural/spawnSixteen" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) { commands.spawn(sixteen[i % 16])({ [index]: i, [value]: i }); } diff --git a/packages/ecs/src/perftest/suite.perf-browser-test.ts b/packages/ecs/src/perftest/suite.perf-browser-test.ts index a3ff5451..104f74af 100644 --- a/packages/ecs/src/perftest/suite.perf-browser-test.ts +++ b/packages/ecs/src/perftest/suite.perf-browser-test.ts @@ -18,6 +18,7 @@ import { indexBenchmarks } from "./indexes.js"; import { structReplace, valueBenchmarks } from "./values.js"; import { observedGpuBenchmarks } from "./observed-gpu.js"; import { uploadBenchmarks } from "./uploads.js"; +import { renderOrderBenchmarks } from "./render-order.js"; /** Every worker's copy bandwidth together, through browser workers running the calibration script. */ const browserThreadsBandwidth = async (): Promise => { @@ -103,6 +104,7 @@ const benchmarks = (device: GPUDevice): readonly Benchmark[] => [ ...transferBenchmarks, ...observedGpuBenchmarks(device, run => gpuFastest(device, run)), ...uploadBenchmarks(device, run => gpuFastest(device, run)), + ...renderOrderBenchmarks(device, run => gpuFastest(device, run)), ...[[100_000, 10, "100K chains of 10"], [1_000, 1_000, "1K chains of 1,000"]].map(([roots, depth, chains]): Benchmark => ({ name: `up depth ${depth} gpu`, operation: upOperation(String(chains)), diff --git a/packages/ecs/src/perftest/suite.ts b/packages/ecs/src/perftest/suite.ts index 62f10769..631588b9 100644 --- a/packages/ecs/src/perftest/suite.ts +++ b/packages/ecs/src/perftest/suite.ts @@ -3,6 +3,7 @@ import { Archetype } from "../interface/archetype/archetype.js"; import { Component } from "../interface/component/component.js"; import { Resource } from "../interface/resource/resource.js"; import { Entity } from "../interface/entity/entity.js"; +import type { I32 } from "../interface/math/i32/index.js"; import type { Schema } from "../interface/schema/schema.js"; import { System } from "../interface/system/system.js"; import type { World } from "../interface/world/world.js"; @@ -56,7 +57,7 @@ const moving = Archetype.create([position, velocity, index]); const following = Archetype.create([position, velocity, index, target]); /** `count` entities, each targeting the one spawned before it. */ -export const seed = System.create({ name: "suite/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +export const seed = System.create({ name: "suite/seed" }, ({ commands }, input: { readonly count: number }) => { let previous = commands.spawn(moving)({ [position]: [0, 0, 0], [velocity]: [0, 0, 0], [index]: 0 }); for (let i = 1; i < input.count; i++) { previous = commands.spawn(following)({ [position]: [i, i * 0.5, -i], [velocity]: [1, 0, 0], [index]: i, [target]: previous }); @@ -64,14 +65,14 @@ export const seed = System.create({ name: "suite/seed", input: { type: "object", }); /** Despawns a tenth of the entities and spawns as many replacements. */ -export const churn = System.create({ name: "suite/churn", input: { type: "object", properties: { group: { type: "integer" } } }, components: { read: [index, position] } }, (row, { input, commands }) => { +export const churn = System.create({ name: "suite/churn", components: { read: [index, position] } }, (row, { commands }, input: { readonly group: number }) => { if (row[index] % 10 !== input.group) return; commands.despawn(row.entity); commands.spawn(moving)({ [position]: row[position], [velocity]: [0, 0, 0], [index]: row[index] }); }); /** Adds the tag to one tenth and removes it from another: a fifth of the entities move archetype. */ -export const retag = System.create({ name: "suite/retag", input: { type: "object", properties: { group: { type: "integer" } } }, components: { read: [index] } }, (row, { input, commands }) => { +export const retag = System.create({ name: "suite/retag", components: { read: [index] } }, (row, { commands }, input: { readonly group: number }) => { const group = row[index] % 10; if (group === input.group) commands.add(row.entity, tag); else if (group === (input.group + 5) % 10) commands.remove(row.entity, tag); @@ -81,9 +82,8 @@ export const retag = System.create({ name: "suite/retag", input: { type: "object export const poke = System.create({ name: "suite/poke", kernel: "cpu", - input: { type: "object", properties: { group: { type: "integer" } } }, components: { read: [index], write: [position] }, -}, (row, { input }) => { +}, (row, _context, input: { readonly group: I32 }) => { if (row[index] % 100 === input.group) { const [x, y, z] = row[position]; row[position] = [x + 1, y, z]; @@ -94,9 +94,8 @@ export const poke = System.create({ export const pokeCluster = System.create({ name: "suite/pokeCluster", kernel: "cpu", - input: { type: "object", properties: { group: { type: "integer" } } }, components: { read: [index], write: [position] }, -}, (row, { input }) => { +}, (row, _context, input: { readonly group: I32 }) => { const group = row[index] - input.group * 10_000; if (group >= 0 && group < 10_000) { const [x, y, z] = row[position]; diff --git a/packages/ecs/src/perftest/systems.ts b/packages/ecs/src/perftest/systems.ts index c32a515c..4553240f 100644 --- a/packages/ecs/src/perftest/systems.ts +++ b/packages/ecs/src/perftest/systems.ts @@ -4,6 +4,7 @@ import { always } from "../interface/resources/always.js"; import { eachTick, ticksAfter } from "./observer.js"; import { System } from "../interface/system/system.js"; import { backToBack, createWorld } from "../implementation/index.js"; +import { retainedRuns } from "../implementation/world/reports.js"; import type { Benchmark } from "./benchmark.js"; import { idle } from "./suite.js"; @@ -57,7 +58,8 @@ const churning = async (churn: (() => System) | null, after: boolean, stretches world.add(again); await ticksAfter(world, 2); world.remove(again); - await ticksAfter(world, 2); + // The world keeps the reports of its last runs and ticks, so it holds what it will hold once it has that many. + await ticksAfter(world, retainedRuns); await world.run(idle); const before = await world.tables(); let added: System | null = null; diff --git a/packages/ecs/src/perftest/transfers.ts b/packages/ecs/src/perftest/transfers.ts index 1abfc5c0..0594df18 100644 --- a/packages/ecs/src/perftest/transfers.ts +++ b/packages/ecs/src/perftest/transfers.ts @@ -23,19 +23,13 @@ Resource.register(sink, { thread: "transfersBenchmark" }); /** Bytes `take` took, on the claimed thread. */ export const taken = { bytes: 0 }; -/** - * On the claimed thread: takes a buffer and a bitmap, as an upload would, closing the bitmap. Its input's fields take - * any value, `{ allOf: [] }`, typed `unknown`, which it narrows. - */ +/** On the claimed thread: takes a buffer and a bitmap, as an upload would, closing the bitmap. */ export const take = System.create({ - name: "transfers/take", input: { type: "object", properties: { buffer: { allOf: [] }, bitmap: { allOf: [] } } }, resources: { read: [sink] }, -}, ({ input, resources }) => { + name: "transfers/take", resources: { read: [sink] }, +}, ({ resources }, input: { readonly buffer: ArrayBuffer; readonly bitmap: ImageBitmap }) => { if (resources[sink] !== null) return; - if (input.buffer instanceof ArrayBuffer) taken.bytes += input.buffer.byteLength; - if (typeof ImageBitmap === "function" && input.bitmap instanceof ImageBitmap) { - taken.bytes += input.bitmap.width * input.bitmap.height * 4; - input.bitmap.close(); - } + taken.bytes += input.buffer.byteLength + input.bitmap.width * input.bitmap.height * 4; + input.bitmap.close(); }); /** Every system the joined threads run, which they import so the world can place them there. */ diff --git a/packages/ecs/src/perftest/uploads.ts b/packages/ecs/src/perftest/uploads.ts index 4ed04278..564c3d33 100644 --- a/packages/ecs/src/perftest/uploads.ts +++ b/packages/ecs/src/perftest/uploads.ts @@ -26,9 +26,8 @@ const spawn = System.create({ name: "uploads/spawn" }, ({ commands }) => { const drag = System.create({ name: "uploads/drag", components: { write: [place] }, - input: { type: "object", properties: { shape: Entity.schema, x: { type: "number" } } }, target: "shape", -}, (row, { input }) => { +}, (row, _context, input: { readonly shape: Entity; readonly x: number }) => { row[place] = [input.x, 1, 0]; }); /** Draws nothing: its run uploads what it reads. */ diff --git a/packages/ecs/src/perftest/values.ts b/packages/ecs/src/perftest/values.ts index 8555da41..0ed06a7e 100644 --- a/packages/ecs/src/perftest/values.ts +++ b/packages/ecs/src/perftest/values.ts @@ -36,7 +36,7 @@ const Node = Archetype.create([pose, shown, extent, order]); const rows = 100_000; -const seed = System.create({ name: "values/seed", input: { type: "object", properties: { count: { type: "integer" } } } }, ({ input, commands }) => { +const seed = System.create({ name: "values/seed" }, ({ commands }, input: { readonly count: number }) => { for (let i = 0; i < input.count; i++) commands.spawn(Node)({ [pose]: { tx: i, ty: 0, rotation: 0, sx: 1, sy: 1 }, [shown]: { tx: 0, ty: 0, rotation: 0, sx: 1, sy: 1 }, [extent]: [1, 1], [order]: i }); }); diff --git a/packages/ecs/src/samples/browser.test.ts b/packages/ecs/src/samples/browser.test.ts new file mode 100644 index 00000000..8c698a8d --- /dev/null +++ b/packages/ecs/src/samples/browser.test.ts @@ -0,0 +1,19 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +/// +import { existsSync, readdirSync, readFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; +import { expect, it } from "vitest"; + +const samples = fileURLToPath(new URL(".", import.meta.url)); + +it("the sample browser has a tab for each sample folder, titled as its page", () => { + const page = readFileSync(`${samples}index.html`, "utf8"); + const tabs = [...page.matchAll(/([^<]+)<\/a>/g)].map(([, name, title]) => ({ name, title })); + const folders = readdirSync(samples, { withFileTypes: true }) + .filter(entry => entry.isDirectory() && existsSync(`${samples}${entry.name}/index.html`)) + .map(entry => { + const title = /([^<]+)<\/title>/.exec(readFileSync(`${samples}${entry.name}/index.html`, "utf8"))?.[1]; + return { name: entry.name, title }; + }); + expect(tabs).toEqual(folders); +}); diff --git a/packages/ecs/src/samples/browser.ts b/packages/ecs/src/samples/browser.ts new file mode 100644 index 00000000..b99c475a --- /dev/null +++ b/packages/ecs/src/samples/browser.ts @@ -0,0 +1,20 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +// The sample browser: a tab per sample, the one the URL's hash names shown in the frame, the first when it names none. + +const tabs = [...document.querySelectorAll<HTMLAnchorElement>('nav a[href^="#"]')]; +const frame = document.querySelector("iframe"); +const alone = document.querySelector<HTMLAnchorElement>("#alone"); + +const show = (): void => { + const chosen = tabs.find(tab => tab.hash === location.hash) ?? tabs[0]; + if (chosen === undefined || frame === null) return; + const page = `./${chosen.hash.slice(1)}/`; + for (const tab of tabs) if (tab === chosen) tab.setAttribute("aria-current", "page"); else tab.removeAttribute("aria-current"); + if (frame.getAttribute("src") !== page) frame.setAttribute("src", page); + if (alone !== null) alone.href = page; + document.title = `${chosen.textContent ?? ""} · ECS Samples`; +}; + +addEventListener("hashchange", show); +show(); diff --git a/packages/ecs/src/samples/index.html b/packages/ecs/src/samples/index.html new file mode 100644 index 00000000..973ce70d --- /dev/null +++ b/packages/ecs/src/samples/index.html @@ -0,0 +1,34 @@ +<!doctype html> +<html lang="en"> +<head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>ECS Samples + + + + +

+ + + + diff --git a/packages/ecs/src/samples/particles/gpu/particles.ts b/packages/ecs/src/samples/particles/gpu/particles.ts index 27f71235..01d6f315 100644 --- a/packages/ecs/src/samples/particles/gpu/particles.ts +++ b/packages/ecs/src/samples/particles/gpu/particles.ts @@ -49,8 +49,7 @@ const random = (seed: number): (() => number) => { /** The particle count, and the field's width and height in pixels. Entity i goes to archetype i % 4. */ export const spawn = System.create({ name: "gpuParticles/spawn", - input: { type: "object", properties: { count: { type: "integer" }, width: { type: "number" }, height: { type: "number" } } }, -}, ({ input: { count, width, height }, commands }) => { +}, ({ commands }, { count, width, height }: { readonly count: number; readonly width: number; readonly height: number }) => { const next = random(7); for (let i = 0; i < count; i++) { const p = [next() * width, next() * height, 0] as const; diff --git a/packages/ecs/src/samples/particles/particles.ts b/packages/ecs/src/samples/particles/particles.ts index f927712e..0bd4ece2 100644 --- a/packages/ecs/src/samples/particles/particles.ts +++ b/packages/ecs/src/samples/particles/particles.ts @@ -59,9 +59,8 @@ export const integrate = System.create({ // Transaction carrying frame time into the ECS. export const setDelta = System.create({ name: "particles/setDelta", - input: { type: "object", properties: { seconds: { type: "number" } } }, resources: { write: [delta] }, -}, ({ resources, input }) => { +}, ({ resources }, input: { readonly seconds: number }) => { resources[delta] = input.seconds; }); diff --git a/packages/ecs/src/samples/todo/README.md b/packages/ecs/src/samples/todo/README.md index 04313d52..449c27dc 100644 --- a/packages/ecs/src/samples/todo/README.md +++ b/packages/ecs/src/samples/todo/README.md @@ -3,7 +3,7 @@ A todo app whose world drives every change to the DOM through systems, while its elements know nothing about the ECS. It's the experiment for `ecs-dom.md`: vanilla custom elements, no framework, no persistence. -`pnpm sample`, then open `/todo/`. Add, toggle, edit (double-click), delete, move (drag, or Alt+↑ and Alt+↓), mark all +`pnpm dev`, then the Todos tab. Add, toggle, edit (double-click), delete, move (drag, or Alt+↑ and Alt+↓), mark all done or none, clear those done, filter, and generate 10 or 1,000 todos. ## How the DOM is driven @@ -141,17 +141,22 @@ What the current API does and doesn't offer the DOM path, each with the change i to back in an empty world, at most 0.3 µs more with the app's observer systems added, and 2.5 µs in the app with 1K todos, where a task round trip takes 7.6 µs. Node's `run idle` is 0.33 µs. A run made right after a frame, as an event's is, takes - 0.02–0.10 ms, which is the CPU coming back from idle, not the ECS. Toggling one todo of 10K takes 62 µs, of which - the profile gives 64% to `mergeEntries` (`index-hash.ts`), keeping `completion/byDone`'s two buckets of 5K entries in - entity order, and 16% to `forEachPassingRange` (`ranges.ts`), the change filters scanning the 1024 row ticks of the - one chunk that changed, for each observer system with `changed`. The rest is the sync point, the observer pass and - the DOM writes, about 13 µs. *Proposals* in the next two findings. -- **A low-cardinality hash index costs O(bucket) per change:** toggling one todo moves it between `byDone`'s buckets, - and each bucket's entries shift to stay in entity order: 26 µs a toggle at 1K, 62 at 10K, 400 at 100K. *Proposal:* - unordered buckets with swap-remove and a position per entity, sorted only when a list is read in entity order; or - a count-only index for what reads only `length`. -- **A change filter scans a whole chunk for one changed row,** 1024 ticks for each system with `changed`. *Proposal:* - a second level of ticks per 32 or 64 rows, or a list of the rows changed per chunk. + 0.02–0.10 ms, which is the CPU coming back from idle, not the ECS. Toggling one todo of 10K took 62 µs, of which + the profile gave 64% to keeping `completion/byDone`'s two buckets of 5K entries in entity order, which the next + finding removes, and 16% to `forEachPassingRange` (`ranges.ts`), the change filters scanning the 1024 row ticks of + the one chunk that changed, for each observer system with `changed`, which the finding after it removes. The rest is + the sync point, the observer pass and the DOM writes, about 13 µs. +- **A low-cardinality hash index costs O(1) amortized per change:** toggling one todo moves it between `byDone`'s + buckets, which are left unsorted, its entry leaving a hole in one and joining a tail of the other, until a list is + read by position (`index-hash.ts`). In node, a toggle and a run reading both buckets' lengths, as the summary does, + take 5.5, 5.3 and 5.5 µs at 1K, 10K and 100K todos (`bucket move … length`), against 9.1, 64 and 423 µs with every + bucket kept in entity order; reading each bucket's first rows after it, as `clearCompleted` reads the done ones, + sorts both buckets first: 6.2, 12.7 and 69 µs (`bucket move … in order`), against 9.4, 65 and 418. +- **A change filter finds one changed row by its block:** it reads the chunk's sixteen block ticks and the 64 row ticks + of the block that changed, not the chunk's 1024. In a profile of 40,000 toggles of one todo of 10K, the change + filters take 5.0% of samples, against 6.9% scanning the chunk, most of it the chunk checks of observer systems + filtering on two or three components; in node, eight systems filtering on one changed row of 100K take 5.0 µs a + run, against 34.9 µs scanning the chunk (`changed one 100K`). *No change needed.* - **Nothing awaits the next tick.** Tests watch `clock` changes through an observer, after a first run that passes it regardless, and a world ticks only while a system that isn't an observer is added. *Proposal:* none for consumers; a test helper in `src/observe` would do. diff --git a/packages/ecs/src/samples/todo/features/completion/model.ts b/packages/ecs/src/samples/todo/features/completion/model.ts index 68299fbc..8bdd164d 100644 --- a/packages/ecs/src/samples/todo/features/completion/model.ts +++ b/packages/ecs/src/samples/todo/features/completion/model.ts @@ -6,9 +6,8 @@ import { byDone } from "./data.js"; /** Marks every todo done, or every one not done. Only todos that change are written. */ export const toggleAll = System.create({ name: "completion/toggleAll", - input: { type: "object", properties: { done: { type: "boolean" } } }, components: { write: [done], has: [todo] }, -}, (row, { input }) => { +}, (row, _context, input: { readonly done: boolean }) => { if (row[done] !== input.done) row[done] = input.done; }); diff --git a/packages/ecs/src/samples/todo/features/filter/model.ts b/packages/ecs/src/samples/todo/features/filter/model.ts index 0c3305b0..d6144bc9 100644 --- a/packages/ecs/src/samples/todo/features/filter/model.ts +++ b/packages/ecs/src/samples/todo/features/filter/model.ts @@ -1,11 +1,10 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. import { System } from "../../../../index.js"; -import { filter, filterSchema } from "./data.js"; +import { filter, type Filter } from "./data.js"; export const setFilter = System.create({ name: "filter/set", - input: { type: "object", properties: { filter: filterSchema } }, resources: { write: [filter] }, -}, ({ input, resources }) => { +}, ({ resources }, input: { readonly filter: Filter }) => { resources[filter] = input.filter; }); diff --git a/packages/ecs/src/samples/todo/features/generate/model.ts b/packages/ecs/src/samples/todo/features/generate/model.ts index d0ef7b1e..d938e0b1 100644 --- a/packages/ecs/src/samples/todo/features/generate/model.ts +++ b/packages/ecs/src/samples/todo/features/generate/model.ts @@ -10,9 +10,8 @@ export const generatedLabel = (n: number): string => `${tasks[n % tasks.length]} /** Adds `count` todos with generated labels at the end of the list. */ export const addGenerated = System.create({ name: "generate/add", - input: { type: "object", properties: { count: { type: "integer", minimum: 0 } } }, lookup: { ordered: { index: ordered, read: [order] } }, -}, ({ input, lookup, commands }) => { +}, ({ lookup, commands }, input: { readonly count: number }) => { const all = lookup.ordered(); const first = orderAfter(all); const spawn = commands.spawn(Todo); diff --git a/packages/ecs/src/samples/todo/features/todos/model.ts b/packages/ecs/src/samples/todo/features/todos/model.ts index df8ef70c..6dcfa1e2 100644 --- a/packages/ecs/src/samples/todo/features/todos/model.ts +++ b/packages/ecs/src/samples/todo/features/todos/model.ts @@ -2,37 +2,33 @@ import { Entity, System } from "../../../../index.js"; import { Todo, done, label, order, orderAfter, ordered, todo } from "./data.js"; -const text = { type: "string" } as const; - /** Adds a todo at the end of the list, with its label trimmed; a blank label adds nothing. */ export const addTodo = System.create({ name: "todos/add", - input: { type: "object", properties: { label: text } }, lookup: { ordered: { index: ordered, read: [order] } }, -}, ({ input, lookup, commands }) => { +}, ({ lookup, commands }, input: { readonly label: string }) => { const trimmed = input.label.trim(); if (trimmed === "") return; commands.spawn(Todo)({ [label]: trimmed, [done]: false, [order]: orderAfter(lookup.ordered()) }); }); -const one = { entity: Entity.schema } as const; +/** The todo a system on one acts on. */ +type One = { readonly entity: Entity }; export const toggleTodo = System.create({ name: "todos/toggle", - input: { type: "object", properties: one }, target: "entity", components: { write: [done], has: [todo] }, -}, row => { +}, (row, _context, _input: One) => { row[done] = !row[done]; }); /** Gives a todo a new label, trimmed; a blank one deletes it. */ export const editTodo = System.create({ name: "todos/edit", - input: { type: "object", properties: { ...one, label: text } }, target: "entity", components: { write: [label], has: [todo] }, -}, (row, { input, commands }) => { +}, (row, { commands }, input: One & { readonly label: string }) => { const trimmed = input.label.trim(); if (trimmed === "") commands.despawn(row.entity); else row[label] = trimmed; @@ -40,10 +36,9 @@ export const editTodo = System.create({ export const deleteTodo = System.create({ name: "todos/delete", - input: { type: "object", properties: one }, target: "entity", components: { has: [todo] }, -}, (row, { commands }) => { +}, (row, { commands }, _input: One) => { commands.despawn(row.entity); }); @@ -53,9 +48,8 @@ export const deleteTodo = System.create({ */ export const moveTodo = System.create({ name: "todos/move", - input: { type: "object", properties: { entity: Entity.schema, before: Entity.schema } }, lookup: { todos: { read: [order], has: [todo] }, ordered: { index: ordered, read: [order] } }, -}, ({ input, lookup, commands }) => { +}, ({ lookup, commands }, input: { readonly entity: Entity; readonly before: Entity }) => { const { entity, before } = input; if (entity === before || lookup.todos(entity) === null) return; const next = before === Entity.none ? null : lookup.todos(before); diff --git a/packages/ecs/src/samples/transfers/README.md b/packages/ecs/src/samples/transfers/README.md new file mode 100644 index 00000000..845a9ffc --- /dev/null +++ b/packages/ecs/src/samples/transfers/README.md @@ -0,0 +1,35 @@ +# Transfers sample + +Shows how values that can't be copied between threads reach the thread that uses them. A canvas and decoded textures +are moved, not copied, to the world's worker, which draws sprites with them every frame. Components and resources +holding such values are registered with `transfer: true`, and a run's input carries them to the world's thread. + +`pnpm dev`, then the Transferred Sprites tab. Numbered discs appear one at a time, 48 in all, and bounce around the canvas; +the overlay counts the sprites and frames. + +## Threads + +- **Main** (`main.ts`) owns the page. It starts both workers, hands each a port to the world, and joins the world. It + moves its `` to the world's thread with `transferControlToOffscreen()`, as the input of a run of + `attachSurface`, and adds `show`, an observer system that writes the counts into the overlay after each frame that + changes them. +- **The world's worker** (`world.ts`) creates the world, which ticks every frame, and shares it on the ports main + sends. The canvas becomes its `surface` resource. +- **The loader's worker** (`loader.ts`) joins the world and makes one image every 120 ms: it draws a disc of its own + hue, encodes it as a PNG, decodes it to an `ImageBitmap` and runs `addSprite` with it. The bitmap is detached in the + loader once the run is posted. + +## Data and systems + +`sprites.ts` holds them, and every thread imports it: + +- `texture`, a component registered with `transfer: true`, holds a sprite's `ImageBitmap`; `position` and `velocity` + are `Vec2`s, in pixels and pixels per second. +- `surface`, a resource registered with `transfer: true`, holds the `OffscreenCanvas`; `stats` counts sprites and + frames. +- `attachSurface` and `addSprite` take what main and the loader move in their input. `frame`, the composite the world + adds, clears the canvas, moves each sprite, bouncing off the edges, and draws each one's texture where it is. + +## Tests + +`build.test.ts`: the sample builds as an application, with the world's worker and the loader's. diff --git a/packages/ecs/src/samples/transfers/sprites.ts b/packages/ecs/src/samples/transfers/sprites.ts index 33e5f479..e00513a4 100644 --- a/packages/ecs/src/samples/transfers/sprites.ts +++ b/packages/ecs/src/samples/transfers/sprites.ts @@ -3,7 +3,7 @@ import { Component, Resource, System, Vec2, clock } from '../../index.js'; // The transfers sample's keys and systems, which every thread imports: the world, on a worker, draws sprites into the // canvas main moved to it, each with a texture a loader worker decoded and moved to it. Values move through runs whose -// input holds them; an input field's schema `{ allOf: [] }` takes any value, typed `unknown`, which bodies narrow. +// input holds them. export const texture = Symbol("transferSprites/texture"); export const position = Symbol("transferSprites/position"); @@ -34,11 +34,9 @@ Resource.register(stats, { type: { type: "object", properties: { sprites: { type: "integer" }, frames: { type: "integer" } }, default: { sprites: 0, frames: 0 } }, }); -const any = { allOf: [] } as const; - /** From main: the canvas the world draws into. */ -export const attachSurface = System.create({ name: "transferSprites/attachSurface", input: { type: "object", properties: { canvas: any } }, resources: { write: [surface] } }, ({ input, resources }) => { - if (typeof OffscreenCanvas === "function" && input.canvas instanceof OffscreenCanvas) resources[surface] = input.canvas; +export const attachSurface = System.create({ name: "transferSprites/attachSurface", resources: { write: [surface] } }, ({ resources }, input: { readonly canvas: OffscreenCanvas }) => { + resources[surface] = input.canvas; }); /** A deterministic number in [0, 1) for each seed. */ @@ -49,10 +47,10 @@ const random = (seed: number): number => { /** From the loader: a sprite with the texture it decoded, somewhere on the canvas, moving. */ export const addSprite = System.create({ - name: "transferSprites/addSprite", input: { type: "object", properties: { bitmap: any, seed: { type: "integer" } } }, resources: { read: [surface], write: [stats] }, -}, ({ input, resources, commands }) => { + name: "transferSprites/addSprite", resources: { read: [surface], write: [stats] }, +}, ({ resources, commands }, input: { readonly bitmap: ImageBitmap; readonly seed: number }) => { const canvas = resources[surface]; - if (canvas === null || typeof ImageBitmap !== "function" || !(input.bitmap instanceof ImageBitmap)) return; + if (canvas === null) return; const angle = random(input.seed) * Math.PI * 2; const speed = 60 + random(input.seed + 1) * 120; commands.spawn([texture, position, velocity])({ diff --git a/packages/ecs/threads.md b/packages/ecs/threads.md index 2031ffb1..e24f224e 100644 --- a/packages/ecs/threads.md +++ b/packages/ecs/threads.md @@ -181,9 +181,10 @@ All derived from declarations and, for terminal systems, the plugin's analysis o - **Archetype records:** the table holds, by the world's archetype number, the offset of that archetype's record. Each record is allocated once, since an archetype's storages are fixed, and never moves; only the table grows. A record holds `capacity`, the `entities` offset, a version, its flags and its count, and per storage its component number, its - flags (`ticked`, linear, remote with the joined thread's number), its column, tick and chunk-tick offsets and its - chunk count; a remote storage's column is its tags. Offsets are `Uint32`, since memory reaches 4 GiB. The resource - archetype has a record too, flagged, and the header holds the resource entity's index. + flags (`ticked`, linear, remote with the joined thread's number), its column, tick and chunk-tick offsets, its + chunk count and its block-tick offsets, sixteen blocks a chunk; a remote storage's column is its tags. Offsets are + `Uint32`, since memory reaches 4 GiB. The resource archetype has a record too, flagged, and the header holds the + resource entity's index, and a count of flushes that wrote an existing archetype's record again. - **Entity-table record:** the `cursor` offset, which never moves, the free list's offset and capacity, the `archetype`, `row` and `generation` offsets and capacity, and a version. The table's length is `cursor[2]`. Its arrays move whenever they grow, even without memory growing, since a grown array's old block is reused. @@ -195,7 +196,9 @@ All derived from declarations and, for terminal systems, the plugin's analysis o - **Reads:** a thread checks a record's version, one load, before using what it cached for that archetype, and keeps per-archetype column views and kernel columns, rebuilt only when that version moves. The whole-buffer view serves the header and records; columns stay views of their own, as kernels and layouts take them. On an entity-table version - change, a worker updates the `entities` object its spawners hold in place. + change, a worker updates the `entities` object its spawners hold in place. A joined thread's index lookups check + that count once a run instead, with its mirror's archetypes and its own views (memory grown, keys named): rows they + gave stay bound while none moved, and are bound again to the views they find otherwise, a rebuilt mirror's included. - **Messages:** only new components' schemas are posted, none for a value that isn't data, since a message closes the pool's windows. A new archetype is found in the table, its record naming its component numbers, the first time a task names it. @@ -255,7 +258,9 @@ only on node, so browser bundles never load them, and the node worker's url isn' so a browser build bundles one worker (`worker-platform.ts`). Death is a `Worker`'s `error` or `messageerror`, a node worker's `error`, `messageerror` or `exit`, or, for a lent worker, what its owner reports or either `MessagePort`'s `close` where the platform has it, whichever comes first; nothing polls, so a quiet world does no work. A browser -`Worker` ended from outside its port reports nothing, so a lent worker's owner reports the end of one it ends. +`Worker` ended from outside its port reports nothing, so a lent worker's owner reports the end of one it ends, before +ending it. The worker's port closing is reported only once the owner answers a `closed` on its channel, which comes +after anything it reported first, so an owner's report wins over the close its ending causes. - **Platforms:** a world's pool comes from a platform, `{ count, start }`: `count` is what the `workers` resource's `null` resolves to, its hardware threads, `navigator.hardwareConcurrency` or `os.availableParallelism()`, minus the @@ -453,8 +458,11 @@ and is reported once in a sync point's failures. Once the thread is gone, they r - **Stop lists** (`stop-lists.ts`): before apply, the world's thread writes one block per sync point with stops for that thread: per finalizer, in the world's finalizer order, a group of its stops as columns: each entity, its pre-apply archetype and row, the tag of each object it reads, and a copy of each linear struct it reads, in that component's - layout, and of each string's record, by its handle, let go once the thread acknowledged the block: it may finalize - sync points after the row's own record was let go and freed. The world copies a column at a time, finding each column's storage once per run of an archetype. Blocks are + layout, and of each string's record, by its handle; then, once per group, the resources it reads that aren't its + thread's objects, as they were before apply: a copy of each string resource's record, by its handle, then each linear + resource's value in its layout, from the resource entity's row; a constant needs none. A finalizer reading no such + resource adds no bytes. The records' copies are let go once the thread acknowledged the block: it may finalize sync + points after the row's or resource's own record was let go and freed. The world copies a column at a time, finding each column's storage once per run of an archetype. Blocks are linked in sync order, the first from the log header's stops word, each block's next offset stored once that block is whole, so a thread reads only whole blocks of published syncs. A block is freed once the thread acknowledged a later one, since the thread reads the next offset from the newest block it ran. The world wakes the thread after each sync @@ -463,14 +471,18 @@ and is reported once in a sync point's failures. Once the thread is gone, they r entry, before the sync's entries apply, or at replay's end for a sync with no entries for it; its finalizers read the linear structs from the block, constants as they are, and its objects by each stop's tag: the start or `null` as tagged, any other at the stop's row in its mirror. After its log lapped, it runs them before it rebuilds, reading a - value it wrote or a settled slot's by entity in its old rows, and a live slot's from its staging array. Resources are - read where they are. Then it stores "finalized through sync N" in the log header. A stop list naming a finalizer + value it wrote or a settled slot's by entity in its old rows, and a live slot's from its staging array. Its thread's + object resources are read as it holds them: at the resource entity's row in its mirror as replayed so far, which may + be behind the entity table, its old rows before a rebuild, and the start where no row holds it, as before its first + rebuild. That's what the rebuild keeps: only its systems write those values, and the world gives them only their start, + when they're added or claimed. Its string and linear resources are read from the group's copies, so every resource + it reads is as of before its stops' sync point applied. Then it stores "finalized through sync N" in the log header. A stop list naming a finalizer whose setup hasn't arrived holds replay before its sync, as a component without a key does; one this thread never imported fails each of its groups. What its finalizers throw is posted, and a later run's report lists it as a `"finalizer"` failure. - **The guarantee:** each such finalizer runs exactly once per stop, on its thread, before that thread's next task or observer post after the sync point and before the entity's index is reused, seeing its declared struct and string - reads and its thread's objects as of before apply. + reads, its thread's objects and every resource it reads as of before apply. - **Held indexes:** the index of an entity despawned with such stops is held, counted by index for each thread that hasn't acknowledged it, apart from `released`, so the leak scan never sees it and the free list never has it; the sync point after a thread acknowledges releases it once no thread holds it. A thread that's gone has its held indexes @@ -583,9 +595,10 @@ its tails before it resolves, with their failures in its report. - **Fallback snapshot.** Sometimes a conflicting writer is ready before the tail has finished its reads, as when main is late. The world's thread then evaluates the tail's head as the tail would, copies the declared linear struct reads of the passing rows from the position the tail reached, with their position, archetype, row and entity, and its - linear resources, a string's handles among them, into a tail buffer, a column at a time; exchanges the progress word for a sentinel; and waits until - the tail is done through what it had claimed, at most one chunk, spinning briefly before it waits. A claim that - returns the sentinel reads the rest from the copy, rows from its own position on. The values are the same either + linear resources, a string's handles among them, into a tail buffer, a column at a time; adds a sentinel to the + progress word, which then still shows whether a claim came first; and waits until the tail is done through what it + had claimed, at most one chunk, spinning briefly before it waits. A claim that returns the sentinel or more reads the + rest from the copy, rows from its own position on. The values are the same either way, so this stays deterministic. A prompt main never copies. A string's handle in the copy names its record as of the copy, which outlives the tail: a tick posts its tails once the last tick's are done, and a run waits for them, so a tail ends before the second sync point after the one it reads, and a record let go is freed only then. The copy lets the world's work after its writer @@ -596,8 +609,9 @@ its tails before it resolves, with their failures in its report. - **The tail's own objects** are read where they are: while a tail is in flight, the thread's replay stops at the sync it names, which the world stores in the log header as it posts it, so the next sync point's moves wait in the log until the tail finishes. Rows read from the copy take their objects from the mirror at the copy's rows, or, once its - log lapped and it rebuilt past that sync, by entity. Objects it writes from the copy go back to the mirror after its - body; the world marks their tags the thread's own when it copies, and stamps them changed at the tail's tick, as the + log lapped and it rebuilt past that sync, by entity. Its object resources stay at the copy's resource row, rebuilt or + not, since the resource entity moves only when resources are added, which waits for tails. Objects it writes from the + copy go back to the mirror after its body; the world marks their tags the thread's own when it copies, and stamps them changed at the tail's tick, as the task's completion names them by entity, before the thread's next task. - **What waits for a tail to finish:** the next task on its thread, and what moves rows a tail may still read outside a sync point: registration, resources added between runs, a GPU device change, and joined threads joining or leaving. @@ -629,8 +643,8 @@ is in the input, and the system, which writes the component, runs on that thread point, and a staged slot of a joined thread's once it replays that sync point, which the world wakes it for. Releasing closes an `ImageBitmap`, `VideoFrame`, `AudioData`, `MessagePort` or `RTCDataChannel`, cancels or aborts a stream, and leaves an `ArrayBuffer`'s or an `OffscreenCanvas`'s memory to the collector. -- **Types:** an input schema can't name a Transferable type yet; a field declared `{ allOf: [] }` takes any value, - typed `unknown`, which the body narrows. +- **Types:** an input field is typed as its class, `{ readonly canvas: OffscreenCanvas }`, whose schema the plugin + gives as one taking any value. ### Without cross-origin isolation @@ -910,8 +924,7 @@ that's gone rejects; an input no task took is released once its run finishes; a its thread. The sample (`src/samples/transfers`) runs the world on a worker drawing into the OffscreenCanvas main moved to it, main showing its state in the DOM, and a loader worker moving each texture it decoded to the world's thread. -- Interface: none. An input schema can't name a Transferable type, so a run's input can't be typed with one; proposed - for review, and meanwhile a field `{ allOf: [] }` takes one as `unknown`. +- Interface: none; an input field is typed as its Transferable class. - Contract: a run's Transferable input moves, never copied, to the thread its system runs on, and the run lands once it arrived, in call order with that thread's other runs; a run whose input moved to a thread that's gone rejects; a value whose command drops is released on its thread. diff --git a/packages/ecs/vite.sample.config.ts b/packages/ecs/vite.sample.config.ts index 6610e675..f7c583d1 100644 --- a/packages/ecs/vite.sample.config.ts +++ b/packages/ecs/vite.sample.config.ts @@ -3,9 +3,11 @@ import { defineConfig } from "vite"; import { kernelPlugin } from "./src/implementation/compiler/index.ts"; import { isolation, sources } from "./vitest.plugins.ts"; -/** Serves `src/samples`, one folder per sample, cross-origin isolated for shared memory: `pnpm sample` opens /particles/. */ +/** + * Serves `src/samples`, one folder per sample with the sample browser at the root, cross-origin isolated for shared + * memory. It opens no browser. + */ export default defineConfig({ root: "src/samples", plugins: [kernelPlugin(), isolation, sources], - server: { open: "/particles/" }, });