ecs: order on System.render, one paint-ordered draw across archetypes

packages/ecs/design.md

@@ -56,7 +56,7 @@ Rule: anything that doesn't affect the API or correctness belongs in phase 2+.
   - **Subset errors** are reported at build time. An ESLint rule shares the same validator, so errors also show in the editor.
 - **GPU systems.** The built-in `gpuDevice` resource, `GPUDevice | null`, is the one device the world's GPU storage lives on, for compute and render alike: GPU buffers belong to one device, so a second device means a second world. It's an ordinary resource, given as `World.create({ gpuDevice: device })` or set later by a main-thread system; the world never destroys it. After each run the world follows it: a new device gets the GPU storage and `kernel: "gpu"` systems, after what's ahead on the old device is read back; with null, GPU kernels run on the CPU again. Render systems run only while there's a device, as if their `is` listed `gpuDevice`. A lost device makes the world set `gpuDevice` to null, as its own sync point, so nothing fails: GPU kernels run on the CPU from the CPU's copies, render systems are skipped, and systems watching `changed: [gpuDevice]` see it; GPU writes not yet read back are lost with the device.
   - **`kernel: "gpu"` declares a GPU kernel.** It runs as a WGSL compute shader compiled from the same Program IR as WASM in a world with a device, and as a `"cpu"` kernel in one without, so it stays portable. Each rule is checked as early as it can be known: types rule out lookups, change filters and commands; the build rejects loops and anything that keeps the body from lowering to a Program; registration rejects f64 or byte values, in every world, since such a declaration can never hold.
-  - **Render systems** (`System.render`) run once per run on the main thread in schedule order, read-only. They record into the run's encoder over the GPU copies of the components their accessors name, which are what they read, one table per matched archetype. `Gpu.wgsl(system)` returns their shaders' declarations: the world's bind group at the group they name and a function per component (`fn position(row: u32) -> vec3<f32>`), from schemas alone, so shaders can be written before a world exists. Pipelines take `Gpu.layout(device)`, one object per device, so they can be built before a world exists; bind groups and buffers stay the world's, so they can be replaced as storage grows.
+  - **Render systems** (`System.render`) run once per run on the main thread in schedule order, read-only. They record into the run's encoder over the GPU copies of the components their accessors name, which are what they read, one table per matched archetype; with `order`, a sorted index, one table of every matching row in its order, whose accessors resolve an instance's position through an order block of slot and row to its archetype's columns. `Gpu.wgsl(system)` returns their shaders' declarations: the world's bind group at the group they name and a function per component (`fn position(row: u32) -> vec3<f32>`), from schemas alone, so shaders can be written before a world exists. Pipelines take `Gpu.layout(device)`, one object per device, so they can be built before a world exists; bind groups and buffers stay the world's, so they can be replaced as storage grows.
   - **Residency:** a separate GPU heap holds copies of only the storages GPU systems use. Each copy tracks rows the CPU changed since its upload and rows the GPU wrote since its readback. GPU systems upload what the CPU changed first, in runs of the chunks it stamped since the last upload: the components GPU systems use keep ticks, and kernels writing them keep their chunk stamps. CPU systems read back what the GPU wrote first. A sync point reads back only archetypes whose existing rows it moves or writes (despawns, adds, removes); spawns only append rows, which are uploaded alone, and a copy that outgrows its block moves on the GPU. GPU-to-GPU chains and rendering never copy back.
   - **Change filters:** a GPU kernel's writes stamp each written column once with the run's tick, and `changed` filters pass a whole column when its stamp is newer than their `lastRun`, so writes on the GPU cost nothing per row on the CPU.
   - **Ordering:** a queue write lands before recorded but unsubmitted work, so recorded work is submitted before each upload. Each run's encoder is submitted at its end, before the sync point; `run` resolves then, not when the GPU finishes.

packages/ecs/followups.md

@@ -35,12 +35,6 @@ Open items awaiting a decision or work. Remove an item once it's decided or done
   per-row object such as a texture's bind group, and a row body can't learn its GPU row. A consumer drawing an image
   per row keeps its own instance buffer, 4,000 draws in 2.1 ms a tick. Possible: tables giving their entities in GPU
   row order. Not needed until a consumer's profile says so.
-- **Paint order across render tables:** a render system draws each matching archetype as its own table in row order,
-  so 2D content of several kinds (shapes, images, paths) can't be blended back to front without the consumer keeping
-  its own sorted instance list. Proposal: `order: index` on `System.render`, giving one table spanning its archetypes
-  whose `bind` sets a buffer of the index's rows in sorted order with each archetype's column base addresses; every
-  column already lives in one heap, so accessors resolve a sorted instance to its row. Not needed while consumers draw
-  every kind with one shader from their own list.
 - **Report retention:** the last 256 runs keep their reports (`retainedRuns` in `reports.ts`), a fixed internal
   constant; a later run takes the oldest one's entity. Recommended: keep it internal until a consumer needs another; then
   a world option or resource.

packages/ecs/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@adobe/ecs",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "Adobe ECS",
   "license": "MIT",
   "type": "module",

packages/ecs/src/implementation/world/index-sorted.ts

@@ -585,12 +585,16 @@ const split = (table: SortedTable, k: number, tick: number): void => {
     reindexFrom(table, k, table.blocks);
 };
 
-/** Updates where entity index `entity` is: its archetype and row, at its entry. */
-export const relocateSorted = (table: SortedTable, entity: number, archetype: number, row: number): void => {
+/** Updates where entity index `entity` is: its archetype and row, at its entry. Returns whether either changed. */
+export const relocateSorted = (table: SortedTable, entity: number, archetype: number, row: number): boolean => {
     const base = entryOf(table, entity);
-    if (base < 0) return;
-    table.entries[base + table.words + 1] = archetype;
-    table.entries[base + table.words + 2] = row;
+    if (base < 0) return false;
+    const at = base + table.words;
+    const { entries } = table;
+    if (entries[at + 1] === archetype && entries[at + 2] === row) return false;
+    entries[at + 1] = archetype;
+    entries[at + 2] = row;
+    return true;
 };
 
 /**

packages/ecs/src/implementation/world/indexes.ts

@@ -51,6 +51,11 @@ export type WorldIndex = {
      * it was built; 0 before.
      */
     changed: number;
+    /**
+     * For a sorted index, the tick of the last refresh at which an entry moved to another archetype or row with its key
+     * and place unchanged, or it was built; 0 before.
+     */
+    moved: number;
     /** By watched component: how many lookups watch it, and the tick its ticks were read up to. */
     readonly watchers: Map<number, number>;
     readonly scanned: Map<number, number>;
@@ -124,6 +129,7 @@ export const acquireIndex = (state: State, definition: IndexDefinition): WorldIn
         position: -1,
         refreshed: 0,
         changed: 0,
+        moved: 0,
         watchers: new Map(),
         scanned: new Map(),
         hash: sorted ? null : createHashTable(state.memory, words),
@@ -434,6 +440,7 @@ const finishRefresh = async (state: State, index: WorldIndex, end: number, tick:
 /** Builds an index again: in phases, across the pool when there is one, for a large hash index keyed by structs; else here. */
 const rebuildAny = (state: State, index: WorldIndex, tick: number, failures: Failure[], pool: Pool | null): Promise<void> | null => {
     index.changed = tick;
+    index.moved = tick;
     const phased = rebuildPhased(state, index, tick, failures, pool);
     if (phased !== false) return phased;
     rebuild(state, index, tick, failures);
@@ -597,7 +604,7 @@ const patchSorted = (state: State, index: WorldIndex, sorted: SortedTable, count
         const fresh = index.objects && unknownValue(index, key);
         if (fresh) encodeRow(index, archetype, row, key, 0, true);
         if (old >= 0 && !fresh && sameStoredKey(sorted, old, key, 0)) {
-            relocateSorted(sorted, entity, archetypeId, row);
+            if (relocateSorted(sorted, entity, archetypeId, row)) index.moved = tick;
             continue;
         }
         if (objects && !fresh) holdWords(index, key, 0, true);

packages/ecs/src/implementation/world/perf.md

@@ -142,6 +142,10 @@ GPU copies uploading only the chunks CPU writes changed, against `a4eef10e`, chr
 - **An upload writes each run of adjacent chunks stamped since its last upload** or holding rows a sync point appended or disturbed, clamped to the row count; a column with GPU writes the CPU hasn't read back writes only those rows. A whole-column stamp (`columnChanged`) uploads nothing: only GPU kernels make one, and their writes are on the GPU already.
 - **A WASM kernel's column addresses are kept per load as well as per archetype version:** a kernel loaded again to stamp a component that already kept ticks went on passing the muted column's address, 0, and wrote its stamps over the entity table there.
 
+Render systems with `order`, against `9daaa3f8`, chromium, three runs each: new `gpu ordered render unchanged 50K`, 50K shapes and images, two archetypes interleaved in depth, drawn as one table in a sorted index's order with nothing changed, 0.030–0.036 ms a run, against 0.045–0.049 ms for the same 50K drawn without `order` as two tables, which writes both tables' records each run. New `gpu ordered render one key 50K`, a target system moving one item to another depth and then the ordered draw, 0.268–0.294 ms a run, against 0.042–0.050 ms unordered: listing the 50K entries in order costs 0.167–0.175 ms (3.4 ns an entry) and writing the 200 KB block 0.067–0.070 ms, timed inside the run.
+- **The order block is listed and written only when the index's entries changed or moved rows,** by the index's `changed` tick and a `moved` tick its refresh stamps when it relocates an entry with its key unchanged, or when the matches grew; the tables block only when a column's address changed; the record only when one of its four words did. An unchanged run writes nothing to the queue and allocates nothing: the table, its `bind`, the tables list and the context are reused.
+- **An entry is one u32:** the matching archetype's slot in the high bits, as few as the matches need and at least one, and its row in the rest, so the shader masks by the record's shift.
+
 ## Where the gains came from
 
 Step figures marked (vitest) are from the node vitest lane: read them as relative.