Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/01-packages_bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ body:
- editor
- env
- graphics-2d
- graphics-3d
- input
- music
- network
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/03-feature_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ body:
- editor
- env
- graphics-2d
- graphics-3d
- input
- music
- network
Expand Down
4 changes: 4 additions & 0 deletions .github/issue-labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ modules:graphics-2d:
- "### Which (package|utils|package or utils) is this (bug
report|feature request) for\\?\\n\\ngraphics-2d\\n"

modules:graphics-3d:
- "### Which (package|utils|package or utils) is this (bug
report|feature request) for\\?\\n\\ngraphics-3d\\n"

modules:input:
- "### Which (package|utils|package or utils) is this (bug
report|feature request) for\\?\\n\\ninput\\n"
Expand Down
6 changes: 6 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ modules:graphics-2d:
- modules/graphics-2d/*
- modules/graphics-2d/**/*

modules:graphics-3d:
- changed-files:
- any-glob-to-any-file:
- modules/graphics-3d/*
- modules/graphics-3d/**/*

modules:input:
- changed-files:
- any-glob-to-any-file:
Expand Down
4 changes: 4 additions & 0 deletions .github/labels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@
description: "Related to Graphics2D library"
color: "1c3f78"

- name: "modules:graphics-3d"
description: "Related to Graphics3D library"
color: "1c3f78"

- name: "modules:input"
description: "Related to Input library"
color: "1c3f78"
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/alpha-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ on:
- "@nanoforge-dev/editor-lib"
- "@nanoforge-dev/env"
- "@nanoforge-dev/graphics-2d"
- "@nanoforge-dev/graphics-3d"
- "@nanoforge-dev/input"
- "@nanoforge-dev/music"
- "@nanoforge-dev/network"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pre-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
- name: Create release PR
uses: ./node_modules/@nanoforge-dev/actions/actions/create-packages-release-pr
with:
packages: "@nanoforge-dev/asset,@nanoforge-dev/common,@nanoforge-dev/core,@nanoforge-dev/ecs,@nanoforge-dev/editor-lib,@nanoforge-dev/env,@nanoforge-dev/graphics-2d,@nanoforge-dev/input,@nanoforge-dev/music,@nanoforge-dev/network,@nanoforge-dev/scene,@nanoforge-dev/sound"
packages: "@nanoforge-dev/asset,@nanoforge-dev/common,@nanoforge-dev/core,@nanoforge-dev/ecs,@nanoforge-dev/editor-lib,@nanoforge-dev/env,@nanoforge-dev/graphics-2d,@nanoforge-dev/graphics-3d,@nanoforge-dev/input,@nanoforge-dev/music,@nanoforge-dev/network,@nanoforge-dev/scene,@nanoforge-dev/sound"
version: ${{ inputs.version }}
branch-format: "releases/nanoforge@{version}"
commit-format: "chore: release @nanoforge-dev/engine@{version}"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- name: Release packages
uses: ./node_modules/@nanoforge-dev/actions/actions/release-packages
with:
packages: "@nanoforge-dev/asset,@nanoforge-dev/common,@nanoforge-dev/core,@nanoforge-dev/ecs,@nanoforge-dev/editor-lib,@nanoforge-dev/env,@nanoforge-dev/graphics-2d,@nanoforge-dev/input,@nanoforge-dev/music,@nanoforge-dev/network,@nanoforge-dev/scene,@nanoforge-dev/sound"
packages: "@nanoforge-dev/asset,@nanoforge-dev/common,@nanoforge-dev/core,@nanoforge-dev/ecs,@nanoforge-dev/editor-lib,@nanoforge-dev/env,@nanoforge-dev/graphics-2d,@nanoforge-dev/graphics-3d,@nanoforge-dev/input,@nanoforge-dev/music,@nanoforge-dev/network,@nanoforge-dev/scene,@nanoforge-dev/sound"
tag-format: "{version}"
latest: true
env:
Expand Down
1 change: 1 addition & 0 deletions docs/docs/getting-started/1-index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@ This documentation covers the engine, its packages, and the surrounding usage gu

- [Registry](../registry)
- [Network](../network)
- [Graphics 3D](../graphics-3d)
102 changes: 102 additions & 0 deletions docs/docs/graphics-3d/advanced.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
title: "Graphics 3D: advanced"
icon: "wand-magic-sparkles"
description: "Full Three.js access, cameras, picking and editor events in @nanoforge-dev/graphics-3d"
---

# Graphics 3D: advanced

The library adds no layer between you and Three.js. Everything below is plain Three.js on the objects of `Context.graphics`.

## Importing Three.js

| Import | Content |
| -------------------------------------- | ----------------------------------------------------------------- |
| `@nanoforge-dev/graphics-3d/three` | `three/webgpu`: the whole core, `WebGPURenderer`, node materials. |
| `@nanoforge-dev/graphics-3d/three/tsl` | `three/tsl`: the Three.js Shading Language. |
| `three/addons/...` | Add-ons (`GLTFLoader`, `OrbitControls`…), from `three` directly. |

The subpaths only re-export `three`. `three` is a peer dependency that the app installs, so the library, the game and the add-ons resolve the same copy. Add-ons import `three`, which shares its core classes with `three/webgpu`. For add-on types, add `@types/three` at the same version.

### Loading a model

Load project files through `ctx.assets`. It returns the URL the game serves the file at.

```ts
import { GLTFLoader } from "three/addons/loaders/GLTFLoader.js";

const file = ctx.assets.getAsset("models/robot.glb");
const gltf = await new GLTFLoader().loadAsync(file.path);
ctx.graphics.scene.add(gltf.scene);
```

Do not await asset loads in a library's `__init`: start them and add the objects when they arrive.

### Node materials and TSL

`WebGPURenderer` compiles node materials to WGSL (WebGPU) and GLSL (WebGL2) from one source.

```ts
import { Mesh, MeshStandardNodeMaterial, SphereGeometry } from "@nanoforge-dev/graphics-3d/three";
import { color, sin, time } from "@nanoforge-dev/graphics-3d/three/tsl";

const material = new MeshStandardNodeMaterial();
material.colorNode = color("#ff6040").mul(sin(time).mul(0.5).add(0.5));
ctx.graphics.scene.add(new Mesh(new SphereGeometry(), material));
```

## Cameras

Assign any camera to `graphics.camera`. The viewport is applied to it at once, and again on every change. For a `PerspectiveCamera`, the library sets `aspect`. For an `OrthographicCamera`, you choose `left`/`right`/`top`/`bottom` in design proportions.

```ts
import { OrthographicCamera } from "@nanoforge-dev/graphics-3d/three";

graphics.camera = new OrthographicCamera(-16, 16, 9, -9, 0.1, 100);
```

After changing `fov`, `near` or `far`, call `camera.updateProjectionMatrix()`.

## Picking

`raycast` takes page coordinates, the ones pointer events give:

```ts
canvas.addEventListener("pointerdown", (event) => {
const [hit] = ctx.graphics.raycast(event.clientX, event.clientY);
if (hit) console.log("clicked", hit.object.name, "at", hit.point);
});
```

`worldToScreen` goes the other way, for example to place an HTML label over an object.

## Render loop

Never call `renderer.setAnimationLoop` or render from `requestAnimationFrame`. The engine renders once per tick, and the editor relies on this: pause freezes the image, and step draws exactly one frame. To draw something else than `scene` through `camera`, such as a post-processing pipeline, replace the library's render step. The library still calls it once per tick, and for editor screenshots:

```ts
import { RenderPipeline } from "@nanoforge-dev/graphics-3d/three";
import { pass, saturation } from "@nanoforge-dev/graphics-3d/three/tsl";

const { renderer, scene, camera } = ctx.graphics;
const postProcessing = new RenderPipeline(renderer);
const scenePass = pass(scene, camera);
postProcessing.outputNode = saturation(scenePass.getTextureNode(), 0.2);

ctx.graphics.render = () => postProcessing.render();
```

Assign `undefined` to go back to the default render. The custom render is reset when the game restarts.

## Editor bridge

When the editor starts the game, the library adds to the editor protocol:

| Name | Direction | Payload | Purpose |
| ------------------ | --------------- | ---------------------- | -------------------------------------------------------------------------------------- |
| `graphicsCamera` | feature | `boolean` | Asked for in the editor's `welcome`. |
| `graphics-camera` | engine → editor | `{ view, projection }` | The active camera's matrices (16 numbers, column-major), on start and on every change. |
| `graphics-capture` | editor → engine | none | Renders a frame now, even while paused. |
| `graphics-capture` | engine → editor | PNG data URL | That frame. WebGL and WebGPU canvases are only readable right after a render. |

The projection includes the viewport crop, so its normalized device coordinates span the canvas. The canvas is drawn `visibleWidth` × `visibleHeight` CSS px at (`contentLeft`, `contentTop`) of the container (see the `viewport` event).
97 changes: 97 additions & 0 deletions docs/docs/graphics-3d/editor-proposal.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
---
title: "Graphics 3D: editor proposal"
icon: "pen-ruler"
description: "Proposal for the editor's 3D Scene screen and plugin needs, for the editor's requirements session"
---

# Graphics 3D: editor proposal

Input for the editor's requirements session on 3D (see `editor/CLAUDE.md`: a plugin starts with a requirements session and a spec in `docs/specs/<plugin>.md`). Nothing here is built on the editor side yet.

**Goal:** a beginner edits a 3D scene without code, in the same way the 2D Scene screen works today.

## What the engine provides now

`@nanoforge-dev/graphics-3d` ([docs](./index)) adds to the editor bridge:

| Name | Kind | Payload | Use in the editor |
| ------------------ | ------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `graphicsCamera` | feature | `boolean` | Ask for it in `welcome` while a 3D tool over the running game is active, like `viewport` in `GameMoveOverlay`. |
| `graphics-camera` | event | `{ view, projection }` | Active camera matrices (column-major), sent once on start and then on every change. Together with `viewport` they map pointer ↔ world. |
| `graphics-capture` | command | none | Ask for a screenshot: the engine renders a frame, even while paused. |
| `graphics-capture` | event | PNG data URL | The frame. |

The library also guarantees the following:

- **Rendering:** it renders only from the engine tick, so pause freezes the image and step draws one frame.
- **Pop-out:** it creates its canvas in `container.ownerDocument`, so pop-out keeps working. A same-document iframe move was tested in the POC; a real `window.open` window was not.
- **Restarts:** `__init`/`__clear` survive repeated restarts, as tested with 10 cycles. Each clear disposes GPU objects, removes the canvas and deletes `window.__THREE__`.

## Findings that change the brief

- **Screenshot:** `plugins/viewport/src/game/screenshot.ts` draws every canvas with `drawImage` at an arbitrary time. A WebGL or WebGPU canvas may be blank then, which never happened with Konva. When the app has graphics-3d, use `graphics-capture` instead.
- **New feature flag:** `graphicsCamera` must be added to the editor's `mergeFeatures`, or it is silently dropped.
- **Telling 2D from 3D:** both libraries use the key `graphics`, so `hello.libraries` cannot tell them apart. The editor must look at the app's dependencies (`@nanoforge-dev/graphics-3d` vs `graphics-2d`). It could also look at the `graphics-camera` answer.
- **The viewport type is 2D:** `EngineViewport` (`packages/protocol`) maps game units to px in 2D. For 3D, it still gives the canvas rectangle, and `graphics-camera` gives the rest.
- **No hot reload today:** a source change while watching rebuilds the app, but nothing restarts the running game. Restart is the manual `runtime.restart`. Asset changes also trigger a rebuild, with no filter. "Hot reload" for 3D therefore means a rebuild plus a restart (manual, or automatic if the editor decides so). The engine side is ready for that.
- **Class names:** `build-scene.ts` matches shapes by class name only (`Rect`, `Circle`), and the bundler renames a component class whose name clashes with another bundled class (`Sphere` → `Sphere2`), which the ECS then fails to match. Three.js ships `Sphere`, `Plane`, `Box3`… That is why the 3D components are named with a `3D` suffix.
- **Vector preset:** `@preset vector` already supports `x`/`y`/`z` (slots `x, y, z, w`, minimum 2). A 3D position needs no new preset.

## Components (not in the engine library)

ECS components and systems for 3D are built in [components-systems](https://github.com/NanoForge-dev/components-systems), not in the library:

- transform: `Position3D`, `Rotation3D` (degrees), `Scale3D`;
- shapes: `Box3D`, `Sphere3D`, `Plane3D`;
- `Model3D` (`@asset model .glb .gltf`), `Camera3D`, `Light3D`;
- the escape hatch `Object3DComponent`;
- a sync system;
- a pure function that maps component params to Three.js objects.

The editor's Scene screen should import that same pure function, so the game and the Scene screen never drift.

## Proposed editor work

### 1. 3D Scene screen (`SCENE_EDITORS`)

- **Building the scene:** build it from `main.ts` literals without running the game, as `build-scene.ts` does in 2D. Find the position by the `Position3D` component (numeric `x`/`y`/`z`), and turn component params into Three.js objects with the shared mapping function.
- **Rendering:** render with its own `WebGPURenderer`, its own scene and an editor `OrbitControls` camera that is not the game camera.
- **Picking and gizmos:** pick with `Raycaster`; move, rotate and scale with `TransformControls`.
- **Writing a gizmo edit, while stopped:** write `Position3D` / `Rotation3D` / `Scale3D` args to code with `ecs.apply({ kind: "setArgs", … })`, undoable, as `SceneView` does for `Position`.
- **Writing a gizmo edit, while playing:** apply it live with `live.setField` (`ecs-set-component`), then _Apply to code_.

### 2. Move tool over the running game (`VIEWPORT_OVERLAYS`)

While the tool is active, ask for `{ viewport: true, graphicsCamera: true }`. Then:

1. Pointer → NDC on the canvas rectangle (`contentLeft`, `contentTop`, `visibleWidth`, `visibleHeight`).
2. Unproject with `inverse(projection)`, then `inverse(view)`, which gives a world ray.
3. Pick against the entities' bounds, or drag on a plane (the ground, or a plane facing the camera through the object).
4. Write with `live.setField`.

### 3. Drag and drop

- **Model file:** dropping a `.glb`/`.gltf` from the file manager onto the 3D Scene screen spawns an entity with `Position3D` at the drop point (a ray on the `y = 0` plane) and `Model3D("<asset path>")`.
- **Component:** dropping a component from the catalog onto an object adds it to that entity.

### 4. Inspector contributions

| Extension point | Proposal |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------ |
| `ecs.paramPresets` | `euler`: slots `x, y, z`, all numbers in degrees, shown with a wrap at ±180 and a dial. Used by `Rotation3D`. |
| `ecs.paramPresets` | `vector` with a uniform-scale lock for `Scale3D` (like `size`'s ratio lock). |
| `ecs.fieldEditors` | `ref` editors for `@nanoforge-dev/graphics-3d/three#Color` and `#Vector3`, for user components that take Three.js objects as params. |
| `color` preset | Already fits the `color` string params of the shape components. |

### 5. Toolbar

- **Screenshot:** use `graphics-capture` for 3D apps.
- **Pop-out:** already works.
- **Zoom / pixel-perfect:** these are 2D-only. Hide them for 3D, or map zoom to camera zoom.

## Questions for the session

1. Should a source change restart the game automatically while playing, or stay manual?
2. Should the 3D Scene screen live in the ECS plugin (next to the 2D one) or in its own plugin?
3. Picking in the running game: are bounds-based picks on the editor side enough, or should we add a `graphics-pick` command answered by the engine (exact meshes)?
4. Should the editor camera be saved per project, and should there be a "view from game camera" toggle?
Loading
Loading