Skip to content

fix(mediaplayer): (Fixes broken video!) present D3D12 frames via a typeless shared texture#907

Merged
dooly123 merged 1 commit into
BasisVR:developerfrom
towneh:fix/mediaplayer-d3d12-srgb
Jun 28, 2026
Merged

fix(mediaplayer): (Fixes broken video!) present D3D12 frames via a typeless shared texture#907
dooly123 merged 1 commit into
BasisVR:developerfrom
towneh:fix/mediaplayer-d3d12-srgb

Conversation

@towneh

@towneh towneh commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

On D3D12, video playback through the media player showed a black screen with a repeating D3D12 error:

Creating a default shader resource view with non compatible format (dxgi-fmt=91 for a resource that uses dxgi-fmt=87)

followed by Attempting to bind a NULL or empty texture every frame.

Root cause: the D3D12 present path handed Unity the decoded BGRA frame as a typed B8G8R8A8_UNORM resource. Unity wraps it with CreateExternalTexture and, in a linear colour space, builds a B8G8R8A8_UNORM_SRGB shader resource view — which D3D12 refuses to create over a typed UNORM resource. The view fails and the texture binds as NULL. D3D11 tolerated this cast, so it only surfaced after the move to D3D12. The D3D11 path already sidesteps it by copying each frame into a separate TYPELESS texture; the D3D12 path didn't.

Fix: mirror the D3D11 approach on D3D12 — allocate a TYPELESS shared texture on the decode device, copy the due ring slot into it each present, and hand that to Unity, which can then cast whichever SRV its colour space needs. This also fixes a second issue where the D3D12 path pinned a single ring slot for the lifetime of the stream instead of presenting the newest due frame.

Because the decode-device copy has no implicit sync with Unity's D3D12 device (which samples the surface directly), the render thread polls an event query until the copy retires, releases the surface's keyed mutex only then, and publishes the frame only once Unity holds the external texture — so Unity never samples a half-written or not-yet-available copy. A device-loss or multi-second GPU stall is surfaced as an engine error rather than exposing an in-flight surface. The shared-texture size is cached only after the full setup succeeds, so a transient allocation/share failure retries rather than dead-ending.

D3D11 is unaffected — all new work is gated on the D3D12 graphics API.

Verified: in-editor on Windows/D3D12, playing an HTTPS MPEG-TS stream — correct video, orientation, colour, and audio, with no per-frame stutter from the completion wait.

Required checks

All boxes below must be ticked before this PR can merge. If a check is genuinely N/A, tick it anyway and explain under Notes.

  • Tested — I built and ran this locally. The change works in the editor and (where relevant) in a built player.
  • Transform access is combined and limited — In hot paths, transform reads/writes go through TransformAccessArray or are otherwise batched. I have not added per-frame transform.position / transform.rotation / transform.localPosition calls inside loops. Whenever I need both position and rotation, I use the combined APIs — SetPositionAndRotation / SetLocalPositionAndRotation for writes, GetPositionAndRotation / GetLocalPositionAndRotation for reads — instead of two separate property accesses; the combined call does one local-to-world matrix traversal instead of two.
  • Addressables used for asset/memory loading — Any new asset loads go through Addressables. No new Resources.Load, no direct asset references that pull large content into memory on scene load.
  • No new GetComponent / AddComponent where avoidable — Where unavoidable, the result is cached on a field, and any GetComponent<T> is replaced with TryGetComponent<T>(out var x) — bare GetComponent will be denied. TryGetComponent is the modern API (Unity 2019.2+) and skips the Editor-only GC allocation GetComponent causes when a component is missing: Unity wraps the null return in a managed "fake null" object so its overloaded == operator can still detect destroyed C++ objects, and constructing that wrapper allocates; TryGetComponent returns a bool plus out parameter and never builds the wrapper. None of these calls run inside Update, LateUpdate, FixedUpdate, jobs, or other per-frame code paths.
  • Per-frame work is scheduled through BasisEventDriver — Any new per-frame work hooks into BasisEventDriver rather than adding standalone Update / LateUpdate / FixedUpdate callbacks on a MonoBehaviour.
  • Anything added to BasisEventDriver is bulletproof, or guarded by try/catchBasisEventDriver runs the single per-frame tick that drives the whole framework (network apply, local player sim, blendshapes, JigglePhysics, nameplates, and more) as one sequential chain. An unhandled exception anywhere in that chain aborts the rest of the tick, so every step after the throwing one is silently skipped for that frame. New work added to the driver must either be guaranteed not to throw, or be wrapped in a try/catch that contains the failure and surfaces it through BasisDebug — logged once / rate-limited, never every frame (see the existing HVRBasisBuiltInAddresses.Simulate() guard for the pattern). Expect this to be scrutinized closely in review.
  • Considered jobification — I asked whether this work can be moved to a Unity Job (Burst-compiled where possible). If it can, it is. If it cannot, the reason is in Notes.
  • No needless { get; set; } properties or access lockdowns — Public fields are fine; Basis is meant to be read and modified freely, so don't wall things off private/internal without a real reason. Don't wrap a field in { get; set; } when the accessors do nothing — property accessors have a real performance cost vs direct field access, and the lead maintainer prefers plain fields (or a method / setter-only property when only the setter needs logic) over a noop-getter pair. For .Instance singletons, callers reassigning Type.Instance is allowed; if that would break your code, log a warning or throw — don't block the assignment. Locking down access is not your call.
  • Camera access goes through BasisLocalCameraDriver — Code that needs the local camera (transform, projection, rig data, etc.) pulls it from BasisLocalCameraDriver rather than looking one up itself. Don't roll a separate camera discovery path.
  • Logging uses BasisDebug — All new logging calls go through BasisDebug.Log / BasisDebug.LogWarning / BasisDebug.LogError (with an appropriate LogTag) instead of UnityEngine.Debug.Log / Debug.LogWarning / Debug.LogError. BasisDebug routes through Basis's tagged, color-coded logger and respects the project-wide LoggingDisabled toggle so logging can be killed at runtime; bare Debug.Log calls bypass that and will be denied.
  • No scene-wide discovery for dependencies — New code is architected so it does not need FindObjectOfType / FindObjectsOfType / GameObject.Find / FindGameObjectsWithTag to locate what it depends on. References are wired in — registered through an existing manager/driver, injected at init, or passed in by the caller — rather than discovered by scanning the scene at runtime. If a scene scan is genuinely unavoidable, justify it under Notes.
  • No allocations in hot paths — Per-frame code (Update / LateUpdate / FixedUpdate, simulation loops, jobs, anything called once per frame or more) does not allocate. No new on reference types, no LINQ, no string concatenation/interpolation, no boxing, no foreach over interface-typed collections. Allocate once at init and reuse the buffer.
  • No debugging in hot paths — No log calls of any kind on per-frame paths, including BasisDebug. Hot-path logging floods the console and incurs cost on every frame regardless of whether the message is filtered out downstream. If a hot-path log is needed while iterating, gate it behind #if UNITY_EDITOR and remove (or leave gated) before merge.
  • Hot-path collection access is optimized — Cache .Count (lists) / .Length (arrays) into a local int before the loop instead of re-reading the property each iteration. Prefer T[] (with a separate length int when the array is over-sized) over List<T> where the data is hot — Unity's mono BCL doesn't expose CollectionsMarshal.AsSpan(List<T>), so a list can't be fed into Span<T> / unsafe paths cleanly. Where the perf justifies it, drop into Span<T> / ref locals / Unsafe.As / unsafe pointer code to skip bounds checks and copies, and call out the invariants you're relying on under Notes so reviewers can sanity-check them.

Testing details

Tick the platforms you actually tested on. Leave the rest unticked — these are informational and do not block merge.

  • Windows
  • Linux
  • Android
  • iOS
  • macOS

Input / control mode coverage:

  • Tested in VR (note headset under Notes)
  • Tested in desktop / non-VR mode
  • Tested with phone controls (mobile touch input)
  • N/A — change does not touch player/XR/input code

Where applicable, confirm these flows still work after your changes:

  • Hot-switching (desktop ↔ VR mode swap at runtime)
  • Avatar swapping
  • Server swapping (joining / leaving / changing servers)
  • N/A — change does not touch any of the above

Notes

This is a native plugin change only — basis_media_native (C++/Direct3D), Native~/windows/basis_win_decode.cpp. There is no C# gameplay code in this PR, so the C#/Unity-specific required checks (transform access, Addressables, GetComponent, BasisEventDriver, jobification, properties, camera driver, BasisDebug, scene discovery, hot-path allocations/collections) are N/A and ticked on that basis. Native logging continues to use the engine's existing basis_engine_set_error path.

Verified in-editor on Windows with the D3D12 backend, playing an HTTPS MPEG-TS stream (correct video, orientation, colour, audio; no stutter from the per-present completion wait). Not separately tested in a VR headset or across a runtime desktop↔VR hot-switch — but the D3D11 path is byte-for-byte unchanged (all new work is gated on the D3D12 graphics API), and the D3D12 path rebuilds its shared textures on any graphics-device or video-size change.

The per-present completion wait polls an ID3D11Query (event query) until the decode-device copy retires — sub-millisecond in practice, since the render thread is blocked inside the plugin event so the copy can't be lapped. The surface is single-buffered and Unity samples it directly, so the wait is what guarantees a complete frame; the keyed mutex is released only on confirmed completion. A device-loss or multi-second stall is treated as a wedged GPU and surfaced as an engine error rather than exposing an in-flight write.

@towneh towneh force-pushed the fix/mediaplayer-d3d12-srgb branch 4 times, most recently from 6a6509c to 8753a66 Compare June 27, 2026 17:26
On D3D12 the present path opened the decoded BGRA ring buffer (a typed
B8G8R8A8_UNORM resource) and handed it straight to Unity. Unity wraps it
with an sRGB shader resource view (B8G8R8A8_UNORM_SRGB) for a linear
colour space, which D3D12 refuses to create over a typed UNORM resource:

  Creating a default shader resource view with non compatible format
  (dxgi-fmt=91 for a resource that uses dxgi-fmt=87)

The view then fails and the texture binds as NULL every frame. The D3D11
path already avoids this by copying the due frame into a separate TYPELESS
texture; the D3D12 path did not.

Mirror that for D3D12: allocate a TYPELESS shared texture on the decode
device and copy the due ring slot into it each present, then hand that to
Unity, which can cast the UNORM or sRGB SRV its colour space needs. This
also corrects the D3D12 path pinning a single ring slot for the lifetime
of the stream instead of presenting the newest due frame.

The decode-device copy has no implicit handoff sync with Unity's D3D12
device, so the render thread polls an event query until the copy retires,
releases the surface's keyed mutex only then, and publishes only once Unity
holds the external texture, so Unity never samples a half-written or absent
frame. The shared-texture size is cached only after the full output path
succeeds, so a transient create/share failure retries instead of leaving no
usable output.

D3D11 behaviour is unchanged; all new work is gated on the D3D12 graphics
API.
@towneh towneh force-pushed the fix/mediaplayer-d3d12-srgb branch from 8753a66 to 6c8a92d Compare June 27, 2026 17:38
@towneh towneh requested a review from dooly123 June 27, 2026 17:40
@towneh towneh added the bug Something isn't working label Jun 27, 2026
@towneh towneh changed the title fix(mediaplayer): present D3D12 frames via a typeless shared texture fix(mediaplayer): (Fixes broken video!) present D3D12 frames via a typeless shared texture Jun 27, 2026
@dooly123 dooly123 merged commit 56ef776 into BasisVR:developer Jun 28, 2026
14 checks passed
@towneh towneh deleted the fix/mediaplayer-d3d12-srgb branch June 28, 2026 06:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants