[EtienneLescot]

Refs #1340

Problem. On Ubuntu 24.04, libva 2.20 makes the encoder ladder skip VA-API, so it tries h264_vulkan. When the GPU advertises Vulkan video encode but has no H.264 encode queue (AMD Carrizo in @PyVoyager's report), ffmpeg 8.1 logs "Encoding of h264 is not supported by this device" and segfaults while freeing the half-built encoder. The helper dies before reaching libopenh264.

Fix. After creating the Vulkan device, ask Vulkan for a queue family with VK_QUEUE_VIDEO_ENCODE_BIT_KHR and H.264 encode. That is the same lookup ffmpeg makes. If there is none, the rung is skipped with a readable reason and avcodec_open2 is never called. No Vulkan headers needed: only the first four pointer fields of AVVulkanDeviceContext are read.

Measured on a Vivobook (Ubuntu 24.04, libva 2.20, Radeon 610M, Mesa 25.2.8), ladder_encodes_a_playable_mp4:

• RADV, which has the queue: Vulkan still selected, 85 fps. Output MP4 is byte-identical before and after (sha256 5d89de10…).
• llvmpipe, no encode queue: vulkan unavailable: the Vulkan device has no H.264 encode queue, then software selected.
• cargo test --release in pipewire-capture: 104 passed.

Not reproduced: the Carrizo segfault itself, for lack of that GPU.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Hardware capture setup now checks that the selected Vulkan device supports H.264 video encoding. Devices without the required capability are rejected with an error instead of proceeding with an incompatible configuration.

[coderabbitai]

No actionable comments were generated in the recent review.

Walkthrough

The encoder now checks Vulkan queue-family capabilities for video encoding and H.264. After creating a Vulkan hardware device, attach_hardware returns an error if no queue family supports both.

Changes: Vulkan H.264 Queue Validation

Layer / File(s)	Summary
Query Vulkan H.264 queue support electron/native/pipewire-capture/src/encoder.rs	Adds vulkan_can_encode_h264, which checks queue-family video capabilities and returns false if the capability query is unavailable.
Gate Vulkan hardware attachment electron/native/pipewire-capture/src/encoder.rs	After creating the Vulkan device, attach_hardware returns an error if no queue supports both video encoding and H.264.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Possibly related issues

• getopenscreen/openscreen#1340 — Reports a Linux recording crash while bundled FFmpeg opens a video codec. This change avoids continuing with Vulkan when no H.264 encode queue is available, though the issue’s exact cause is unconfirmed.

Possibly related PRs

• getopenscreen/openscreen#1476 — Changes the VA-API branch of attach_hardware; this PR changes its Vulkan branch.

Merge Risk: Minimal · up to 8b2ef

The change adds an early capability check so unsupported Vulkan devices can fall back to software encoding. No concrete merge-blocking risk is identified.

Pre-merge checks: 5

Check name	Status	Explanation
Title check		The title names the fix and its scope: the h264_vulkan rung is skipped when the Vulkan device has no H.264 encode queue. The raw summary says attach_hardware now rejects such a device after a queue-fa…
Description check		The description follows the template's main sections: Summary, Related issue, Type of change, Release impact, Desktop impact, Screenshots, and Testing. It gives a clear problem statement, the fix, and…
Docstring Coverage		Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Title check

Explanation

The title names the fix and its scope: the h264_vulkan rung is skipped when the Vulkan device has no H.264 encode queue. The raw summary says attach_hardware now rejects such a device after a queue-family probe, which matches the title. The title uses the conventional fix(linux) prefix and is 70 characters, within a concise length.

Full details: Description check

Explanation

The description follows the template's main sections: Summary, Related issue, Type of change, Release impact, Desktop impact, Screenshots, and Testing. It gives a clear problem statement, the fix, and measured test results. It uses "Refs #1340" instead of the template's "Fixes #" line, which fits a partial fix. Several checkboxes are unmarked: Type of change, Release impact, and Desktop impact (Linux is implied by the title and body but not checked). The Screenshots section is omitted, which is acceptable since the change is not visual. The description is mostly complete, so it passes.

Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 2 functions across 1 files.

Finishing Touches

Generate docstrings

• Commit to this branch
• Create a new PR

Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

Clippy (1.99.1)

Clippy execution failed

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

Recent review info

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 44bdd85f-48a5-446e-8303-3f0b13c6b61e

Commits

Reviewing files that changed from the base of the PR and between 8f00d51 and 8b2efb0.

Files selected for processing: 1

• electron/native/pipewire-capture/src/encoder.rs

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.