Support VDDK imports into RBD storage#13294
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## 4.22 #13294 +/- ##
==========================================
Coverage 17.67% 17.68%
- Complexity 15792 15827 +35
==========================================
Files 5922 5923 +1
Lines 533165 533595 +430
Branches 65208 65259 +51
==========================================
+ Hits 94242 94361 +119
- Misses 428276 428583 +307
- Partials 10647 10651 +4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@blueorangutan package kvm |
|
@andrijapanicsb a [SL] Jenkins job has been kicked to build packages. It will be bundled with kvm SystemVM template(s). I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18103 |
|
@blueorangutan help |
|
@andrijapanicsb [SL] I understand these words: "help", "hello", "thanks", "package", "test" Blessed contributors for kicking Trillian test jobs: ['rohityadavcloud', 'shwstppr', 'Damans227', 'vishesh92', 'Pearl1594', 'harikrishna-patnala', 'nvazquez', 'DaanHoogland', 'weizhouapache', 'borisstoyanov', 'vladimirpetrov', 'kiranchavala', 'andrijapanicsb', 'NuxRo', 'rajujith', 'sureshanaparti', 'abh1sar', 'sudo87', 'RosiKyu'] |
|
@blueorangutan test ol9 kvm-ol9 keepEnv |
|
@andrijapanicsb a [SL] Trillian-Jenkins test job (ol9 mgmt + kvm-ol9) has been kicked to run smoke tests |
|
[SF] Trillian test result (tid-16237)
|
There was a problem hiding this comment.
Pull request overview
This pull request extends the existing VMware→KVM import (virt-v2v / VDDK) pipeline to support Ceph RBD as a destination, adding a destination-aware writer model that preserves current QCOW2 behavior while enabling both direct-to-RBD and staged-to-RBD imports. It also adds KVM host capability detection to ensure direct RBD mode is only used when the conversion host supports the necessary tooling (qemu-img RBD + virt-v2v in-place).
Changes:
- Add direct VDDK→RBD import path (qemu-img convert into raw RBD + in-place virt-v2v finalization) and staged QCOW2→RBD copy support.
- Propagate destination pool types and VMware identity/disk metadata (VMware moRef + source disk paths) through the management server → agent command flow.
- Add host capability detection + expanded unit tests for direct/staged RBD scenarios.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| vmware-base/src/main/java/com/cloud/hypervisor/vmware/util/VmwareHelper.java | Populate VMware moRef into the unmanaged VM transfer object for more reliable VDDK disk selection. |
| server/src/test/java/org/apache/cloudstack/vm/UnmanagedVMsManagerImplTest.java | Add tests covering staged VDDK validation bypass and direct-RBD host selection / pool validation. |
| server/src/main/java/org/apache/cloudstack/vm/UnmanagedVMsManagerImpl.java | Add RBD-aware validation, host selection, and propagation of pool types + VMware VDDK source disk metadata. |
| plugins/hypervisors/kvm/src/test/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtImportConvertedInstanceCommandWrapperTest.java | Add tests for RBD destination handling and RBD disk metadata extraction. |
| plugins/hypervisors/kvm/src/test/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtConvertInstanceCommandWrapperTest.java | Add test ensuring direct RBD VDDK conversion fails cleanly when host lacks support. |
| plugins/hypervisors/kvm/src/test/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtCheckConvertInstanceCommandWrapperTest.java | Add tests for direct RBD VDDK support checks. |
| plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtReadyCommandWrapper.java | Report additional host capabilities (virt-v2v in-place, qemu-img rbd, etc.) in host details. |
| plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtImportConvertedInstanceCommandWrapper.java | Handle forced RBD conversion shortcut and pass pool types to select correct storage backend for copies. |
| plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtConvertInstanceCommandWrapper.java | Implement direct VDDK→RBD conversion (nbdkit-vddk + qemu-img convert + virt-v2v in-place finalization). |
| plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtCheckConvertInstanceCommandWrapper.java | Enforce direct RBD VDDK capability checks when requested. |
| plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java | Add detection helpers for virt-v2v in-place support and qemu-img RBD support. |
| PendingReleaseNotes | Document the new RBD import capability. |
| engine/orchestration/src/main/java/com/cloud/agent/manager/AgentManagerImpl.java | Persist newly reported host capabilities into host details on reconnect. |
| core/src/main/java/com/cloud/agent/api/ImportConvertedInstanceCommand.java | Extend command to carry destination pool types (in addition to UUIDs). |
| core/src/main/java/com/cloud/agent/api/ConvertInstanceCommand.java | Add VMware VDDK source disk metadata list for direct-to-RBD conversion. |
| core/src/main/java/com/cloud/agent/api/CheckConvertInstanceCommand.java | Add flag to explicitly request direct RBD capability validation. |
| api/src/main/java/org/apache/cloudstack/vm/UnmanagedInstanceTO.java | Add vmwareMoref field to carry VMware managed object reference through the flow. |
| api/src/main/java/org/apache/cloudstack/api/command/admin/vm/ImportVmCmd.java | Clarify API parameter behavior/requirements for direct RBD VDDK import mode. |
| api/src/main/java/com/cloud/host/Host.java | Add new host detail keys for RBD + in-place conversion capabilities. |
| api/src/main/java/com/cloud/agent/api/to/VmwareVddkSourceDiskTO.java | New TO carrying VMware source disk ID/path/capacity/position. |
| api/src/main/java/com/cloud/agent/api/to/RemoteInstanceTO.java | Add vmwareMoref field so agents can target the correct VM via VDDK. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| } | ||
|
|
||
| private void probeRbdQemuAccess(KVMStoragePool pool, String temporaryConvertUuid) { | ||
| String probeName = temporaryConvertUuid + "-probe-" + UUID.randomUUID(); |
| Storage to grow to enormous sizes as Linux Distributions keep growing in | ||
| size while a stripped down Linux should fit on a 2.88MB floppy. | ||
|
|
||
| 4.23.0.0: |
| Storage.StoragePoolType poolType = getDestinationStoragePoolType(destinationStoragePoolTypes, i); | ||
| KVMStoragePool destinationPool = storagePoolMgr.getStoragePool(poolType, poolPath); | ||
| if (destinationPool == null) { | ||
| String err = String.format("Could not find a storage pool by URI: %s", poolPath); |
Companion main/4.23 PR: #13270
This PR is intended to keep the 4.22 and main branches aligned for the VDDK-to-RBD import feature.
Description
This PR extends the existing VMware-to-KVM VDDK import flow with Ceph RBD support, while preserving the current VDDK/QCOW2 behaviour.
The existing VDDK/QCOW2 writer remains supported in both existing modes:
forceconverttopool=true, CloudStack continues to letvirt-v2vwrite QCOW2 output directly into the selected supported primary storage pool using-o local -os <pool-path> -of qcow2.forceconverttopool=false, CloudStack continues to use the staged flow:virt-v2vwrites finalized QCOW2 disks to temporary conversion storage, and the import step then moves/copies those disks to the final destination pool.This change adds a destination-aware writer model for VDDK imports. The current file/QCOW2 writer path remains, and a new Ceph RBD/raw writer path is added. This gives the VDDK import flow a cleaner foundation for adding more destination-specific writers later.
For Ceph RBD, two modes are supported:
Direct RBD mode
When
usevddk=true,forceconverttopool=true, andconvertinstancepoolidpoints to an RBD primary storage pool, CloudStack can write the VMware disk data directly into raw RBD images.This path does not use the normal
virt-v2v -o local -of qcow2writer, because RBD is not a local filesystem target. Instead, the conversion host:nbdkitqemu-img convertvirt-v2v-in-place, orvirt-v2v --in-placewhen supportedThe in-place finalization happens only on the destination RBD images. The VMware source disks are not modified.
Staged RBD mode
When
usevddk=trueandforceconverttopool=false, RBD can still be used as the final destination through the existing staged model.In this mode:
virt-v2vcreates finalized QCOW2 disks on temporary conversion storageThis mode does not require in-place
virt-v2v, because regularvirt-v2vhas already finalized the guest while creating the temporary QCOW2 disks.Direct RBD mode requires newer conversion-host tooling. The practical recommended baseline is EL9-family KVM hosts, such as Oracle Linux 9, Rocky Linux 9, AlmaLinux 9, or RHEL 9, and Ubuntu 24.04-style hosts where
virt-v2v-in-placeorvirt-v2v --in-placeis available. EL8 and Ubuntu 22.04-style hosts should use staged RBD import unless compatible in-place tooling has been explicitly installed and detected by CloudStack.The implementation adds KVM host capability detection for:
virt-v2vsupportqemu-imgRBD supportDirect RBD mode is rejected early when the selected conversion host does not support the required in-place finalization path. Staged RBD remains available in that case.
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
N/A
How Has This Been Tested?
Focused compile and unit test coverage was run for the changed API/core/server/KVM paths, including:
The tests cover direct RBD host capability checks, rejection when direct RBD support is missing, staged RBD destination handling, RBD destination pool type propagation, RBD disk metadata, and server-side validation for RBD storage pools.
How did you try to break this feature and the system with this change?
Negative tests were added for hosts without direct VDDK/RBD support, and staged RBD import is validated separately so that lack of in-place
virt-v2vblocks only direct RBD mode, not RBD import as a whole. Existing VDDK/QCOW2 direct and staged flows are kept on the existing writer path unless the selected forced conversion pool is RBD.