Extend Libvirt provider docs, including SAP HANA SKU

source/guides/coriolis-getting-started.md

@@ -106,7 +106,7 @@ For further information, please check **[Coriolis Licensing](coriolis-license.md
 <li class="platform"><img src="../_static/images/virt-icon1.png" alt="Red Hat OpenShift Virtualization"><span>Red Hat OpenShift Virtualization</span></li>
 <li class="platform"><img src="../_static/images/redhat.svg" alt="Red Hat Virtualization"><span>Red Hat Virtualization (legacy RHV)</span></li>
 <li class="platform"><img class="logo-light" src="../_static/images/stackit-32.svg" alt="StackIT"><img class="logo-dark" src="../_static/images/stackit-32-white.svg" alt=""><span>StackIT</span></li>
-<li class="platform"><img src="../_static/images/suse.svg" alt="SUSE Virtualization"><span>SUSE Virtualization</span></li>
+<li class="platform"><img src="../_static/images/18700703.png" alt="SUSE Virtualization"><span>SUSE Virtualization</span></li>
 <li class="platform"><img src="../_static/images/suse.svg" alt="SUSE Linux (KVM)"><span>SUSE Linux (KVM)</span></li>
 <li class="platform"><img src="../_static/images/vmware.svg" alt="VMware vSphere"><span>VMware vSphere</span></li>
 <li class="platform"><img class="logo-light" src="../_static/images/vhi-128.svg" alt="Virtuozzo Hybrid Infrastructure"><img class="logo-dark" src="../_static/images/vhi-128-dark.svg" alt=""><span>Virtuozzo Hybrid Infrastructure (VHI)</span></li>
@@ -188,7 +188,7 @@ For more information regarding each supported platform, please check the corresp
 <li class="platform"><img src="../_static/images/nutanix.svg" alt=""><a href="../platforms/nutanix-as-a-source-cloud.html" title="Nutanix AHV Coriolis Plugin">Nutanix AHV</a></li>
 <li class="platform"><img class="logo-light" src="../_static/images/stackit-32.svg" alt=""><img class="logo-dark" src="../_static/images/stackit-32-white.svg" alt=""><a href="../plugins/stackit-coriolis-plugin.html" title="StackIT Coriolis Plugin">StackIT</a></li>
 <li class="platform"><img src="../_static/images/18700703.png" alt=""><a href="../plugins/kubevirt-harvester-coriolis-plugin.html" title="SUSE Virtualization">SUSE Virtualization</a></li>
-<li class="platform"><img src="../_static/images/18700703.png" alt=""><a href="../platforms/suse-linux-kvm-target-platform.html" title="SUSE Linux (KVM)">SUSE Linux (KVM)</a></li>
+<li class="platform"><img src="../_static/images/suse.svg" alt=""><a href="../platforms/suse-linux-kvm-target-platform.html" title="SUSE Linux (KVM)">SUSE Linux (KVM)</a></li>
 </ul>
 </div>
 <div class="destination-platform">

source/platforms/index.md

@@ -19,6 +19,7 @@ proxmox-as-a-destination-cloud
 microcloud-lxd-as-a-destination-cloud
 kubevirt-harvester-as-a-destination-cloud
 suse-linux-kvm-target-platform
+suse-linux-kvm-sap-hana-target-platform
 stackit-as-a-source-cloud
 stackit-as-a-destination-cloud
 cloudstack-as-a-destination-cloud

source/platforms/suse-linux-kvm-sap-hana-target-platform.md

@@ -0,0 +1,190 @@
+# SUSE Linux (KVM) as a destination cloud (SAP HANA SKU)
+
+SAP HANA workloads require a separate licensing model and endpoint, which
+provide SAP HANA-specific configuration options needed to meet performance,
+support, and compliance requirements.
+
+See the [**basic SKU documentation**](./suse-linux-kvm-target-platform.md)
+before getting started.
+
+```{note}
+The SAP HANA SKU may also be used for other instances that require advanced
+features.
+```
+
+## Configuration options
+
+The SAP HANA endpoint uses the same `[libvirt_migration_provider]` section
+as the [**basic SKU**](./suse-linux-kvm-target-platform.md#configuration-options).
+The options below are the advanced settings from the SAP provider: huge pages,
+NUMA pinning, guest clock and timers, vhostmd, raw disk passthrough, and
+disk-controller passthrough.
+
+`hugepage_size`, `numa_node_count`, `allocate_entire_numa_nodes`, and
+`enable_vhostmd` can also be set per transfer in the target environment.
+When a transfer does not set them, they fall back to the values in this section.
+
+```ini
+[libvirt_migration_provider]
+
+# Configure replica instances to use huge pages of the given size (KB).
+# Note that the huge pages must be preallocated. The possible size depends
+# on the host CPU architecture, 2MB and 1GB being the most common values on
+# x86-64. If unset, the VMs will not use huge pages. The VM memory capacity
+# must be a multiple of the hugepage size.
+hugepage_size = 0
+
+# The amount of NUMA nodes to use for the VM. If set to 0, no explicit NUMA
+# topology will be defined. If set to a positive value, the VM CPUs and
+# memory will be spread across the given number of host NUMA nodes, pinning
+# VM CPUs to unused host CPUs.
+numa_node_count = 0
+
+# Allocate the specified amount of host NUMA nodes entirely to the VM,
+# regardless of the number of source VM vCPUs.
+allocate_entire_numa_nodes = false
+
+# Value for the Libvirt <cpu check="..."> attribute. Set to 'none' to
+# disable CPU compatibility checks, or 'partial' / 'full' as required by
+# your environment.
+cpu_check = none
+
+# A list of host CPUs that will not be used by pinned VM vCPUs.
+#
+# Contains comma separated ranges of CPUs in Libvirt format.
+#
+# Example: 1-4,^3,5
+#
+# In this example, CPUs 1,2,4 and 5 will be reserved.
+# reserved_cpus =
+
+# Number of memory pages to reserved per host NUMA node.
+#
+# Each entry will contain comma separated key:value pairs describing the
+# NUMA node ID, the page size in KB and the number of reserved pages.
+#
+# Example:
+#
+#     reserved_memory_pages = node:0,size:2048,count:64
+#     reserved_memory_pages = node:0,size:4,count:10485760
+#     reserved_memory_pages = node:1,size:1048576,count:2
+#     reserved_memory_pages = node:1,size:4,count:10485760
+#
+# In this example we're reserving 10GB of memory using standard 4K page
+# size on both NUMA nodes, 2 GB of memory in 1GB pages on NUMA node 1 and
+# 128MB of memory in 2MB pages on NUMA node 0.
+# reserved_memory_pages =
+
+# Defines the NUMA scheduling strategy. If enabled, the scheduler will
+# favor NUMA nodes that are more loaded. If disabled, we'll try to spread
+# the resources across NUMA nodes, favoring nodes that are less loaded.
+# pack_numa_nodes = false
+
+# Clock offset for replica VMs. Accepted values are 'utc' and 'localtime'.
+# When unset (the default), the provider picks 'localtime' for Windows
+# guests and 'utc' for all other OS types.
+# clock_offset =
+
+# VM timer configuration. Each entry is a set of comma-separated key:value
+# pairs describing one timer. Windows guests additionally get a hypervclock
+# timer appended unless one is already listed.
+# Example:
+#   cpu_timers = name:rtc,tickpolicy:catchup
+#   cpu_timers = name:pit,tickpolicy:delay
+#   cpu_timers = name:hpet,present:no
+# cpu_timers =
+
+# Attach the vhostmd metrics image as a read-only block device to replica
+# instances. This exposes KVM host metrics to workloads such as SAP HANA via
+# the vm-dump-metrics utility. Requires vhostmd to be running on the Libvirt
+# host and the metrics disk at vhostmd_device_path to exist.
+enable_vhostmd = false
+
+# Path to the vhostmd metrics disk on the Libvirt host. The file disk in the
+# domain XML will use this path as its source. Only used when enable_vhostmd
+# is True.
+vhostmd_device_path = /dev/shm/vhostmd0
+
+# Specifies how pinned sibling CPUs should be defined in the Libvirt domain
+# configuration.
+#
+# If sibling CPU floating is enabled, the VM vCPU will be allowed to float
+# between sibling host CPUs.
+#
+#     <vcpupin vcpu='0' cpuset='0,16'/>
+#     <vcpupin vcpu='1' cpuset='0,16'/>
+#
+# If sibling CPU floating is disabled, the VM vCPU will be pinned to a
+# single host CPU.
+#
+#     <vcpupin vcpu='0' cpuset='0'/>
+#     <vcpupin vcpu='1' cpuset='16/>
+pinned_cpu_float_between_siblings = false
+
+# The name of the pool used for raw passthrough disks. Will be hidden if
+# 'passthrough_disks' is empty. Note that we aren't using Libvirt storage
+# pools for raw passthrough disks.
+raw_disk_pool_name = raw-disks
+
+# A list of disk paths that can be attached to replica instances when the
+# user selects the pool specified by "raw_disk_pool_name".
+#
+# Each entry must specify the target host address and the disk path on that
+# host. The disks will be attached using virtio emulated devices (PCI
+# passthrough cannot be used with individual disks, only entire
+# controllers).
+#
+# Example:
+#     passthrough_disks = host:192.168.1.10,path:/dev/disk/by-id/wwn-0x5c50071daf
+#     passthrough_disks = host:192.168.1.10,path:/dev/disk/by-id/wwn-0x5d911034a9
+#     passthrough_disks = host:192.168.1.11,path:/dev/disk/by-id/wwn-0x5911034a97
+# passthrough_disks =
+
+# A list containing PCI addresses of disk controllers that can be attached
+# to replica instances.
+#
+# Each entry must specify the target host address and the PCI address of
+# the controller on that host. The entire controller will be exposed to the
+# instance using PCI passthrough, avoiding virtualization overhead. Once
+# attached, the controller as well as the associated disks will no longer
+# be visible to the host.
+#
+# The controllers are exposed to Coriolis users as storage backends
+# (pools). Coriolis will pick a suitable disk from the specified controller
+# when performing disk transfers.
+#
+# Fibre Channel HBAs often share an IO-MMU group (typically two functions
+# of the same adapter). All PCI devices that belong to the same IO-MMU
+# group must be listed here so they can be attached together. Only the
+# first listed device in each group is exposed as a selectable storage
+# pool.
+#
+# Example:
+#     passthrough_disk_controllers = host:192.168.1.10,address:0000:af:00.0
+#     passthrough_disk_controllers = host:192.168.1.10,address:0000:af:00.1
+#     passthrough_disk_controllers = host:192.168.1.11,address:0000:3b:00.0
+# passthrough_disk_controllers =
+```
+
+## Storage controllers exposed over PCI passthrough
+
+PCI passthrough requires the same kernel parameters as described by the
+[**SR-IOV section**](./suse-linux-kvm-target-platform.md#sr-iov)
+
+Use the `passthrough_disk_controllers` setting to whitelist storage controllers
+that can be attached to migrated VMs.
+
+All devices that belong to a IO-MMU group must be attached together to the
+same VM. In case of Fibre Channel HBAs, make sure to whitelist all the HBAs
+that belong to the same IO-MMU group.
+
+Note that only the first HBA of a group will be reported as a Coriolis storage
+backend. Coriolis will automatically "detach" the devices from the host, set them
+to use the VFIO driver and attach them to transfer workers or replica instances.
+
+If a replica instance is deleted and you wish to reuse the controller for
+another instance, use the "free_libvirt_resources.py" script from the appliance
+console to release it and the corresponding disks.
+
+Use the `virsh nodedev-reattach` command to expose the storage controller to
+the host again, passing `pci_<address_with_underscores>` as parameter.

source/platforms/suse-linux-kvm-target-platform.md

@@ -11,7 +11,11 @@ When using SUSE Linux as a KVM-based virtualization platform, Coriolis integrate
 
 Coriolis connects to the libvirt service through **SSH**(TCP/22) on each SUSE Linux KVM host to perform the operations required during migration workflows. In this design, each SUSE Linux KVM host must expose a functional libvirt endpoint that Coriolis can reach and authenticate against.
 
-**Note:** Migration to SUSE Linux on KVM applies to standard virtual machines only. SAP HANA workloads require a separate licensing model and endpoint, which provide SAP HANA-specific configuration options needed to meet performance, support, and compliance requirements.
+```{note} Migration to SUSE Linux on KVM applies to standard virtual machines only. SAP HANA workloads require a separate licensing model and endpoint, which provide SAP HANA-specific configuration options needed to meet performance, support, and compliance requirements.
+
+See the [**SAP HANA SKU document**](./suse-linux-kvm-sap-hana-target-platform.md)
+for more details.
+```
 
 ### Endpoint connection parameters
 
@@ -197,3 +201,116 @@ sqlite_database_file = /opt/coriolis/libvirt_provider_db.sqlite
 # Attach a persistent TPM device to replica instances.
 add_tpm_device = true
 ```
+
+## Libvirt host prerequisites
+
+### Storage pool
+
+The provider relies on Libvirt storage pools, allowing it to transparently use
+a variety of storage backends: local directory, Ceph, LVM, etc.
+
+Note that the minion VM image is expected to reside in a "directory"
+storage pool. To speed up minion VM deployments, the provider will create QCOW2
+images pointing to the specified image.
+
+Create a directory storage pool like so:
+
+```bash
+pool_name=local-dir-pool
+pool_path=/var/lib/libvirt/local-dir-pool
+
+sudo virsh pool-define-as \
+  --name $pool_name \
+  --type dir \
+  --target $pool_path
+
+sudo mkdir -p $pool_path
+sudo chown -R libvirt-qemu:kvm $pool_path
+sudo chmod 755 $pool_path
+sudo virsh pool-start $pool_name
+sudo virsh pool-autostart $pool_name
+```
+
+Then copy the desired minion image to that directory. When initiating transfers,
+the user will be prompted to select an image from that location.
+
+### Networks
+
+The minion VM network is expected to have DHCP enabled and be accessible from
+the appliance side.
+
+This example uses the `br1` bridge, which must be pre-configured:
+
+```bash
+cat > br1-network.xml <<EOF
+<network>
+  <name>br1-network</name>
+  <forward mode='bridge'/>
+  <bridge name='br1'/>
+</network>
+EOF
+
+virsh net-define br1-network.xml
+virsh net-start br1-network
+virsh net-autostart br1-network
+```
+
+### SR-IOV
+
+The Libvirt Coriolis provider can assign SR-IOV VFs to replica instances.
+
+Follow this guide to configure a `hostdev` Libvirt network, which can then
+be passed to Coriolis transfers.
+
+#### Host configuration
+
+First, enable SR-IOV and VT-d in the BIOS configuration.
+
+Then add the following to the list of kernel parameters to allow VM passthrough
+devices:
+
+```text
+intel_iommu=on iommu=pt
+```
+
+Some devices may not be mapped correctly and also require the following:
+
+```text
+pci=realloc pci=assign-busses
+```
+
+Preallocate the desired number of VFs:
+
+```bash
+echo 8 > /sys/class/net/p1p1/device/sriov_numvfs
+```
+
+To make the VFs persistent, consider using a Systemd service or Netplan
+configuration, depending on the Linux distribution.
+
+#### Libvirt network
+
+Libvirt networks can be configured to expose SR-IOV VFs to the connected VMs.
+
+Set the forward mode to `hostdev` and enable `managed` mode to automatically
+pick a VF. Then specify the desired PF to expose.
+
+```bash
+cat > vfio-p1p1-network.xml <<EOF
+<network>
+  <name>vfio-p1p1</name>
+  <forward mode='hostdev' managed='yes'>
+    <pf dev='p1p1'/>
+  </forward>
+</network>
+EOF
+
+virsh net-define vfio-p1p1-network.xml
+virsh net-start vfio-p1p1
+virsh net-autostart vfio-p1p1
+```
+
+#### Guest configuration
+
+The migrated VM needs to include the according VF interface drivers, which
+can be handled through user scripts during OS morphing.