Skip to content

Commit 9e2fd06

Browse files
soleenakpm00
authored andcommitted
liveupdate: luo_core: Live Update Orchestrator
Patch series "Live Update Orchestrator", v8. This series introduces the Live Update Orchestrator, a kernel subsystem designed to facilitate live kernel updates using a kexec-based reboot. This capability is critical for cloud environments, allowing hypervisors to be updated with minimal downtime for running virtual machines. LUO achieves this by preserving the state of selected resources, such as memory, devices and their dependencies, across the kernel transition. As a key feature, this series includes support for preserving memfd file descriptors, which allows critical in-memory data, such as guest RAM or any other large memory region, to be maintained in RAM across the kexec reboot. The other series that use LUO, are VFIO [1], IOMMU [2], and PCI [3] preservations. Github repo of this series [4]. The core of LUO is a framework for managing the lifecycle of preserved resources through a userspace-driven interface. Key features include: - Session Management Userspace agent (i.e. luod [5]) creates named sessions, each represented by a file descriptor (via centralized agent that controls /dev/liveupdate). The lifecycle of all preserved resources within a session is tied to this FD, ensuring automatic kernel cleanup if the controlling userspace agent crashes or exits unexpectedly. - File Preservation A handler-based framework allows specific file types (demonstrated here with memfd) to be preserved. Handlers manage the serialization, restoration, and lifecycle of their specific file types. - File-Lifecycle-Bound State A new mechanism for managing shared global state whose lifecycle is tied to the preservation of one or more files. This is crucial for subsystems like IOMMU or HugeTLB, where multiple file descriptors may depend on a single, shared underlying resource that must be preserved only once. - KHO Integration LUO drives the Kexec Handover framework programmatically to pass its serialized metadata to the next kernel. The LUO state is finalized and added to the kexec image just before the reboot is triggered. In the future this step will also be removed once stateless KHO is merged [6]. - Userspace Interface Control is provided via ioctl commands on /dev/liveupdate for creating and retrieving sessions, as well as on session file descriptors for managing individual files. - Testing The series includes a set of selftests, including userspace API validation, kexec-based lifecycle tests for various session and file scenarios, and a new in-kernel test module to validate the FLB logic. Introduce LUO, a mechanism intended to facilitate kernel updates while keeping designated devices operational across the transition (e.g., via kexec). The primary use case is updating hypervisors with minimal disruption to running virtual machines. For userspace side of hypervisor update we have copyless migration. LUO is for updating the kernel. This initial patch lays the groundwork for the LUO subsystem. Further functionality, including the implementation of state transition logic, integration with KHO, and hooks for subsystems and file descriptors, will be added in subsequent patches. Create a character device at /dev/liveupdate. A new uAPI header, <uapi/linux/liveupdate.h>, will define the necessary structures. The magic number for IOCTL is registered in Documentation/userspace-api/ioctl/ioctl-number.rst. Link: https://lkml.kernel.org/r/20251125165850.3389713-1-pasha.tatashin@soleen.com Link: https://lkml.kernel.org/r/20251125165850.3389713-2-pasha.tatashin@soleen.com Link: https://lore.kernel.org/all/20251018000713.677779-1-vipinsh@google.com/ [1] Link: https://lore.kernel.org/linux-iommu/20250928190624.3735830-1-skhawaja@google.com [2] Link: https://lore.kernel.org/linux-pci/20250916-luo-pci-v2-0-c494053c3c08@kernel.org [3] Link: https://github.com/googleprodkernel/linux-liveupdate/tree/luo/v8 [4] Link: https://tinyurl.com/luoddesign [5] Link: https://lore.kernel.org/all/20251020100306.2709352-1-jasonmiu@google.com [6] Link: https://lore.kernel.org/all/20251115233409.768044-1-pasha.tatashin@soleen.com [7] Link: https://github.com/soleen/linux/blob/luo/v8b03/diff.v7.v8 [8] Signed-off-by: Pasha Tatashin <pasha.tatashin@soleen.com> Reviewed-by: Pratyush Yadav <pratyush@kernel.org> Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org> Tested-by: David Matlack <dmatlack@google.com> Cc: Aleksander Lobakin <aleksander.lobakin@intel.com> Cc: Alexander Graf <graf@amazon.com> Cc: Alice Ryhl <aliceryhl@google.com> Cc: Andriy Shevchenko <andriy.shevchenko@linux.intel.com> Cc: anish kumar <yesanishhere@gmail.com> Cc: Anna Schumaker <anna.schumaker@oracle.com> Cc: Bartosz Golaszewski <bartosz.golaszewski@linaro.org> Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Borislav Betkov <bp@alien8.de> Cc: Chanwoo Choi <cw00.choi@samsung.com> Cc: Chen Ridong <chenridong@huawei.com> Cc: Chris Li <chrisl@kernel.org> Cc: Christian Brauner <brauner@kernel.org> Cc: Daniel Wagner <wagi@kernel.org> Cc: Danilo Krummrich <dakr@kernel.org> Cc: Dan Williams <dan.j.williams@intel.com> Cc: David Hildenbrand <david@redhat.com> Cc: David Jeffery <djeffery@redhat.com> Cc: David Rientjes <rientjes@google.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Guixin Liu <kanie@linux.alibaba.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Hugh Dickins <hughd@google.com> Cc: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Ira Weiny <ira.weiny@intel.com> Cc: Jann Horn <jannh@google.com> Cc: Jason Gunthorpe <jgg@nvidia.com> Cc: Jens Axboe <axboe@kernel.dk> Cc: Joanthan Cameron <Jonathan.Cameron@huawei.com> Cc: Joel Granados <joel.granados@kernel.org> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Lennart Poettering <lennart@poettering.net> Cc: Leon Romanovsky <leon@kernel.org> Cc: Leon Romanovsky <leonro@nvidia.com> Cc: Lukas Wunner <lukas@wunner.de> Cc: Marc Rutland <mark.rutland@arm.com> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Matthew Maurer <mmaurer@google.com> Cc: Miguel Ojeda <ojeda@kernel.org> Cc: Myugnjoo Ham <myungjoo.ham@samsung.com> Cc: Parav Pandit <parav@nvidia.com> Cc: Randy Dunlap <rdunlap@infradead.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Saeed Mahameed <saeedm@nvidia.com> Cc: Samiullah Khawaja <skhawaja@google.com> Cc: Song Liu <song@kernel.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Stuart Hayes <stuart.w.hayes@gmail.com> Cc: Tejun Heo <tj@kernel.org> Cc: Thomas Gleinxer <tglx@linutronix.de> Cc: Thomas Weißschuh <linux@weissschuh.net> Cc: Vincent Guittot <vincent.guittot@linaro.org> Cc: William Tu <witu@nvidia.com> Cc: Yoann Congal <yoann.congal@smile.fr> Cc: Zijun Hu <quic_zijuhu@quicinc.com> Cc: Pratyush Yadav <ptyadav@amazon.de> Cc: Zhu Yanjun <yanjun.zhu@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
1 parent 7bd3643 commit 9e2fd06

6 files changed

Lines changed: 220 additions & 0 deletions

File tree

Documentation/userspace-api/ioctl/ioctl-number.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -385,6 +385,8 @@ Code Seq# Include File Comments
385385
0xB8 01-02 uapi/misc/mrvl_cn10k_dpi.h Marvell CN10K DPI driver
386386
0xB8 all uapi/linux/mshv.h Microsoft Hyper-V /dev/mshv driver
387387
<mailto:linux-hyperv@vger.kernel.org>
388+
0xBA 00-0F uapi/linux/liveupdate.h Pasha Tatashin
389+
<mailto:pasha.tatashin@soleen.com>
388390
0xC0 00-0F linux/usb/iowarrior.h
389391
0xCA 00-0F uapi/misc/cxl.h Dead since 6.15
390392
0xCA 10-2F uapi/misc/ocxl.h

include/linux/liveupdate.h

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
/* SPDX-License-Identifier: GPL-2.0 */
2+
3+
/*
4+
* Copyright (c) 2025, Google LLC.
5+
* Pasha Tatashin <pasha.tatashin@soleen.com>
6+
*/
7+
#ifndef _LINUX_LIVEUPDATE_H
8+
#define _LINUX_LIVEUPDATE_H
9+
10+
#include <linux/bug.h>
11+
#include <linux/list.h>
12+
#include <linux/types.h>
13+
14+
#ifdef CONFIG_LIVEUPDATE
15+
16+
/* Return true if live update orchestrator is enabled */
17+
bool liveupdate_enabled(void);
18+
19+
/* Called during kexec to tell LUO that entered into reboot */
20+
int liveupdate_reboot(void);
21+
22+
#else /* CONFIG_LIVEUPDATE */
23+
24+
static inline bool liveupdate_enabled(void)
25+
{
26+
return false;
27+
}
28+
29+
static inline int liveupdate_reboot(void)
30+
{
31+
return 0;
32+
}
33+
34+
#endif /* CONFIG_LIVEUPDATE */
35+
#endif /* _LINUX_LIVEUPDATE_H */

include/uapi/linux/liveupdate.h

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
2+
3+
/*
4+
* Userspace interface for /dev/liveupdate
5+
* Live Update Orchestrator
6+
*
7+
* Copyright (c) 2025, Google LLC.
8+
* Pasha Tatashin <pasha.tatashin@soleen.com>
9+
*/
10+
11+
#ifndef _UAPI_LIVEUPDATE_H
12+
#define _UAPI_LIVEUPDATE_H
13+
14+
#include <linux/ioctl.h>
15+
#include <linux/types.h>
16+
17+
/**
18+
* DOC: General ioctl format
19+
*
20+
* The ioctl interface follows a general format to allow for extensibility. Each
21+
* ioctl is passed in a structure pointer as the argument providing the size of
22+
* the structure in the first u32. The kernel checks that any structure space
23+
* beyond what it understands is 0. This allows userspace to use the backward
24+
* compatible portion while consistently using the newer, larger, structures.
25+
*
26+
* ioctls use a standard meaning for common errnos:
27+
*
28+
* - ENOTTY: The IOCTL number itself is not supported at all
29+
* - E2BIG: The IOCTL number is supported, but the provided structure has
30+
* non-zero in a part the kernel does not understand.
31+
* - EOPNOTSUPP: The IOCTL number is supported, and the structure is
32+
* understood, however a known field has a value the kernel does not
33+
* understand or support.
34+
* - EINVAL: Everything about the IOCTL was understood, but a field is not
35+
* correct.
36+
* - ENOENT: A provided token does not exist.
37+
* - ENOMEM: Out of memory.
38+
* - EOVERFLOW: Mathematics overflowed.
39+
*
40+
* As well as additional errnos, within specific ioctls.
41+
*/
42+
43+
/* The ioctl type, documented in ioctl-number.rst */
44+
#define LIVEUPDATE_IOCTL_TYPE 0xBA
45+
46+
#endif /* _UAPI_LIVEUPDATE_H */

kernel/liveupdate/Kconfig

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,4 +51,25 @@ config KEXEC_HANDOVER_ENABLE_DEFAULT
5151
The default behavior can still be overridden at boot time by
5252
passing 'kho=off'.
5353

54+
config LIVEUPDATE
55+
bool "Live Update Orchestrator"
56+
depends on KEXEC_HANDOVER
57+
help
58+
Enable the Live Update Orchestrator. Live Update is a mechanism,
59+
typically based on kexec, that allows the kernel to be updated
60+
while keeping selected devices operational across the transition.
61+
These devices are intended to be reclaimed by the new kernel and
62+
re-attached to their original workload without requiring a device
63+
reset.
64+
65+
Ability to handover a device from current to the next kernel depends
66+
on specific support within device drivers and related kernel
67+
subsystems.
68+
69+
This feature primarily targets virtual machine hosts to quickly update
70+
the kernel hypervisor with minimal disruption to the running virtual
71+
machines.
72+
73+
If unsure, say N.
74+
5475
endmenu

kernel/liveupdate/Makefile

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# SPDX-License-Identifier: GPL-2.0
22

3+
luo-y := \
4+
luo_core.o
5+
36
obj-$(CONFIG_KEXEC_HANDOVER) += kexec_handover.o
47
obj-$(CONFIG_KEXEC_HANDOVER_DEBUG) += kexec_handover_debug.o
58
obj-$(CONFIG_KEXEC_HANDOVER_DEBUGFS) += kexec_handover_debugfs.o
9+
10+
obj-$(CONFIG_LIVEUPDATE) += luo.o

kernel/liveupdate/luo_core.c

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
// SPDX-License-Identifier: GPL-2.0
2+
3+
/*
4+
* Copyright (c) 2025, Google LLC.
5+
* Pasha Tatashin <pasha.tatashin@soleen.com>
6+
*/
7+
8+
/**
9+
* DOC: Live Update Orchestrator (LUO)
10+
*
11+
* Live Update is a specialized, kexec-based reboot process that allows a
12+
* running kernel to be updated from one version to another while preserving
13+
* the state of selected resources and keeping designated hardware devices
14+
* operational. For these devices, DMA activity may continue throughout the
15+
* kernel transition.
16+
*
17+
* While the primary use case driving this work is supporting live updates of
18+
* the Linux kernel when it is used as a hypervisor in cloud environments, the
19+
* LUO framework itself is designed to be workload-agnostic. Live Update
20+
* facilitates a full kernel version upgrade for any type of system.
21+
*
22+
* For example, a non-hypervisor system running an in-memory cache like
23+
* memcached with many gigabytes of data can use LUO. The userspace service
24+
* can place its cache into a memfd, have its state preserved by LUO, and
25+
* restore it immediately after the kernel kexec.
26+
*
27+
* Whether the system is running virtual machines, containers, a
28+
* high-performance database, or networking services, LUO's primary goal is to
29+
* enable a full kernel update by preserving critical userspace state and
30+
* keeping essential devices operational.
31+
*
32+
* The core of LUO is a mechanism that tracks the progress of a live update,
33+
* along with a callback API that allows other kernel subsystems to participate
34+
* in the process. Example subsystems that can hook into LUO include: kvm,
35+
* iommu, interrupts, vfio, participating filesystems, and memory management.
36+
*
37+
* LUO uses Kexec Handover to transfer memory state from the current kernel to
38+
* the next kernel. For more details see
39+
* Documentation/core-api/kho/concepts.rst.
40+
*/
41+
42+
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
43+
44+
#include <linux/kobject.h>
45+
#include <linux/liveupdate.h>
46+
#include <linux/miscdevice.h>
47+
48+
static struct {
49+
bool enabled;
50+
} luo_global;
51+
52+
static int __init early_liveupdate_param(char *buf)
53+
{
54+
return kstrtobool(buf, &luo_global.enabled);
55+
}
56+
early_param("liveupdate", early_liveupdate_param);
57+
58+
/* Public Functions */
59+
60+
/**
61+
* liveupdate_reboot() - Kernel reboot notifier for live update final
62+
* serialization.
63+
*
64+
* This function is invoked directly from the reboot() syscall pathway
65+
* if kexec is in progress.
66+
*
67+
* If any callback fails, this function aborts KHO, undoes the freeze()
68+
* callbacks, and returns an error.
69+
*/
70+
int liveupdate_reboot(void)
71+
{
72+
return 0;
73+
}
74+
75+
/**
76+
* liveupdate_enabled - Check if the live update feature is enabled.
77+
*
78+
* This function returns the state of the live update feature flag, which
79+
* can be controlled via the ``liveupdate`` kernel command-line parameter.
80+
*
81+
* @return true if live update is enabled, false otherwise.
82+
*/
83+
bool liveupdate_enabled(void)
84+
{
85+
return luo_global.enabled;
86+
}
87+
88+
struct luo_device_state {
89+
struct miscdevice miscdev;
90+
};
91+
92+
static const struct file_operations luo_fops = {
93+
.owner = THIS_MODULE,
94+
};
95+
96+
static struct luo_device_state luo_dev = {
97+
.miscdev = {
98+
.minor = MISC_DYNAMIC_MINOR,
99+
.name = "liveupdate",
100+
.fops = &luo_fops,
101+
},
102+
};
103+
104+
static int __init liveupdate_ioctl_init(void)
105+
{
106+
if (!liveupdate_enabled())
107+
return 0;
108+
109+
return misc_register(&luo_dev.miscdev);
110+
}
111+
late_initcall(liveupdate_ioctl_init);

0 commit comments

Comments
 (0)