Skip to content

Commit df989b0

Browse files
nathanchancenathanchance
committed
Merge 7.0 Kbuild changes into kbuild-fixes
kbuild-fixes needs to be based on 6.19 to apply some fixes for 62089b8 ("kbuild: rpm-pkg: Generate debuginfo package manually") which landed in 6.19-rc1 but the new material of 7.0 needs fixes merged as well. Signed-off-by: Nathan Chancellor <nathan@kernel.org>
2 parents 05f7e89 + d8ad80a commit df989b0

63 files changed

Lines changed: 1515 additions & 626 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Documentation/dev-tools/container.rst‎

Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
.. SPDX-License-Identifier: GPL-2.0-only
2+
.. Copyright (C) 2025 Guillaume Tucker
3+
4+
====================
5+
Containerized Builds
6+
====================
7+
8+
The ``container`` tool can be used to run any command in the kernel source tree
9+
from within a container. Doing so facilitates reproducing builds across
10+
various platforms, for example when a test bot has reported an issue which
11+
requires a specific version of a compiler or an external test suite. While
12+
this can already be done by users who are familiar with containers, having a
13+
dedicated tool in the kernel tree lowers the barrier to entry by solving common
14+
problems once and for all (e.g. user id management). It also makes it easier
15+
to share an exact command line leading to a particular result. The main use
16+
case is likely to be kernel builds but virtually anything can be run: KUnit,
17+
checkpatch etc. provided a suitable image is available.
18+
19+
20+
Options
21+
=======
22+
23+
Command line syntax::
24+
25+
scripts/container -i IMAGE [OPTION]... CMD...
26+
27+
Available options:
28+
29+
``-e, --env-file ENV_FILE``
30+
31+
Path to an environment file to load in the container.
32+
33+
``-g, --gid GID``
34+
35+
Group id to use inside the container.
36+
37+
``-i, --image IMAGE``
38+
39+
Container image name (required).
40+
41+
``-r, --runtime RUNTIME``
42+
43+
Container runtime name. Supported runtimes: ``docker``, ``podman``.
44+
45+
If not specified, the first one found on the system will be used
46+
i.e. Podman if present, otherwise Docker.
47+
48+
``-s, --shell``
49+
50+
Run the container in an interactive shell.
51+
52+
``-u, --uid UID``
53+
54+
User id to use inside the container.
55+
56+
If the ``-g`` option is not specified, the user id will also be used for
57+
the group id.
58+
59+
``-v, --verbose``
60+
61+
Enable verbose output.
62+
63+
``-h, --help``
64+
65+
Show the help message and exit.
66+
67+
68+
Usage
69+
=====
70+
71+
It's entirely up to the user to choose which image to use and the ``CMD``
72+
arguments are passed directly as an arbitrary command line to run in the
73+
container. The tool will take care of mounting the source tree as the current
74+
working directory and adjust the user and group id as needed.
75+
76+
The container image which would typically include a compiler toolchain is
77+
provided by the user and selected via the ``-i`` option. The container runtime
78+
can be selected with the ``-r`` option, which can be either ``docker`` or
79+
``podman``. If none is specified, the first one found on the system will be
80+
used while giving priority to Podman. Support for other runtimes may be added
81+
later depending on their popularity among users.
82+
83+
By default, commands are run non-interactively. The user can abort a running
84+
container with SIGINT (Ctrl-C). To run commands interactively with a TTY, the
85+
``--shell`` or ``-s`` option can be used. Signals will then be received by the
86+
shell directly rather than the parent ``container`` process. To exit an
87+
interactive shell, use Ctrl-D or ``exit``.
88+
89+
.. note::
90+
91+
The only host requirement aside from a container runtime is Python 3.10 or
92+
later.
93+
94+
.. note::
95+
96+
Out-of-tree builds are not fully supported yet. The ``O=`` option can
97+
however already be used with a relative path inside the source tree to keep
98+
separate build outputs. A workaround to build outside the tree is to use
99+
``mount --bind``, see the examples section further down.
100+
101+
102+
Environment Variables
103+
=====================
104+
105+
Environment variables are not propagated to the container so they have to be
106+
either defined in the image itself or via the ``-e`` option using an
107+
environment file. In some cases it makes more sense to have them defined in
108+
the Containerfile used to create the image. For example, a Clang-only compiler
109+
toolchain image may have ``LLVM=1`` defined.
110+
111+
The local environment file is more useful for user-specific variables added
112+
during development. It is passed as-is to the container runtime so its format
113+
may vary. Typically, it will look like the output of ``env``. For example::
114+
115+
INSTALL_MOD_STRIP=1
116+
SOME_RANDOM_TEXT=One upon a time
117+
118+
Please also note that ``make`` options can still be passed on the command line,
119+
so while this can't be done since the first argument needs to be the
120+
executable::
121+
122+
scripts/container -i docker.io/tuxmake/korg-clang LLVM=1 make # won't work
123+
124+
this will work::
125+
126+
scripts/container -i docker.io/tuxmake/korg-clang make LLVM=1
127+
128+
129+
User IDs
130+
========
131+
132+
This is an area where the behaviour will vary slightly depending on the
133+
container runtime. The goal is to run commands as the user invoking the tool.
134+
With Podman, a namespace is created to map the current user id to a different
135+
one in the container (1000 by default). With Docker, while this is also
136+
possible with recent versions it requires a special feature to be enabled in
137+
the daemon so it's not used here for simplicity. Instead, the container is run
138+
with the current user id directly. In both cases, this will provide the same
139+
file permissions for the kernel source tree mounted as a volume. The only
140+
difference is that when using Docker without a namespace, the user id may not
141+
be the same as the default one set in the image.
142+
143+
Say, we're using an image which sets up a default user with id 1000 and the
144+
current user calling the ``container`` tool has id 1234. The kernel source
145+
tree was checked out by this same user so the files belong to user 1234. With
146+
Podman, the container will be running as user id 1000 with a mapping to id 1234
147+
so that the files from the mounted volume appear to belong to id 1000 inside
148+
the container. With Docker and no namespace, the container will be running
149+
with user id 1234 which can access the files in the volume but not in the user
150+
1000 home directory. This shouldn't be an issue when running commands only in
151+
the kernel tree but it is worth highlighting here as it might matter for
152+
special corner cases.
153+
154+
.. note::
155+
156+
Podman's `Docker compatibility
157+
<https://podman-desktop.io/docs/migrating-from-docker/managing-docker-compatibility>`__
158+
mode to run ``docker`` commands on top of a Podman backend is more complex
159+
and not fully supported yet. As such, Podman will take priority if both
160+
runtimes are available on the system.
161+
162+
163+
Examples
164+
========
165+
166+
The TuxMake project provides a variety of prebuilt container images available
167+
on `Docker Hub <https://hub.docker.com/u/tuxmake>`__. Here's the shortest
168+
example to build a kernel using a TuxMake Clang image::
169+
170+
scripts/container -i docker.io/tuxmake/korg-clang -- make LLVM=1 defconfig
171+
scripts/container -i docker.io/tuxmake/korg-clang -- make LLVM=1 -j$(nproc)
172+
173+
.. note::
174+
175+
When running a command with options within the container, it should be
176+
separated with a double dash ``--`` to not confuse them with the
177+
``container`` tool options. Plain commands with no options don't strictly
178+
require the double dashes e.g.::
179+
180+
scripts/container -i docker.io/tuxmake/korg-clang make mrproper
181+
182+
To run ``checkpatch.pl`` in a ``patches`` directory with a generic Perl image::
183+
184+
scripts/container -i perl:slim-trixie scripts/checkpatch.pl patches/*
185+
186+
As an alternative to the TuxMake images, the examples below refer to
187+
``kernel.org`` images which are based on the `kernel.org compiler toolchains
188+
<https://mirrors.edge.kernel.org/pub/tools/>`__. These aren't (yet) officially
189+
available in any public registry but users can build their own locally instead
190+
using this `experimental repository
191+
<https://gitlab.com/gtucker/korg-containers>`__ by running ``make
192+
PREFIX=kernel.org/``.
193+
194+
To build just ``bzImage`` using Clang::
195+
196+
scripts/container -i kernel.org/clang -- make bzImage -j$(nproc)
197+
198+
Same with GCC 15 as a particular version tag::
199+
200+
scripts/container -i kernel.org/gcc:15 -- make bzImage -j$(nproc)
201+
202+
For an out-of-tree build, a trick is to bind-mount the destination directory to
203+
a relative path inside the source tree::
204+
205+
mkdir -p $HOME/tmp/my-kernel-build
206+
mkdir -p build
207+
sudo mount --bind $HOME/tmp/my-kernel-build build
208+
scripts/container -i kernel.org/gcc -- make mrproper
209+
scripts/container -i kernel.org/gcc -- make O=build defconfig
210+
scripts/container -i kernel.org/gcc -- make O=build -j$(nproc)
211+
212+
To run KUnit in an interactive shell and get the full output::
213+
214+
scripts/container -s -i kernel.org/gcc:kunit -- \
215+
tools/testing/kunit/kunit.py \
216+
run \
217+
--arch=x86_64 \
218+
--cross_compile=x86_64-linux-
219+
220+
To just start an interactive shell::
221+
222+
scripts/container -si kernel.org/gcc bash
223+
224+
To build the HTML documentation, which requires the ``kdocs`` image built with
225+
``make PREFIX=kernel.org/ extra`` as it's not a compiler toolchain::
226+
227+
scripts/container -i kernel.org/kdocs make htmldocs

‎Documentation/dev-tools/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ Documentation/process/debugging/index.rst
3838
gpio-sloppy-logic-analyzer
3939
autofdo
4040
propeller
41+
container
4142

4243

4344
.. only:: subproject and html

0 commit comments

Comments
 (0)