Skip to content

Commit abbb993

Browse files
knurdJonathan Corbet
authored andcommitted
docs: verify/bisect: use git switch, tag kernel, and various fixes
Various small improvements and fixes: * Use the more modern 'git switch' instead of 'git checkout', which makes it more obvious what's happening (among others due to the --discard-changes parameter that is more clear than --force). * Provide a hint how a mainline version number and one from a stable series look like. * When trying to validate the bisection result with a revert, add a special tag to facilitate the identification. * Sync version numbers used in various examples for consistency: stick to 6.0.13, 6.0.15, and 6.1.5. * Fix a few typos and oddities. Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/85029aa004447b0eeb5043fb014630f2acafacec.1712647788.git.linux@leemhuis.info
1 parent e9c44c1 commit abbb993

1 file changed

Lines changed: 67 additions & 50 deletions

File tree

Documentation/admin-guide/verify-bugs-and-bisect-regressions.rst

Lines changed: 67 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,8 @@ aspects, all of which might be essential in your present case.]*
3838
**In case you want to check if a bug is present in code currently supported by
3939
developers**, execute just the *preparations* and *segment 1*; while doing so,
4040
consider the newest Linux kernel you regularly use to be the 'working' kernel.
41-
In the following example that's assumed to be 6.0.13, which is why the sources
42-
of 6.0 will be used to prepare the .config file.
41+
In the following example that's assumed to be 6.0, which is why its sources
42+
will be used to prepare the .config file.
4343

4444
**In case you face a regression**, follow the steps at least till the end of
4545
*segment 2*. Then you can submit a preliminary report -- or continue with
@@ -61,7 +61,7 @@ will be considered the 'good' release and used to prepare the .config file.
6161
cd ~/linux/
6262
git remote add -t master stable \
6363
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
64-
git checkout --detach v6.0
64+
git switch --detach v6.0
6565
# * Hint: if you used an existing clone, ensure no stale .config is around.
6666
make olddefconfig
6767
# * Ensure the former command picked the .config of the 'working' kernel.
@@ -87,7 +87,7 @@ will be considered the 'good' release and used to prepare the .config file.
8787
a) Checking out latest mainline code::
8888

8989
cd ~/linux/
90-
git checkout --force --detach mainline/master
90+
git switch --discard-changes --detach mainline/master
9191

9292
b) Build, install, and boot a kernel::
9393

@@ -125,7 +125,7 @@ will be considered the 'good' release and used to prepare the .config file.
125125
a) Start by checking out the sources of the 'good' version::
126126

127127
cd ~/linux/
128-
git checkout --force --detach v6.0
128+
git switch --discard-changes --detach v6.0
129129

130130
b) Build, install, and boot a kernel as described earlier in *segment 1,
131131
section b* -- just feel free to skip the 'du' commands, as you have a rough
@@ -157,11 +157,12 @@ will be considered the 'good' release and used to prepare the .config file.
157157
works with the newly built kernel. If it does, tell Git by executing
158158
``git bisect good``; if it does not, run ``git bisect bad`` instead.
159159

160-
All three commands will make Git checkout another commit; then re-execute
160+
All three commands will make Git check out another commit; then re-execute
161161
this step (e.g. build, install, boot, and test a kernel to then tell Git
162162
the outcome). Do so again and again until Git shows which commit broke
163163
things. If you run short of disk space during this process, check the
164-
"Supplementary tasks" section below.
164+
section 'Supplementary tasks: cleanup during and after the process'
165+
below.
165166

166167
d) Once your finished the bisection, put a few things away::
167168

@@ -172,12 +173,15 @@ will be considered the 'good' release and used to prepare the .config file.
172173

173174
e) Try to verify the bisection result::
174175

175-
git checkout --force --detach mainline/master
176+
git switch --discard-changes --detach mainline/master
176177
git revert --no-edit cafec0cacaca0
178+
cp ~/kernel-config-working .config
179+
./scripts/config --set-str CONFIG_LOCALVERSION '-local-cafec0cacaca0-reverted'
177180

178181
This is optional, as some commits are impossible to revert. But if the
179182
second command worked flawlessly, build, install, and boot one more kernel
180-
kernel, which should not show the regression.
183+
kernel; just this time skip the first command copying the base .config file
184+
over, as that already has been taken care off.
181185

182186
* **Supplementary tasks**: cleanup during and after the process.
183187

@@ -208,7 +212,7 @@ Step-by-step guide on how to verify bugs and bisect regressions
208212
===============================================================
209213

210214
This guide describes how to set up your own Linux kernels for investigating bugs
211-
or regressions you intent to report. How far you want to follow the instructions
215+
or regressions you intend to report. How far you want to follow the instructions
212216
depends on your issue:
213217

214218
Execute all steps till the end of *segment 1* to **verify if your kernel problem
@@ -240,12 +244,17 @@ to get things rolling again.
240244
For further details on how to report Linux kernel issues or regressions check
241245
out Documentation/admin-guide/reporting-issues.rst, which works in conjunction
242246
with this document. It among others explains why you need to verify bugs with
243-
the latest 'mainline' kernel, even if you face a problem with a kernel from a
244-
'stable/longterm' series; for users facing a regression it also explains that
245-
sending a preliminary report after finishing segment 2 might be wise, as the
246-
regression and its culprit might be known already. For further details on
247-
what actually qualifies as a regression check out
248-
Documentation/admin-guide/reporting-regressions.rst.
247+
the latest 'mainline' kernel (e.g. versions like 6.0, 6.1-rc1, or 6.1-rc6),
248+
even if you face a problem with a kernel from a 'stable/longterm' series
249+
(say 6.0.13).
250+
251+
For users facing a regression that document also explains why sending a
252+
preliminary report after segment 2 might be wise, as the regression and its
253+
culprit might be known already. For further details on what actually qualifies
254+
as a regression check out Documentation/admin-guide/reporting-regressions.rst.
255+
256+
If you run into any problems while following this guide or have ideas how to
257+
improve it, :ref:`please let the kernel developers know <submit_improvements>`.
249258

250259
.. _introprep_bissbs:
251260

@@ -286,7 +295,7 @@ Preparations: set up everything to build your own kernels
286295
Do you follow this guide to verify if a bug is present in the code developers
287296
care for? Then consider the mainline release your 'working' kernel (the newest
288297
one you regularly use) is based on to be the 'good' version; if your 'working'
289-
kernel for example is 6.0.11, then your 'good' kernel is 6.0.
298+
kernel for example is 6.0.13, then your 'good' kernel is 6.0.
290299

291300
In case you face a regression, it depends on the version range where the
292301
regression was introduced:
@@ -295,14 +304,14 @@ Preparations: set up everything to build your own kernels
295304
6.1-rc1? Then henceforth regard 6.0 as the last known 'good' version
296305
and 6.1-rc1 as the first 'bad' one.
297306

298-
* Some function stopped working when updating from 6.0.11 to 6.1.4? Then for
299-
the time being consider 6.0 as the last 'good' version and 6.1.4 as
307+
* Some function stopped working when updating from 6.0.13 to 6.1.5? Then for
308+
the time being consider 6.0 as the last 'good' version and 6.1.5 as
300309
the 'bad' one. Note, at this point it is merely assumed that 6.0 is fine;
301310
this assumption will be checked in segment 2.
302311

303-
* A feature you used in 6.0.11 does not work at all or worse in 6.1.13? In
312+
* A feature you used in 6.0.13 does not work at all or worse in 6.1.15? In
304313
that case you want to bisect within a stable/longterm series: consider
305-
6.0.11 as the last known 'good' version and 6.0.13 as the first 'bad'
314+
6.0.13 as the last known 'good' version and 6.0.15 as the first 'bad'
306315
one. Note, in this case you still want to compile and test a mainline kernel
307316
as explained in segment 1: the outcome will determine if you need to report
308317
your issue to the regular developers or the stable team.
@@ -367,15 +376,15 @@ Preparations: set up everything to build your own kernels
367376
* Start preparing a kernel build configuration (the '.config' file).
368377

369378
Before doing so, ensure you are still running the 'working' kernel an earlier
370-
step told you to boot; if you are unsure, check the current kernel release
379+
step told you to boot; if you are unsure, check the current kernelrelease
371380
identifier using ``uname -r``.
372381

373382
Afterwards check out the source code for the version earlier established as
374383
'good'. In the following example command this is assumed to be 6.0; note that
375384
the version number in this and all later Git commands needs to be prefixed
376385
with a 'v'::
377386

378-
git checkout --detach v6.0
387+
git switch --discard-changes --detach v6.0
379388

380389
Now create a build configuration file::
381390

@@ -505,7 +514,7 @@ be a waste of time. [:ref:`details<introlatestcheck_bisref>`]
505514
* Check out the latest Linux codebase::
506515

507516
cd ~/linux/
508-
git checkout --force --detach mainline/master
517+
git switch --discard-changes --detach mainline/master
509518

510519
[:ref:`details<checkoutmaster_bisref>`]
511520

@@ -617,7 +626,7 @@ be a waste of time. [:ref:`details<introlatestcheck_bisref>`]
617626
cd ~/linux/
618627
git remote set-branches --add stable linux-6.0.y
619628
git fetch stable
620-
git checkout --force --detach linux-6.0.y
629+
git switch --discard-changes --detach linux-6.0.y
621630

622631
Now use the checked out code to build and install another kernel using the
623632
commands the earlier steps already described in more detail::
@@ -669,7 +678,7 @@ otherwise would be a waste of time. [:ref:`details<introworkingcheck_bisref>`]
669678
'good' (once again assumed to be 6.0 here)::
670679

671680
cd ~/linux/
672-
git checkout --detach v6.0
681+
git switch --discard-changes --detach v6.0
673682

674683
Now use the checked out code to configure, build, and install another kernel
675684
using the commands the previous subsection explained in more detail::
@@ -703,7 +712,7 @@ Segment 3: perform the bisection and validate the result
703712
With all the preparations and precaution builds taken care of, you are now ready
704713
to begin the bisection. This will make you build quite a few kernels -- usually
705714
about 15 in case you encountered a regression when updating to a newer series
706-
(say from 6.0.11 to 6.1.3). But do not worry, due to the trimmed build
715+
(say from 6.0.13 to 6.1.5). But do not worry, due to the trimmed build
707716
configuration created earlier this works a lot faster than many people assume:
708717
overall on average it will often just take about 10 to 15 minutes to compile
709718
each kernel on commodity x86 machines.
@@ -745,7 +754,7 @@ each kernel on commodity x86 machines.
745754
If compilation fails for some reason, run ``git bisect skip`` and restart
746755
executing the stack of commands from the beginning.
747756

748-
In case you skipped the "test latest codebase" step in the guide, check its
757+
In case you skipped the 'test latest codebase' step in the guide, check its
749758
description as for why the 'df [...]' and 'make -s kernelrelease [...]'
750759
commands are here.
751760

@@ -823,16 +832,16 @@ each kernel on commodity x86 machines.
823832
Begin by checking out the latest codebase depending on the range you bisected:
824833

825834
* Did you face a regression within a stable/longterm series (say between
826-
6.0.11 and 6.0.13) that does not happen in mainline? Then check out the
835+
6.0.13 and 6.0.15) that does not happen in mainline? Then check out the
827836
latest codebase for the affected series like this::
828837

829838
git fetch stable
830-
git checkout --force --detach linux-6.0.y
839+
git switch --discard-changes --detach linux-6.0.y
831840

832841
* In all other cases check out latest mainline::
833842

834843
git fetch mainline
835-
git checkout --force --detach mainline/master
844+
git switch --discard-changes --detach mainline/master
836845

837846
If you bisected a regression within a stable/longterm series that also
838847
happens in mainline, there is one more thing to do: look up the mainline
@@ -846,21 +855,27 @@ each kernel on commodity x86 machines.
846855

847856
git revert --no-edit cafec0cacaca0
848857

849-
If that fails, give up trying and move on to the next step. But if it works,
850-
build a kernel again using the familiar command sequence::
858+
If that fails, give up trying and move on to the next step; if it works,
859+
adjust the tag to facilitate the identification and prevent accidentally
860+
overwriting another kernel::
851861

852862
cp ~/kernel-config-working .config
863+
./scripts/config --set-str CONFIG_LOCALVERSION '-local-cafec0cacaca0-reverted'
864+
865+
Build a kernel using the familiar command sequence, just without copying the
866+
the base .config over::
867+
853868
make olddefconfig &&
854-
make -j $(nproc --all) &&
869+
make -j $(nproc --all)
855870
# * Check if the free space suffices holding another kernel:
856871
df -h /boot/ /lib/modules/
857872
sudo make modules_install
858873
command -v installkernel && sudo make install
859-
Make -s kernelrelease | tee -a ~/kernels-built
874+
make -s kernelrelease | tee -a ~/kernels-built
860875
reboot
861876

862-
Now check one last time if the feature that made you perform a bisection work
863-
with that kernel.
877+
Now check one last time if the feature that made you perform a bisection works
878+
with that kernel: if everything went well, it should not show the regression.
864879

865880
[:ref:`details<revert_bisref>`]
866881

@@ -934,10 +949,12 @@ This concludes the step-by-step guide.
934949

935950
Did you run into trouble following any of the above steps not cleared up by the
936951
reference section below? Did you spot errors? Or do you have ideas how to
937-
improve the guide? Then please take a moment and let the maintainer of this
952+
improve the guide?
953+
954+
If any of that applies, please take a moment and let the maintainer of this
938955
document know by email (Thorsten Leemhuis <linux@leemhuis.info>), ideally while
939956
CCing the Linux docs mailing list (linux-doc@vger.kernel.org). Such feedback is
940-
vital to improve this document further, which is in everybody's interest, as it
957+
vital to improve this text further, which is in everybody's interest, as it
941958
will enable more people to master the task described here -- and hopefully also
942959
improve similar guides inspired by this one.
943960

@@ -1059,18 +1076,18 @@ Bisection range
10591076

10601077
Establishing the range of commits to be checked is mostly straightforward,
10611078
except when a regression occurred when switching from a release of one stable
1062-
series to a release of a later series (e.g. from 6.0.11 to 6.1.4). In that case
1079+
series to a release of a later series (e.g. from 6.0.13 to 6.1.5). In that case
10631080
Git will need some hand holding, as there is no straight line of descent.
10641081

10651082
That's because with the release of 6.0 mainline carried on to 6.1 while the
10661083
stable series 6.0.y branched to the side. It's therefore theoretically possible
1067-
that the issue you face with 6.1.4 only worked in 6.0.11, as it was fixed by a
1084+
that the issue you face with 6.1.5 only worked in 6.0.13, as it was fixed by a
10681085
commit that went into one of the 6.0.y releases, but never hit mainline or the
10691086
6.1.y series. Thankfully that normally should not happen due to the way the
10701087
stable/longterm maintainers maintain the code. It's thus pretty safe to assume
10711088
6.0 as a 'good' kernel. That assumption will be tested anyway, as that kernel
10721089
will be built and tested in the segment '2' of this guide; Git would force you
1073-
to do this as well, if you tried bisecting between 6.0.11 and 6.1.13.
1090+
to do this as well, if you tried bisecting between 6.0.13 and 6.1.15.
10741091

10751092
[:ref:`back to step-by-step guide <rangecheck_bissbs>`]
10761093

@@ -1117,7 +1134,7 @@ These commands install a few packages that are often, but not always needed. You
11171134
for example might want to skip installing the development headers for ncurses,
11181135
which you will only need in case you later might want to adjust the kernel build
11191136
configuration using make the targets 'menuconfig' or 'nconfig'; likewise omit
1120-
the headers of Qt6 is you do not plan to adjust the .config using 'xconfig'.
1137+
the headers of Qt6 if you do not plan to adjust the .config using 'xconfig'.
11211138

11221139
You furthermore might need additional libraries and their development headers
11231140
for tasks not covered in this guide -- for example when building utilities from
@@ -1184,7 +1201,7 @@ First, execute the following command to retrieve the latest mainline codebase::
11841201
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
11851202

11861203
Now deepen your clone's history to the second predecessor of the mainline
1187-
release of your 'good' version. In case the latter are 6.0 or 6.0.11, 5.19 would
1204+
release of your 'good' version. In case the latter are 6.0 or 6.0.13, 5.19 would
11881205
be the first predecessor and 5.18 the second -- hence deepen the history up to
11891206
that version::
11901207

@@ -1490,7 +1507,7 @@ highly recommended for these reasons:
14901507

14911508
Your report might be ignored if you send it to the wrong party -- and even
14921509
when you get a reply there is a decent chance that developers tell you to
1493-
evaluate which of the two cases it is before they take a closer look.
1510+
evaluate which of the two cases it is before they take a closer look.
14941511

14951512
[:ref:`back to step-by-step guide <introlatestcheck_bissbs>`]
14961513

@@ -1552,8 +1569,8 @@ by modifying your search terms or using another line from the error messages.
15521569

15531570
In the end, most issues you run into have likely been encountered and
15541571
reported by others already. That includes issues where the cause is not your
1555-
system, but lies in the code. If you run into one of those, you might thus find a
1556-
solution (e.g. a patch) or workaround for your issue, too.
1572+
system, but lies in the code. If you run into one of those, you might thus find
1573+
a solution (e.g. a patch) or workaround for your issue, too.
15571574

15581575
Package your kernel up
15591576
~~~~~~~~~~~~~~~~~~~~~~
@@ -1767,8 +1784,8 @@ multitude of reasons why this might happen. Some ideas where to look:
17671784

17681785
Note, if you found and fixed problems with the .config file, you want to use it
17691786
to build another kernel from the latest codebase, as your earlier tests with
1770-
mainline and the latest version from an affected stable/longterm series were most
1771-
likely flawed.
1787+
mainline and the latest version from an affected stable/longterm series were
1788+
most likely flawed.
17721789

17731790
[:ref:`back to step-by-step guide <recheckworking_bissbs>`]
17741791

@@ -1911,7 +1928,7 @@ Now remove the boot entry for the kernel from your bootloader's configuration;
19111928
the steps to do that vary quite a bit between Linux distributions.
19121929

19131930
Note, be careful with wildcards like '*' when deleting files or directories
1914-
for kernels manually: you might accidentally remove files of a 6.0.11 kernel
1931+
for kernels manually: you might accidentally remove files of a 6.0.13 kernel
19151932
when all you want is to remove 6.0 or 6.0.1.
19161933

19171934
[:ref:`back to step-by-step guide <makeroom_bissbs>`]

0 commit comments

Comments
 (0)