Skip to content

Commit 0674324

Browse files
torvaldstorvalds
committed
Merge tag 'linux-kselftest-next-5.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
Pull kselftest updates from Shuah Khan: - speed up headers_install done during selftest build - add generic make nesting support - add support to select individual tests: Selftests build/install generates run_kselftest.sh script to run selftests on a target system. Currently the script doesn't have support for selecting individual tests. Add support for it. With this enhancement, user can select test collections (or tests) individually. e.g: run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep Additionally adds a way to list all known tests with "-l", usage with "-h", and perform a dry run without running tests with "-n". * tag 'linux-kselftest-next-5.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: doc: dev-tools: kselftest.rst: Update examples and paths selftests/run_kselftest.sh: Make each test individually selectable selftests: Extract run_kselftest.sh and generate stand-alone test list selftests: Add missing gitignore entries selftests: more general make nesting support selftests: use "$(MAKE)" instead of "make" for headers_install
2 parents bbf6259 + 4c85113 commit 0674324

7 files changed

Lines changed: 135 additions & 41 deletions

File tree

‎Documentation/dev-tools/kselftest.rst‎

Lines changed: 22 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -125,32 +125,41 @@ Note that some tests will require root privileges.
125125
Install selftests
126126
=================
127127

128-
You can use the kselftest_install.sh tool to install selftests in the
129-
default location, which is tools/testing/selftests/kselftest, or in a
130-
user specified location.
128+
You can use the "install" target of "make" (which calls the `kselftest_install.sh`
129+
tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`),
130+
or in a user specified location via the `INSTALL_PATH` "make" variable.
131131

132132
To install selftests in default location::
133133

134-
$ cd tools/testing/selftests
135-
$ ./kselftest_install.sh
134+
$ make -C tools/testing/selftests install
136135

137136
To install selftests in a user specified location::
138137

139-
$ cd tools/testing/selftests
140-
$ ./kselftest_install.sh install_dir
138+
$ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path
141139

142140
Running installed selftests
143141
===========================
144142

145-
Kselftest install as well as the Kselftest tarball provide a script
146-
named "run_kselftest.sh" to run the tests.
143+
Found in the install directory, as well as in the Kselftest tarball,
144+
is a script named `run_kselftest.sh` to run the tests.
147145

148146
You can simply do the following to run the installed Kselftests. Please
149147
note some tests will require root privileges::
150148

151-
$ cd kselftest
149+
$ cd kselftest_install
152150
$ ./run_kselftest.sh
153151

152+
To see the list of available tests, the `-l` option can be used::
153+
154+
$ ./run_kselftest.sh -l
155+
156+
The `-c` option can be used to run all the tests from a test collection, or
157+
the `-t` option for specific single tests. Either can be used multiple times::
158+
159+
$ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep
160+
161+
For other features see the script usage output, seen with the `-h` option.
162+
154163
Packaging selftests
155164
===================
156165

@@ -160,9 +169,9 @@ different system. To package selftests, run::
160169
$ make -C tools/testing/selftests gen_tar
161170

162171
This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By
163-
default, `.gz` format is used. The tar format can be overridden by specifying
164-
a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_ option
165-
is supported, such as::
172+
default, `.gz` format is used. The tar compression format can be overridden by
173+
specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_
174+
option is supported, such as::
166175

167176
$ make -C tools/testing/selftests gen_tar FORMAT=.xz
168177

‎tools/testing/selftests/Makefile‎

Lines changed: 11 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -88,10 +88,10 @@ endif
8888
# of the targets gets built.
8989
FORCE_TARGETS ?=
9090

91-
# Clear LDFLAGS and MAKEFLAGS if called from main
92-
# Makefile to avoid test build failures when test
93-
# Makefile doesn't have explicit build rules.
94-
ifeq (1,$(MAKELEVEL))
91+
# Clear LDFLAGS and MAKEFLAGS when implicit rules are missing. This provides
92+
# implicit rules to sub-test Makefiles which avoids build failures in test
93+
# Makefile that don't have explicit build rules.
94+
ifeq (,$(LINK.c))
9595
override LDFLAGS =
9696
override MAKEFLAGS =
9797
endif
@@ -206,6 +206,7 @@ KSFT_INSTALL_PATH := $(abspath $(KSFT_INSTALL_PATH))
206206
# Avoid changing the rest of the logic here and lib.mk.
207207
INSTALL_PATH := $(KSFT_INSTALL_PATH)
208208
ALL_SCRIPT := $(INSTALL_PATH)/run_kselftest.sh
209+
TEST_LIST := $(INSTALL_PATH)/kselftest-list.txt
209210

210211
install: all
211212
ifdef INSTALL_PATH
@@ -214,6 +215,8 @@ ifdef INSTALL_PATH
214215
install -m 744 kselftest/module.sh $(INSTALL_PATH)/kselftest/
215216
install -m 744 kselftest/runner.sh $(INSTALL_PATH)/kselftest/
216217
install -m 744 kselftest/prefix.pl $(INSTALL_PATH)/kselftest/
218+
install -m 744 run_kselftest.sh $(INSTALL_PATH)/
219+
rm -f $(TEST_LIST)
217220
@ret=1; \
218221
for TARGET in $(TARGETS); do \
219222
BUILD_TARGET=$$BUILD/$$TARGET; \
@@ -222,33 +225,18 @@ ifdef INSTALL_PATH
222225
ret=$$((ret * $$?)); \
223226
done; exit $$ret;
224227

225-
@# Ask all targets to emit their test scripts
226-
echo "#!/bin/sh" > $(ALL_SCRIPT)
227-
echo "BASE_DIR=\$$(realpath \$$(dirname \$$0))" >> $(ALL_SCRIPT)
228-
echo "cd \$$BASE_DIR" >> $(ALL_SCRIPT)
229-
echo ". ./kselftest/runner.sh" >> $(ALL_SCRIPT)
230-
echo "ROOT=\$$PWD" >> $(ALL_SCRIPT)
231-
echo "if [ \"\$$1\" = \"--summary\" ]; then" >> $(ALL_SCRIPT)
232-
echo " logfile=\$$BASE_DIR/output.log" >> $(ALL_SCRIPT)
233-
echo " cat /dev/null > \$$logfile" >> $(ALL_SCRIPT)
234-
echo "fi" >> $(ALL_SCRIPT)
235228

236-
@# While building run_kselftest.sh skip also non-existent TARGET dirs:
229+
@# Ask all targets to emit their test scripts
230+
@# While building kselftest-list.text skip also non-existent TARGET dirs:
237231
@# they could be the result of a build failure and should NOT be
238232
@# included in the generated runlist.
239233
for TARGET in $(TARGETS); do \
240234
BUILD_TARGET=$$BUILD/$$TARGET; \
241235
[ ! -d $(INSTALL_PATH)/$$TARGET ] && echo "Skipping non-existent dir: $$TARGET" && continue; \
242-
echo "[ -w /dev/kmsg ] && echo \"kselftest: Running tests in $$TARGET\" >> /dev/kmsg" >> $(ALL_SCRIPT); \
243-
echo "cd $$TARGET" >> $(ALL_SCRIPT); \
244-
echo -n "run_many" >> $(ALL_SCRIPT); \
245236
echo -n "Emit Tests for $$TARGET\n"; \
246-
$(MAKE) -s --no-print-directory OUTPUT=$$BUILD_TARGET -C $$TARGET emit_tests >> $(ALL_SCRIPT); \
247-
echo "" >> $(ALL_SCRIPT); \
248-
echo "cd \$$ROOT" >> $(ALL_SCRIPT); \
237+
$(MAKE) -s --no-print-directory OUTPUT=$$BUILD_TARGET COLLECTION=$$TARGET \
238+
-C $$TARGET emit_tests >> $(TEST_LIST); \
249239
done;
250-
251-
chmod u+x $(ALL_SCRIPT)
252240
else
253241
$(error Error: set INSTALL_PATH to use install)
254242
endif

‎tools/testing/selftests/firmware/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# SPDX-License-Identifier: GPL-2.0-only
2+
fw_namespace

‎tools/testing/selftests/lib.mk‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -47,9 +47,9 @@ ARCH ?= $(SUBARCH)
4747
khdr:
4848
ifndef KSFT_KHDR_INSTALL_DONE
4949
ifeq (1,$(DEFAULT_INSTALL_HDR_PATH))
50-
make --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install
50+
$(MAKE) --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install
5151
else
52-
make --no-builtin-rules INSTALL_HDR_PATH=$$OUTPUT/usr \
52+
$(MAKE) --no-builtin-rules INSTALL_HDR_PATH=$$OUTPUT/usr \
5353
ARCH=$(ARCH) -C $(top_srcdir) headers_install
5454
endif
5555
endif
@@ -107,9 +107,8 @@ endif
107107
emit_tests:
108108
for TEST in $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) $(TEST_PROGS); do \
109109
BASENAME_TEST=`basename $$TEST`; \
110-
echo " \\"; \
111-
echo -n " \"$$BASENAME_TEST\""; \
112-
done; \
110+
echo "$(COLLECTION):$$BASENAME_TEST"; \
111+
done
113112

114113
# define if isn't already. It is undefined in make O= case.
115114
ifeq ($(RM),)

‎tools/testing/selftests/netfilter/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# SPDX-License-Identifier: GPL-2.0-only
2+
nf-queue

‎tools/testing/selftests/ptrace/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
# SPDX-License-Identifier: GPL-2.0-only
22
get_syscall_info
33
peeksiginfo
4+
vmaccess

‎tools/testing/selftests/run_kselftest.sh‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
#!/bin/sh
2+
# SPDX-License-Identifier: GPL-2.0
3+
#
4+
# Run installed kselftest tests.
5+
#
6+
BASE_DIR=$(realpath $(dirname $0))
7+
cd $BASE_DIR
8+
TESTS="$BASE_DIR"/kselftest-list.txt
9+
if [ ! -r "$TESTS" ] ; then
10+
echo "$0: Could not find list of tests to run ($TESTS)" >&2
11+
available=""
12+
else
13+
available="$(cat "$TESTS")"
14+
fi
15+
16+
. ./kselftest/runner.sh
17+
ROOT=$PWD
18+
19+
usage()
20+
{
21+
cat <<EOF
22+
Usage: $0 [OPTIONS]
23+
-s | --summary Print summary with detailed log in output.log
24+
-t | --test COLLECTION:TEST Run TEST from COLLECTION
25+
-c | --collection COLLECTION Run all tests from COLLECTION
26+
-l | --list List the available collection:test entries
27+
-d | --dry-run Don't actually run any tests
28+
-h | --help Show this usage info
29+
EOF
30+
exit $1
31+
}
32+
33+
COLLECTIONS=""
34+
TESTS=""
35+
dryrun=""
36+
while true; do
37+
case "$1" in
38+
-s | --summary)
39+
logfile="$BASE_DIR"/output.log
40+
cat /dev/null > $logfile
41+
shift ;;
42+
-t | --test)
43+
TESTS="$TESTS $2"
44+
shift 2 ;;
45+
-c | --collection)
46+
COLLECTIONS="$COLLECTIONS $2"
47+
shift 2 ;;
48+
-l | --list)
49+
echo "$available"
50+
exit 0 ;;
51+
-n | --dry-run)
52+
dryrun="echo"
53+
shift ;;
54+
-h | --help)
55+
usage 0 ;;
56+
"")
57+
break ;;
58+
*)
59+
usage 1 ;;
60+
esac
61+
done
62+
63+
# Add all selected collections to the explicit test list.
64+
if [ -n "$COLLECTIONS" ]; then
65+
for collection in $COLLECTIONS ; do
66+
found="$(echo "$available" | grep "^$collection:")"
67+
if [ -z "$found" ] ; then
68+
echo "No such collection '$collection'" >&2
69+
exit 1
70+
fi
71+
TESTS="$TESTS $found"
72+
done
73+
fi
74+
# Replace available test list with explicitly selected tests.
75+
if [ -n "$TESTS" ]; then
76+
valid=""
77+
for test in $TESTS ; do
78+
found="$(echo "$available" | grep "^${test}$")"
79+
if [ -z "$found" ] ; then
80+
echo "No such test '$test'" >&2
81+
exit 1
82+
fi
83+
valid="$valid $found"
84+
done
85+
available="$(echo "$valid" | sed -e 's/ /\n/g')"
86+
fi
87+
88+
collections=$(echo "$available" | cut -d: -f1 | uniq)
89+
for collection in $collections ; do
90+
[ -w /dev/kmsg ] && echo "kselftest: Running tests in $collection" >> /dev/kmsg
91+
tests=$(echo "$available" | grep "^$collection:" | cut -d: -f2)
92+
($dryrun cd "$collection" && $dryrun run_many $tests)
93+
done

0 commit comments

Comments
 (0)