@@ -5,143 +5,115 @@ Dynamic debug
55Introduction
66============
77
8- This document describes how to use the dynamic debug (dyndbg) feature.
8+ Dynamic debug allows you to dynamically enable/disable kernel
9+ debug-print code to obtain additional kernel information.
910
10- Dynamic debug is designed to allow you to dynamically enable/disable
11- kernel code to obtain additional kernel information. Currently, if
12- ``CONFIG_DYNAMIC_DEBUG `` is set, then all ``pr_debug() ``/``dev_dbg() `` and
13- ``print_hex_dump_debug() ``/``print_hex_dump_bytes() `` calls can be dynamically
14- enabled per-callsite.
11+ If ``/proc/dynamic_debug/control `` exists, your kernel has dynamic
12+ debug. You'll need root access (sudo su) to use this.
1513
16- If you do not want to enable dynamic debug globally (i.e. in some embedded
17- system), you may set ``CONFIG_DYNAMIC_DEBUG_CORE `` as basic support of dynamic
18- debug and add ``ccflags := -DDYNAMIC_DEBUG_MODULE `` into the Makefile of any
19- modules which you'd like to dynamically debug later.
20-
21- If ``CONFIG_DYNAMIC_DEBUG `` is not set, ``print_hex_dump_debug() `` is just
22- shortcut for ``print_hex_dump(KERN_DEBUG) ``.
23-
24- For ``print_hex_dump_debug() ``/``print_hex_dump_bytes() ``, format string is
25- its ``prefix_str `` argument, if it is constant string; or ``hexdump ``
26- in case ``prefix_str `` is built dynamically.
14+ Dynamic debug provides:
2715
28- Dynamic debug has even more useful features:
16+ * a Catalog of all *prdbgs * in your kernel.
17+ ``cat /proc/dynamic_debug/control `` to see them.
2918
30- * Simple query language allows turning on and off debugging
31- statements by matching any combination of 0 or 1 of:
19+ * a Simple query/command language to alter * prdbgs * by selecting on
20+ any combination of 0 or 1 of:
3221
3322 - source filename
3423 - function name
3524 - line number (including ranges of line numbers)
3625 - module name
3726 - format string
38-
39- * Provides a debugfs control file: ``<debugfs>/dynamic_debug/control ``
40- which can be read to display the complete list of known debug
41- statements, to help guide you
42-
43- Controlling dynamic debug Behaviour
44- ===================================
45-
46- The behaviour of ``pr_debug() ``/``dev_dbg() `` are controlled via writing to a
47- control file in the 'debugfs' filesystem. Thus, you must first mount
48- the debugfs filesystem, in order to make use of this feature.
49- Subsequently, we refer to the control file as:
50- ``<debugfs>/dynamic_debug/control ``. For example, if you want to enable
51- printing from source file ``svcsock.c ``, line 1603 you simply do::
52-
53- nullarbor:~ # echo 'file svcsock.c line 1603 +p' >
54- <debugfs>/dynamic_debug/control
55-
56- If you make a mistake with the syntax, the write will fail thus::
57-
58- nullarbor:~ # echo 'file svcsock.c wtf 1 +p' >
59- <debugfs>/dynamic_debug/control
60- -bash: echo: write error: Invalid argument
61-
62- Note, for systems without 'debugfs' enabled, the control file can be
63- found in ``/proc/dynamic_debug/control ``.
27+ - class name (as known/declared by each module)
6428
6529Viewing Dynamic Debug Behaviour
6630===============================
6731
68- You can view the currently configured behaviour of all the debug
69- statements via::
32+ You can view the currently configured behaviour in the *prdbg * catalog::
7033
71- nullarbor:~ # cat <debugfs> /dynamic_debug/control
34+ :#> head -n7 /proc /dynamic_debug/control
7235 # filename:lineno [module]function flags format
73- net/sunrpc/svc_rdma.c:323 [svcxprt_rdma]svc_rdma_cleanup =_ "SVCRDMA Module Removed, deregister RPC RDMA transport\012"
74- net/sunrpc/svc_rdma.c:341 [svcxprt_rdma]svc_rdma_init =_ "\011max_inline : %d\012"
75- net/sunrpc/svc_rdma.c:340 [svcxprt_rdma]svc_rdma_init =_ "\011sq_depth : %d\012"
76- net/sunrpc/svc_rdma.c:338 [svcxprt_rdma]svc_rdma_init =_ "\011max_requests : %d\012"
77- ...
36+ init/main.c:1179 [main]initcall_blacklist =_ "blacklisting initcall %s\012
37+ init/main.c:1218 [main]initcall_blacklisted =_ "initcall %s blacklisted\012"
38+ init/main.c:1424 [main]run_init_process =_ " with arguments:\012"
39+ init/main.c:1426 [main]run_init_process =_ " %s\012"
40+ init/main.c:1427 [main]run_init_process =_ " with environment:\012"
41+ init/main.c:1429 [main]run_init_process =_ " %s\012"
7842
43+ The 3rd space-delimited column shows the current flags, preceded by
44+ a ``= `` for easy use with grep/cut. ``=p `` shows enabled callsites.
7945
80- You can also apply standard Unix text manipulation filters to this
81- data, e.g.::
46+ Controlling dynamic debug Behaviour
47+ ===================================
8248
83- nullarbor:~ # grep -i rdma <debugfs>/dynamic_debug/control | wc -l
84- 62
49+ The behaviour of * prdbg * sites are controlled by writing
50+ query/commands to the control file. Example::
8551
86- nullarbor:~ # grep -i tcp <debugfs>/dynamic_debug/control | wc -l
87- 42
52+ # grease the interface
53+ :#> alias ddcmd='echo $* > /proc/dynamic_debug/control'
8854
89- The third column shows the currently enabled flags for each debug
90- statement callsite (see below for definitions of the flags). The
91- default value, with no flags enabled, is ``=_ ``. So you can view all
92- the debug statement callsites with any non-default flags::
55+ :#> ddcmd '-p; module main func run* +p'
56+ :#> grep =p /proc/dynamic_debug/control
57+ init/main.c:1424 [main]run_init_process =p " with arguments:\012"
58+ init/main.c:1426 [main]run_init_process =p " %s\012"
59+ init/main.c:1427 [main]run_init_process =p " with environment:\012"
60+ init/main.c:1429 [main]run_init_process =p " %s\012"
9361
94- nullarbor:~ # awk '$3 != "=_"' <debugfs>/dynamic_debug/control
95- # filename:lineno [module]function flags format
96- net/sunrpc/svcsock.c:1603 [sunrpc]svc_send p "svc_process: st_sendto returned %d\012"
62+ Error messages go to console/syslog::
63+
64+ :#> ddcmd mode foo +p
65+ dyndbg: unknown keyword "mode"
66+ dyndbg: query parse failed
67+ bash: echo: write error: Invalid argument
68+
69+ If debugfs is also enabled and mounted, ``dynamic_debug/control `` is
70+ also under the mount-dir, typically ``/sys/kernel/debug/ ``.
9771
9872Command Language Reference
9973==========================
10074
101- At the lexical level, a command comprises a sequence of words separated
75+ At the basic lexical level, a command is a sequence of words separated
10276by spaces or tabs. So these are all equivalent::
10377
104- nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
105- <debugfs>/dynamic_debug/control
106- nullarbor:~ # echo -n ' file svcsock.c line 1603 +p ' >
107- <debugfs>/dynamic_debug/control
108- nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
109- <debugfs>/dynamic_debug/control
78+ :#> ddcmd file svcsock.c line 1603 +p
79+ :#> ddcmd "file svcsock.c line 1603 +p"
80+ :#> ddcmd ' file svcsock.c line 1603 +p '
11081
11182Command submissions are bounded by a write() system call.
11283Multiple commands can be written together, separated by ``; `` or ``\n ``::
11384
114- ~# echo "func pnpacpi_get_resources +p; func pnp_assign_mem +p" \
115- > <debugfs>/dynamic_debug/control
85+ :#> ddcmd "func pnpacpi_get_resources +p; func pnp_assign_mem +p"
86+ :#> ddcmd <<"EOC"
87+ func pnpacpi_get_resources +p
88+ func pnp_assign_mem +p
89+ EOC
90+ :#> cat query-batch-file > /proc/dynamic_debug/control
11691
117- If your query set is big, you can batch them too::
92+ You can also use wildcards in each query term. The match rule supports
93+ ``* `` (matches zero or more characters) and ``? `` (matches exactly one
94+ character). For example, you can match all usb drivers::
11895
119- ~# cat query-batch- file > <debugfs>/dynamic_debug/control
96+ :#> ddcmd file "drivers/usb/*" +p # "" to suppress shell expansion
12097
121- Another way is to use wildcards. The match rule supports ``* `` (matches
122- zero or more characters) and ``? `` (matches exactly one character). For
123- example, you can match all usb drivers::
124-
125- ~# echo "file drivers/usb/* +p" > <debugfs>/dynamic_debug/control
126-
127- At the syntactical level, a command comprises a sequence of match
128- specifications, followed by a flags change specification::
98+ Syntactically, a command is pairs of keyword values, followed by a
99+ flags change or setting::
129100
130101 command ::= match-spec* flags-spec
131102
132- The match-spec's are used to choose a subset of the known pr_debug()
133- callsites to which to apply the flags-spec. Think of them as a query
134- with implicit ANDs between each pair. Note that an empty list of
135- match-specs will select all debug statement callsites.
103+ The match-spec's select * prdbgs * from the catalog, upon which to apply
104+ the flags-spec, all constraints are ANDed together. An absent keyword
105+ is the same as keyword "*".
106+
136107
137- A match specification comprises a keyword, which controls the
138- attribute of the callsite to be compared, and a value to compare
139- against. Possible keywords are:::
108+ A match specification is a keyword, which selects the attribute of
109+ the callsite to be compared, and a value to compare against. Possible
110+ keywords are:::
140111
141112 match-spec ::= 'func' string |
142113 'file' string |
143114 'module' string |
144115 'format' string |
116+ 'class' string |
145117 'line' line-range
146118
147119 line-range ::= lineno |
@@ -203,6 +175,16 @@ format
203175 format "nfsd: SETATTR" // a neater way to match a format with whitespace
204176 format 'nfsd: SETATTR' // yet another way to match a format with whitespace
205177
178+ class
179+ The given class_name is validated against each module, which may
180+ have declared a list of known class_names. If the class_name is
181+ found for a module, callsite & class matching and adjustment
182+ proceeds. Examples::
183+
184+ class DRM_UT_KMS # a DRM.debug category
185+ class JUNK # silent non-match
186+ // class TLD_* # NOTICE: no wildcard in class names
187+
206188line
207189 The given line number or range of line numbers is compared
208190 against the line number of each ``pr_debug() `` callsite. A single
@@ -228,17 +210,16 @@ of the characters::
228210The flags are::
229211
230212 p enables the pr_debug() callsite.
231- f Include the function name in the printed message
232- l Include line number in the printed message
233- m Include module name in the printed message
234- t Include thread ID in messages not generated from interrupt context
235- _ No flags are set. (Or'd with others on input)
213+ _ enables no flags.
236214
237- For ``print_hex_dump_debug() `` and ``print_hex_dump_bytes() ``, only ``p `` flag
238- have meaning, other flags ignored.
215+ Decorator flags add to the message-prefix, in order:
216+ t Include thread ID, or <intr>
217+ m Include module name
218+ f Include the function name
219+ l Include line number
239220
240- For display, the flags are preceded by `` = ``
241- (mnemonic: what the flags are currently equal to) .
221+ For `` print_hex_dump_debug() `` and `` print_hex_dump_bytes() ``, only
222+ the `` p `` flag has meaning, other flags are ignored .
242223
243224Note the regexp ``^[-+=][flmpt_]+$ `` matches a flags specification.
244225To clear all flags at once, use ``=_ `` or ``-flmpt ``.
@@ -313,45 +294,39 @@ For ``CONFIG_DYNAMIC_DEBUG`` kernels, any settings given at boot-time (or
313294enabled by ``-DDEBUG `` flag during compilation) can be disabled later via
314295the debugfs interface if the debug messages are no longer needed::
315296
316- echo "module module_name -p" > <debugfs> /dynamic_debug/control
297+ echo "module module_name -p" > /proc /dynamic_debug/control
317298
318299Examples
319300========
320301
321302::
322303
323304 // enable the message at line 1603 of file svcsock.c
324- nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
325- <debugfs>/dynamic_debug/control
305+ :#> ddcmd 'file svcsock.c line 1603 +p'
326306
327307 // enable all the messages in file svcsock.c
328- nullarbor:~ # echo -n 'file svcsock.c +p' >
329- <debugfs>/dynamic_debug/control
308+ :#> ddcmd 'file svcsock.c +p'
330309
331310 // enable all the messages in the NFS server module
332- nullarbor:~ # echo -n 'module nfsd +p' >
333- <debugfs>/dynamic_debug/control
311+ :#> ddcmd 'module nfsd +p'
334312
335313 // enable all 12 messages in the function svc_process()
336- nullarbor:~ # echo -n 'func svc_process +p' >
337- <debugfs>/dynamic_debug/control
314+ :#> ddcmd 'func svc_process +p'
338315
339316 // disable all 12 messages in the function svc_process()
340- nullarbor:~ # echo -n 'func svc_process -p' >
341- <debugfs>/dynamic_debug/control
317+ :#> ddcmd 'func svc_process -p'
342318
343319 // enable messages for NFS calls READ, READLINK, READDIR and READDIR+.
344- nullarbor:~ # echo -n 'format "nfsd: READ" +p' >
345- <debugfs>/dynamic_debug/control
320+ :#> ddcmd 'format "nfsd: READ" +p'
346321
347322 // enable messages in files of which the paths include string "usb"
348- nullarbor:~ # echo -n 'file *usb* +p' > <debugfs> /dynamic_debug/control
323+ :#> ddcmd 'file *usb* +p' > /proc /dynamic_debug/control
349324
350325 // enable all messages
351- nullarbor:~ # echo -n '+p' > <debugfs> /dynamic_debug/control
326+ :#> ddcmd '+p' > /proc /dynamic_debug/control
352327
353328 // add module, function to all enabled messages
354- nullarbor:~ # echo -n '+mf' > <debugfs> /dynamic_debug/control
329+ :#> ddcmd '+mf' > /proc /dynamic_debug/control
355330
356331 // boot-args example, with newlines and comments for readability
357332 Kernel command line: ...
@@ -364,3 +339,38 @@ Examples
364339 dyndbg="file init/* +p #cmt ; func parse_one +p"
365340 // enable pr_debugs in 2 functions in a module loaded later
366341 pc87360.dyndbg="func pc87360_init_device +p; func pc87360_find +p"
342+
343+ Kernel Configuration
344+ ====================
345+
346+ Dynamic Debug is enabled via kernel config items::
347+
348+ CONFIG_DYNAMIC_DEBUG=y # build catalog, enables CORE
349+ CONFIG_DYNAMIC_DEBUG_CORE=y # enable mechanics only, skip catalog
350+
351+ If you do not want to enable dynamic debug globally (i.e. in some embedded
352+ system), you may set ``CONFIG_DYNAMIC_DEBUG_CORE `` as basic support of dynamic
353+ debug and add ``ccflags := -DDYNAMIC_DEBUG_MODULE `` into the Makefile of any
354+ modules which you'd like to dynamically debug later.
355+
356+
357+ Kernel *prdbg * API
358+ ==================
359+
360+ The following functions are cataloged and controllable when dynamic
361+ debug is enabled::
362+
363+ pr_debug()
364+ dev_dbg()
365+ print_hex_dump_debug()
366+ print_hex_dump_bytes()
367+
368+ Otherwise, they are off by default; ``ccflags += -DDEBUG `` or
369+ ``#define DEBUG `` in a source file will enable them appropriately.
370+
371+ If ``CONFIG_DYNAMIC_DEBUG `` is not set, ``print_hex_dump_debug() `` is
372+ just a shortcut for ``print_hex_dump(KERN_DEBUG) ``.
373+
374+ For ``print_hex_dump_debug() ``/``print_hex_dump_bytes() ``, format string is
375+ its ``prefix_str `` argument, if it is constant string; or ``hexdump ``
376+ in case ``prefix_str `` is built dynamically.
0 commit comments