Skip to content

Commit f7d3276

Browse files
committed
bpf: Have __bpf_trace_run() use rcu_read_lock_dont_migrate()
In order to switch the protection of tracepoint callbacks from preempt_disable() to srcu_read_lock_fast() the BPF callback from tracepoints needs to have migration prevention as the BPF programs expect to stay on the same CPU as they execute. Put together the RCU protection with migration prevention and use rcu_read_lock_dont_migrate() in __bpf_trace_run(). This will allow tracepoints callbacks to be preemptible. Link: https://lore.kernel.org/all/CAADnVQKvY026HSFGOsavJppm3-Ajm-VsLzY-OeFUe+BaKMRnDg@mail.gmail.com/ Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: "Paul E. McKenney" <paulmck@kernel.org> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20260126231256.335034877@kernel.org Suggested-by: Alexei Starovoitov <alexei.starovoitov@gmail.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
1 parent 1671827 commit f7d3276

1 file changed

Lines changed: 2 additions & 3 deletions

File tree

kernel/trace/bpf_trace.c

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2062,7 +2062,7 @@ void __bpf_trace_run(struct bpf_raw_tp_link *link, u64 *args)
20622062
struct bpf_run_ctx *old_run_ctx;
20632063
struct bpf_trace_run_ctx run_ctx;
20642064

2065-
cant_sleep();
2065+
rcu_read_lock_dont_migrate();
20662066
if (unlikely(this_cpu_inc_return(*(prog->active)) != 1)) {
20672067
bpf_prog_inc_misses_counter(prog);
20682068
goto out;
@@ -2071,13 +2071,12 @@ void __bpf_trace_run(struct bpf_raw_tp_link *link, u64 *args)
20712071
run_ctx.bpf_cookie = link->cookie;
20722072
old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx);
20732073

2074-
rcu_read_lock();
20752074
(void) bpf_prog_run(prog, args);
2076-
rcu_read_unlock();
20772075

20782076
bpf_reset_run_ctx(old_run_ctx);
20792077
out:
20802078
this_cpu_dec(*(prog->active));
2079+
rcu_read_unlock_migrate();
20812080
}
20822081

20832082
#define UNPACK(...) __VA_ARGS__

0 commit comments

Comments
 (0)