Skip to content

Commit 1ace9ba

Browse files
Alexei Starovoitovanakryiko
authored andcommitted
bpf: Prevent reentrance into call_rcu_tasks_trace()
call_rcu_tasks_trace() is not safe from in_nmi() and not reentrant. To prevent deadlock on raw_spin_lock_rcu_node(rtpcp) or memory corruption defer to irq_work when IRQs are disabled. call_rcu_tasks_generic() protects itself with local_irq_save(). Note when bpf_async_cb->refcnt drops to zero it's safe to reuse bpf_async_cb->worker for a different irq_work callback, since bpf_async_schedule_op() -> irq_work_queue(&cb->worker); is only called when refcnt >= 1. Fixes: 1bfbc26 ("bpf: Enable bpf_timer and bpf_wq in any context") Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260205190233.912-1-alexei.starovoitov@gmail.com
1 parent a2c86aa commit 1ace9ba

1 file changed

Lines changed: 13 additions & 1 deletion

File tree

kernel/bpf/helpers.c

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1276,12 +1276,24 @@ static void bpf_async_cb_rcu_tasks_trace_free(struct rcu_head *rcu)
12761276
bpf_async_cb_rcu_free(rcu);
12771277
}
12781278

1279+
static void worker_for_call_rcu(struct irq_work *work)
1280+
{
1281+
struct bpf_async_cb *cb = container_of(work, struct bpf_async_cb, worker);
1282+
1283+
call_rcu_tasks_trace(&cb->rcu, bpf_async_cb_rcu_tasks_trace_free);
1284+
}
1285+
12791286
static void bpf_async_refcount_put(struct bpf_async_cb *cb)
12801287
{
12811288
if (!refcount_dec_and_test(&cb->refcnt))
12821289
return;
12831290

1284-
call_rcu_tasks_trace(&cb->rcu, bpf_async_cb_rcu_tasks_trace_free);
1291+
if (irqs_disabled()) {
1292+
cb->worker = IRQ_WORK_INIT(worker_for_call_rcu);
1293+
irq_work_queue(&cb->worker);
1294+
} else {
1295+
call_rcu_tasks_trace(&cb->rcu, bpf_async_cb_rcu_tasks_trace_free);
1296+
}
12851297
}
12861298

12871299
static void bpf_async_cancel_and_free(struct bpf_async_kern *async);

0 commit comments

Comments
 (0)