Skip to content

Commit 6b67c8a

Browse files
Shrikanth HegdePeter Zijlstra
authored andcommitted
sched/fair: Move checking for nohz cpus after time check
Current code does. - Read nohz.nr_cpus - Check if the time has passed to do NOHZ idle balance Instead do this. - Check if the time has passed to do NOHZ idle balance - Read nohz.nr_cpus This will skip the read most of the time in normal system usage. i.e when there are nohz.nr_cpus (system is not 100% busy). Note that when there are no idle CPUs(100% busy), even if the flag gets set to NOHZ_STATS_KICK | NOHZ_NEXT_KICK, find_new_ilb will fail and there will be no NOHZ idle balance. In such cases there will be a very narrow window where, kick_ilb will be called un-necessarily. However current functionality is still retained. Note: This patch doesn't solve any cacheline overheads. No improvement in performance apart from saving a few cycles of reading nohz.nr_cpus Reviewed-and-tested-by: K Prateek Nayak <kprateek.nayak@amd.com> Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org> Link: https://patch.msgid.link/20260115073524.376643-2-sshegde@linux.ibm.com
1 parent 553255c commit 6b67c8a

1 file changed

Lines changed: 16 additions & 7 deletions

File tree

kernel/sched/fair.c

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -12441,20 +12441,29 @@ static void nohz_balancer_kick(struct rq *rq)
1244112441
*/
1244212442
nohz_balance_exit_idle(rq);
1244312443

12444-
/*
12445-
* None are in tickless mode and hence no need for NOHZ idle load
12446-
* balancing:
12447-
*/
12448-
if (likely(!atomic_read(&nohz.nr_cpus)))
12449-
return;
12450-
1245112444
if (READ_ONCE(nohz.has_blocked_load) &&
1245212445
time_after(now, READ_ONCE(nohz.next_blocked)))
1245312446
flags = NOHZ_STATS_KICK;
1245412447

12448+
/*
12449+
* Most of the time system is not 100% busy. i.e nohz.nr_cpus > 0
12450+
* Skip the read if time is not due.
12451+
*
12452+
* If none are in tickless mode, there maybe a narrow window
12453+
* (28 jiffies, HZ=1000) where flags maybe set and kick_ilb called.
12454+
* But idle load balancing is not done as find_new_ilb fails.
12455+
* That's very rare. So read nohz.nr_cpus only if time is due.
12456+
*/
1245512457
if (time_before(now, nohz.next_balance))
1245612458
goto out;
1245712459

12460+
/*
12461+
* None are in tickless mode and hence no need for NOHZ idle load
12462+
* balancing:
12463+
*/
12464+
if (likely(!atomic_read(&nohz.nr_cpus)))
12465+
return;
12466+
1245812467
if (rq->nr_running >= 2) {
1245912468
flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK;
1246012469
goto out;

0 commit comments

Comments
 (0)