Skip to content

Commit 4b038a9

Browse files
hao-leetehcaster
authored andcommitted
slub: keep empty main sheaf as spare in __pcs_replace_empty_main()
When __pcs_replace_empty_main() fails to obtain a full sheaf directly from the barn, it may either: - Refill an empty sheaf obtained via barn_get_empty_sheaf(), or - Allocate a brand new full sheaf via alloc_full_sheaf(). After reacquiring the per-CPU lock, if pcs->main is still empty and pcs->spare is NULL, the current code donates the empty main sheaf to the barn via barn_put_empty_sheaf() and installs the full sheaf as pcs->main, leaving pcs->spare unpopulated. Instead, keep the existing empty main sheaf locally as the spare: pcs->spare = pcs->main; pcs->main = full; This populates pcs->spare earlier, which can reduce future barn traffic. Suggested-by: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Hao Li <haolee.swjtu@gmail.com> Reviewed-by: Harry Yoo <harry.yoo@oracle.com> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Tested-by: Zhao Liu <zhao1.liu@intel.com>
1 parent d907bf4 commit 4b038a9

1 file changed

Lines changed: 4 additions & 1 deletion

File tree

mm/slub.c

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5049,7 +5049,10 @@ __pcs_replace_empty_main(struct kmem_cache *s, struct slub_percpu_sheaves *pcs,
50495049
*/
50505050

50515051
if (pcs->main->size == 0) {
5052-
barn_put_empty_sheaf(barn, pcs->main);
5052+
if (!pcs->spare)
5053+
pcs->spare = pcs->main;
5054+
else
5055+
barn_put_empty_sheaf(barn, pcs->main);
50535056
pcs->main = full;
50545057
return pcs;
50555058
}

0 commit comments

Comments
 (0)