Skip to content

Commit 8a1968b

Browse files
ryncsnakpm00
authored andcommitted
mm/shmem, swap: fix race of truncate and swap entry split
The helper for shmem swap freeing is not handling the order of swap entries correctly. It uses xa_cmpxchg_irq to erase the swap entry, but it gets the entry order before that using xa_get_order without lock protection, and it may get an outdated order value if the entry is split or changed in other ways after the xa_get_order and before the xa_cmpxchg_irq. And besides, the order could grow and be larger than expected, and cause truncation to erase data beyond the end border. For example, if the target entry and following entries are swapped in or freed, then a large folio was added in place and swapped out, using the same entry, the xa_cmpxchg_irq will still succeed, it's very unlikely to happen though. To fix that, open code the Xarray cmpxchg and put the order retrieval and value checking in the same critical section. Also, ensure the order won't exceed the end border, skip it if the entry goes across the border. Skipping large swap entries crosses the end border is safe here. Shmem truncate iterates the range twice, in the first iteration, find_lock_entries already filtered such entries, and shmem will swapin the entries that cross the end border and partially truncate the folio (split the folio or at least zero part of it). So in the second loop here, if we see a swap entry that crosses the end order, it must at least have its content erased already. I observed random swapoff hangs and kernel panics when stress testing ZSWAP with shmem. After applying this patch, all problems are gone. Link: https://lkml.kernel.org/r/20260120-shmem-swap-fix-v3-1-3d33ebfbc057@tencent.com Fixes: 809bc86 ("mm: shmem: support large folio swap out") Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Nhat Pham <nphamcs@gmail.com> Acked-by: Chris Li <chrisl@kernel.org> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Baoquan He <bhe@redhat.com> Cc: Barry Song <baohua@kernel.org> Cc: Hugh Dickins <hughd@google.com> Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
1 parent 6c79021 commit 8a1968b

1 file changed

Lines changed: 34 additions & 11 deletions

File tree

mm/shmem.c

Lines changed: 34 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -962,17 +962,29 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap)
962962
* being freed).
963963
*/
964964
static long shmem_free_swap(struct address_space *mapping,
965-
pgoff_t index, void *radswap)
965+
pgoff_t index, pgoff_t end, void *radswap)
966966
{
967-
int order = xa_get_order(&mapping->i_pages, index);
968-
void *old;
967+
XA_STATE(xas, &mapping->i_pages, index);
968+
unsigned int nr_pages = 0;
969+
pgoff_t base;
970+
void *entry;
969971

970-
old = xa_cmpxchg_irq(&mapping->i_pages, index, radswap, NULL, 0);
971-
if (old != radswap)
972-
return 0;
973-
free_swap_and_cache_nr(radix_to_swp_entry(radswap), 1 << order);
972+
xas_lock_irq(&xas);
973+
entry = xas_load(&xas);
974+
if (entry == radswap) {
975+
nr_pages = 1 << xas_get_order(&xas);
976+
base = round_down(xas.xa_index, nr_pages);
977+
if (base < index || base + nr_pages - 1 > end)
978+
nr_pages = 0;
979+
else
980+
xas_store(&xas, NULL);
981+
}
982+
xas_unlock_irq(&xas);
983+
984+
if (nr_pages)
985+
free_swap_and_cache_nr(radix_to_swp_entry(radswap), nr_pages);
974986

975-
return 1 << order;
987+
return nr_pages;
976988
}
977989

978990
/*
@@ -1124,8 +1136,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,
11241136
if (xa_is_value(folio)) {
11251137
if (unfalloc)
11261138
continue;
1127-
nr_swaps_freed += shmem_free_swap(mapping,
1128-
indices[i], folio);
1139+
nr_swaps_freed += shmem_free_swap(mapping, indices[i],
1140+
end - 1, folio);
11291141
continue;
11301142
}
11311143

@@ -1191,12 +1203,23 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,
11911203
folio = fbatch.folios[i];
11921204

11931205
if (xa_is_value(folio)) {
1206+
int order;
11941207
long swaps_freed;
11951208

11961209
if (unfalloc)
11971210
continue;
1198-
swaps_freed = shmem_free_swap(mapping, indices[i], folio);
1211+
swaps_freed = shmem_free_swap(mapping, indices[i],
1212+
end - 1, folio);
11991213
if (!swaps_freed) {
1214+
/*
1215+
* If found a large swap entry cross the end border,
1216+
* skip it as the truncate_inode_partial_folio above
1217+
* should have at least zerod its content once.
1218+
*/
1219+
order = shmem_confirm_swap(mapping, indices[i],
1220+
radix_to_swp_entry(folio));
1221+
if (order > 0 && indices[i] + (1 << order) > end)
1222+
continue;
12001223
/* Swap was replaced by page: retry */
12011224
index = indices[i];
12021225
break;

0 commit comments

Comments
 (0)