Skip to content

Commit 8c49fba

Browse files
Kiryl Shutsemauakpm00
authored andcommitted
mm/rmap: mlock large folios in try_to_unmap_one()
Currently, try_to_unmap_once() only tries to mlock small folios. Use logic similar to folio_referenced_one() to mlock large folios: only do this for fully mapped folios and under page table lock that protects all page table entries. [akpm@linux-foundation.org: s/CROSSSED/CROSSED/] Link: https://lkml.kernel.org/r/20250923110711.690639-4-kirill@shutemov.name Signed-off-by: Kiryl Shutsemau <kas@kernel.org> Reviewed-by: Shakeel Butt <shakeel.butt@linux.dev> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: David Hildenbrand <david@redhat.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
1 parent a288020 commit 8c49fba

1 file changed

Lines changed: 28 additions & 3 deletions

File tree

mm/rmap.c

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1870,6 +1870,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
18701870
unsigned long nr_pages = 1, end_addr;
18711871
unsigned long pfn;
18721872
unsigned long hsz = 0;
1873+
int ptes = 0;
18731874

18741875
/*
18751876
* When racing against e.g. zap_pte_range() on another cpu,
@@ -1910,10 +1911,34 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
19101911
*/
19111912
if (!(flags & TTU_IGNORE_MLOCK) &&
19121913
(vma->vm_flags & VM_LOCKED)) {
1914+
ptes++;
1915+
1916+
/*
1917+
* Set 'ret' to indicate the page cannot be unmapped.
1918+
*
1919+
* Do not jump to walk_abort immediately as additional
1920+
* iteration might be required to detect fully mapped
1921+
* folio an mlock it.
1922+
*/
1923+
ret = false;
1924+
1925+
/* Only mlock fully mapped pages */
1926+
if (pvmw.pte && ptes != pvmw.nr_pages)
1927+
continue;
1928+
1929+
/*
1930+
* All PTEs must be protected by page table lock in
1931+
* order to mlock the page.
1932+
*
1933+
* If page table boundary has been cross, current ptl
1934+
* only protect part of ptes.
1935+
*/
1936+
if (pvmw.flags & PVMW_PGTABLE_CROSSED)
1937+
goto walk_done;
1938+
19131939
/* Restore the mlock which got missed */
1914-
if (!folio_test_large(folio))
1915-
mlock_vma_folio(folio, vma);
1916-
goto walk_abort;
1940+
mlock_vma_folio(folio, vma);
1941+
goto walk_done;
19171942
}
19181943

19191944
if (!pvmw.pte) {

0 commit comments

Comments
 (0)