Skip to content

Commit 929bf01

Browse files
Li ZheAlex Williamson
authored andcommitted
mm: introduce num_pages_contiguous()
Let's add a simple helper for determining the number of contiguous pages that represent contiguous PFNs. In an ideal world, this helper would be simpler or not even required. Unfortunately, on some configs we still have to maintain (SPARSEMEM without VMEMMAP), the memmap is allocated per memory section, and we might run into weird corner cases of false positives when blindly testing for contiguous pages only. One example of such false positives would be a memory section-sized hole that does not have a memmap. The surrounding memory sections might get "struct pages" that are contiguous, but the PFNs are actually not. This helper will, for example, be useful for determining contiguous PFNs in a GUP result, to batch further operations across returned "struct page"s. VFIO will utilize this interface to accelerate the VFIO DMA map process. Implementation based on Linus' suggestions to avoid new usage of nth_page() where avoidable. Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Suggested-by: Jason Gunthorpe <jgg@ziepe.ca> Signed-off-by: Li Zhe <lizhe.67@bytedance.com> Co-developed-by: David Hildenbrand <david@redhat.com> Signed-off-by: David Hildenbrand <david@redhat.com> Link: https://lore.kernel.org/r/20250814064714.56485-2-lizhe.67@bytedance.com Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
1 parent fd94619 commit 929bf01

2 files changed

Lines changed: 42 additions & 1 deletion

File tree

include/linux/mm.h

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1833,7 +1833,12 @@ static inline unsigned long memdesc_section(memdesc_flags_t mdf)
18331833
{
18341834
return (mdf.f >> SECTIONS_PGSHIFT) & SECTIONS_MASK;
18351835
}
1836-
#endif
1836+
#else /* !SECTION_IN_PAGE_FLAGS */
1837+
static inline unsigned long memdesc_section(memdesc_flags_t mdf)
1838+
{
1839+
return 0;
1840+
}
1841+
#endif /* SECTION_IN_PAGE_FLAGS */
18371842

18381843
/**
18391844
* folio_pfn - Return the Page Frame Number of a folio.

include/linux/mm_inline.h

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -617,4 +617,40 @@ static inline bool vma_has_recency(const struct vm_area_struct *vma)
617617
return true;
618618
}
619619

620+
/**
621+
* num_pages_contiguous() - determine the number of contiguous pages
622+
* that represent contiguous PFNs
623+
* @pages: an array of page pointers
624+
* @nr_pages: length of the array, at least 1
625+
*
626+
* Determine the number of contiguous pages that represent contiguous PFNs
627+
* in @pages, starting from the first page.
628+
*
629+
* In some kernel configs contiguous PFNs will not have contiguous struct
630+
* pages. In these configurations num_pages_contiguous() will return a num
631+
* smaller than ideal number. The caller should continue to check for pfn
632+
* contiguity after each call to num_pages_contiguous().
633+
*
634+
* Returns the number of contiguous pages.
635+
*/
636+
static inline size_t num_pages_contiguous(struct page **pages, size_t nr_pages)
637+
{
638+
struct page *cur_page = pages[0];
639+
unsigned long section = memdesc_section(cur_page->flags);
640+
size_t i;
641+
642+
for (i = 1; i < nr_pages; i++) {
643+
if (++cur_page != pages[i])
644+
break;
645+
/*
646+
* In unproblematic kernel configs, page_to_section() == 0 and
647+
* the whole check will get optimized out.
648+
*/
649+
if (memdesc_section(cur_page->flags) != section)
650+
break;
651+
}
652+
653+
return i;
654+
}
655+
620656
#endif

0 commit comments

Comments
 (0)