BACKPORT: FROMGIT: hugetlb/userfaultfd: unshare all pmds for hugetlbfs when register wp

Huge pmd sharing for hugetlbfs is racy with userfaultfd-wp because
userfaultfd-wp is always based on pgtable entries, so they cannot be
shared.

Walk the hugetlb range and unshare all such mappings if there is, right
before UFFDIO_REGISTER will succeed and return to userspace.

This will pair with want_pmd_share() in hugetlb code so that huge pmd
sharing is completely disabled for userfaultfd-wp registered range.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Peter Xu <[email protected]>
Reviewed-by: Mike Kravetz <[email protected]>
Cc: Peter Xu <[email protected]>
Cc: Andrea Arcangeli <[email protected]>
Cc: Axel Rasmussen <[email protected]>
Cc: Mike Rapoport <[email protected]>
Cc: Kirill A. Shutemov <[email protected]>
Cc: Matthew Wilcox (Oracle) <[email protected]>
Cc: Adam Ruprecht <[email protected]>
Cc: Alexander Viro <[email protected]>
Cc: Alexey Dobriyan <[email protected]>
Cc: Anshuman Khandual <[email protected]>
Cc: Cannon Matthews <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Chinwen Chang <[email protected]>
Cc: David Rientjes <[email protected]>
Cc: "Dr . David Alan Gilbert" <[email protected]>
Cc: Huang Ying <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Jann Horn <[email protected]>
Cc: Jerome Glisse <[email protected]>
Cc: Lokesh Gidra <[email protected]>
Cc: Michael Ellerman <[email protected]>
Cc: "Michal Koutn" <[email protected]>
Cc: Michel Lespinasse <[email protected]>
Cc: Mina Almasry <[email protected]>
Cc: Nicholas Piggin <[email protected]>
Cc: Oliver Upton <[email protected]>
Cc: Shaohua Li <[email protected]>
Cc: Shawn Anastasio <[email protected]>
Cc: Steven Price <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Vlastimil Babka <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Stephen Rothwell <[email protected]>

(cherry picked from commit 267bda5c9993856b86f91a998df632b29cf517e2
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Link: https://lore.kernel.org/patchwork/patch/1382208/
Conflicts:
	mm/hugetlb.c

(CONFIG_CMA not CP'ed in this kernel. Manual rebase)
Signed-off-by: Lokesh Gidra <[email protected]>
Bug: 160737021
Bug: 169683130
Change-Id: I99d541ce45aaf924fa912f00dafa4caefe307755
This commit is contained in:
Peter Xu 2021-03-18 17:01:47 +11:00 • committed by Michael Bestas
commit 5f1b26c2cd
3 changed files with 59 additions and 0 deletions

View file

@ -15,6 +15,7 @@
#include <linux/sched/signal.h>
#include <linux/sched/mm.h>
#include <linux/mm.h>
#include <linux/mmu_notifier.h>
#include <linux/poll.h>
#include <linux/slab.h>
#include <linux/seq_file.h>
@ -1499,6 +1500,9 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
vma->vm_userfaultfd_ctx.ctx = ctx;
vm_write_end(vma);
if (is_vm_hugetlb_page(vma) && uffd_disable_huge_pmd_share(vma))
hugetlb_unshare_all_pmds(vma);
skip:
prev = vma;
start = vma->vm_end;