This reverts commit f783633cc8.
Bug: 128240262
Change-Id: I2d5961521043932a767f828432f28e3418d7a77f
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit f447ae0acd.
Bug: 128240262
Change-Id: I362db97f606279ce66add0b6c3726438e80a597f
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 1d0d2c5e71.
Bug: 128240262
Change-Id: Ie3d9a7f340ae93ef13475c41cbcb2b52f2ac720c
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 7224f45453.
Bug: 128240262
Change-Id: Ifdf06b5394bcfb111468fd7f220434a343fb48e8
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit f9eb248e24.
Bug: 128240262
Change-Id: Ia6ca7bfc525b9e238d49df4812e1b66307674198
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 662540931e.
Bug: 128240262
Change-Id: I7293939d08e6b9603d82464f4538a573b3407a88
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 06219e6c87.
Bug: 128240262
Change-Id: If31a4c81badd891e6ca5740dbd022b5edbe47254
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 728a43dfff.
Bug: 128240262
Change-Id: Ida9fb7e41a7905755470e20f5c72867bb3dad03f
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit ff6cfddf7a.
Bug: 128240262
Change-Id: I317785d46d335aab974e194e5ea0d744513baecd
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 5d1dccddd7.
Bug: 128240262
Change-Id: Iea60ab5360ae83ca0efcbb47a101b3870691a93f
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit b7efa36bf5.
Bug: 128240262
Change-Id: I21a63bb3e600beb422a8fc5240a4c3bed611fadc
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 2536f11b7d.
Bug: 128240262
Change-Id: I54aa09970c5967cfa7a93deca52f28773e27c041
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit ec5147dea1.
Bug: 128240262
Change-Id: Iadc1c38e82431020b79194aff4bb25e7d9d66125
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit d56126780c.
Bug: 128240262
Change-Id: Ia5a417e52de006fba4f8b1b51d9ae4db36fd9035
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 9db216b5c9.
Bug: 128240262
Change-Id: I210789054d394b929de6d9444040f1d2aac14917
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 6e9deb2ea7.
Bug: 128240262
Change-Id: I1060d8e43cebca1826a489fa3271279501c8647c
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 52e2e1f805.
Bug: 128240262
Change-Id: I96fe4f7e02f5d6716e20b1f8bc03b686dd189f09
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit f556cd74a7.
Bug: 128240262
Change-Id: Icc3edff62de261aee10658b1567b85ff7b5d58dd
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit 2842d9234e.
Bug: 128240262
Change-Id: I3ba5c0ab739af15061dd376fcb15366e9644ed4f
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit bcca0756b3.
Reason for revert: remove SPF non upstream code
Bug: 140544941
Test: boot
Change-Id: Id97756b85be0a1690e000fd24f125f21915d20da
Signed-off-by: Martin Liu <liumartin@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit ad3023acf9.
Bug: 128240262
Change-Id: I80f12ab9b25478a13b69d9c8fe7b46b3f65b1197
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit a9e3a1ab5d.
Reason for revert: remove SPF non upstream code
Bug: 140544941
Test: boot
Change-Id: I912a8891ac6cf3e72c7b7aa27df2922554b31491
Signed-off-by: Martin Liu <liumartin@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit c41742ec7c.
Bug: 128240262
Change-Id: Ide6a3bb060fd1792c1ac5880cfc2298931ad07ed
Signed-off-by: Minchan Kim <minchan@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
This reverts commit b37bae60c5.
Reason for revert: remove SPF non upstream code
Bug: 140544941
Test: boot
Change-Id: I466435dabfed767085934109a43bf7ca3da855a3
Signed-off-by: Martin Liu <liumartin@google.com>
[dereference23: Forward port to msm-5.4]
Signed-off-by: Alexander Winkowski <dereference23@outlook.com>
The previous logic was flawed, since it was limiting charge to 1A
when charging control was disabled. Default to thermal limit to mimic
what restrict_chg node does.
Change-Id: I18fb4f18ade276b561171f3217ddafa0e48a4555
There was an asterisk missing for one of the function parameters in the
upstreamed patch.
Fixes: e8ba376301a36 ("BACKPORT: FROMGIT: hugetlb: pass vma into
huge_pte_alloc() and huge_pmd_share()")
Signed-off-by: Lokesh Gidra <lokeshgidra@google.com>
Bug: 160737021
Bug: 169683130
Change-Id: I110563bc38e60a829fe7808f69dc0aa0f203a50e
With the more strict (but greatly simplified) page reuse logic in
do_wp_page(), we can safely go back to the world where cow is not
enforced with writes.
This essentially reverts commit 17839856fd58 ("gup: document and work
around 'COW can break either way' issue"). There are some context
differences due to some changes later on around it:
2170ecfa7688 ("drm/i915: convert get_user_pages() --> pin_user_pages()", 2020-06-03)
376a34efa4ee ("mm/gup: refactor and de-duplicate gup_fast() code", 2020-06-03)
Some lines moved back and forth with those, but this revert patch should
have striped out and covered all the enforced cow bits anyways.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
(cherry picked from commit a308c71bf1e6e19cc2e4ced31853ee0fc7cb439a)
Bug: 173684178
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Change-Id: Ib2308e4ff3f2e9283e5c93b096047cf276edf27f
In RHEL's gating selftests we've encountered memory corruption in the
uffd event test even with upstream kernel:
# ./userfaultfd anon 128 4
nr_pages: 32768, nr_pages_per_cpu: 32768
bounces: 3, mode: rnd racing read, userfaults: 6240 missing (6240) 14729 wp (14729)
bounces: 2, mode: racing read, userfaults: 1444 missing (1444) 28877 wp (28877)
bounces: 1, mode: rnd read, userfaults: 6055 missing (6055) 14699 wp (14699)
bounces: 0, mode: read, userfaults: 82 missing (82) 25196 wp (25196)
testing uffd-wp with pagemap (pgsize=4096): done
testing uffd-wp with pagemap (pgsize=2097152): done
testing events (fork, remap, remove): ERROR: nr 32427 memory corruption 0 1 (errno=0, line=963)
ERROR: faulting process failed (errno=0, line=1117)
It can be easily reproduced when global thp enabled, which is the
default for RHEL.
It's also known as a side effect of commit 0db282ba2c12 ("selftest: use
mmap instead of posix_memalign to allocate memory", 2021-07-23), which
is imho right itself on using mmap() to make sure the addresses will be
untagged even on arm.
The problem is, for each test we allocate buffers using two
allocate_area() calls. We assumed these two buffers won't affect each
other, however they could, because mmap() could have found that the two
buffers are near each other and having the same VMA flags, so they got
merged into one VMA.
It won't be a big problem if thp is not enabled, but when thp is
agressively enabled it means when initializing the src buffer it could
accidentally setup part of the dest buffer too when there's a shared THP
that overlaps the two regions. Then some of the dest buffer won't be
able to be trapped by userfaultfd missing mode, then it'll cause memory
corruption as described.
To fix it, do release_pages() after initializing the src buffer.
Since the previous two release_pages() calls are after
uffd_test_ctx_clear() which will unmap all the buffers anyway (which is
stronger than release pages; as unmap() also tear town pgtables), drop
them as they shouldn't really be anything useful.
We can mark the Fixes tag upon 0db282ba2c12 as it's reported to only
happen there, however the real "Fixes" IMHO should be 8ba6e8640844, as
before that commit we'll always do explicit release_pages() before
registration of uffd, and 8ba6e8640844 changed that logic by adding
extra unmap/map and we didn't release the pages at the right place.
Meanwhile I don't have a solid glue anyway on whether posix_memalign()
could always avoid triggering this bug, hence it's safer to attach this
fix to commit 8ba6e8640844.
Link: https://lkml.kernel.org/r/20210923232512.210092-1-peterx@redhat.com
Fixes: 8ba6e8640844 ("userfaultfd/selftests: reinitialize test context in each test")
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1994931
Signed-off-by: Peter Xu <peterx@redhat.com>
Reported-by: Li Wang <liwan@redhat.com>
Tested-by: Li Wang <liwang@redhat.com>
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
(cherry picked from commit 8913970c19915bbe773d97d42989cd85b7fdc098)
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Change-Id: I1378a52661d17c5cf6e8a0e84e3216556160c1b8
mfill_atomic_install_pte() checks page->mapping to detect whether one page
is used in the page cache. However as pointed out by Matthew, the page
can logically be a tail page rather than always the head in the case of
uffd minor mode with UFFDIO_CONTINUE. It means we could wrongly install
one pte with shmem thp tail page assuming it's an anonymous page.
It's not that clear even for anonymous page, since normally anonymous
pages also have page->mapping being setup with the anon vma. It's safe
here only because the only such caller to mfill_atomic_install_pte() is
always passing in a newly allocated page (mcopy_atomic_pte()), whose
page->mapping is not yet setup. However that's not extremely obvious
either.
For either of above, use page_mapping() instead.
Bug: 254441685
Link: https://lkml.kernel.org/r/Y2K+y7wnhC4vbnP2@x1n
Fixes: 153132571f02 ("userfaultfd/shmem: support UFFDIO_CONTINUE for shmem")
Signed-off-by: Peter Xu <peterx@redhat.com>
Reported-by: Matthew Wilcox <willy@infradead.org>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
(cherry picked from commit 93b0d9178743a68723babe8448981f658aebc58e)
Signed-off-by: Lee Jones <joneslee@google.com>
Change-Id: I03246130310cc7f3486843ed945ef92cab966cdc
Userfaultfd self-test fails occasionally, indicating a memory corruption.
Analyzing this problem indicates that there is a real bug since mmap_lock
is only taken for read in mwriteprotect_range() and defers flushes, and
since there is insufficient consideration of concurrent deferred TLB
flushes in wp_page_copy(). Although the PTE is flushed from the TLBs in
wp_page_copy(), this flush takes place after the copy has already been
performed, and therefore changes of the page are possible between the time
of the copy and the time in which the PTE is flushed.
To make matters worse, memory-unprotection using userfaultfd also poses a
problem. Although memory unprotection is logically a promotion of PTE
permissions, and therefore should not require a TLB flush, the current
userrfaultfd code might actually cause a demotion of the architectural PTE
permission: when userfaultfd_writeprotect() unprotects memory region, it
unintentionally *clears* the RW-bit if it was already set. Note that this
unprotecting a PTE that is not write-protected is a valid use-case: the
userfaultfd monitor might ask to unprotect a region that holds both
write-protected and write-unprotected PTEs.
The scenario that happens in selftests/vm/userfaultfd is as follows:
cpu0 cpu1 cpu2
---- ---- ----
[ Writable PTE
cached in TLB ]
userfaultfd_writeprotect()
[ write-*unprotect* ]
mwriteprotect_range()
mmap_read_lock()
change_protection()
change_protection_range()
...
change_pte_range()
[ *clear* “write”-bit ]
[ defer TLB flushes ]
[ page-fault ]
...
wp_page_copy()
cow_user_page()
[ copy page ]
[ write to old
page ]
...
set_pte_at_notify()
A similar scenario can happen:
cpu0 cpu1 cpu2 cpu3
---- ---- ---- ----
[ Writable PTE
cached in TLB ]
userfaultfd_writeprotect()
[ write-protect ]
[ deferred TLB flush ]
userfaultfd_writeprotect()
[ write-unprotect ]
[ deferred TLB flush]
[ page-fault ]
wp_page_copy()
cow_user_page()
[ copy page ]
... [ write to page ]
set_pte_at_notify()
This race exists since commit 292924b26024 ("userfaultfd: wp: apply
_PAGE_UFFD_WP bit"). Yet, as Yu Zhao pointed, these races became apparent
since commit 09854ba94c6a ("mm: do_wp_page() simplification") which made
wp_page_copy() more likely to take place, specifically if page_count(page)
> 1.
To resolve the aforementioned races, check whether there are pending
flushes on uffd-write-protected VMAs, and if there are, perform a flush
before doing the COW.
Further optimizations will follow to avoid during uffd-write-unprotect
unnecassary PTE write-protection and TLB flushes.
Bug: 254441685
Link: https://lkml.kernel.org/r/20210304095423.3825684-1-namit@vmware.com
Fixes: 09854ba94c6a ("mm: do_wp_page() simplification")
Signed-off-by: Nadav Amit <namit@vmware.com>
Suggested-by: Yu Zhao <yuzhao@google.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Tested-by: Peter Xu <peterx@redhat.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Pavel Emelyanov <xemul@openvz.org>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Will Deacon <will@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: <stable@vger.kernel.org> [5.9+]
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
(cherry picked from commit 6ce64428d62026a10cb5d80138ff2f90cc21d367)
Signed-off-by: Lee Jones <joneslee@google.com>
Change-Id: Ie35334aa739acfade88b74c9e5dde5c8a387925d
Missed calling flush_cache_range() before removing the sharing PMD
entrires, otherwise data consistence issue may be occurred on some
architectures whose caches are strict and require a virtual>physical
translation to exist for a virtual address. Thus add it.
Now no architectures enabling PMD sharing will be affected, since they do
not have a VIVT cache. That means this issue can not be happened in
practice so far.
Bug: 254441685
Link: https://lkml.kernel.org/r/47441086affcabb6ecbe403173e9283b0d904b38.1650956489.git.baolin.wang@linux.alibaba.com
Link: https://lkml.kernel.org/r/419b0e777c9e6d1454dcd906e0f5b752a736d335.1650781755.git.baolin.wang@linux.alibaba.com
Fixes: 6dfeaff93be1 ("hugetlb/userfaultfd: unshare all pmds for hugetlbfs when register wp")
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Muchun Song <songmuchun@bytedance.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
(cherry picked from commit 9c8bbfaca1bce84664403fd7dddbef6b3ff0a05a)
Signed-off-by: Lee Jones <joneslee@google.com>
Change-Id: Ib7d886d4f8bc18087771b9999bb7d9941a879581
