android_kernel_motorola_sm6375/include/asm-generic
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Jann Horn 48b00ceb54 mm/khugepaged: fix GUP-fast interaction by sending IPI
commit 2ba99c5e08812494bc57f319fb562f527d9bacd8 upstream.

Since commit 70cbc3cc78a99 ("mm: gup: fix the fast GUP race against THP
collapse"), the lockless_pages_from_mm() fastpath rechecks the pmd_t to
ensure that the page table was not removed by khugepaged in between.

However, lockless_pages_from_mm() still requires that the page table is
not concurrently freed.  Fix it by sending IPIs (if the architecture uses
semi-RCU-style page table freeing) before freeing/reusing page tables.

Link: https://lkml.kernel.org/r/20221129154730.2274278-2-jannh@google.com
Link: https://lkml.kernel.org/r/20221128180252.1684965-2-jannh@google.com
Link: https://lkml.kernel.org/r/20221125213714.4115729-2-jannh@google.com
Fixes: ba76149f47 ("thp: khugepaged")
Signed-off-by: Jann Horn <jannh@google.com>
Reviewed-by: Yang Shi <shy828301@gmail.com>
Acked-by: David Hildenbrand <david@redhat.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
[manual backport: two of the three places in khugepaged that can free
ptes were refactored into a common helper between 5.15 and 6.0;
TLB flushing was refactored between 5.4 and 5.10]
Signed-off-by: Jann Horn <jannh@google.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-12-14 11:30:42 +01:00
..
bitops locking/atomic: Make test_and_*_bit() ordered on failure 2022-08-25 11:18:31 +02:00
vdso lib/vdso: Make __arch_update_vdso_data() logic understandable 2020-03-05 16:43:49 +01:00
4level-fixup.h mm: make the __PAGETABLE_PxD_FOLDED defines non-empty 2018-11-02 08:31:52 +01:00
5level-fixup.h include/asm-generic/5level-fixup.h: fix variable 'p4d' set but not used 2019-08-13 16:06:52 -07:00
asm-offsets.h
asm-prototypes.h
atomic-instrumented.h locking/atomics: Check atomic headers with sha1sum 2019-02-13 08:07:31 +01:00
atomic-long.h locking/atomics: Check atomic headers with sha1sum 2019-02-13 08:07:31 +01:00
atomic.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36 2019-05-24 17:27:11 +02:00
atomic64.h locking/atomic: Use s64 for atomic64 2019-06-03 12:32:56 +02:00
audit_change_attr.h
audit_dir_write.h
audit_read.h
audit_signal.h
audit_write.h
barrier.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36 2019-05-24 17:27:11 +02:00
bitops-instrumented.h asm-generic, x86: add bitops instrumentation for KASAN 2019-07-12 11:05:42 -07:00
bitops.h
bitsperlong.h
bug.h bug: move WARN_ON() "cut here" into exception handler 2019-09-25 17:51:41 -07:00
bugs.h
cache.h
cacheflush.h asm-generic/nds32: don't redefine cacheflush primitives 2020-01-17 19:48:43 +01:00
checksum.h
cmpxchg-local.h
cmpxchg.h
compat.h asm-generic: Move common compat types to asm-generic/compat.h 2018-08-29 15:42:20 +02:00
current.h
delay.h
device.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 428 2019-06-05 17:37:16 +02:00
div64.h asm-generic changes for v5.4 2019-09-17 14:30:30 -07:00
dma-contiguous.h
dma-mapping.h dma-mapping: bypass indirect calls for dma-direct 2018-12-13 21:06:18 +01:00
dma.h
early_ioremap.h
emergency-restart.h
error-injection.h error-injection: Consolidate override function definition 2019-08-07 13:52:43 +01:00
exec.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36 2019-05-24 17:27:11 +02:00
export.h module: Remove leftover '#undef' from export header 2019-09-12 15:29:46 +02:00
extable.h
fb.h
fixmap.h mm: introduce common STRUCT_PAGE_MAX_SHIFT define 2018-12-14 15:05:45 -08:00
flat.h binfmt_flat: remove the persistent argument from flat_get_addr_from_rp 2019-06-24 09:16:47 +10:00
ftrace.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
futex.h futex: Cleanup generic SMP variant of arch_futex_atomic_op_inuser() 2019-07-22 11:20:10 +02:00
getorder.h asm-generic: fix -Wtype-limits compiler warnings 2019-08-03 07:02:01 -07:00
gpio.h
hardirq.h
hugetlb.h hugetlb: allow to free gigantic pages regardless of the configuration 2019-05-14 09:47:47 -07:00
hw_irq.h
ide_iops.h
int-ll64.h
io.h asm-generic: don't provide __ioremap 2019-08-30 21:46:27 +02:00
ioctl.h
iomap.h iomap: introduce io{read|write}64_{lo_hi|hi_lo} 2019-01-22 13:39:59 +01:00
irq.h
irq_regs.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
irq_work.h
irqflags.h
Kbuild asm-generic: make simd.h a mandatory include/asm header 2019-07-31 21:29:24 +10:00
kdebug.h
kmap_types.h
kprobes.h
kvm_para.h
linkage.h
local.h
local64.h
mcs_spinlock.h
memory_model.h
mm-arch-hooks.h
mm_hooks.h x86/mpx, mm/core: Fix recursive munmap() corruption 2019-05-09 10:37:17 +02:00
mmiowb.h asm-generic/mmiowb: Allow mmiowb_set_pending() when preemptible() 2020-07-29 10:18:40 +02:00
mmiowb_types.h asm-generic/mmiowb: Add generic implementation of mmiowb() tracking 2019-04-08 11:59:39 +01:00
mmu.h
mmu_context.h
module.h
mshyperv.h x86/Hyper-V: Report crash data in die() when panic_on_oops is set 2020-04-23 10:36:24 +02:00
msi.h
page.h asm-generic/page.h: fix typo in #error text requiring a real asm/page.h 2019-02-22 21:53:25 +01:00
param.h
parport.h
pci.h
pci_iomap.h
percpu.h percpu: remove PER_CPU_DEF_ATTRIBUTES macro 2018-10-31 08:54:14 -07:00
pgalloc.h mm: treewide: clarify pgtable_page_{ctor,dtor}() naming 2019-09-26 10:10:44 -07:00
pgtable-nop4d-hack.h x86/mm: Validate kernel_physical_mapping_init() PTE population 2018-12-05 09:03:06 +01:00
pgtable-nop4d.h x86/mm: Validate kernel_physical_mapping_init() PTE population 2018-12-05 09:03:06 +01:00
pgtable-nopmd.h mm: make the __PAGETABLE_PxD_FOLDED defines non-empty 2018-11-02 08:31:52 +01:00
pgtable-nopud.h x86/mm: Validate kernel_physical_mapping_init() PTE population 2018-12-05 09:03:06 +01:00
pgtable.h arch: pgtable: define MAX_POSSIBLE_PHYSMEM_BITS where needed 2020-12-02 08:49:50 +01:00
preempt.h sched/preempt: Use CONFIG_PREEMPTION where appropriate 2019-07-31 19:03:34 +02:00
qrwlock.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
qrwlock_types.h
qspinlock.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
qspinlock_types.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
resource.h
seccomp.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
sections.h asm-generic: sections: refactor memory_intersects 2022-09-05 10:27:43 +02:00
serial.h
set_memory.h
shmparam.h treewide: remove SPDX "WITH Linux-syscall-note" from kernel-space headers 2019-05-14 19:52:48 -07:00
signal.h
simd.h
spinlock.h
statfs.h
string.h
switch_to.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36 2019-05-24 17:27:11 +02:00
syscall.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 193 2019-05-30 11:29:21 -07:00
syscalls.h
termios-base.h
termios.h
timex.h
tlb.h mm/khugepaged: fix GUP-fast interaction by sending IPI 2022-12-14 11:30:42 +01:00
tlbflush.h
topology.h include/asm-generic/topology.h: guard cpumask_of_node() macro argument 2020-06-03 08:21:27 +02:00
trace_clock.h
uaccess.h asm-generic: optimize generic uaccess for 8-byte loads and stores 2019-04-23 21:51:41 +02:00
unaligned.h
user.h
vga.h
vmlinux.lds.h vmlinux.lds.h: Fix placement of '.data..decrypted' section 2022-11-25 17:42:09 +01:00
vtime.h
word-at-a-time.h
xor.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 47 2019-05-24 17:27:13 +02:00