android_kernel_motorola_sm6375/mm
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Yin Fengwei b53d209d71 mm/thp: check and bail out if page in deferred queue already
commit 81e506bec9be1eceaf5a2c654e28ba5176ef48d8 upstream.

Kernel build regression with LLVM was reported here:
https://lore.kernel.org/all/Y1GCYXGtEVZbcv%2F5@dev-arch.thelio-3990X/ with
commit f35b5d7d676e ("mm: align larger anonymous mappings on THP
boundaries").  And the commit f35b5d7d676e was reverted.

It turned out the regression is related with madvise(MADV_DONTNEED)
was used by ld.lld. But with none PMD_SIZE aligned parameter len.
trace-bpfcc captured:
531607  531732  ld.lld          do_madvise.part.0 start: 0x7feca9000000, len: 0x7fb000, behavior: 0x4
531607  531793  ld.lld          do_madvise.part.0 start: 0x7fec86a00000, len: 0x7fb000, behavior: 0x4

If the underneath physical page is THP, the madvise(MADV_DONTNEED) can
trigger split_queue_lock contention raised significantly. perf showed
following data:
    14.85%     0.00%  ld.lld           [kernel.kallsyms]           [k]
       entry_SYSCALL_64_after_hwframe
           11.52%
                entry_SYSCALL_64_after_hwframe
                do_syscall_64
                __x64_sys_madvise
                do_madvise.part.0
                zap_page_range
                unmap_single_vma
                unmap_page_range
                page_remove_rmap
                deferred_split_huge_page
                __lock_text_start
                native_queued_spin_lock_slowpath

If THP can't be removed from rmap as whole THP, partial THP will be
removed from rmap by removing sub-pages from rmap.  Even the THP head page
is added to deferred queue already, the split_queue_lock will be acquired
and check whether the THP head page is in the queue already.  Thus, the
contention of split_queue_lock is raised.

Before acquire split_queue_lock, check and bail out early if the THP
head page is in the queue already. The checking without holding
split_queue_lock could race with deferred_split_scan, but it doesn't
impact the correctness here.

Test result of building kernel with ld.lld:
commit 7b5a0b664ebe (parent commit of f35b5d7d676e):
time -f "\t%E real,\t%U user,\t%S sys" make LD=ld.lld -skj96 allmodconfig all
        6:07.99 real,   26367.77 user,  5063.35 sys

commit f35b5d7d676e:
time -f "\t%E real,\t%U user,\t%S sys" make LD=ld.lld -skj96 allmodconfig all
        7:22.15 real,   26235.03 user,  12504.55 sys

commit f35b5d7d676e with the fixing patch:
time -f "\t%E real,\t%U user,\t%S sys" make LD=ld.lld -skj96 allmodconfig all
        6:08.49 real,   26520.15 user,  5047.91 sys

Link: https://lkml.kernel.org/r/20221223135207.2275317-1-fengwei.yin@intel.com
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Acked-by: David Rientjes <rientjes@google.com>
Reviewed-by: "Huang, Ying" <ying.huang@intel.com>
Cc: Feng Tang <feng.tang@intel.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Xing Zhengjun <zhengjun.xing@linux.intel.com>
Cc: Yang Shi <shy828301@gmail.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-03-11 16:44:05 +01:00
..
kasan panic: Consolidate open-coded panic_on_warn checks 2023-02-06 07:52:50 +01:00
backing-dev.c mm: bdi: initialize bdi_min_ratio when bdi is unregistered 2021-12-14 14:49:00 +01:00
balloon_compaction.c
cleancache.c
cma.c
cma.h
cma_debug.c
compaction.c mm, compaction: fix fast_isolate_around() to stay within boundaries 2023-01-18 11:41:44 +01:00
debug.c
debug_page_ref.c
dmapool.c
early_ioremap.c
fadvise.c
failslab.c
filemap.c mm: fs: initialize fsdata passed to write_begin/write_end interface 2022-11-25 17:42:22 +01:00
frame_vector.c v4l2: don't fall back to follow_pfn() if pin_user_pages_fast() fails 2022-12-08 11:23:06 +01:00
frontswap.c
gup.c mm/hugetlb: fix races when looking up a CONT-PTE/PMD size hugetlb page 2022-12-19 12:24:15 +01:00
gup_benchmark.c
highmem.c
hmm.c
huge_memory.c mm/thp: check and bail out if page in deferred queue already 2023-03-11 16:44:05 +01:00
hugetlb.c mm/hugetlb: fix races when looking up a CONT-PTE/PMD size hugetlb page 2022-12-19 12:24:15 +01:00
hugetlb_cgroup.c
hwpoison-inject.c
init-mm.c
internal.h mm/thp: fix vma_address() if virtual address below file offset 2021-06-30 08:47:52 -04:00
interval_tree.c
Kconfig
Kconfig.debug
khugepaged.c mm/khugepaged: fix collapse_pte_mapped_thp() to allow anon_vma 2023-01-24 07:18:01 +01:00
kmemleak-test.c
kmemleak.c Revert "mm: kmemleak: take a full lowmem check in kmemleak_*_phys()" 2022-09-15 12:04:49 +02:00
ksm.c
list_lru.c
maccess.c
madvise.c mm: fix madivse_pageout mishandling on non-LRU page 2022-10-05 10:37:43 +02:00
Makefile
memblock.c Revert "mm: Always release pages to the buddy allocator in memblock_free_late()." 2023-02-22 12:50:39 +01:00
memcontrol.c mm: memcontrol: deprecate charge moving 2023-03-11 16:44:05 +01:00
memfd.c memfd: fix F_SEAL_WRITE after shmem huge page allocated 2022-03-08 19:07:49 +01:00
memory-failure.c
memory.c mm: hugetlb: fix missing cache flush in copy_huge_page_from_user() 2022-05-15 19:54:47 +02:00
memory_hotplug.c mm/memory_hotplug: use "unsigned long" for PFN in zone_for_pfn_range() 2021-09-22 12:26:43 +02:00
mempolicy.c migrate: hugetlb: check for hugetlb shared PMD in node migration 2023-02-22 12:50:34 +01:00
mempool.c
memremap.c
memtest.c
migrate.c mm/migrate_device.c: flush TLB while holding PTL 2022-10-05 10:37:43 +02:00
mincore.c
mlock.c
mm_init.c
mmap.c mm: Fix TLB flush for not-first PFNMAP mappings in unmap_region() 2022-09-20 12:28:00 +02:00
mmu_context.c
mmu_gather.c mm/khugepaged: fix GUP-fast interaction by sending IPI 2022-12-14 11:30:42 +01:00
mmu_notifier.c
mmzone.c arm: remove CONFIG_ARCH_HAS_HOLES_MEMORYMODEL 2022-05-15 19:54:46 +02:00
mprotect.c
mremap.c mm/mremap: hold the rmap lock in write mode when moving page table entries. 2022-08-25 11:17:20 +02:00
msync.c
nommu.c
oom_kill.c oom_kill.c: futex: delay the OOM reaper to allow time for proper futex cleanup 2022-04-27 13:50:48 +02:00
page-writeback.c
page_alloc.c mm: prevent page_frag_alloc() from corrupting the memory 2022-10-05 10:37:43 +02:00
page_counter.c
page_ext.c
page_idle.c
page_io.c mm: fix unexpected zeroed page mapping with zram swap 2022-05-12 12:23:48 +02:00
page_isolation.c
page_owner.c
page_poison.c
page_vma_mapped.c mm/thp: another PVMW_SYNC fix in page_vma_mapped_walk() 2021-06-30 08:47:55 -04:00
pagewalk.c mm: pagewalk: Fix race between unmap and page walker 2022-10-15 07:54:36 +02:00
percpu-internal.h
percpu-km.c
percpu-stats.c
percpu-vm.c
percpu.c
pgtable-generic.c mm/thp: fix __split_huge_pmd_locked() on shmem migration entry 2021-06-30 08:47:52 -04:00
process_vm_access.c
readahead.c
rmap.c mm/rmap: Fix anon_vma->degree ambiguity leading to double-reuse 2022-09-05 10:27:46 +02:00
rodata_test.c
shmem.c shmem: fix a race between shmem_unused_huge_shrink and shmem_evict_inode 2022-01-27 09:19:29 +01:00
shuffle.c
shuffle.h
slab.c
slab.h mm: kmemleak: slob: respect SLAB_NOLEAKTRACE flag 2021-11-26 10:47:21 +01:00
slab_common.c mm: slab: fix kmem_cache_create failed when sysfs node not destroyed 2021-07-25 14:35:14 +02:00
slob.c
slub.c mm/slub: fix to return errno if kmalloc() fails 2022-09-28 11:04:04 +02:00
sparse-vmemmap.c
sparse.c
swap.c
swap_cgroup.c
swap_slots.c
swap_state.c
swapfile.c mm: swap: properly update readahead statistics in unuse_pte_range() 2023-02-22 12:50:29 +01:00
truncate.c mm/thp: unmap_mapping_page() to fix THP truncate_cleanup_page() 2021-06-30 08:47:53 -04:00
usercopy.c mm/usercopy: return 1 from hardened_usercopy __setup() handler 2022-04-15 14:18:30 +02:00
userfaultfd.c mm: userfaultfd: fix missing cache flush in mcopy_atomic_pte() and __mcopy_atomic() 2022-05-15 19:54:47 +02:00
util.c random: move randomize_page() into mm where it belongs 2022-06-22 14:11:17 +02:00
vmacache.c
vmalloc.c
vmpressure.c
vmscan.c mm,vmscan: fix divide by zero in get_scan_count 2021-09-22 12:26:37 +02:00
vmstat.c arm: remove CONFIG_ARCH_HAS_HOLES_MEMORYMODEL 2022-05-15 19:54:46 +02:00
workingset.c
z3fold.c mm/z3fold: fix potential memory leak in z3fold_destroy_pool() 2021-07-14 16:53:47 +02:00
zbud.c
zpool.c
zsmalloc.c zsmalloc: fix races between asynchronous zspage free and page migration 2022-06-06 08:33:50 +02:00
zswap.c