Rik van Riel
2ff6ddf19c
x86/mm/tlb: Leave lazy TLB mode at page table free time
...
Andy discovered that speculative memory accesses while in lazy
TLB mode can crash a system, when a CPU tries to dereference a
speculative access using memory contents that used to be valid
page table memory, but have since been reused for something else
and point into la-la land.
The latter problem can be prevented in two ways. The first is to
always send a TLB shootdown IPI to CPUs in lazy TLB mode, while
the second one is to only send the TLB shootdown at page table
freeing time.
The second should result in fewer IPIs, since operationgs like
mprotect and madvise are very common with some workloads, but
do not involve page table freeing. Also, on munmap, batching
of page table freeing covers much larger ranges of virtual
memory than the batching of unmapped user pages.
Tested-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Rik van Riel <riel@surriel.com>
Acked-by: Dave Hansen <dave.hansen@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: efault@gmx.de
Cc: kernel-team@fb.com
Cc: luto@kernel.org
Link: http://lkml.kernel.org/r/20180716190337.26133-3-riel@surriel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-07-17 09:35:31 +02:00
..
kasan
kasan: fix shadow_size calculation error in kasan_module_alloc
2018-07-03 17:32:19 -07:00
backing-dev.c
bdi: Fix another oops in wb_workfn()
2018-06-22 12:08:07 -06:00
balloon_compaction.c
bootmem.c
mm: docs: fix parameter names mismatch
2018-02-06 18:32:48 -08:00
cleancache.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
cma.c
Revert "mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE"
2018-05-24 10:07:50 -07:00
cma.h
cma_debug.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
compaction.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
debug.c
mm: teach dump_page() to correctly output poisoned struct pages
2018-07-03 17:32:19 -07:00
debug_page_ref.c
dmapool.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
early_ioremap.c
fadvise.c
mm: add ksys_fadvise64_64() helper; remove in-kernel call to sys_fadvise64_64()
2018-04-02 20:16:10 +02:00
failslab.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
filemap.c
mm: use new return type vm_fault_t
2018-06-07 17:34:36 -07:00
frame_vector.c
frontswap.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
gup.c
mm: do not bug_on on incorrect length in __mm_populate()
2018-07-14 11:11:10 -07:00
gup_benchmark.c
treewide: kvzalloc() -> kvcalloc()
2018-06-12 16:19:22 -07:00
highmem.c
hmm.c
mm: introduce MEMORY_DEVICE_FS_DAX and CONFIG_DEV_PAGEMAP_OPS
2018-05-22 06:59:39 -07:00
huge_memory.c
treewide: kmalloc() -> kmalloc_array()
2018-06-12 16:19:22 -07:00
hugetlb.c
mm: hugetlb: yield when prepping struct pages
2018-07-03 17:32:19 -07:00
hugetlb_cgroup.c
mm: rename page_counter's count/limit into usage/max
2018-06-07 17:34:35 -07:00
hwpoison-inject.c
init-mm.c
mm: Allocate the mm_cpumask (mm->cpu_bitmap[]) dynamically based on nr_cpu_ids
2018-07-17 09:35:30 +02:00
internal.h
Changes for 4.18:
2018-06-05 13:24:20 -07:00
interval_tree.c
Kconfig
libnvdimm for 4.18
2018-06-08 17:21:52 -07:00
Kconfig.debug
khugepaged.c
page cache: use xa_lock
2018-04-11 10:28:39 -07:00
kmemleak-test.c
kmemleak.c
mm: kernel-doc: add missing parameter descriptions
2018-04-05 21:36:27 -07:00
ksm.c
mm/ksm.c: ignore STABLE_FLAG of rmap_item->address in rmap_walk_ksm()
2018-06-15 07:55:23 +09:00
list_lru.c
mm: make counting of list_lru_one::nr_items lockless
2018-04-05 21:36:27 -07:00
maccess.c
mm: docs: fix parameter names mismatch
2018-02-06 18:32:48 -08:00
madvise.c
Makefile
mm: restructure memfd code
2018-06-07 17:34:35 -07:00
memblock.c
mm/memblock.c: do not complain about top-down allocations for !MEMORY_HOTREMOVE
2018-07-14 11:11:10 -07:00
memcontrol.c
mm: fix oom_kill event handling
2018-06-15 07:55:25 +09:00
memfd.c
mm: restructure memfd code
2018-06-07 17:34:35 -07:00
memory-failure.c
mm, migrate: remove reason argument from new_page_t
2018-04-11 10:28:32 -07:00
memory.c
x86/mm/tlb: Leave lazy TLB mode at page table free time
2018-07-17 09:35:31 +02:00
memory_hotplug.c
mm: move is_pageblock_removable_nolock() to mm/memory_hotplug.c
2018-06-07 17:34:36 -07:00
mempolicy.c
mm: unclutter THP migration
2018-04-11 10:28:32 -07:00
mempool.c
mempool: Add mempool_init()/mempool_exit()
2018-05-14 13:14:23 -06:00
memtest.c
migrate.c
mm: migrate: fix double call of radix_tree_replace_slot()
2018-05-11 17:28:45 -07:00
mincore.c
mlock.c
mm, mlock, vmscan: no more skipping pagevecs
2018-02-21 15:35:42 -08:00
mm_init.c
mmap.c
mm: do not bug_on on incorrect length in __mm_populate()
2018-07-14 11:11:10 -07:00
mmu_context.c
mmu_notifier.c
mmzone.c
mprotect.c
sched/numa: avoid trapping faults and attempting migration of file-backed dirty pages
2018-04-11 10:28:31 -07:00
mremap.c
mremap: remove LATENCY_LIMIT from mremap to reduce the number of TLB shootdowns
2018-06-15 07:55:24 +09:00
msync.c
nobootmem.c
nommu.c
mm: use new return type vm_fault_t
2018-06-07 17:34:36 -07:00
oom_kill.c
mm: fix oom_kill event handling
2018-06-15 07:55:25 +09:00
page-writeback.c
writeback: safer lock nesting
2018-04-20 17:18:35 -07:00
page_alloc.c
mm: zero unavailable pages before memmap init
2018-07-14 11:02:20 -07:00
page_counter.c
memcg: introduce memory.min
2018-06-07 17:34:36 -07:00
page_ext.c
page_idle.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
page_io.c
page_isolation.c
mm, migrate: remove reason argument from new_page_t
2018-04-11 10:28:32 -07:00
page_owner.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
page_poison.c
mm/page_poison.c: make early_page_poison_param() __init
2018-04-05 21:36:26 -07:00
page_vma_mapped.c
pagewalk.c
mm: kernel-doc: add missing parameter descriptions
2018-04-05 21:36:27 -07:00
percpu-internal.h
percpu-km.c
percpu: allow select gfp to be passed to underlying allocators
2018-02-18 05:33:01 -08:00
percpu-stats.c
treewide: Use array_size() in vmalloc()
2018-06-12 16:19:22 -07:00
percpu-vm.c
percpu: allow select gfp to be passed to underlying allocators
2018-02-18 05:33:01 -08:00
percpu.c
arch: remove obsolete architecture ports
2018-04-02 20:20:12 -07:00
pgtable-generic.c
process_vm_access.c
mm: docs: add blank lines to silence sphinx "Unexpected indentation" errors
2018-02-06 18:32:48 -08:00
quicklist.c
readahead.c
mm: split ->readpages calls to avoid non-contiguous pages lists
2018-06-01 18:37:32 -07:00
rmap.c
mm: do not drop unused pages when userfaultd is running
2018-07-14 11:11:09 -07:00
rodata_test.c
shmem.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
slab.c
treewide: kzalloc() -> kcalloc()
2018-06-12 16:19:22 -07:00
slab.h
slab, slub: skip unnecessary kasan_cache_shutdown()
2018-04-05 21:36:24 -07:00
slab_common.c
slub: fix failure when we delete and create a slab cache
2018-06-28 11:16:44 -07:00
slob.c
slab: __GFP_ZERO is incompatible with a constructor
2018-06-07 17:34:34 -07:00
slub.c
slub: fix failure when we delete and create a slab cache
2018-06-28 11:16:44 -07:00
sparse-vmemmap.c
sparse.c
mm/sparse.c: pass the __highest_present_section_nr + 1 to alloc_func()
2018-06-07 17:34:35 -07:00
swap.c
mm: introduce MEMORY_DEVICE_FS_DAX and CONFIG_DEV_PAGEMAP_OPS
2018-05-22 06:59:39 -07:00
swap_cgroup.c
swap_slots.c
treewide: kvzalloc() -> kvcalloc()
2018-06-12 16:19:22 -07:00
swap_state.c
treewide: kvzalloc() -> kvcalloc()
2018-06-12 16:19:22 -07:00
swapfile.c
mm/swapfile.c: fix swap_count comment about nonexistent SWAP_HAS_CONT
2018-06-15 07:55:23 +09:00
truncate.c
page cache: use xa_lock
2018-04-11 10:28:39 -07:00
usercopy.c
userfaultfd.c
userfaultfd: prevent non-cooperative events vs mcopy_atomic races
2018-06-07 17:34:38 -07:00
util.c
mm: kvmalloc does not fallback to vmalloc for incompatible gfp flags
2018-06-07 17:34:38 -07:00
vmacache.c
vmalloc.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
vmpressure.c
mm/vmpressure.c: convert to use match_string() helper
2018-06-07 17:34:36 -07:00
vmscan.c
memcg: introduce memory.min
2018-06-07 17:34:36 -07:00
vmstat.c
Revert mm/vmstat.c: fix vmstat_update() preemption BUG
2018-06-28 11:16:44 -07:00
workingset.c
page cache: use xa_lock
2018-04-11 10:28:39 -07:00
z3fold.c
z3fold: fix reclaim lock-ups
2018-05-11 17:28:45 -07:00
zbud.c
mm: docs: fix parameter names mismatch
2018-02-06 18:32:48 -08:00
zpool.c
mm/zpool.c: zpool_evictable: fix mismatch in parameter name and kernel-doc
2018-02-21 15:35:43 -08:00
zsmalloc.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00
zswap.c
mm: use octal not symbolic permissions
2018-06-15 07:55:25 +09:00