Rik van Riel
2084140594
mm: fix TLB flush race between migration, and change_protection_range ...
There are a few subtle races, between change_protection_range (used by
mprotect and change_prot_numa) on one side, and NUMA page migration and
compaction on the other side.
The basic race is that there is a time window between when the PTE gets
made non-present (PROT_NONE or NUMA), and the TLB is flushed.
During that time, a CPU may continue writing to the page.
This is fine most of the time, however compaction or the NUMA migration
code may come in, and migrate the page away.
When that happens, the CPU may continue writing, through the cached
translation, to what is no longer the current memory location of the
process.
This only affects x86, which has a somewhat optimistic pte_accessible.
All other architectures appear to be safe, and will either always flush,
or flush whenever there is a valid mapping, even with no permissions
(SPARC).
The basic race looks like this:
CPU A CPU B CPU C
load TLB entry
make entry PTE/PMD_NUMA
fault on entry
read/write old page
start migrating page
change PTE/PMD to new page
read/write old page [*]
flush TLB
reload TLB from new entry
read/write new page
lose data
[*] the old page may belong to a new user at this point!
The obvious fix is to flush remote TLB entries, by making sure that
pte_accessible aware of the fact that PROT_NONE and PROT_NUMA memory may
still be accessible if there is a TLB flush pending for the mm.
This should fix both NUMA migration and compaction.
[mgorman@suse.de: fix build]
Signed-off-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Cc: Alex Thorlton <athorlton@sgi.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-12-18 19:04:51 -08:00
bitops/find: clarify and extend documentation
2013-11-13 12:09:22 +09:00
asm-generic: move cmpxchg*_local defs to cmpxchg.h
2013-03-13 06:11:05 +01:00
Create asm-generic/barrier.h
2012-03-28 18:30:03 +01:00
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
bug.h: Fix up CONFIG_BUG=n implicit function declarations.
2012-06-25 10:32:49 -07:00
asm-generic headers: Allow yet more arch overrides in checksum.h
2013-02-11 20:00:33 +05:30
asm-generic: Add default clkdev.h
2012-10-03 21:33:53 +02:00
generic: Use raw local irq variant for generic cmpxchg
2013-02-19 08:43:37 +01:00
asm-generic: move cmpxchg*_local defs to cmpxchg.h
2013-03-13 06:11:05 +01:00
cputime: Generic on-demand virtual cputime accounting
2013-01-27 19:23:27 +01:00
cputime: Move default nsecs_to_cputime() to jiffies based cputime file
2013-01-27 19:23:25 +01:00
cputime_nsecs: use math64.h for nsec resolution conversion helpers
2013-04-26 18:58:12 +02:00
common: dma-mapping: add support for generic dma_mmap_* calls
2012-07-30 12:25:46 +02:00
asm-generic/dma-mapping-broken.h: Provide dma_alloc_attrs()/dma_free_attrs()
2012-12-25 20:14:54 +01:00
common: dma-mapping: introduce dma_get_sgtable() function
2012-07-30 12:25:46 +02:00
Split arch_align_stack() out from asm-generic/system.h
2012-03-28 18:30:03 +01:00
bitops: Add missing parentheses to new get_order macro
2012-02-24 10:39:27 -08:00
GPIO bulk changes for the v3.13 development cycle
2013-11-12 15:50:46 +09:00
mm: Fix generic hugetlb pte check return type.
2013-10-02 20:02:35 -04:00
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
asm-generic: architecture independent readq/writeq for 32bit environment
2012-02-21 16:47:28 -08:00
asm-generic: architecture independent readq/writeq for 32bit environment
2012-02-21 16:47:28 -08:00
kernel: Fix s390 absolute memory access for /dev/mem
2013-05-22 09:45:57 +02:00
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
[PARISC] fix compile break caused by iomap: make IOPORT/PCI mapping functions conditional
2012-02-27 09:43:30 -06:00
UAPI: Set up uapi/asm/Kbuild.asm
2012-10-02 18:01:56 +01:00
asm-generic: remove km_type definitions
2012-07-24 15:27:30 +08:00
KVM: add kvm_para_available to asm-generic/kvm_para.h
2013-06-05 13:21:29 +03:00
__page_to_pfn: Fix typo in comment
2013-10-14 15:28:29 +02:00
asm-generic/mmu.h: Add support for FDPIC
2012-12-09 23:14:14 +01:00
Make most arch asm/module.h files use asm-generic/module.h
2012-09-28 14:31:03 +09:30
arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not
2013-06-26 12:10:55 +02:00
arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not
2013-06-26 12:10:55 +02:00
arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not
2013-06-26 12:10:55 +02:00
The following changes since commit 3ee72ca992
2012-01-10 17:39:40 -08:00
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
include: remove __dev* attributes.
2013-01-03 15:57:16 -08:00
PCI: work around Stratus ftServer broken PCIe hierarchy
2012-04-30 15:21:02 -06:00
PCI: collapse pcibios_resource_to_bus
2012-02-23 20:19:04 -07:00
[PARISC] fix compile break caused by iomap: make IOPORT/PCI mapping functions conditional
2012-02-27 09:43:30 -06:00
mm: fix TLB flush race between migration, and change_protection_range
2013-12-18 19:04:51 -08:00
sched: Remove PREEMPT_NEED_RESCHED from generic code
2013-12-11 15:52:32 +01:00
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
vmlinux.lds: add comments for global variables and clean up useless declarations
2013-07-03 16:07:34 -07:00
constify copy_siginfo_to_user{,32}()
2013-11-09 00:16:29 -05:00
unify default ptrace_signal_deliver
2012-11-29 00:01:23 -05:00
crypto: create generic version of ablk_helper
2013-09-24 06:02:24 +10:00
ARM: 7430/1: sizes.h: move from asm-generic to <linux/sizes.h>
2012-06-28 17:14:34 +01:00
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
Split the switch_to() wrapper out of asm-generic/system.h
2012-03-28 18:30:03 +01:00
asm/syscall.h: add syscall_get_arch
2012-04-14 11:13:19 +10:00
burying unused conditionals
2013-02-14 09:21:15 -05:00
UAPI: (Scripted) Disintegrate include/asm-generic
2012-10-04 18:20:15 +01:00
Fix TLB gather virtual address range invalidation corner cases
2013-08-16 08:52:46 -07:00
BUG: headers with BUG/BUG_ON etc. need linux/bug.h
2012-03-04 17:54:34 -05:00
tracing,x86: Add a TSC trace_clock
2012-11-13 15:48:27 -05:00
asm-generic: uaccess s/might_sleep/might_fault/
2013-05-28 09:41:05 +02:00
We get rid of the general module prefix confusion with a binary config option,
2013-05-05 10:58:06 -07:00
kernel: add support for init_array constructors
2013-10-17 15:05:17 +10:30
include/asm-generic/vtime.h: avoid zero-length file
2013-09-30 14:31:02 -07:00
word-at-a-time: provide generic big-endian zero_bytemask implementation
2013-12-12 10:39:01 -08:00
asm-generic: xor: mark static functions as __maybe_unused
2012-10-03 21:21:06 +02:00
2084140594