Nadav Amit
16af97dc5a
mm: migrate: prevent racy access to tlb_flush_pending
...
Patch series "fixes of TLB batching races", v6.
It turns out that Linux TLB batching mechanism suffers from various
races. Races that are caused due to batching during reclamation were
recently handled by Mel and this patch-set deals with others. The more
fundamental issue is that concurrent updates of the page-tables allow
for TLB flushes to be batched on one core, while another core changes
the page-tables. This other core may assume a PTE change does not
require a flush based on the updated PTE value, while it is unaware that
TLB flushes are still pending.
This behavior affects KSM (which may result in memory corruption) and
MADV_FREE and MADV_DONTNEED (which may result in incorrect behavior). A
proof-of-concept can easily produce the wrong behavior of MADV_DONTNEED.
Memory corruption in KSM is harder to produce in practice, but was
observed by hacking the kernel and adding a delay before flushing and
replacing the KSM page.
Finally, there is also one memory barrier missing, which may affect
architectures with weak memory model.
This patch (of 7):
Setting and clearing mm->tlb_flush_pending can be performed by multiple
threads, since mmap_sem may only be acquired for read in
task_numa_work(). If this happens, tlb_flush_pending might be cleared
while one of the threads still changes PTEs and batches TLB flushes.
This can lead to the same race between migration and
change_protection_range() that led to the introduction of
tlb_flush_pending. The result of this race was data corruption, which
means that this patch also addresses a theoretically possible data
corruption.
An actual data corruption was not observed, yet the race was was
confirmed by adding assertion to check tlb_flush_pending is not set by
two threads, adding artificial latency in change_protection_range() and
using sysctl to reduce kernel.numa_balancing_scan_delay_ms.
Link: http://lkml.kernel.org/r/20170802000818.4760-2-namit@vmware.com
Fixes: 2084140594 ("mm: fix TLB flush race between migration, and
change_protection_range")
Signed-off-by: Nadav Amit <namit@vmware.com>
Acked-by: Mel Gorman <mgorman@suse.de>
Acked-by: Rik van Riel <riel@redhat.com>
Acked-by: Minchan Kim <minchan@kernel.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-08-10 15:54:07 -07:00
..
bpf
bpf: fix bpf_prog_get_info_by_fd to dump correct xlated_prog_len
2017-07-29 23:29:41 -07:00
cgroup
cpuset: fix a deadlock due to incomplete patching of cpusets_enabled()
2017-08-02 17:16:12 -07:00
configs
config: android-base: disable CONFIG_NFSD and CONFIG_NFS_FS
2017-06-09 11:47:38 +02:00
debug
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>
2017-03-02 08:42:34 +01:00
events
Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2017-07-21 11:12:48 -07:00
gcov
gcov: support GCC 7.1
2017-05-12 15:57:15 -07:00
irq
genirq/cpuhotplug: Revert "Set force affinity flag on hotplug migration"
2017-07-27 15:40:02 +02:00
livepatch
livepatch: Fix stacking of patches with respect to RCU
2017-06-20 10:42:19 +02:00
locking
Merge branch 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2017-07-21 11:11:23 -07:00
power
mm: fix global NR_SLAB_.*CLAIMABLE counter reads
2017-08-10 15:54:06 -07:00
printk
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/pmladek/printk
2017-07-05 11:11:26 -07:00
rcu
rcu: Remove RCU CPU stall warnings from Tiny RCU
2017-06-08 18:52:45 -07:00
sched
sched/core: Fix some documentation build warnings
2017-07-25 11:17:02 +02:00
time
timers: Fix overflow in get_next_timer_interrupt
2017-08-01 14:20:53 +02:00
trace
trace: fix the errors caused by incompatible type of RCU variables
2017-07-20 09:27:29 -04:00
.gitignore
acct.c
sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>
2017-03-02 08:42:39 +01:00
async.c
async: Adjust system_state checks
2017-05-23 10:01:37 +02:00
audit.c
Merge branch 'stable-4.13' of git://git.infradead.org/users/pcmoore/audit
2017-07-20 10:22:26 -07:00
audit.h
audit: style fix
2017-06-12 18:07:43 -04:00
audit_fsnotify.c
Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs
2017-05-03 11:05:15 -07:00
audit_tree.c
Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs
2017-05-03 11:05:15 -07:00
audit_watch.c
Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs
2017-05-03 11:05:15 -07:00
auditfilter.c
audit: kernel generated netlink traffic should have a portid of 0
2017-05-02 10:16:05 -04:00
auditsc.c
Merge branch 'stable-4.13' of git://git.infradead.org/users/pcmoore/audit
2017-07-05 11:24:05 -07:00
backtracetest.c
bounds.c
capability.c
compat.c
Merge branch 'misc.compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2017-07-06 20:57:13 -07:00
configs.c
context_tracking.c
cpu.c
smp/hotplug: Replace BUG_ON and react useful
2017-07-11 22:25:44 +02:00
cpu_pm.c
crash_core.c
kdump: protect vmcoreinfo data under the crash memory
2017-07-12 16:26:00 -07:00
crash_dump.c
cred.c
doc: ReSTify credentials.txt
2017-05-18 10:30:19 -06:00
delayacct.c
sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>
2017-03-02 08:42:39 +01:00
dma.c
elfcore.c
exec_domain.c
exit.c
kernel/exit.c: avoid undefined behaviour when calling wait4()
2017-07-10 16:32:36 -07:00
extable.c
lib/extable.c: use bsearch() library function in search_extable()
2017-07-10 16:32:35 -07:00
fork.c
mm: migrate: prevent racy access to tlb_flush_pending
2017-08-10 15:54:07 -07:00
freezer.c
futex.c
futex: Remove unnecessary warning from get_futex_key
2017-08-09 14:00:54 -07:00
futex_compat.c
groups.c
kernel/groups.c: use sort library function
2017-07-10 16:32:34 -07:00
hung_task.c
kernel/hung_task.c: defer showing held locks
2017-05-08 17:15:10 -07:00
irq_work.c
jump_label.c
jump_label: Reorder hotplug lock and jump_label_lock
2017-05-26 10:10:45 +02:00
kallsyms.c
kernel/kallsyms.c: replace all_var with IS_ENABLED(CONFIG_KALLSYMS_ALL)
2017-07-10 16:32:34 -07:00
kcmp.c
kcmp: add KCMP_EPOLL_TFD mode to compare epoll target files
2017-07-12 16:26:01 -07:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c
kcov: simplify interrupt check
2017-05-08 17:15:12 -07:00
kexec.c
kdump: protect vmcoreinfo data under the crash memory
2017-07-12 16:26:00 -07:00
kexec_core.c
kdump: protect vmcoreinfo data under the crash memory
2017-07-12 16:26:00 -07:00
kexec_file.c
kexec_file: adjust declaration of kexec_purgatory
2017-07-12 16:26:02 -07:00
kexec_internal.h
kexec_file: adjust declaration of kexec_purgatory
2017-07-12 16:26:02 -07:00
kmod.c
kmod: throttle kmod thread limit
2017-07-14 15:05:13 -07:00
kprobes.c
kprobes: Ensure that jprobe probepoints are at function entry
2017-07-08 11:05:35 +02:00
ksysfs.c
kexec: move vmcoreinfo out of the kernel's .bss section
2017-07-12 16:25:59 -07:00
kthread.c
cgroup, kthread: close race window where new kthreads can be migrated to non-root cgroups
2017-03-17 10:18:47 -04:00
latencytop.c
sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>
2017-03-02 08:42:39 +01:00
Makefile
kernel/watchdog: split up config options
2017-07-12 16:26:02 -07:00
membarrier.c
Fix: Disable sys_membarrier when nohz_full is enabled
2017-01-23 11:32:16 -08:00
memremap.c
mm, memory_hotplug: replace for_device by want_memblock in arch_add_memory
2017-07-06 16:24:32 -07:00
module-internal.h
module.c
Modules updates for v4.13
2017-07-12 17:22:01 -07:00
module_signing.c
notifier.c
kernel/notifier.c: simplify expression
2017-02-24 17:46:56 -08:00
nsproxy.c
perf: Add PERF_RECORD_NAMESPACES to include namespaces related info
2017-03-13 15:57:41 -03:00
padata.c
padata: Avoid nested calls to cpus_read_lock() in pcrypt_init_padata()
2017-05-26 10:10:37 +02:00
panic.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>
2017-03-02 08:42:34 +01:00
params.c
boot/param: Move next_arg() function to lib/cmdline.c for later reuse
2017-04-18 10:37:13 +02:00
pid.c
pid: kill pidhash_size in pidhash_init()
2017-08-02 16:34:46 -07:00
pid_namespace.c
pid_ns: Sleep in TASK_INTERRUPTIBLE in zap_pid_ns_processes
2017-05-13 17:26:01 -05:00
profile.c
sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>
2017-03-02 08:42:39 +01:00
ptrace.c
ptrace: Properly initialize ptracer_cred on fork
2017-05-23 07:40:44 -05:00
range.c
reboot.c
relay.c
Merge branch 'work.splice' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2017-05-02 11:38:06 -07:00
resource.c
seccomp.c
seccomp: Switch from atomic_t to recount_t
2017-06-26 09:24:00 -07:00
signal.c
Fix compat_sys_sigpending breakage
2017-08-06 11:48:27 -07:00
smp.c
smp, cpumask: Use non-atomic cpumask_{set,clear}_cpu()
2017-05-23 10:01:32 +02:00
smpboot.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>
2017-03-02 08:42:35 +01:00
smpboot.h
softirq.c
sched/core: Remove 'task' parameter and rename tsk_restore_flags() to current_restore_flags()
2017-04-11 09:06:32 +02:00
stacktrace.c
stacktrace/x86: add function for detecting reliable stack traces
2017-03-08 09:18:02 +01:00
stop_machine.c
stop_machine: Provide stop_machine_cpuslocked()
2017-05-26 10:10:36 +02:00
sys.c
fix a braino in compat_sys_getrlimit()
2017-07-12 09:15:00 -07:00
sys_ni.c
sysctl.c
kernel/watchdog: split up config options
2017-07-12 16:26:02 -07:00
sysctl_binary.c
kernel/sysctl_binary.c: check name array length in deprecated_sysctl_warning()
2017-07-12 16:26:00 -07:00
task_work.c
taskstats.c
taskstats: add e/u/stime for TGID command
2017-05-08 17:15:12 -07:00
test_kprobes.c
torture.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/clock.h>
2017-03-02 08:42:27 +01:00
tracepoint.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>
2017-03-02 08:42:35 +01:00
tsacct.c
sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>
2017-03-02 08:42:39 +01:00
ucount.c
ucount: Remove the atomicity from ucount->count
2017-03-06 15:26:37 -06:00
uid16.c
sched/headers: Prepare to remove <linux/cred.h> inclusion from <linux/sched.h>
2017-03-02 08:42:31 +01:00
up.c
user-return-notifier.c
user.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/user.h>
2017-03-02 08:42:29 +01:00
user_namespace.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/signal.h>
2017-03-02 08:42:29 +01:00
utsname.c
sched/headers: Prepare to move the task_lock()/unlock() APIs to <linux/sched/task.h>
2017-03-02 08:42:38 +01:00
utsname_sysctl.c
sched/headers: Remove <linux/rwsem.h> from <linux/sched.h>
2017-03-03 01:45:36 +01:00
watchdog.c
kernel/watchdog.c: use better pr_fmt prefix
2017-07-14 15:05:13 -07:00
watchdog_hld.c
kernel/watchdog: split up config options
2017-07-12 16:26:02 -07:00
workqueue.c
workqueue: Work around edge cases for calc of pool's cpumask
2017-07-28 11:05:52 -04:00
workqueue_internal.h