Peter Zijlstra
e274795ea7
locking/mutex: Fix mutex handoff
...
While reviewing the ww_mutex patches, I noticed that it was still
possible to (incorrectly) succeed for (incorrect) code like:
mutex_lock(&a);
mutex_lock(&a);
This was possible if the second mutex_lock() would block (as expected)
but then receive a spurious wakeup. At that point it would find itself
at the front of the queue, request a handoff and instantly claim
ownership and continue, since owner would point to itself.
Avoid this scenario and simplify the code by introducing a third low
bit to signal handoff pickup. So once we request handoff, unlock
clears the handoff bit and sets the pickup bit along with the new
owner.
This also removes the need for the .handoff argument to
__mutex_trylock(), since that becomes superfluous with PICKUP.
In order to guarantee enough low bits, ensure task_struct alignment is
at least L1_CACHE_BYTES (which seems a good ideal regardless).
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Fixes: 9d659ae14b ("locking/mutex: Add lock handoff to avoid starvation")
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-01-14 11:14:38 +01:00
..
bpf
bpf: do not use KMALLOC_SHIFT_MAX
2017-01-10 18:31:54 -08:00
configs
config: android: enable CONFIG_SECCOMP
2016-10-11 15:06:32 -07:00
debug
kdb: call vkdb_printf() from vprintk_default() only when wanted
2016-12-14 16:04:08 -08:00
events
Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2016-12-23 16:49:12 -08:00
gcov
gcov: add support for gcc version >= 6
2016-07-15 14:54:27 +09:00
irq
genirq/affinity: Fix node generation from cpumask
2016-12-15 12:32:35 +01:00
livepatch
livepatch/module: make TAINT_LIVEPATCH module-specific
2016-08-26 14:42:08 +02:00
locking
locking/mutex: Fix mutex handoff
2017-01-14 11:14:38 +01:00
power
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
printk
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
rcu
rcu: Don't kick unless grace period or request
2016-11-14 10:46:31 -08:00
sched
ktime: Cleanup ktime_set() usage
2016-12-25 17:21:22 +01:00
time
ktime: Cleanup ktime_set() usage
2016-12-25 17:21:22 +01:00
trace
clocksource: Use a plain u64 instead of cycle_t
2016-12-25 11:04:12 +01:00
.gitignore
acct.c
async.c
audit.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2016-12-17 18:44:00 -08:00
audit.h
audit_log_{name,link_denied}: constify struct path
2016-12-05 19:00:38 -05:00
audit_fsnotify.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2016-12-17 18:44:00 -08:00
audit_tree.c
Merge branch 'stable-4.10' of git://git.infradead.org/users/pcmoore/audit
2017-01-05 23:06:06 -08:00
audit_watch.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2016-12-17 18:44:00 -08:00
auditfilter.c
audit: add support for session ID user filter
2016-11-29 15:10:12 -05:00
auditsc.c
Merge branch 'stable-4.10' of git://git.infradead.org/users/pcmoore/audit
2016-12-14 14:06:40 -08:00
backtracetest.c
bounds.c
capability.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
cgroup.c
cgroup: add support for eBPF programs
2016-11-25 16:25:52 -05:00
cgroup_freezer.c
cgroup_pids.c
cgroup: Use lld instead of ld when printing pids controller events_limit
2016-06-21 15:03:36 -04:00
compat.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
configs.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
context_tracking.c
cpu.c
smp/hotplug: Undo tglxs brainfart
2016-12-26 17:30:24 -08:00
cpu_pm.c
cpuset.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
crash_dump.c
cred.c
cred: Reject inodes with invalid ids in set_create_file_as()
2016-06-30 18:05:09 -05:00
delayacct.c
dma.c
elfcore.c
exec_domain.c
exit.c
sched/wait, RCU: Introduce rcuwait machinery
2017-01-14 11:14:33 +01:00
extable.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
fork.c
locking/mutex: Fix mutex handoff
2017-01-14 11:14:38 +01:00
freezer.c
freezer, oom: check TIF_MEMDIE on the correct task
2016-07-28 16:07:41 -07:00
futex.c
ktime: Get rid of the union
2016-12-25 17:21:22 +01:00
futex_compat.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
groups.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
hung_task.c
hung_task: decrement sysctl_hung_task_warnings only if it is positive
2016-12-12 18:55:09 -08:00
irq_work.c
jump_label.c
powerpc updates for 4.8 #2
2016-08-05 09:00:54 -04:00
kallsyms.c
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
locking/mutex: Allow MUTEX_SPIN_ON_OWNER when DEBUG_MUTEXES
2016-10-25 11:31:51 +02:00
Kconfig.preempt
kcov.c
kcov: make kcov work properly with KASLR enabled
2016-12-20 09:48:47 -08:00
kexec.c
kexec: allow architectures to override boot mapping
2016-08-02 19:35:27 -04:00
kexec_core.c
kexec: add cond_resched into kimage_alloc_crash_control_pages
2016-12-14 16:04:07 -08:00
kexec_file.c
ima: on soft reboot, save the measurement list
2016-12-20 09:48:44 -08:00
kexec_internal.h
kexec_file: Allow arch-specific memory walking for kexec_add_buffer
2016-11-30 23:14:57 +11:00
kmod.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
kprobes.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
ksysfs.c
kexec: add a kexec_crash_loaded() function
2016-08-02 19:35:30 -04:00
kthread.c
kthread: add __printf attributes
2016-12-12 18:55:06 -08:00
latencytop.c
Makefile
Merge branch 'akpm' (patches from Andrew)
2016-12-14 17:25:18 -08:00
membarrier.c
memremap.c
mm: fix devm_memremap_pages crash, use mem_hotplug_{begin, done}
2017-01-10 18:31:54 -08:00
module-internal.h
module.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
module_signing.c
notifier.c
nsproxy.c
padata.c
padata: Remove unused but set variables
2016-10-25 11:08:10 +08:00
panic.c
taint/module: Clean up global and module taint flags handling
2016-11-26 11:18:01 -08:00
params.c
pid.c
remove lots of IS_ERR_VALUE abuses
2016-05-27 15:26:11 -07:00
pid_namespace.c
Merge branch 'nsfs-ioctls' into HEAD
2016-09-22 20:00:36 -05:00
profile.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
ptrace.c
ptrace: Don't allow accessing an undumpable mm
2016-11-22 12:57:38 -06:00
range.c
reboot.c
relay.c
relay: check array offset before using it
2016-12-14 16:04:08 -08:00
resource.c
seccomp.c
Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2016-12-14 13:57:44 -08:00
signal.c
signal: protect SIGNAL_UNKILLABLE from unintentional clearing.
2017-01-10 18:31:55 -08:00
smp.c
kernel/smp: Tell the user we're bringing up secondary CPUs
2016-10-26 12:02:35 +02:00
smpboot.c
kthread/smpboot: do not park in kthread_create_on_cpu()
2016-10-11 15:06:33 -07:00
smpboot.h
softirq.c
softirq: Display IRQ_POLL for irq-poll statistics
2016-10-21 15:45:47 -06:00
stacktrace.c
stop_machine.c
locking/core, stop_machine: Yield the CPU during stop machine()
2016-11-16 10:15:09 +01:00
sys.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
sys_ni.c
move aio compat to fs/aio.c
2016-12-22 22:58:37 -05:00
sysctl.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
sysctl_binary.c
sysctl: add KERN_CONT to deprecated_sysctl_warning()
2016-12-14 16:04:07 -08:00
task_work.c
task_work: use READ_ONCE/lockless_dereference, avoid pi_lock if !task_works
2016-08-02 19:35:02 -04:00
taskstats.c
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
2016-11-15 10:54:36 -05:00
test_kprobes.c
torture.c
torture: Convert torture_shutdown() to hrtimer
2016-08-22 10:01:49 -07:00
tracepoint.c
tracing: Have the reg function allow to fail
2016-12-09 09:13:30 -05:00
tsacct.c
ucount.c
mntns: Add a limit on the number of mount namespaces.
2016-08-31 07:28:35 -05:00
uid16.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
up.c
smp: Add function to execute a function synchronously on a CPU
2016-09-05 13:52:39 +02:00
user-return-notifier.c
user.c
user_namespace.c
Merge branch 'nsfs-ioctls' into HEAD
2016-09-22 20:00:36 -05:00
utsname.c
Merge branch 'nsfs-ioctls' into HEAD
2016-09-22 20:00:36 -05:00
utsname_sysctl.c
watchdog.c
kernel/watchdog.c: move hardlockup detector to separate file
2016-12-14 16:04:08 -08:00
watchdog_hld.c
kernel/watchdog.c: move hardlockup detector to separate file
2016-12-14 16:04:08 -08:00
workqueue.c
Merge branch 'for-4.9' into for-4.10
2016-10-19 12:12:40 -04:00
workqueue_internal.h