Nicolai Hähnle
659cf9f582
locking/ww_mutex: Optimize ww-mutexes by waking at most one waiter for backoff when acquiring the lock
...
The wait list is sorted by stamp order, and the only waiting task that may
have to back off is the first waiter with a context.
The regular slow path does not have to wake any other tasks at all, since
all other waiters that would have to back off were either woken up when
the waiter was added to the list, or detected the condition before they
added themselves.
Median timings taken of a contention-heavy GPU workload:
Without this series:
real 0m59.900s
user 0m7.516s
sys 2m16.076s
With changes up to and including this patch:
real 0m52.946s
user 0m7.272s
sys 1m55.964s
Signed-off-by: Nicolai Hähnle <Nicolai.Haehnle@amd.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Maarten Lankhorst <dev@mblankhorst.nl>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: dri-devel@lists.freedesktop.org
Link: http://lkml.kernel.org/r/1482346000-9927-9-git-send-email-nhaehnle@gmail.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-01-14 11:14:44 +01:00
..
bpf
bpf: do not use KMALLOC_SHIFT_MAX
2017-01-10 18:31:54 -08:00
configs
config: android: enable CONFIG_SECCOMP
2016-10-11 15:06:32 -07:00
debug
kdb: call vkdb_printf() from vprintk_default() only when wanted
2016-12-14 16:04:08 -08:00
events
Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2016-12-23 16:49:12 -08:00
gcov
irq
genirq/affinity: Fix node generation from cpumask
2016-12-15 12:32:35 +01:00
livepatch
locking
locking/ww_mutex: Optimize ww-mutexes by waking at most one waiter for backoff when acquiring the lock
2017-01-14 11:14:44 +01:00
power
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
printk
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
rcu
rcu: Don't kick unless grace period or request
2016-11-14 10:46:31 -08:00
sched
ktime: Cleanup ktime_set() usage
2016-12-25 17:21:22 +01:00
time
ktime: Cleanup ktime_set() usage
2016-12-25 17:21:22 +01:00
trace
clocksource: Use a plain u64 instead of cycle_t
2016-12-25 11:04:12 +01:00
.gitignore
acct.c
async.c
audit.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2016-12-17 18:44:00 -08:00
audit.h
audit_log_{name,link_denied}: constify struct path
2016-12-05 19:00:38 -05:00
audit_fsnotify.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2016-12-17 18:44:00 -08:00
audit_tree.c
Merge branch 'stable-4.10' of git://git.infradead.org/users/pcmoore/audit
2017-01-05 23:06:06 -08:00
audit_watch.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2016-12-17 18:44:00 -08:00
auditfilter.c
audit: add support for session ID user filter
2016-11-29 15:10:12 -05:00
auditsc.c
Merge branch 'stable-4.10' of git://git.infradead.org/users/pcmoore/audit
2016-12-14 14:06:40 -08:00
backtracetest.c
bounds.c
capability.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
cgroup.c
cgroup: add support for eBPF programs
2016-11-25 16:25:52 -05:00
cgroup_freezer.c
cgroup_pids.c
compat.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
configs.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
context_tracking.c
cpu.c
smp/hotplug: Undo tglxs brainfart
2016-12-26 17:30:24 -08:00
cpu_pm.c
cpuset.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
crash_dump.c
cred.c
delayacct.c
dma.c
elfcore.c
exec_domain.c
exit.c
sched/wait, RCU: Introduce rcuwait machinery
2017-01-14 11:14:33 +01:00
extable.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
fork.c
locking/mutex: Fix mutex handoff
2017-01-14 11:14:38 +01:00
freezer.c
futex.c
ktime: Get rid of the union
2016-12-25 17:21:22 +01:00
futex_compat.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
groups.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
hung_task.c
hung_task: decrement sysctl_hung_task_warnings only if it is positive
2016-12-12 18:55:09 -08:00
irq_work.c
jump_label.c
kallsyms.c
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
locking/mutex: Allow MUTEX_SPIN_ON_OWNER when DEBUG_MUTEXES
2016-10-25 11:31:51 +02:00
Kconfig.preempt
kcov.c
kcov: make kcov work properly with KASLR enabled
2016-12-20 09:48:47 -08:00
kexec.c
kexec_core.c
kexec: add cond_resched into kimage_alloc_crash_control_pages
2016-12-14 16:04:07 -08:00
kexec_file.c
ima: on soft reboot, save the measurement list
2016-12-20 09:48:44 -08:00
kexec_internal.h
kexec_file: Allow arch-specific memory walking for kexec_add_buffer
2016-11-30 23:14:57 +11:00
kmod.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
kprobes.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
ksysfs.c
kthread.c
kthread: add __printf attributes
2016-12-12 18:55:06 -08:00
latencytop.c
Makefile
Merge branch 'akpm' (patches from Andrew)
2016-12-14 17:25:18 -08:00
membarrier.c
memremap.c
mm: fix devm_memremap_pages crash, use mem_hotplug_{begin, done}
2017-01-10 18:31:54 -08:00
module-internal.h
module.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
module_signing.c
notifier.c
nsproxy.c
padata.c
padata: Remove unused but set variables
2016-10-25 11:08:10 +08:00
panic.c
taint/module: Clean up global and module taint flags handling
2016-11-26 11:18:01 -08:00
params.c
pid.c
pid_namespace.c
Merge branch 'nsfs-ioctls' into HEAD
2016-09-22 20:00:36 -05:00
profile.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
ptrace.c
ptrace: Don't allow accessing an undumpable mm
2016-11-22 12:57:38 -06:00
range.c
reboot.c
relay.c
relay: check array offset before using it
2016-12-14 16:04:08 -08:00
resource.c
seccomp.c
Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2016-12-14 13:57:44 -08:00
signal.c
signal: protect SIGNAL_UNKILLABLE from unintentional clearing.
2017-01-10 18:31:55 -08:00
smp.c
kernel/smp: Tell the user we're bringing up secondary CPUs
2016-10-26 12:02:35 +02:00
smpboot.c
kthread/smpboot: do not park in kthread_create_on_cpu()
2016-10-11 15:06:33 -07:00
smpboot.h
softirq.c
softirq: Display IRQ_POLL for irq-poll statistics
2016-10-21 15:45:47 -06:00
stacktrace.c
stop_machine.c
locking/core, stop_machine: Yield the CPU during stop machine()
2016-11-16 10:15:09 +01:00
sys.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
sys_ni.c
move aio compat to fs/aio.c
2016-12-22 22:58:37 -05:00
sysctl.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
sysctl_binary.c
sysctl: add KERN_CONT to deprecated_sysctl_warning()
2016-12-14 16:04:07 -08:00
task_work.c
taskstats.c
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
2016-11-15 10:54:36 -05:00
test_kprobes.c
torture.c
tracepoint.c
tracing: Have the reg function allow to fail
2016-12-09 09:13:30 -05:00
tsacct.c
ucount.c
uid16.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
up.c
smp: Add function to execute a function synchronously on a CPU
2016-09-05 13:52:39 +02:00
user-return-notifier.c
user.c
user_namespace.c
Merge branch 'nsfs-ioctls' into HEAD
2016-09-22 20:00:36 -05:00
utsname.c
Merge branch 'nsfs-ioctls' into HEAD
2016-09-22 20:00:36 -05:00
utsname_sysctl.c
watchdog.c
kernel/watchdog.c: move hardlockup detector to separate file
2016-12-14 16:04:08 -08:00
watchdog_hld.c
kernel/watchdog.c: move hardlockup detector to separate file
2016-12-14 16:04:08 -08:00
workqueue.c
Merge branch 'for-4.9' into for-4.10
2016-10-19 12:12:40 -04:00
workqueue_internal.h