Will Deacon
0bce9c46bf
mutex: Place lock in contended state after fastpath_lock failure
...
ARM recently moved to asm-generic/mutex-xchg.h for its mutex
implementation after the previous implementation was found to be missing
some crucial memory barriers. However, this has revealed some problems
running hackbench on SMP platforms due to the way in which the
MUTEX_SPIN_ON_OWNER code operates.
The symptoms are that a bunch of hackbench tasks are left waiting on an
unlocked mutex and therefore never get woken up to claim it. This boils
down to the following sequence of events:
Task A Task B Task C Lock value
0 1
1 lock() 0
2 lock() 0
3 spin(A) 0
4 unlock() 1
5 lock() 0
6 cmpxchg(1,0) 0
7 contended() -1
8 lock() 0
9 spin(C) 0
10 unlock() 1
11 cmpxchg(1,0) 0
12 unlock() 1
At this point, the lock is unlocked, but Task B is in an uninterruptible
sleep with nobody to wake it up.
This patch fixes the problem by ensuring we put the lock into the
contended state if we fail to acquire it on the fastpath, ensuring that
any blocked waiters are woken up when the mutex is released.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: <stable@vger.kernel.org>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-6e9lrw2avczr0617fzl5vqb8@git.kernel.org
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2012-08-13 18:46:54 +02:00
..
bitops
Add #includes needed to permit the removal of asm/system.h
2012-03-28 18:30:03 +01:00
4level-fixup.h
atomic-long.h
atomic.h
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
atomic64.h
audit_change_attr.h
audit: support the "standard" <asm-generic/unistd.h>
2011-05-04 14:41:28 -04:00
audit_dir_write.h
audit: support the "standard" <asm-generic/unistd.h>
2011-05-04 14:41:28 -04:00
audit_read.h
audit: support the "standard" <asm-generic/unistd.h>
2011-05-04 14:41:28 -04:00
audit_signal.h
audit_write.h
audit: support the "standard" <asm-generic/unistd.h>
2011-05-04 14:41:28 -04:00
auxvec.h
barrier.h
Create asm-generic/barrier.h
2012-03-28 18:30:03 +01:00
bitops.h
bitops: remove minix bitops from asm/bitops.h
2011-03-23 19:46:22 -07:00
bitsperlong.h
vsprintf: further optimize decimal conversion
2012-05-31 17:49:27 -07:00
bug.h
bug.h: Fix up CONFIG_BUG=n implicit function declarations.
2012-06-25 10:32:49 -07:00
bugs.h
cache.h
cacheflush.h
asm-generic/cacheflush.h: flush icache when copying to user pages
2011-05-25 08:39:37 -07:00
checksum.h
Add extra arch overrides to asm-generic/checksum.h
2011-11-01 07:34:21 -07:00
cmpxchg-local.h
cmpxchg.h
asm-generic: add linux/types.h to cmpxchg.h
2012-04-02 14:41:27 -07:00
cputime.h
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2012-01-06 08:44:54 -08:00
current.h
delay.h
asm-generic: delay.h fix udelay and ndelay for 8 bit args
2011-07-22 18:45:33 +02:00
device.h
div64.h
dma-coherent.h
common: dma-mapping: add support for generic dma_mmap_* calls
2012-07-30 12:25:46 +02:00
dma-contiguous.h
mm: cma: fix condition check when setting global cma area
2012-07-06 12:02:04 +02:00
dma-mapping-broken.h
dma-mapping-common.h
common: dma-mapping: introduce dma_get_sgtable() function
2012-07-30 12:25:46 +02:00
dma.h
emergency-restart.h
errno-base.h
errno.h
exec.h
Split arch_align_stack() out from asm-generic/system.h
2012-03-28 18:30:03 +01:00
fb.h
fcntl.h
c/r: fcntl: add F_GETOWNER_UIDS option
2012-07-30 17:25:21 -07:00
ftrace.h
futex.h
getorder.h
bitops: Add missing parentheses to new get_order macro
2012-02-24 10:39:27 -08:00
gpio.h
gpiolib: Remove 'const' from data argument of gpiochip_find()
2012-05-18 23:01:05 -06:00
hardirq.h
hw_irq.h
ide_iops.h
int-l64.h
int-ll64.h
io-64-nonatomic-hi-lo.h
asm-generic: architecture independent readq/writeq for 32bit environment
2012-02-21 16:47:28 -08:00
io-64-nonatomic-lo-hi.h
asm-generic: architecture independent readq/writeq for 32bit environment
2012-02-21 16:47:28 -08:00
io.h
lib: use generic pci_iomap on all architectures
2012-01-10 18:04:27 -08:00
ioctl.h
ioctls.h
iomap.h
[PARISC] fix compile break caused by iomap: make IOPORT/PCI mapping functions conditional
2012-02-27 09:43:30 -06:00
ipcbuf.h
irq.h
irq_regs.h
irqflags.h
Kbuild
KVM: Export asm-generic/kvm_para.h
2012-05-29 12:31:01 +03:00
Kbuild.asm
kdebug.h
kmap_types.h
asm-generic: remove km_type definitions
2012-07-24 15:27:30 +08:00
kvm_para.h
KVM: make asm-generic/kvm_para.h have an ifdef __KERNEL__ block
2012-05-21 17:47:52 +03:00
libata-portmap.h
linkage.h
local.h
atomic: use <linux/atomic.h>
2011-07-26 16:49:47 -07:00
local64.h
atomic: use <linux/atomic.h>
2011-07-26 16:49:47 -07:00
memory_model.h
mm: fix __page_to_pfn for a const struct page argument
2011-08-17 13:00:20 -07:00
mm_hooks.h
mman-common.h
coredump: add VM_NODUMP, MADV_NODUMP, MADV_CLEAR_NODUMP
2012-03-23 16:58:42 -07:00
mman.h
mmu.h
mmu_context.h
module.h
msgbuf.h
mutex-dec.h
mutex-null.h
mutex-xchg.h
mutex: Place lock in contended state after fastpath_lock failure
2012-08-13 18:46:54 +02:00
mutex.h
page.h
The following changes since commit 3ee72ca992
2012-01-10 17:39:40 -08:00
param.h
UAPI: Rearrange definition of HZ in asm-generic/param.h
2011-12-13 09:26:45 +00:00
parport.h
pci-bridge.h
PCI: work around Stratus ftServer broken PCIe hierarchy
2012-04-30 15:21:02 -06:00
pci-dma-compat.h
pci.h
PCI: collapse pcibios_resource_to_bus
2012-02-23 20:19:04 -07:00
pci_iomap.h
[PARISC] fix compile break caused by iomap: make IOPORT/PCI mapping functions conditional
2012-02-27 09:43:30 -06:00
percpu.h
pgalloc.h
pgtable-nopmd.h
pgtable-nopud.h
pgtable.h
thp: avoid atomic64_read in pmd_read_atomic for 32bit PAE
2012-06-20 14:39:35 -07:00
poll.h
epoll: introduce POLLFREE to flush ->signalfd_wqh before kfree()
2012-02-24 11:42:50 -08:00
posix_types.h
bury __kernel_nlink_t, make internal nlink_t consistent
2012-05-30 21:04:50 -04:00
ptrace.h
asm-generic/ptrace.h: start a common low level ptrace helper
2011-05-26 17:12:36 -07:00
resource.h
ulimit: raise default hard ulimit on number of files to 4096
2011-05-25 08:39:43 -07:00
rtc.h
rwsem.h
Hexagon: Add locking types and functions
2011-11-01 07:34:20 -07:00
scatterlist.h
sections.h
segment.h
sembuf.h
serial.h
setup.h
shmbuf.h
shmparam.h
siginfo.h
Linux 3.4-rc5
2012-05-04 12:46:40 +10:00
signal-defs.h
signal.h
sizes.h
ARM: 7430/1: sizes.h: move from asm-generic to <linux/sizes.h>
2012-06-28 17:14:34 +01:00
socket.h
net: Add framework to allow sending packets with customized CRC.
2012-02-24 01:37:35 -08:00
sockios.h
spinlock.h
stat.h
statfs.h
asm-generic: Use __BITS_PER_LONG in statfs.h
2012-04-30 12:55:15 -07:00
string.h
swab.h
switch_to.h
Split the switch_to() wrapper out of asm-generic/system.h
2012-03-28 18:30:03 +01:00
syscall.h
asm/syscall.h: add syscall_get_arch
2012-04-14 11:13:19 +10:00
syscalls.h
termbits.h
termios-base.h
termios.h
timex.h
tlb.h
mm/mmu_gather: enable tlb flush range in generic mmu_gather
2012-06-27 19:29:11 -07:00
tlbflush.h
BUG: headers with BUG/BUG_ON etc. need linux/bug.h
2012-03-04 17:54:34 -05:00
topology.h
types.h
consolidate umode_t declarations
2012-01-03 22:55:17 -05:00
uaccess-unaligned.h
uaccess.h
fix default __strnlen_user macro
2011-10-06 19:47:12 -04:00
ucontext.h
unaligned.h
unistd.h
compat: use sys_sendfile64() implementation for sendfile syscall
2012-03-27 13:36:57 -04:00
user.h
vga.h
vmlinux.lds.h
ftrace: Sort all function addresses, not just per page
2012-05-16 19:58:44 -04:00
word-at-a-time.h
word-at-a-time: make the interfaces truly generic
2012-05-26 11:33:40 -07:00
xor.h
sanitize <linux/prefetch.h> usage
2011-05-20 12:50:29 -07:00