Peter Zijlstra
2c61002271
locking/qspinlock: Fix spin_unlock_wait() some more
...
While this prior commit:
54cf809b95 ("locking,qspinlock: Fix spin_is_locked() and spin_unlock_wait()")
... fixes spin_is_locked() and spin_unlock_wait() for the usage
in ipc/sem and netfilter, it does not in fact work right for the
usage in task_work and futex.
So while the 2 locks crossed problem:
spin_lock(A) spin_lock(B)
if (!spin_is_locked(B)) spin_unlock_wait(A)
foo() foo();
... works with the smp_mb() injected by both spin_is_locked() and
spin_unlock_wait(), this is not sufficient for:
flag = 1;
smp_mb(); spin_lock()
spin_unlock_wait() if (!flag)
// add to lockless list
// iterate lockless list
... because in this scenario, the store from spin_lock() can be delayed
past the load of flag, uncrossing the variables and loosing the
guarantee.
This patch reworks spin_is_locked() and spin_unlock_wait() to work in
both cases by exploiting the observation that while the lock byte
store can be delayed, the contender must have registered itself
visibly in other state contained in the word.
It also allows for architectures to override both functions, as PPC
and ARM64 have an additional issue for which we currently have no
generic solution.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Giovanni Gherdovich <ggherdovich@suse.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Pan Xinhui <xinhui.pan@linux.vnet.ibm.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Waiman Long <waiman.long@hpe.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: stable@vger.kernel.org # v4.2 and later
Fixes: 54cf809b95 ("locking,qspinlock: Fix spin_is_locked() and spin_unlock_wait()")
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-06-08 14:29:08 +02:00
..
bitops
bitops: Do not default to __clear_bit() for __clear_bit_unlock()
2016-03-21 10:50:48 +01:00
4level-fixup.h
mm, asm-generic: define PUD_SHIFT in <asm-generic/4level-fixup.h>
2015-02-11 17:06:03 -08:00
asm-offsets.h
asm-generic: Add common asm-offsets.h
2015-06-23 13:35:49 +09:00
atomic-long.h
include/asm-generic/atomic-long.h: force inlining of some atomic_long operations
2016-03-17 15:09:34 -07:00
atomic.h
atomic, arch: Audit atomic_{read,set}()
2015-09-23 09:54:28 +02:00
atomic64.h
atomic: Provide atomic_{or,xor,and}
2015-07-27 14:06:24 +02:00
audit_change_attr.h
audit: Modify a set of system calls in audit class definitions
2014-01-17 17:01:46 -05:00
audit_dir_write.h
audit_read.h
audit_signal.h
audit_write.h
audit: Modify a set of system calls in audit class definitions
2014-01-17 17:01:46 -05:00
barrier.h
asm-generic: implement virt_xxx memory barriers
2016-01-12 20:46:59 +02:00
bitops.h
arch: Prepare for smp_mb__{before,after}_atomic()
2014-04-18 11:40:30 +02:00
bitsperlong.h
bug.h
asm-generic changes for 4.6
2016-03-24 23:13:48 -07:00
bugs.h
cache.h
cacheflush.h
checksum.h
ipv4: Update parameters for csum_tcpudp_magic to their original types
2016-03-13 23:55:13 -04:00
clkdev.h
asm-generic: COMMON_CLK defines __clk_{get,put}
2014-09-25 18:00:45 -07:00
cmpxchg-local.h
LLVMLinux: Remove warning about returning an uninitialized variable
2014-04-09 13:44:35 -07:00
cmpxchg.h
asm-generic: cmpxchg: avoid warnings from macro-ized cmpxchg() implementations
2015-10-15 00:21:13 +02:00
cputime.h
cputime_jiffies.h
sched, time: Fix build error with 64 bit cputime_t on 32 bit systems
2014-10-03 05:46:55 +02:00
cputime_nsecs.h
cputime: Prevent 32bit overflow in time[val|spec]_to_cputime()
2016-02-02 15:24:38 +01:00
current.h
delay.h
device.h
div64.h
__div64_32(): make it overridable at compile time
2015-11-16 14:42:12 -05:00
dma-contiguous.h
asm-generic: Add dma-contiguous.h
2014-09-22 13:35:51 +02:00
dma.h
early_ioremap.h
Merge branch 'akpm' (patches from Andrew)
2015-09-08 17:52:23 -07:00
emergency-restart.h
exec.h
fb.h
fixmap.h
asm-generic: Fix local variable shadow in __set_fixmap_offset
2016-02-16 15:10:44 +00:00
ftrace.h
futex.h
asm-generic/futex: Re-enable preemption in futex_atomic_cmpxchg_inatomic()
2016-04-21 11:06:09 +02:00
getorder.h
gpio.h
gpio: allow setting ARCH_NR_GPIOS from Kconfig
2016-02-19 00:22:54 +01:00
hardirq.h
hugetlb.h
hw_irq.h
ide_iops.h
int-ll64.h
io.h
io-64-nonatomic: Add relaxed accessor variants
2016-05-03 18:23:02 +01:00
ioctl.h
include/asm-generic/ioctl.h: fix _IOC_TYPECHECK sparse error
2014-06-06 16:08:13 -07:00
iomap.h
x86/mm, asm-generic: Add ioremap_wt() for creating Write-Through mappings
2015-06-07 15:28:56 +02:00
irq.h
irq_regs.h
irq_work.h
irq_work: Introduce arch_irq_work_has_interrupt()
2014-09-13 18:38:07 +02:00
irqflags.h
Kbuild.asm
kdebug.h
kmap_types.h
kvm_para.h
libata-portmap.h
linkage.h
local.h
local64.h
mcs_spinlock.h
locking/mcs: Allow architecture specific asm files to be used for contended case
2014-02-09 21:18:52 +01:00
memory_model.h
mm: add PHYS_PFN, use it in __phys_to_pfn()
2016-01-14 16:00:49 -08:00
mm-arch-hooks.h
mm: clean up per architecture MM hook header files
2015-07-17 16:39:53 -07:00
mm_hooks.h
mm/core, x86/mm/pkeys: Differentiate instruction fetches
2016-02-18 19:46:29 +01:00
mmu.h
mmu_context.h
asm-generic: Remove asm-generic arch_bprm_mm_init()
2014-11-22 21:52:08 +01:00
module.h
msi.h
asm-generic: Add msi.h
2014-11-23 13:01:47 +01:00
mutex-dec.h
locking/mutex: Use acquire/release semantics
2015-10-06 17:28:20 +02:00
mutex-null.h
mutex-xchg.h
locking/mutex: Use acquire/release semantics
2015-10-06 17:28:20 +02:00
mutex.h
page.h
asm-generic: page.h: Remove useless get_user_page and free_user_page
2016-02-26 15:24:55 +01:00
param.h
parport.h
pci.h
PCI: Remove unused pcibios_select_root() (again)
2015-06-08 07:56:21 -05:00
pci_iomap.h
PCI: Add pci_iomap_wc() variants
2015-08-25 09:59:45 +02:00
percpu.h
percpu: preffity percpu header files
2014-06-17 19:12:40 -04:00
pgalloc.h
pgtable-nopmd.h
pgtable-nopud.h
pgtable.h
arch: fix has_transparent_hugepage()
2016-05-19 19:12:14 -07:00
preempt.h
sched/preempt: Fix preempt_count manipulations
2016-05-17 12:24:21 +02:00
ptrace.h
qrwlock.h
locking/qrwlock: Make use of _{acquire|release|relaxed}() atomics
2015-08-12 11:59:06 +02:00
qrwlock_types.h
locking/qrwlock: Rename ->lock to ->wait_lock
2015-09-18 09:27:29 +02:00
qspinlock.h
locking/qspinlock: Fix spin_unlock_wait() some more
2016-06-08 14:29:08 +02:00
qspinlock_types.h
locking/qspinlock: Move __ARCH_SPIN_LOCK_UNLOCKED to qspinlock_types.h
2016-02-29 10:02:43 +01:00
resource.h
asm-generic: remove _STK_LIM_MAX
2014-05-15 00:32:09 +01:00
rtc.h
rtc: cmos: century support
2015-09-05 13:19:09 +02:00
rwsem.h
locking/rwsem: Introduce basis for down_write_killable()
2016-04-13 10:42:20 +02:00
seccomp.h
seccomp: Get compat syscalls from asm-generic header
2016-05-13 14:02:00 +02:00
sections.h
asm/sections: add helpers to check for section data
2016-01-16 11:17:24 -08:00
segment.h
serial.h
siginfo.h
SIGNAL: Move generic copy_siginfo() to signal.h
2016-05-13 14:02:10 +02:00
signal.h
simd.h
sizes.h
spinlock.h
statfs.h
string.h
switch_to.h
syscall.h
syscall.h: fix doc text for syscall_get_arch()
2014-09-23 16:20:00 -04:00
syscalls.h
termios-base.h
termios.h
timex.h
tlb.h
treewide: Remove old email address
2015-11-23 09:44:58 +01:00
tlbflush.h
topology.h
trace_clock.h
uaccess-unaligned.h
uaccess.h
asm-generic: {get,put}_user ptr argument evaluate only 1 time
2015-11-08 22:44:42 +09:00
unaligned.h
asm-generic: allow generic unaligned access if the arch supports it
2014-05-08 10:22:23 +02:00
unistd.h
user.h
vga.h
vmlinux.lds.h
x86/asm: Make sure verify_cpu() has a good stack
2016-04-13 11:52:19 +02:00
vtime.h
word-at-a-time.h
Make asm/word-at-a-time.h available on all architectures
2015-07-08 16:41:55 -04:00
xor.h