Will Deacon
59fb586b4a
locking/qspinlock: Remove unbounded cmpxchg() loop from locking slowpath ...
The qspinlock locking slowpath utilises a "pending" bit as a simple form
of an embedded test-and-set lock that can avoid the overhead of explicit
queuing in cases where the lock is held but uncontended. This bit is
managed using a cmpxchg() loop which tries to transition the uncontended
lock word from (0,0,0) -> (0,0,1) or (0,0,1) -> (0,1,1).
Unfortunately, the cmpxchg() loop is unbounded and lockers can be starved
indefinitely if the lock word is seen to oscillate between unlocked
(0,0,0) and locked (0,0,1). This could happen if concurrent lockers are
able to take the lock in the cmpxchg() loop without queuing and pass it
around amongst themselves.
This patch fixes the problem by unconditionally setting _Q_PENDING_VAL
using atomic_fetch_or, and then inspecting the old value to see whether
we need to spin on the current lock owner, or whether we now effectively
hold the lock. The tricky scenario is when concurrent lockers end up
queuing on the lock and the lock becomes available, causing us to see
a lockword of (n,0,0). With pending now set, simply queuing could lead
to deadlock as the head of the queue may not have observed the pending
flag being cleared. Conversely, if the head of the queue did observe
pending being cleared, then it could transition the lock from (n,0,0) ->
(0,0,1) meaning that any attempt to "undo" our setting of the pending
bit could race with a concurrent locker trying to set it.
We handle this race by preserving the pending bit when taking the lock
after reaching the head of the queue and leaving the tail entry intact
if we saw pending set, because we know that the tail is going to be
updated shortly.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Waiman Long <longman@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: boqun.feng@gmail.com
Cc: linux-arm-kernel@lists.infradead.org
Cc: paulmck@linux.vnet.ibm.com
Link: http://lkml.kernel.org/r/1524738868-31318-6-git-send-email-will.deacon@arm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-04-27 09:48:47 +02:00
bpf: sockmap remove dead check
2018-04-20 22:09:51 +02:00
Merge branch 'for-4.17' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq
2018-04-03 18:00:13 -07:00
KVM changes for 4.16
2018-02-10 13:16:35 -08:00
* Fix 2032 time access issues and new compiler warnings
2018-04-12 10:21:19 -07:00
perf: Remove superfluous allocation error check
2018-04-17 09:47:40 -03:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
genirq/affinity: Spread irq vectors among present CPUs as far as possible
2018-04-06 12:19:51 +02:00
livepatch: Allow to call a custom callback when freeing shadow variables
2018-04-17 13:42:48 +02:00
locking/qspinlock: Remove unbounded cmpxchg() loop from locking slowpath
2018-04-27 09:48:47 +02:00
PM / QoS: mark expected switch fall-throughs
2018-04-09 13:49:40 +02:00
New features:
2018-04-10 11:27:30 -07:00
Merge branches 'fixes.2018.02.23a', 'srcu.2018.02.20a' and 'torture.2018.02.20a' into HEAD
2018-02-23 15:15:41 -08:00
Merge branch 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-04-15 12:43:30 -07:00
posix-cpu-timers: Ensure set_process_cpu_timer is always evaluated
2018-04-19 12:54:57 +02:00
Following tracing fixes:
2018-04-26 16:22:47 -07:00
kernel/acct.c: fix the acct->needcheck check in check_free_space()
2018-01-04 16:45:09 -08:00
kernel/async.c: revert "async: simplify lowest_in_progress()"
2018-02-06 18:32:44 -08:00
audit/stable-4.17 PR 20180403
2018-04-06 15:01:25 -07:00
audit: track the owner of the command mutex ourselves
2018-02-23 11:22:22 -05:00
audit: track the owner of the command mutex ourselves
2018-02-23 11:22:22 -05:00
audit: deprecate the AUDIT_FILTER_ENTRY filter
2018-02-15 14:36:29 -05:00
audit: bail before bug check if audit disabled
2018-02-15 14:40:25 -05:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
mm: add kernel_move_pages() helper, move compat syscall to mm/migrate.c
2018-04-02 20:15:32 +02:00
cpu/hotplug: Fix unused function warning
2018-03-15 20:34:40 +01:00
kexec: export PG_swapbacked to VMCOREINFO
2018-04-13 17:10:27 -07:00
delayacct: Account blkio completion on the correct task
2018-01-16 03:29:36 +01:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
get rid of pointless includes of fs_struct.h
2018-02-22 14:28:50 -05:00
kernel: use kernel_wait4() instead of sys_wait4()
2018-04-02 20:14:51 +02:00
extable: Make init_kernel_text() global
2018-02-21 16:54:06 +01:00
error-injection: Fix to prohibit jump optimization
2018-03-12 16:16:00 +01:00
fork: unconditionally clear stack on fork
2018-04-20 17:18:35 -07:00
pids: introduce find_get_task_by_vpid() helper
2018-02-06 18:32:46 -08:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
kernel: make groups_sort calling a responsibility group_info allocators
2017-12-14 16:00:49 -08:00
irq/work: Improve the flag definitions
2018-01-08 19:43:15 +01:00
jump_label: Disable jump labels in __exit code
2018-03-20 08:57:17 +01:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/pmladek/printk
2018-02-01 13:36:15 -08:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
kcov: detect double association with a single task
2018-02-06 18:32:46 -08:00
kexec: call do_kexec_load() in compat syscall directly
2018-04-02 20:15:01 +02:00
kernel/kexec_file.c: allow archs to set purgatory load address
2018-04-13 17:10:28 -07:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
kprobes: Fix random address output of blacklist file
2018-04-25 10:27:56 -04:00
treewide: Remove TIMER_FUNC_TYPE and TIMER_DATA_TYPE casts
2017-11-21 16:35:54 -08:00
error-injection: Support fault injection framework
2018-01-12 17:33:38 -08:00
kernel/memremap: Remove stale devres_free() call
2018-03-06 10:58:54 -08:00
arch: remove obsolete architecture ports
2018-04-02 20:20:12 -07:00
padata: add SPDX identifier
2018-01-05 18:43:00 +11:00
taint: add taint for randstruct
2018-04-11 10:28:35 -07:00
kernel/params.c: downgrade warning for unsafe parameters
2018-04-11 10:28:37 -07:00
xarray: add the xa_lock to the radix_tree_root
2018-04-11 10:28:39 -07:00
Merge branch 'userns-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2018-04-03 19:15:32 -07:00
pids: introduce find_get_task_by_vpid() helper
2018-02-06 18:32:46 -08:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
kernel/reboot.c: add devm_register_reboot_notifier()
2017-11-17 16:10:04 -08:00
kernel/relay.c: limit kmalloc size to KMALLOC_MAX_SIZE
2018-02-21 15:35:43 -08:00
resource: fix integer overflow at reallocation
2018-04-13 17:10:27 -07:00
- Fix seccomp GET_METADATA to deal with field sizes correctly (Tycho Andersen)
2018-02-22 10:50:24 -08:00
Merge branch 'next-general' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2018-04-07 11:11:41 -07:00
smp/core: Use lockdep to assert IRQs are disabled/enabled
2017-11-08 11:13:50 +01:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
softirq: Consolidate common code in tasklet_[hi]_action()
2018-03-09 11:50:55 +01:00
kernel: add ksys_setsid() helper; remove in-kernel call to sys_setsid()
2018-04-02 20:16:06 +02:00
syscalls/core: Prepare CONFIG_ARCH_HAS_SYSCALL_WRAPPER=y for compat syscalls
2018-04-05 16:59:38 +02:00
kernel/sysctl.c: add kdoc comments to do_proc_do{u}intvec_minmax_conv_param
2018-04-11 10:28:38 -07:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
locking/barriers: Convert users of lockless_dereference() to READ_ONCE()
2017-12-17 13:57:15 +01:00
pids: introduce find_get_task_by_vpid() helper
2018-02-06 18:32:46 -08:00
torture: Save a line in stutter_wait(): while -> for
2017-12-11 09:18:30 -08:00
tracepoint: Remove smp_read_barrier_depends() from comment
2017-12-04 10:52:56 -08:00
headers: untangle kmemleak.h from mm.h
2018-04-05 21:36:27 -07:00
fs: add do_fchownat(), ksys_fchown() helpers and ksys_{,l}chown() wrappers
2018-04-02 20:15:59 +02:00
kernel: provide ksys_*() wrappers for syscalls called by kernel/uid16.c
2018-04-02 20:15:30 +02:00
kernel: use kernel_wait4() instead of sys_wait4()
2018-04-02 20:14:51 +02:00
efivarfs: Limit the rate for non-root to read files
2018-02-22 10:21:02 -08:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2017-11-16 12:20:15 -08:00
uts: create "struct uts_namespace" from kmem_cache
2018-04-11 10:28:35 -07:00
Merge branch 'linus' into sched/core, to pick up fixes
2017-11-08 10:17:15 +01:00
Merge branch 'linus' into core/urgent, to pick up dependent commits
2017-11-04 08:53:04 +01:00
Merge branch 'for-4.17' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq
2018-04-03 18:00:13 -07:00
Merge branch 'for-4.14-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq
2017-11-06 12:26:49 -08:00
59fb586b4a