No description
  • C 98.2%
  • Assembly 1%
  • Makefile 0.3%
  • Shell 0.2%
  • Python 0.1%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Peter Zijlstra 7aa54be297 locking/qspinlock, x86: Provide liveness guarantee
On x86 we cannot do fetch_or() with a single instruction and thus end up
using a cmpxchg loop, this reduces determinism. Replace the fetch_or()
with a composite operation: tas-pending + load.

Using two instructions of course opens a window we previously did not
have. Consider the scenario:

	CPU0		CPU1		CPU2

 1)	lock
	  trylock -> (0,0,1)

 2)			lock
			  trylock /* fail */

 3)	unlock -> (0,0,0)

 4)					lock
					  trylock -> (0,0,1)

 5)			  tas-pending -> (0,1,1)
			  load-val <- (0,1,0) from 3

 6)			  clear-pending-set-locked -> (0,0,1)

			  FAIL: _2_ owners

where 5) is our new composite operation. When we consider each part of
the qspinlock state as a separate variable (as we can when
_Q_PENDING_BITS == 8) then the above is entirely possible, because
tas-pending will only RmW the pending byte, so the later load is able
to observe prior tail and lock state (but not earlier than its own
trylock, which operates on the whole word, due to coherence).

To avoid this we need 2 things:

 - the load must come after the tas-pending (obviously, otherwise it
   can trivially observe prior state).

 - the tas-pending must be a full word RmW instruction, it cannot be an XCHGB for
   example, such that we cannot observe other state prior to setting
   pending.

On x86 we can realize this by using "LOCK BTS m32, r32" for
tas-pending followed by a regular load.

Note that observing later state is not a problem:

 - if we fail to observe a later unlock, we'll simply spin-wait for
   that store to become visible.

 - if we observe a later xchg_tail(), there is no difference from that
   xchg_tail() having taken place before the tas-pending.

Suggested-by: Will Deacon <will.deacon@arm.com>
Reported-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: andrea.parri@amarulasolutions.com
Cc: longman@redhat.com
Fixes: 59fb586b4a ("locking/qspinlock: Remove unbounded cmpxchg() loop from locking slowpath")
Link: https://lkml.kernel.org/r/20181003130957.183726335@infradead.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-10-16 17:33:54 +02:00
arch locking/qspinlock, x86: Provide liveness guarantee 2018-10-16 17:33:54 +02:00
block blk-mq: I/O and timer unplugs are inverted in blktrace 2018-09-27 13:12:44 -06:00
certs export.h: remove VMLINUX_SYMBOL() and VMLINUX_SYMBOL_STR() 2018-08-22 23:21:44 +09:00
crypto
Documentation Merge branch 'x86/build' into locking/core, to pick up dependent patches and unify jump-label work 2018-10-16 17:30:11 +02:00
drivers Merge branch 'linus' into x86/core, to pick up fixes 2018-10-04 08:23:03 +02:00
firmware
fs Merge branch 'linus' into x86/core, to pick up fixes 2018-10-04 08:23:03 +02:00
include Merge branch 'x86/build' into locking/core, to pick up dependent patches and unify jump-label work 2018-10-16 17:30:11 +02:00
init jump_label: Annotate entries that operate on __init code earlier 2018-09-27 17:56:48 +02:00
ipc ipc/shm: properly return EIDRM in shm_lock() 2018-09-04 16:45:02 -07:00
kernel locking/qspinlock, x86: Provide liveness guarantee 2018-10-16 17:33:54 +02:00
lib lib/Kconfig.debug: fix three typos in help text 2018-09-04 16:45:02 -07:00
LICENSES
mm Merge branch 'linus' into x86/core, to pick up fixes 2018-10-04 08:23:03 +02:00
net ip_tunnel: be careful when accessing the inner header 2018-09-24 12:27:04 -07:00
samples
scripts kbuild/Makefile: Prepare for using macros in inline assembly code to work around asm() related GCC inlining bugs 2018-10-04 10:57:09 +02:00
security Revert "uapi/linux/keyctl.h: don't use C++ reserved keyword as a struct member name" 2018-09-25 13:28:58 +02:00
sound ASoC: Fixes for v4.19 2018-09-17 18:59:21 +02:00
tools Merge branch 'x86/build' into locking/core, to pick up dependent patches and unify jump-label work 2018-10-16 17:30:11 +02:00
usr initramfs: move gen_initramfs_list.sh from scripts/ to usr/ 2018-08-22 23:21:44 +09:00
virt KVM: Remove obsolete kvm_unmap_hva notifier backend 2018-09-07 15:06:02 +02:00
.clang-format
.cocciconfig
.get_maintainer.ignore
.gitattributes
.gitignore
.mailmap libnvdimm-for-4.19_misc 2018-08-25 18:13:10 -07:00
COPYING
CREDITS
Kbuild
Kconfig
MAINTAINERS fbdev fixes for v4.19-rc7: 2018-10-02 05:19:43 -07:00
Makefile kbuild/Makefile: Prepare for using macros in inline assembly code to work around asm() related GCC inlining bugs 2018-10-04 10:57:09 +02:00
README

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.
See Documentation/00-INDEX for a list of what is contained in each file.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.