Paul E. McKenney
b8c17e6664
rcu: Maintain special bits at bottom of ->dynticks counter
...
Currently, IPIs are used to force other CPUs to invalidate their TLBs
in response to a kernel virtual-memory mapping change. This works, but
degrades both battery lifetime (for idle CPUs) and real-time response
(for nohz_full CPUs), and in addition results in unnecessary IPIs due to
the fact that CPUs executing in usermode are unaffected by stale kernel
mappings. It would be better to cause a CPU executing in usermode to
wait until it is entering kernel mode to do the flush, first to avoid
interrupting usemode tasks and second to handle multiple flush requests
with a single flush in the case of a long-running user task.
This commit therefore reserves a bit at the bottom of the ->dynticks
counter, which is checked upon exit from extended quiescent states.
If it is set, it is cleared and then a new rcu_eqs_special_exit() macro is
invoked, which, if not supplied, is an empty single-pass do-while loop.
If this bottom bit is set on -entry- to an extended quiescent state,
then a WARN_ON_ONCE() triggers.
This bottom bit may be set using a new rcu_eqs_special_set() function,
which returns true if the bit was set, or false if the CPU turned
out to not be in an extended quiescent state. Please note that this
function refuses to set the bit for a non-nohz_full CPU when that CPU
is executing in usermode because usermode execution is tracked by RCU
as a dyntick-idle extended quiescent state only for nohz_full CPUs.
Reported-by: Andy Lutomirski <luto@amacapital.net>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
2017-04-18 11:19:22 -07:00
..
bpf
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
2017-03-04 17:31:39 -08:00
cgroup
scripts/spelling.txt: add "disble(d)" pattern and fix typo instances
2017-03-09 17:01:09 -08:00
configs
config: android-base: enable hardened usercopy and kernel ASLR
2017-02-27 18:43:46 -08:00
debug
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>
2017-03-02 08:42:34 +01:00
events
scripts/spelling.txt: add "disble(d)" pattern and fix typo instances
2017-03-09 17:01:09 -08:00
gcov
irq
sched/headers: Prepare to move the get_task_struct()/put_task_struct() and related APIs from <linux/sched.h> to <linux/sched/task.h>
2017-03-02 08:42:40 +01:00
livepatch
locking
Merge branch 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2017-03-07 14:33:11 -08:00
power
Merge branch 'WIP.sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2017-03-03 10:16:38 -08:00
printk
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task_stack.h>
2017-03-02 08:42:36 +01:00
rcu
rcu: Maintain special bits at bottom of ->dynticks counter
2017-04-18 11:19:22 -07:00
sched
Power management updates for v4.11-rc2
2017-03-09 16:30:37 -08:00
time
Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2017-03-07 14:45:22 -08:00
trace
scripts/spelling.txt: add "overide" pattern and fix typo instances
2017-03-09 17:01:09 -08:00
.gitignore
acct.c
sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>
2017-03-02 08:42:39 +01:00
async.c
audit.c
Merge branch 'stable-4.11' of git://git.infradead.org/users/pcmoore/audit
2017-02-21 13:25:50 -08:00
audit.h
Merge branch 'stable-4.11' of git://git.infradead.org/users/pcmoore/audit
2017-02-21 13:25:50 -08:00
audit_fsnotify.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2016-12-17 18:44:00 -08:00
audit_tree.c
Merge branch 'stable-4.10' of git://git.infradead.org/users/pcmoore/audit
2017-01-05 23:06:06 -08:00
audit_watch.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2016-12-17 18:44:00 -08:00
auditfilter.c
auditsc.c
Merge branch 'stable-4.11' of git://git.infradead.org/users/pcmoore/audit
2017-02-21 13:25:50 -08:00
backtracetest.c
bounds.c
capability.c
capability: export has_capability
2017-01-12 07:01:56 -07:00
compat.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
configs.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
context_tracking.c
cpu.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>
2017-03-02 08:42:35 +01:00
cpu_pm.c
crash_dump.c
cred.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/coredump.h>
2017-03-02 08:42:28 +01:00
delayacct.c
sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>
2017-03-02 08:42:39 +01:00
dma.c
elfcore.c
exec_domain.c
exit.c
userfaultfd: non-cooperative: rollback userfaultfd_exit
2017-03-09 17:01:09 -08:00
extable.c
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2017-02-22 10:15:09 -08:00
fork.c
sched/headers, RCU: Move rcu_copy_process() from <linux/sched/task.h> to kernel/fork.c
2017-03-03 01:43:46 +01:00
freezer.c
futex.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/mm.h>
2017-03-02 08:42:28 +01:00
futex_compat.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
groups.c
Replace <asm/uaccess.h> with <linux/uaccess.h> globally
2016-12-24 11:46:01 -08:00
hung_task.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>
2017-03-02 08:42:34 +01:00
irq_work.c
jump_label.c
This release has no new tracing features, just clean ups, minor fixes
2017-02-27 13:26:17 -08:00
kallsyms.c
bpf: make jited programs visible in traces
2017-02-17 13:40:05 -05:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c
kcov: make kcov work properly with KASLR enabled
2016-12-20 09:48:47 -08:00
kexec.c
kexec_core.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/pmladek/printk
2017-02-22 17:33:34 -08:00
kexec_file.c
kexec, x86/purgatory: Unbreak it and clean it up
2017-03-10 20:55:09 +01:00
kexec_internal.h
kexec, x86/purgatory: Unbreak it and clean it up
2017-03-10 20:55:09 +01:00
kmod.c
sched/headers, vfs/execve: Prepare to move the do_execve*() prototypes from <linux/sched.h> to <linux/binfmts.h>
2017-03-02 08:42:39 +01:00
kprobes.c
powerpc updates for 4.11 part 1.
2017-02-22 10:30:38 -08:00
ksysfs.c
kernel/ksysfs.c: add __ro_after_init to bin_attribute structure
2017-02-24 17:46:56 -08:00
kthread.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>
2017-03-02 08:42:35 +01:00
latencytop.c
sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>
2017-03-02 08:42:39 +01:00
Makefile
cgroup: move cgroup files under kernel/cgroup/
2016-12-27 14:49:05 -05:00
membarrier.c
Fix: Disable sys_membarrier when nohz_full is enabled
2017-01-23 11:32:16 -08:00
memremap.c
mm, devm_memremap_pages: hold device_hotplug lock over mem_hotplug_{begin, done}
2017-02-24 17:46:53 -08:00
module-internal.h
module.c
Modules updates for v4.11
2017-02-22 17:08:33 -08:00
module_signing.c
notifier.c
kernel/notifier.c: simplify expression
2017-02-24 17:46:56 -08:00
nsproxy.c
padata.c
panic.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>
2017-03-02 08:42:34 +01:00
params.c
pid.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>
2017-03-02 08:42:35 +01:00
pid_namespace.c
sched/headers: Prepare for the reduction of <linux/sched.h>'s signal API dependency
2017-03-02 08:42:37 +01:00
profile.c
sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>
2017-03-02 08:42:39 +01:00
ptrace.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>
2017-03-02 08:42:35 +01:00
range.c
reboot.c
relay.c
lib/vsprintf.c: remove %Z support
2017-02-27 18:43:47 -08:00
resource.c
seccomp.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task_stack.h>
2017-03-02 08:42:36 +01:00
signal.c
sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>
2017-03-02 08:42:39 +01:00
smp.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/idle.h>
2017-03-02 08:42:26 +01:00
smpboot.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>
2017-03-02 08:42:35 +01:00
smpboot.h
softirq.c
stacktrace.c
stacktrace, lockdep: Fix address, newline ugliness
2017-02-08 08:21:31 +01:00
stop_machine.c
sys.c
sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>
2017-03-02 08:42:39 +01:00
sys_ni.c
move aio compat to fs/aio.c
2016-12-22 22:58:37 -05:00
sysctl.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/coredump.h>
2017-03-02 08:42:28 +01:00
sysctl_binary.c
sysctl: add KERN_CONT to deprecated_sysctl_warning()
2016-12-14 16:04:07 -08:00
task_work.c
taskstats.c
test_kprobes.c
torture.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/clock.h>
2017-03-02 08:42:27 +01:00
tracepoint.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>
2017-03-02 08:42:35 +01:00
tsacct.c
sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>
2017-03-02 08:42:39 +01:00
ucount.c
ucount: Remove the atomicity from ucount->count
2017-03-06 15:26:37 -06:00
uid16.c
sched/headers: Prepare to remove <linux/cred.h> inclusion from <linux/sched.h>
2017-03-02 08:42:31 +01:00
up.c
user-return-notifier.c
user.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/user.h>
2017-03-02 08:42:29 +01:00
user_namespace.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/signal.h>
2017-03-02 08:42:29 +01:00
utsname.c
sched/headers: Prepare to move the task_lock()/unlock() APIs to <linux/sched/task.h>
2017-03-02 08:42:38 +01:00
utsname_sysctl.c
sched/headers: Remove <linux/rwsem.h> from <linux/sched.h>
2017-03-03 01:45:36 +01:00
watchdog.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>
2017-03-02 08:42:34 +01:00
watchdog_hld.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>
2017-03-02 08:42:34 +01:00
workqueue.c
time: Remove CONFIG_TIMER_STATS
2017-02-10 11:15:08 +01:00
workqueue_internal.h