Paul E. McKenney
8203d6d0ee
rcu: Use single-stage IPI algorithm for RCU expedited grace period
...
The current preemptible-RCU expedited grace-period algorithm invokes
synchronize_sched_expedited() to enqueue all tasks currently running
in a preemptible-RCU read-side critical section, then waits for all the
->blkd_tasks lists to drain. This works, but results in both an IPI and
a double context switch even on CPUs that do not happen to be running
in a preemptible RCU read-side critical section.
This commit implements a new algorithm that causes less OS jitter.
This new algorithm IPIs all online CPUs that are not idle (from an
RCU perspective), but refrains from self-IPIs. If a CPU receiving
this IPI is not in a preemptible RCU read-side critical section (or
is just now exiting one), it pushes quiescence up the rcu_node tree,
otherwise, it sets a flag that will be handled by the upcoming outermost
rcu_read_unlock(), which will then push quiescence up the tree.
The expedited grace period must of course wait on any pre-existing blocked
readers, and newly blocked readers must be queued carefully based on
the state of both the normal and the expedited grace periods. This
new queueing approach also avoids the need to update boost state,
courtesy of the fact that blocked tasks are no longer ever migrated to
the root rcu_node structure.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2015-09-20 21:16:19 -07:00
..
bpf
bpf: fix out of bounds access in verifier log
2015-09-09 14:11:55 -07:00
configs
kconfig: add xenconfig defconfig helper
2015-06-16 11:04:29 +01:00
debug
events
kexec: split kexec_load syscall from kexec core code
2015-09-10 13:29:01 -07:00
gcov
gcov: add support for GCC 5.1
2015-06-30 19:44:57 -07:00
irq
genirq: Remove irq argument from irq flow handlers
2015-09-16 15:47:51 +02:00
livepatch
livepatch: Improve error handling in klp_disable_func()
2015-07-14 22:48:06 +02:00
locking
Merge branch 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2015-09-17 08:45:23 -07:00
power
Merge branch 'for-4.3/core' of git://git.kernel.dk/linux-block
2015-09-02 13:10:25 -07:00
printk
kexec: split kexec_load syscall from kexec core code
2015-09-10 13:29:01 -07:00
rcu
rcu: Use single-stage IPI algorithm for RCU expedited grace period
2015-09-20 21:16:19 -07:00
sched
Mostly stable material, a lot of ARM fixes.
2015-09-18 09:23:08 -07:00
time
Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2015-09-17 10:55:25 -07:00
trace
Mostly this is just clean ups and micro optimizations.
2015-09-08 14:04:14 -07:00
.gitignore
acct.c
async.c
audit.c
Merge branch 'upstream' of git://git.infradead.org/users/pcmoore/audit
2015-09-08 13:34:59 -07:00
audit.h
Merge branch 'upstream' of git://git.infradead.org/users/pcmoore/audit
2015-09-08 13:34:59 -07:00
audit_fsnotify.c
audit: clean simple fsnotify implementation
2015-08-06 16:14:53 -04:00
audit_tree.c
Merge branch 'upstream' of git://git.infradead.org/users/pcmoore/audit
2015-09-08 13:34:59 -07:00
audit_watch.c
Merge branch 'upstream' of git://git.infradead.org/users/pcmoore/audit
2015-09-08 13:34:59 -07:00
auditfilter.c
audit: implement audit by executable
2015-08-06 16:17:25 -04:00
auditsc.c
Merge branch 'upstream' of git://git.infradead.org/users/pcmoore/audit
2015-09-08 13:34:59 -07:00
backtracetest.c
bounds.c
capability.c
cgroup.c
cgroup: fix seq_show_option merge with legacy_name
2015-09-08 15:35:28 -07:00
cgroup_freezer.c
cgroup: allow a cgroup subsystem to reject a fork
2015-07-14 17:29:23 -04:00
cgroup_pids.c
cgroup: pids: fix invalid get/put usage
2015-08-25 14:19:25 -04:00
compat.c
compat: cleanup coding in compat_get_bitmap() and compat_put_bitmap()
2015-06-04 23:57:18 +02:00
configs.c
context_tracking.c
context_tracking: Inherit TIF_NOHZ through forks instead of context switches
2015-05-07 12:02:51 +02:00
cpu.c
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2015-08-31 20:26:22 -07:00
cpu_pm.c
kernel/cpu_pm: fix cpu_cluster_pm_exit comment
2015-09-03 02:42:20 +02:00
cpuset.c
cpuset: use trialcs->mems_allowed as a temp variable
2015-08-10 11:18:41 -04:00
crash_dump.c
cred.c
kernel/cred.c: remove unnecessary kdebug atomic reads
2015-09-10 13:29:01 -07:00
delayacct.c
dma.c
elfcore.c
exec_domain.c
exit.c
kernel: exit: fix typo in comment
2015-08-07 13:59:49 +02:00
extable.c
kernel/extable.c: remove duplicated include
2015-09-10 13:29:01 -07:00
fork.c
userfaultfd: add VM_UFFD_MISSING and VM_UFFD_WP
2015-09-04 16:54:41 -07:00
freezer.c
futex.c
futex: Make should_fail_futex() static
2015-07-20 21:43:54 +02:00
futex_compat.c
groups.c
hung_task.c
irq_work.c
jump_label.c
locking/static_keys: Add selftest
2015-08-03 11:34:16 +02:00
kallsyms.c
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
locking/qrwlock: Rename QUEUE_RWLOCK to QUEUED_RWLOCKS
2015-05-12 09:46:00 +02:00
Kconfig.preempt
kexec.c
kexec: split kexec_load syscall from kexec core code
2015-09-10 13:29:01 -07:00
kexec_core.c
kexec: export KERNEL_IMAGE_SIZE to vmcoreinfo
2015-09-10 13:29:01 -07:00
kexec_file.c
kexec: split kexec_file syscall code to kexec_file.c
2015-09-10 13:29:01 -07:00
kexec_internal.h
kexec: split kexec_file syscall code to kexec_file.c
2015-09-10 13:29:01 -07:00
kmod.c
kmod: handle UMH_WAIT_PROC from system unbound workqueue
2015-09-10 13:29:01 -07:00
kprobes.c
perf/x86/hw_breakpoints: Disallow kernel breakpoints unless kprobe-safe
2015-08-04 10:16:54 +02:00
ksysfs.c
kexec: split kexec_load syscall from kexec core code
2015-09-10 13:29:01 -07:00
kthread.c
kernel/kthread.c:kthread_create_on_node(): clarify documentation
2015-09-04 16:54:41 -07:00
latencytop.c
Makefile
sys_membarrier(): system-wide memory barrier (generic, x86)
2015-09-11 15:21:34 -07:00
membarrier.c
sys_membarrier(): system-wide memory barrier (generic, x86)
2015-09-11 15:21:34 -07:00
memremap.c
add devm_memremap_pages
2015-08-27 19:40:58 -04:00
module-internal.h
module.c
module: weaken locking assertion for oops path.
2015-07-29 06:13:22 +09:30
module_signing.c
PKCS#7: Appropriately restrict authenticated attributes and content type
2015-08-12 17:01:01 +01:00
notifier.c
Merge branch 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2015-09-01 08:40:25 -07:00
nsproxy.c
padata.c
panic.c
kernel/panic/kexec: fix "crash_kexec_post_notifiers" option issue in oops path
2015-06-30 19:44:57 -07:00
params.c
Minor merge needed, due to function move.
2015-07-01 10:49:25 -07:00
pid.c
rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()
2015-07-22 15:27:32 -07:00
pid_namespace.c
profile.c
mm: rename alloc_pages_exact_node() to __alloc_pages_node()
2015-09-08 15:35:28 -07:00
ptrace.c
seccomp: add ptrace options for suspend/resume
2015-07-15 11:52:52 -07:00
range.c
reboot.c
kexec: split kexec_load syscall from kexec core code
2015-09-10 13:29:01 -07:00
relay.c
kernel/relay.c: use kvfree() in relay_free_page_array()
2015-06-30 19:44:59 -07:00
resource.c
mm: enhance region_is_ram() to region_intersects()
2015-08-10 23:07:05 -04:00
seccomp.c
Merge tag 'seccomp-next' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux into next
2015-07-20 17:19:19 +10:00
signal.c
signal: fix information leak in copy_siginfo_to_user
2015-08-07 04:39:40 +03:00
smp.c
smpboot.c
smpboot: allow passing the cpumask on per-cpu thread registration
2015-09-04 16:54:41 -07:00
smpboot.h
softirq.c
stacktrace.c
stop_machine.c
stop_machine: Remove cpu_stop_work's from list in cpu_stop_park()
2015-08-03 12:21:28 +02:00
sys.c
vfs: Commit to never having exectuables on proc and sysfs.
2015-07-10 10:39:25 -05:00
sys_ni.c
sys_membarrier(): system-wide memory barrier (generic, x86)
2015-09-11 15:21:34 -07:00
sysctl.c
sysctl: fix int -> unsigned long assignments in INT_MIN case
2015-09-10 13:29:01 -07:00
sysctl_binary.c
task_work.c
task_work: remove fifo ordering guarantee
2015-09-05 13:46:58 -07:00
taskstats.c
test_kprobes.c
torture.c
rcu: Convert ACCESS_ONCE() to READ_ONCE() and WRITE_ONCE()
2015-05-27 12:56:15 -07:00
tracepoint.c
tsacct.c
uid16.c
up.c
user-return-notifier.c
user.c
user_namespace.c
capabilities: ambient capabilities
2015-09-04 16:54:41 -07:00
utsname.c
utsname_sysctl.c
watchdog.c
watchdog: rename watchdog_suspend() and watchdog_resume()
2015-09-04 16:54:41 -07:00
workqueue.c
Merge branch 'for-4.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq
2015-09-02 08:02:20 -07:00
workqueue_internal.h