Paul E. McKenney
7765be2fec
rcu: Fix RCU_BOOST race handling current->rcu_read_unlock_special ...
The RCU_BOOST commits for TREE_PREEMPT_RCU introduced an other-task
write to a new RCU_READ_UNLOCK_BOOSTED bit in the task_struct structure's
->rcu_read_unlock_special field, but, as noted by Steven Rostedt, without
correctly synchronizing all accesses to ->rcu_read_unlock_special.
This could result in bits in ->rcu_read_unlock_special being spuriously
set and cleared due to conflicting accesses, which in turn could result
in deadlocks between the rcu_node structure's ->lock and the scheduler's
rq and pi locks. These deadlocks would result from RCU incorrectly
believing that the just-ended RCU read-side critical section had been
preempted and/or boosted. If that RCU read-side critical section was
executed with either rq or pi locks held, RCU's ensuing (incorrect)
calls to the scheduler would cause the scheduler to attempt to once
again acquire the rq and pi locks, resulting in deadlock. More complex
deadlock cycles are also possible, involving multiple rq and pi locks
as well as locks from multiple rcu_node structures.
This commit fixes synchronization by creating ->rcu_boosted field in
task_struct that is accessed and modified only when holding the ->lock
in the rcu_node structure on which the task is queued (on that rcu_node
structure's ->blkd_tasks list). This results in tasks accessing only
their own current->rcu_read_unlock_special fields, making unsynchronized
access once again legal, and keeping the rcu_read_unlock() fastpath free
of atomic instructions and memory barriers.
The reason that the rcu_read_unlock() fastpath does not need to access
the new current->rcu_boosted field is that this new field cannot
be non-zero unless the RCU_READ_UNLOCK_BLOCKED bit is set in the
current->rcu_read_unlock_special field. Therefore, rcu_read_unlock()
need only test current->rcu_read_unlock_special: if that is zero, then
current->rcu_boosted must also be zero.
This bug does not affect TINY_PREEMPT_RCU because this implementation
of RCU accesses current->rcu_read_unlock_special with irqs disabled,
thus preventing races on the !SMP systems that TINY_PREEMPT_RCU runs on.
Maybe-reported-by: Dave Jones <davej@redhat.com>
Maybe-reported-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Reported-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Steven Rostedt <rostedt@goodmis.org>
2011-07-19 21:38:52 -07:00
Fix common misspellings
2011-03-31 11:26:23 -03:00
Merge branch 'perf/urgent' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/urgent
2011-06-04 12:13:06 +02:00
gcov: disable CONFIG_CONSTRUCTORS when not needed by CONFIG_GCOV_KERNEL
2011-06-15 20:04:01 -07:00
genirq: Prevent potential NULL dereference in irq_set_irq_wake()
2011-06-10 10:53:42 +02:00
PM / Hibernate: Fix free_unnecessary_pages()
2011-07-06 20:15:23 +02:00
alarmtimers: Return -ENOTSUPP if no RTC device is present
2011-06-21 16:32:28 -07:00
tracing: Fix regression in printk_formats file
2011-06-09 08:42:15 -04:00
netlink: kill loginuid/sessionid/sid members from struct netlink_skb_parms
2011-03-03 10:55:40 -08:00
audit: make functions static
2010-10-30 01:42:19 -04:00
Fix common misspellings
2011-03-31 11:26:23 -03:00
kill path_lookup()
2011-03-14 09:15:23 -04:00
netlink: kill loginuid/sessionid/sid members from struct netlink_skb_parms
2011-03-03 10:55:40 -08:00
audit: acquire creds selectively to reduce atomic op overhead
2011-04-27 15:11:03 +02:00
memcg: remove direct page_cgroup-to-page pointer
2011-03-23 19:46:28 -07:00
Merge branch 'master' into next
2011-05-19 18:51:57 +10:00
cgroup: remove the ns_cgroup
2011-05-26 17:12:34 -07:00
cgroups: add per-thread subsystem callbacks
2011-05-26 17:12:34 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/cmetcalf/linux-tile
2011-05-25 15:35:32 -07:00
llseek: automatically add .llseek fop
2010-10-15 15:53:27 +02:00
Fix common misspellings
2011-03-31 11:26:23 -03:00
cpuset: Fix cpuset_cpus_allowed_fallback(), don't update tsk->rt.nr_cpus_allowed
2011-05-28 17:02:57 +02:00
crash_dump: export is_kdump_kernel to modules, consolidate elfcorehdr_addr, setup_elfcorehdr and saved_max_pfn
2011-03-23 19:47:19 -07:00
Merge branch 'docs-move' of git://git.kernel.org/pub/scm/linux/kernel/git/rdunlap/linux-docs
2011-05-27 10:25:02 -07:00
memcg: clear mm->owner when last possible owner leaves
2011-06-15 20:04:01 -07:00
extable, core_kernel_data(): Make sure all archs define _sdata
2011-05-20 08:56:56 +02:00
mm: Fix boot crash in mm_alloc()
2011-05-29 11:32:28 -07:00
Freezer: Use SMP barriers
2011-05-17 23:19:17 +02:00
futex: Set FLAGS_HAS_TIMEOUT during futex_wait restart setup
2011-04-15 16:34:32 +02:00
userns: user namespaces: convert several capable() calls
2011-03-23 19:47:08 -07:00
userns: user namespaces: convert several capable() calls
2011-03-23 19:47:08 -07:00
hrtimers: Fix typo causing erratic timers
2011-05-25 15:31:58 -07:00
watchdog, hung_task_timeout: Add Kconfig configurable default
2011-04-28 09:13:17 +02:00
irq_work: Use per cpu atomics instead of regular atomics
2010-12-18 15:54:48 +01:00
jump_label: Fix jump_label update for modules
2011-06-29 09:59:17 -04:00
Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2011-03-25 17:52:22 -07:00
arch:Kconfig.locks Remove unused config option.
2011-04-10 17:01:05 +02:00
PM: Remove sysdev suspend, resume and shutdown operations
2011-05-11 21:37:15 +02:00
kfifo: fix scatterlist usage
2010-10-01 10:50:58 -07:00
KEYS/DNS: Fix ____call_usermodehelper() to not lose the session keyring
2011-06-17 09:40:48 -07:00
Merge branch 'for-2.6.38' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu
2011-01-07 17:02:58 -08:00
kernel/ksysfs.c: expose file_caps_enabled in sysfs
2011-04-19 16:45:51 -07:00
cpuset: Fix cpuset_cpus_allowed_fallback(), don't update tsk->rt.nr_cpus_allowed
2011-05-28 17:02:57 +02:00
Fix common misspellings
2011-03-31 11:26:23 -03:00
lockdep: Fix lock_is_held() on recursion
2011-06-07 12:25:50 +02:00
lockdep: Remove unused 'factor' variable from lockdep_stats_show()
2011-03-23 13:54:47 +01:00
cgroup: remove the ns_cgroup
2011-05-26 17:12:34 -07:00
Merge branch 'staging-next' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging-2.6
2011-05-23 12:49:28 -07:00
mutex: Use p->on_cpu for the adaptive spin
2011-04-14 08:52:33 +02:00
mutex: Use p->on_cpu for the adaptive spin
2011-04-14 08:52:33 +02:00
lockdep, mutex: provide mutex_lock_nest_lock
2011-05-25 08:39:17 -07:00
mutex: Use p->on_cpu for the adaptive spin
2011-04-14 08:52:33 +02:00
cgroup: remove the ns_cgroup
2011-05-26 17:12:34 -07:00
Fix common misspellings
2011-03-31 11:26:23 -03:00
move x86 specific oops=panic to generic code
2011-03-22 17:44:11 -07:00
params.c: Use new strtobool function to process boolean inputs
2011-05-19 16:55:28 +09:30
next_pidmap: fix overflow condition
2011-04-18 10:35:30 -07:00
pidns: call pid_ns_prepare_proc() from create_pid_namespace()
2011-03-23 19:46:58 -07:00
Merge branch 'idle-release' of git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-idle-2.6
2011-05-29 11:18:09 -07:00
hrtimers: Avoid touching inactive timer bases
2011-05-23 13:59:54 +02:00
posix-timers: RCU conversion
2011-05-24 12:10:51 +02:00
printk: allocate kernel log buffer earlier
2011-05-25 08:39:48 -07:00
kernel/profile.c: remove some duplicate code from profile_hits()
2011-05-26 17:12:37 -07:00
ptrace: ptrace_resume() shouldn't wake up !TASK_TRACED thread
2011-05-25 19:20:21 +02:00
kernel/range.c: fix clean_sort_range() for the case of full array
2010-11-12 07:55:31 -08:00
rcu: Use WARN_ON_ONCE for DEBUG_OBJECTS_RCU_HEAD warnings
2011-05-05 23:16:57 -07:00
sanitize <linux/prefetch.h> usage
2011-05-20 12:50:29 -07:00
rcu: Converge TINY_RCU expedited and normal boosting
2011-05-05 23:16:58 -07:00
rcu: mark rcutorture boosting callback as being on-stack
2011-05-05 23:16:57 -07:00
rcu: Prevent RCU callbacks from executing before scheduler initialized
2011-07-13 08:17:56 -07:00
rcu: Move RCU_BOOST #ifdefs to header file
2011-06-16 16:12:05 -07:00
rcu: Fix RCU_BOOST race handling current->rcu_read_unlock_special
2011-07-19 21:38:52 -07:00
rcu: use softirq instead of kthreads except when RCU_BOOST=y
2011-06-15 23:07:21 -07:00
Clean up relay_alloc_page_array() slightly by using vzalloc rather than vmalloc and memset
2010-11-05 08:21:34 -07:00
memcg: res_counter_read_u64(): fix potential races on 32-bit machines
2011-03-23 19:46:22 -07:00
resource: ability to resize an allocated resource
2011-07-06 10:54:08 -07:00
rtmutex: Simplify PI algorithm and make highest prio task get lock
2011-01-27 21:13:51 -05:00
rtmutex: tester: Remove the remaining BKL leftovers
2011-02-22 22:07:22 +01:00
rtmutex: Simplify PI algorithm and make highest prio task get lock
2011-01-27 21:13:51 -05:00
rtmutex: Simplify PI algorithm and make highest prio task get lock
2011-01-27 21:13:51 -05:00
sched, cgroups: Fix MIN_SHARES on 64-bit boxen
2011-07-01 10:25:03 +02:00
Fix common misspellings
2011-03-31 11:26:23 -03:00
sched, autogroup: Stop going ahead if autogroup is disabled
2011-02-23 11:33:59 +01:00
sched: Add some clock info to sched_debug
2010-11-23 10:29:08 +01:00
sched: Get rid of lock_depth
2011-04-24 13:18:38 +02:00
sched: Fix ->min_vruntime calculation in dequeue_entity()
2011-05-28 17:02:56 +02:00
sched: Move the second half of ttwu() to the remote cpu
2011-04-14 08:52:41 +02:00
sched: Drop the rq argument to sched_class::select_task_rq()
2011-04-14 08:52:36 +02:00
sched: Check if lowest_mask is initialized in find_lowest_rq()
2011-06-15 11:44:48 +02:00
sched: More sched_domain iterations fixes
2011-05-28 17:02:54 +02:00
sched: Drop the rq argument to sched_class::select_task_rq()
2011-04-14 08:52:36 +02:00
signal.c: fix kernel-doc notation
2011-06-14 19:12:17 -07:00
generic-ipi: Fix kexec boot crash by initializing call_single_queue before enabling interrupts
2011-06-17 10:17:12 +02:00
rcu: Use softirq to address performance regression
2011-06-14 15:25:39 -07:00
rcu: demote SRCU_SYNCHRONIZE_DELAY from kernel-parameter status
2011-01-14 04:56:49 -08:00
kthread: use kthread_create_on_node()
2011-03-22 17:44:01 -07:00
Merge branch 'driver-core-next' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core-2.6
2011-05-19 18:24:11 -07:00
ipc: Add missing sys_ni entries for ipc/compat.c functions
2011-05-20 13:53:02 -07:00
perf: Comment /proc/sys/kernel/perf_event_paranoid to be part of user ABI
2011-06-04 12:22:04 +02:00
open-style analog of vfs_path_lookup()
2011-03-14 09:15:28 -04:00
sysctl_check: drop dead code
2011-03-23 19:46:51 -07:00
taskstats: don't allow duplicate entries in listener mode
2011-06-27 18:00:13 -07:00
kprobes: Fix selftest to clear flags field for reusing probes
2010-10-14 08:55:27 +02:00
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2011-03-15 18:53:35 -07:00
timers: Consider slack value in mod_timer()
2011-06-03 15:02:32 +02:00
jump label: Introduce static_branch() interface
2011-04-04 12:48:08 -04:00
taskstats: use real microsecond granularity for CPU times
2010-10-27 18:03:17 -07:00
userns: user namespaces: convert several capable() calls
2011-03-23 19:47:08 -07:00
Fix common misspellings
2011-03-31 11:26:23 -03:00
userns: add a user_namespace as creator/owner of uts_namespace
2011-03-23 19:46:59 -07:00
user_ns: improve the user_ns on-the-slab packaging
2011-01-13 08:03:18 -08:00
ns proc: Add support for the uts namespace
2011-05-10 14:35:35 -07:00
Fix common misspellings
2011-03-31 11:26:23 -03:00
kernel/watchdog.c: Use proper ANSI C prototypes
2011-05-23 21:07:40 -07:00
Merge branch 'for-2.6.40' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu
2011-05-24 11:53:42 -07:00
7765be2fec