Heiko Carstens
27c379f7f8
generic-ipi: Fix deadlock in __smp_call_function_single
...
Just got my 6 way machine to a state where cpu 0 is in an
endless loop within __smp_call_function_single.
All other cpus are idle.
The call trace on cpu 0 looks like this:
__smp_call_function_single
scheduler_tick
update_process_times
tick_sched_timer
__run_hrtimer
hrtimer_interrupt
clock_comparator_work
do_extint
ext_int_handler
----> timer irq
cpu_idle
__smp_call_function_single() got called from nohz_balancer_kick()
(inlined) with the remote cpu being 1, wait being 0 and the per
cpu variable remote_sched_softirq_cb (call_single_data) of the
current cpu (0).
Then it loops forever when it tries to grab the lock of the
call_single_data, since it is already locked and enqueued on cpu 0.
My theory how this could have happened: for some reason the
scheduler decided to call __smp_call_function_single() on it's own
cpu, and sends an IPI to itself. The interrupt stays pending
since IRQs are disabled. If then the hypervisor schedules the
cpu away it might happen that upon rescheduling both the IPI and
the timer IRQ are pending. If then interrupts are enabled again
it depends which one gets scheduled first.
If the timer interrupt gets delivered first we end up with the
local deadlock as seen in the calltrace above.
Let's make __smp_call_function_single() check if the target cpu is
the current cpu and execute the function immediately just like
smp_call_function_single does. That should prevent at least the
scenario described here.
It might also be that the scheduler is not supposed to call
__smp_call_function_single with the remote cpu being the current
cpu, but that is a different issue.
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: Jens Axboe <jaxboe@fusionio.com>
Cc: Venkatesh Pallipadi <venki@google.com>
Cc: Suresh Siddha <suresh.b.siddha@intel.com>
LKML-Reference: <20100910114729.GB2827@osiris.boeblingen.de.ibm.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-09-10 16:48:40 +02:00
..
debug
Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2010-09-08 11:13:42 -07:00
gcov
gcov: fix null-pointer dereference for certain module types
2010-09-09 18:57:23 -07:00
irq
irq: Add new IRQ flag IRQF_NO_SUSPEND
2010-07-29 13:24:57 +02:00
power
swap: revert special hibernation allocation
2010-09-09 18:57:25 -07:00
time
time: Workaround gcc loop optimization that causes 64bit div errors
2010-08-13 12:03:24 -07:00
trace
Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2010-09-08 11:13:42 -07:00
.gitignore
acct.c
pass a struct path to vfs_statfs
2010-08-09 16:48:42 -04:00
async.c
async: use workqueue for worker pool
2010-07-14 11:29:46 +02:00
audit.c
Merge branch 'for-linus' of git://git.infradead.org/users/eparis/notify
2010-08-10 11:39:13 -07:00
audit.h
Audit: split audit watch Kconfig
2010-07-28 09:58:19 -04:00
audit_tree.c
fanotify: use both marks when possible
2010-07-28 10:18:55 -04:00
audit_watch.c
Revert "fsnotify: store struct file not struct path"
2010-08-12 14:23:04 -07:00
auditfilter.c
audit: do not get and put just to free a watch
2010-07-28 09:58:17 -04:00
auditsc.c
vfs: add helpers to get root and pwd
2010-08-11 00:28:20 -04:00
backtracetest.c
bounds.c
kbuild: move bounds.h to include/generated
2009-12-12 13:08:14 +01:00
capability.c
sched: Remove remaining USER_SCHED code
2010-04-02 20:12:00 +02:00
cgroup.c
cgroups: fix API thinko
2010-09-09 18:57:23 -07:00
cgroup_freezer.c
Freezer / cgroup freezer: Update stale locking comments
2010-05-10 23:18:47 +02:00
compat.c
rlimits: switch more rlimit syscalls to do_prlimit
2010-07-16 09:48:48 +02:00
configs.c
cpu.c
sched: adjust when cpu_active and cpuset configurations are updated during cpu on/offlining
2010-06-08 21:40:36 +02:00
cpuset.c
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2010-08-06 09:39:22 -07:00
cred.c
Add a dummy printk function for the maintenance of unused printks
2010-08-12 09:51:35 -07:00
delayacct.c
dma.c
early_res.c
kmemleak: Add support for NO_BOOTMEM configurations
2010-07-19 11:54:15 +01:00
elfcore.c
elf coredump: add extended numbering support
2010-03-06 11:26:46 -08:00
exec_domain.c
sys_personality: remove the bogus checks in sys_personality()->__set_personality() path
2010-08-09 20:45:05 -07:00
exit.c
Fix unprotected access to task credentials in waitid()
2010-08-17 18:07:43 -07:00
extable.c
fork.c
mm: make the vma list be doubly linked
2010-08-21 08:49:21 -07:00
freezer.c
futex.c
futex: futex_find_get_task remove credentails check
2010-06-30 15:43:44 -07:00
futex_compat.c
futex: Protect pid lookup in compat code with RCU
2009-12-09 14:22:14 +01:00
groups.c
kernel/groups.c: fix integer overflow in groups_search
2010-09-09 18:57:24 -07:00
hrtimer.c
gcc-4.6: kernel/*: Fix unused but set warnings
2010-09-05 14:36:58 +02:00
hung_task.c
hw_breakpoint.c
Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2010-08-06 09:30:52 -07:00
itimer.c
kallsyms.c
kdb: core for kgdb back end (2 of 2)
2010-05-20 21:04:21 -05:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kexec.c
kexec: return -EFAULT on copy_to_user() failures
2010-08-11 08:59:22 -07:00
kfifo.c
kfifo: implement missing __kfifo_skip_r()
2010-08-20 09:34:54 -07:00
kmod.c
Make do_execve() take a const filename pointer
2010-08-17 18:07:43 -07:00
kprobes.c
kprobes: Move enable/disable_kprobe() out from debugfs code
2010-05-08 18:08:30 +02:00
ksysfs.c
sysfs: add struct file* to bin_attr callbacks
2010-05-21 09:37:31 -07:00
kthread.c
kthread: implement kthread_data()
2010-06-29 10:07:09 +02:00
latencytop.c
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
2010-03-30 22:02:32 +09:00
lockdep.c
sched_clock: Add local_clock() API and improve documentation
2010-06-09 10:34:49 +02:00
lockdep_internals.h
lockdep: No need to disable preemption in debug atomic ops
2010-05-04 05:38:16 +02:00
lockdep_proc.c
lockstat: Make lockstat counting per cpu
2010-04-06 00:15:37 +02:00
lockdep_states.h
Makefile
Merge branch 'for-linus' of git://git.infradead.org/users/eparis/notify
2010-08-10 11:39:13 -07:00
module.c
module: cleanup comments, remove noinline
2010-08-05 12:59:13 +09:30
mutex-debug.c
mutex-debug.h
locking: Implement new raw_spinlock
2009-12-14 23:55:32 +01:00
mutex.c
mutex: Fix annotations to include it in kernel-locking docbook
2010-09-03 08:19:51 +02:00
mutex.h
notifier.c
sched: Use lockdep-based checking on rcu_dereference()
2010-02-25 10:34:26 +01:00
ns_cgroup.c
nsproxy.c
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
2010-03-30 22:02:32 +09:00
padata.c
Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
2010-08-04 15:23:14 -07:00
panic.c
lib/bug.c: add oops end marker to WARN implementation
2010-08-11 08:59:22 -07:00
params.c
param: locking for kernel parameters
2010-08-11 23:04:20 +09:30
perf_event.c
perf_events: Fix time tracking for events with pid != -1 and cpu != -1
2010-08-30 12:16:55 +02:00
pid.c
pids: alloc_pidmap: remove the unnecessary boundary checks
2010-08-11 08:59:20 -07:00
pid_namespace.c
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
2010-03-30 22:02:32 +09:00
pm_qos_params.c
PM QoS: Fix inline documentation.
2010-08-26 20:18:43 +02:00
posix-cpu-timers.c
Merge branch 'writable_limits' of git://decibel.fi.muni.cz/~xslaby/linux
2010-08-10 12:07:51 -07:00
posix-timers.c
posix_timer: Move copy_to_user(created_timer_id) down in timer_create()
2010-07-23 15:08:12 +02:00
printk.c
gcc-4.6: printk: use stable variable to dump kmsg buffer
2010-08-09 20:45:06 -07:00
profile.c
numa: in-kernel profiling: use cpu_to_mem() for per cpu allocations
2010-05-27 09:12:57 -07:00
ptrace.c
ptrace: optimize exit_ptrace() for the likely case
2010-08-11 08:59:19 -07:00
range.c
kernel/range: remove unused definition of ARRAY_SIZE()
2010-08-09 20:45:06 -07:00
rcupdate.c
tree/tiny rcu: Add debug RCU head objects
2010-06-14 16:37:26 -07:00
rcutiny.c
tree/tiny rcu: Add debug RCU head objects
2010-06-14 16:37:26 -07:00
rcutiny_plugin.h
rcu: slim down rcutiny by removing rcu_scheduler_active and friends
2010-05-10 11:08:34 -07:00
rcutorture.c
sched_clock: Add local_clock() API and improve documentation
2010-06-09 10:34:49 +02:00
rcutree.c
tree/tiny rcu: Add debug RCU head objects
2010-06-14 16:37:26 -07:00
rcutree.h
rcu: reduce the number of spurious RCU_SOFTIRQ invocations
2010-05-10 11:08:35 -07:00
rcutree_plugin.h
rcu: remove all rcu head initializations, except on_stack initializations
2010-05-11 16:10:47 -07:00
rcutree_trace.c
rcu: reduce the number of spurious RCU_SOFTIRQ invocations
2010-05-10 11:08:35 -07:00
relay.c
kernel/: convert cpu notifier to return encapsulate errno value
2010-05-27 09:12:48 -07:00
res_counter.c
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
2010-03-30 22:02:32 +09:00
resource.c
resource: shared I/O region support
2010-05-11 12:01:10 -07:00
rtmutex-debug.c
sched: Convert pi_lock to raw_spinlock
2009-12-14 23:55:33 +01:00
rtmutex-debug.h
rtmutex-tester.c
rtmutex.c
rtmutes: Convert rtmutex.lock to raw_spinlock
2009-12-14 23:55:33 +01:00
rtmutex.h
rtmutex_common.h
rwsem.c
sched.c
mutex: Improve the scalability of optimistic spinning
2010-08-23 10:56:27 +02:00
sched_clock.c
sched_clock: Add local_clock() API and improve documentation
2010-06-09 10:34:49 +02:00
sched_cpupri.c
sched: No need for bootmem special cases
2010-07-17 12:06:22 +02:00
sched_cpupri.h
sched: No need for bootmem special cases
2010-07-17 12:06:22 +02:00
sched_debug.c
sched: Use correct macro to display sched_child_runs_first in /proc/sched_debug
2010-07-21 21:46:12 +02:00
sched_fair.c
gcc-4.6: kernel/*: Fix unused but set warnings
2010-09-05 14:36:58 +02:00
sched_features.h
sched: Remove ASYM_GRAN feature
2010-03-11 18:32:53 +01:00
sched_idletask.c
sched: Cure load average vs NO_HZ woes
2010-04-23 11:02:02 +02:00
sched_rt.c
sched: task_tick_rt: Remove the obsolete ->signal != NULL check
2010-06-18 10:46:56 +02:00
sched_stats.h
sched: Remove the obsolete exit_state/signal hacks
2010-06-18 10:46:56 +02:00
seccomp.c
semaphore.c
signal.c
CRED: Fix RCU warning due to previous patch fixing __task_cred()'s checks
2010-08-04 11:17:10 -07:00
smp.c
generic-ipi: Fix deadlock in __smp_call_function_single
2010-09-10 16:48:40 +02:00
softirq.c
kernel/: fix BUG_ON checks for cpu notifier callbacks direct call
2010-06-04 15:21:45 -07:00
spinlock.c
locking: Cleanup the name space completely
2009-12-14 23:55:33 +01:00
srcu.c
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
2010-03-30 22:02:32 +09:00
stacktrace.c
stop_machine.c
stop_machine: struct cpu_stopper, remove alignment padding on 64 bits
2010-08-09 20:45:06 -07:00
sys.c
pid: make setpgid() system call use RCU read-side critical section
2010-08-31 17:00:18 -07:00
sys_ni.c
fanotify: sys_fanotify_mark declartion
2010-07-28 09:58:55 -04:00
sysctl.c
gcc-4.6: kernel/*: Fix unused but set warnings
2010-09-05 14:36:58 +02:00
sysctl_binary.c
sysctl: don't use own implementation of hex_to_bin()
2010-05-25 08:07:05 -07:00
sysctl_check.c
taskstats.c
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
2010-03-30 22:02:32 +09:00
test_kprobes.c
time.c
time: Kill off CONFIG_GENERIC_TIME
2010-07-27 12:40:54 +02:00
timeconst.pl
timer.c
kernel/timer.c: fix kernel-doc function parameter warning
2010-08-10 15:33:09 -07:00
tracepoint.c
tracing: Let tracepoints have data passed to tracepoint callbacks
2010-05-14 09:50:34 -04:00
tsacct.c
mm: clean up mm_counter
2010-03-06 11:26:23 -08:00
uid16.c
up.c
user-return-notifier.c
user.c
sched: Remove a stale comment
2010-05-10 08:48:39 +02:00
user_namespace.c
user_ns: Introduce user_nsmap_uid and user_ns_map_gid.
2010-06-16 14:55:34 -07:00
utsname.c
utsname_sysctl.c
wait.c
watchdog.c
lockup_detector: Sync touch_*_watchdog back to old semantics
2010-09-01 10:02:28 +02:00
workqueue.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq
2010-09-07 14:08:17 -07:00
workqueue_sched.h
workqueue: implement concurrency managed dynamic worker pool
2010-06-29 10:07:14 +02:00