Frederic Weisbecker
20ab65e33f
rcu: Exit RCU extended QS on user preemption ...
When exceptions or irq are about to resume userspace, if
the task needs to be rescheduled, the arch low level code
calls schedule() directly.
If we call it, it is because we have the TIF_RESCHED flag:
- It can be set after random local calls to set_need_resched()
(RCU, drm, ...)
- A wake up happened and the CPU needs preemption. This can
happen in several ways:
* Remotely: the remote waking CPU has set TIF_RESCHED and send the
wakee an IPI to schedule the new task.
* Remotely enqueued: the remote waking CPU sends an IPI to the target
and the wake up is made by the target.
* Locally: waking CPU == wakee CPU and the wakeup is done locally.
set_need_resched() is called without IPI.
In the case of local and remotely enqueued wake ups, the tick can
be restarted when we enqueue the new task and RCU can exit the
extended quiescent state at the same time. Then by the time we reach
irq exit path and we call schedule, we are not in RCU user mode.
But if we call schedule() only because something called set_need_resched(),
RCU may still be in user mode when we reach schedule.
Also if a wake up is done remotely, the CPU might see the TIF_RESCHED
flag and call schedule while the IPI has not yet happen to restart the
tick and exit RCU user mode.
We need to manually protect against these corner cases.
Create a new API schedule_user() that calls schedule() inside
rcu_user_exit()-rcu_user_enter() in order to protect it. Archs
will need to rely on it now to implement user preemption safely.
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Alessio Igor Bogani <abogani@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Avi Kivity <avi@redhat.com>
Cc: Chris Metcalf <cmetcalf@tilera.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Geoff Levand <geoff@infradead.org>
Cc: Gilad Ben Yossef <gilad@benyossef.com>
Cc: Hakan Akkan <hakanakkan@gmail.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Kevin Hilman <khilman@ti.com>
Cc: Max Krasnyansky <maxk@qualcomm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephen Hemminger <shemminger@vyatta.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Sven-Thorsten Dietrich <thebigcorporation@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
2012-09-26 15:47:11 +02:00
kernel/debug: Make use of KGDB_REASON_NMI
2012-07-31 08:16:43 -05:00
perf/hwpb: Invoke __perf_event_disable() if interrupts are already disabled
2012-09-04 17:29:53 +02:00
gcov: disable CONSTRUCTORS for UML
2011-07-26 16:49:45 -07:00
Merge branch 'irq-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2012-08-03 10:56:44 -07:00
Revert "NMI watchdog: fix for lockup detector breakage on resume"
2012-08-08 20:49:45 +02:00
rcu: Exit RCU extended QS on user preemption
2012-09-26 15:47:11 +02:00
Merge remote-tracking branch 'tip/core/rcu' into next.2012.09.25b
2012-09-25 10:03:56 -07:00
tracing/syscalls: Fix perf syscall tracing when syscall_nr == -1
2012-08-17 15:19:46 -04:00
Merge branch 'for-linus2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2012-01-08 12:19:57 -08:00
[SCSI] async: make async_synchronize_full() flush all work regardless of domain
2012-07-20 09:07:37 +01:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2012-08-01 10:26:23 -07:00
audit: remove AUDIT_SETUP_CONTEXT as it isn't used
2012-01-17 16:16:57 -05:00
audit: clean up refcounting in audit-tree
2012-08-15 12:55:22 +02:00
get rid of kern_path_parent()
2012-07-14 16:35:02 +04:00
audit: allow interfield comparison in audit rules
2012-01-17 16:17:01 -05:00
seccomp: remove duplicated failure logging
2012-04-14 11:13:20 +10:00
userns: Teach inode_capable to understand inodes whose uids map to other namespaces.
2012-05-15 14:59:24 -07:00
Merge branch 'for-3.6' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
2012-07-24 17:47:44 -07:00
cgroup: convert all non-memcg controllers to the new cftype interface
2012-04-01 12:09:55 -07:00
new helper: sigsuspend()
2012-05-21 23:52:30 -04:00
kernel/configs.c: include MODULE_*() when CONFIG_IKCONFIG_PROC=n
2011-07-25 20:57:15 -07:00
smpboot: Provide infrastructure for percpu hotplug threads
2012-08-13 17:01:07 +02:00
kernel/cpu_pm.c: fix various typos
2012-05-31 17:49:27 -07:00
cpusets: Remove/update outdated comments
2012-07-24 13:53:28 +02:00
Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux
2011-11-06 19:44:47 -08:00
keys: kill task_struct->replacement_session_keyring
2012-05-23 22:11:41 -04:00
KVM: Steal time implementation
2011-07-14 12:59:14 +03:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
posix_types.h: Cleanup stale __NFDBITS and related definitions
2012-07-26 13:36:43 -07:00
extable: Skip sorting if sorted at build time.
2012-04-19 15:06:55 -07:00
uprobes: Fix mmap_region()'s mm->mm_rb corruption if uprobe_mmap() fails
2012-08-21 11:48:12 +02:00
PM / Freezer: Remove references to TIF_FREEZE in comments
2012-03-04 23:08:54 +01:00
futex: Forbid uaddr == uaddr2 in futex_wait_requeue_pi()
2012-07-24 16:02:57 +02:00
futex: Mark get_robust_list as deprecated
2012-03-29 11:37:17 +02:00
userns: Convert in_group_p and in_egroup_p to use kgid_t
2012-05-03 03:29:33 -07:00
hrtimer: Update hrtimer base offsets each hrtimer_interrupt
2012-07-11 23:34:39 +02:00
hung task debugging: Inject NMI when hung and going to panic
2012-04-25 12:39:25 +02:00
irq_work: fix compile failure on tile from missing include
2012-04-13 13:15:16 -04:00
itimer: Use printk_once instead of WARN_ONCE
2012-04-10 11:00:30 +02:00
static keys: Inline the static_key_enabled() function
2012-02-28 20:01:08 +01:00
vsprintf: fix %ps on non symbols when using kallsyms
2012-05-29 16:22:32 -07:00
syscalls, x86: add __NR_kcmp syscall
2012-05-31 17:49:32 -07:00
locking/kconfig: Simplify INLINE_SPIN_UNLOCK usage
2012-03-23 13:18:57 +01:00
locking/kconfig: Simplify INLINE_SPIN_UNLOCK usage
2012-03-23 13:18:57 +01:00
kdump: append newline to the last lien of vmcoreinfo note
2012-07-30 17:25:20 -07:00
[media] kernel:kfifo: export __kfifo_max_r symbol
2012-04-11 18:24:37 -03:00
kmod: avoid deadlock from recursive kmod call
2012-07-30 17:25:20 -07:00
kprobes: return proper error code from register_kprobe()
2012-03-05 15:49:42 -08:00
kernel: ksysfs.c is implicitly using stat.h
2011-10-31 09:20:13 -04:00
kthread: Implement park/unpark facility
2012-08-13 17:01:06 +02:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
brlocks/lglocks: turn into functions
2012-05-29 23:28:41 -04:00
lockdep: Add CPU-idle/offline warning to lockdep-RCU splat
2012-02-21 09:06:06 -08:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
hotplug: Fix UP bug in smpboot hotplug code
2012-08-13 17:01:07 +02:00
Guard check in module loader against integer overflow
2012-05-23 22:28:53 +09:30
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
mutex: Use p->on_cpu for the adaptive spin
2011-04-14 08:52:33 +02:00
sched/rt: Use schedule_preempt_disabled()
2012-03-01 10:28:03 +01:00
mutex: Use p->on_cpu for the adaptive spin
2011-04-14 08:52:33 +02:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
padata: Fix cpu hotplug
2012-03-29 19:52:46 +08:00
panic: fix a possible deadlock in panic()
2012-07-30 17:25:13 -07:00
params: replace printk(KERN_<LVL>...) with pr_<lvl>(...)
2012-05-04 17:28:18 -07:00
mm: add a low limit to alloc_large_system_hash
2012-05-24 00:28:21 -04:00
pid-namespace: limit value of ns_last_pid to (0, max_pid)
2012-09-17 15:00:38 -07:00
[S390] cputime: add sparse checking and cleanup
2011-12-15 14:56:19 +01:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
printk: Fix calculation of length used to discard records
2012-08-12 21:25:50 +03:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
userns: Convert ptrace, kill, set_priority permission checks to work with kuids and kgids
2012-05-03 03:28:51 -07:00
range: fix bogus misuse of module.h to get printk()
2011-10-31 09:20:11 -04:00
rcu: Allow nesting of rcu_idle_enter() and rcu_idle_exit()
2012-02-21 09:06:12 -08:00
rcu: Add PROVE_RCU_DELAY to provoke difficult races
2012-09-23 07:42:49 -07:00
rcu: Move TINY_RCU quiescent state out of extended quiescent state
2012-09-23 07:42:52 -07:00
rcu: Move TINY_PREEMPT_RCU away from raw_local_irq_save()
2012-09-23 07:42:51 -07:00
rcu: Prevent initialization race in rcutorture kthreads
2012-09-23 07:42:23 -07:00
rcu: Switch task's syscall hooks on context switch
2012-09-26 15:47:02 +02:00
rcu: Ignore userspace extended quiescent state by default
2012-09-26 15:47:01 +02:00
rcu: Make RCU_FAST_NO_HZ handle adaptive ticks
2012-09-26 15:44:02 +02:00
Merge remote-tracking branch 'tip/smp/hotplug' into next.2012.09.25b
2012-09-25 10:01:45 -07:00
splice: fix racy pipe->buffers uses
2012-06-13 21:16:42 +02:00
rescounters: add res_counter_uncharge_until()
2012-05-29 16:22:27 -07:00
resource: make sure requested range is included in the root range
2012-07-30 17:25:21 -07:00
lockdep, rtmutex, bug: Show taint flags on error
2011-12-06 08:16:49 +01:00
rtmutex-tester: convert sysdev_class to a regular subsystem
2011-12-14 14:54:22 -08:00
Revert "rcu: Permit rt_mutex_unlock() with irqs disabled"
2011-12-11 10:33:18 -08:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
seccomp: fix build warnings when there is no CONFIG_SECCOMP_FILTER
2012-04-18 12:24:52 +10:00
semaphore: fix improper comment reference to mutex
2012-04-05 17:15:55 -07:00
signal: make sure we don't get stopped with pending task_work
2012-07-22 23:57:54 +04:00
smp: Remove ipi_call_lock[_irq]()/ipi_call_unlock[_irq]()
2012-06-05 17:27:14 +02:00
hotplug: Fix UP bug in smpboot hotplug code
2012-08-13 17:01:07 +02:00
smpboot: Provide infrastructure for percpu hotplug threads
2012-08-13 17:01:07 +02:00
softirq: Use hotplug thread infrastructure
2012-08-13 17:01:07 +02:00
locking/kconfig: Simplify INLINE_SPIN_UNLOCK usage
2012-03-23 13:18:57 +01:00
rcu: Implement per-domain single-threaded call_srcu() state machine
2012-04-30 10:48:25 -07:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux
2011-11-06 19:44:47 -08:00
kernel/sys.c: avoid argv_free(NULL)
2012-07-30 17:25:13 -07:00
syscalls, x86: add __NR_kcmp syscall
2012-05-31 17:49:32 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2012-08-01 10:26:23 -07:00
mm: prepare for removal of obsolete /proc/sys/vm/nr_pdflush_threads
2012-07-31 18:42:40 -07:00
task_work: add a scheduling point in task_work_run()
2012-08-21 09:11:44 -07:00
taskstats: check nla_reserve() return
2012-07-30 17:25:21 -07:00
time: Remove bogus comments
2012-03-15 18:17:55 -07:00
alpha: take a bunch of syscalls into osf_sys.c
2012-08-19 08:41:19 -07:00
static keys: Introduce 'struct static_key', static_key_true()/false() and static_key_slow_[inc|dec]()
2012-02-24 10:05:59 +01:00
[S390] cputime: add sparse checking and cleanup
2011-12-15 14:56:19 +01:00
userns: Convert setting and getting uid and gid system calls to use kuid and kgid
2012-05-03 03:28:41 -07:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
kernel: Map most files to use export.h instead of module.h
2011-10-31 09:20:12 -04:00
userns: Silence silly gcc warning.
2012-05-19 15:44:40 -06:00
userns: Store uid and gid values in struct cred with kuid_t and kgid_t types
2012-05-03 03:28:38 -07:00
userns: Use cred->user_ns instead of cred->user->user_ns
2012-04-07 16:55:51 -07:00
Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux
2011-11-06 19:44:47 -08:00
lockdep/waitqueues: Add better annotation
2011-12-21 10:07:39 +01:00
watchdog: Use hotplug thread infrastructure
2012-08-13 17:01:07 +02:00
workqueue: reimplement work_on_cpu() using system_wq
2012-09-19 10:13:12 -07:00
20ab65e33f