Peter Zijlstra
88ec22d3ed
sched: Remove the cfs_rq dependency from set_task_cpu()
...
In order to remove the cfs_rq dependency from set_task_cpu() we
need to ensure the task is cfs_rq invariant for all callsites.
The simple approach is to substract cfs_rq->min_vruntime from
se->vruntime on dequeue, and add cfs_rq->min_vruntime on
enqueue.
However, this has the downside of breaking FAIR_SLEEPERS since
we loose the old vruntime as we only maintain the relative
position.
To solve this, we observe that we only migrate runnable tasks,
we do this using deactivate_task(.sleep=0) and
activate_task(.wakeup=0), therefore we can restrain the
min_vruntime invariance to that state.
The only other case is wakeup balancing, since we want to
maintain the old vruntime we cannot make it relative on dequeue,
but since we don't migrate inactive tasks, we can do so right
before we activate it again.
This is where we need the new pre-wakeup hook, we need to call
this while still holding the old rq->lock. We could fold it into
->select_task_rq(), but since that has multiple callsites and
would obfuscate the locking requirements, that seems like a
fudge.
This leaves the fork() case, simply make sure that ->task_fork()
leaves the ->vruntime in a relative state.
This covers all cases where set_task_cpu() gets called, and
ensures it sees a relative vruntime.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
LKML-Reference: <20091216170518.191697025@chello.nl>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-16 19:01:58 +01:00
..
gcov
microblaze: Enable GCOV_PROFILE_ALL
2009-09-21 14:29:21 +02:00
irq
genirq: Convert irq_desc.lock to raw_spinlock
2009-12-14 23:55:33 +01:00
power
vt: introduce and use vt_kmsg_redirect() function
2009-12-15 08:53:28 -08:00
time
clockevents: Convert to raw_spinlock
2009-12-14 23:55:34 +01:00
trace
locking: Convert __raw_spin* functions to arch_spin*
2009-12-14 23:55:32 +01:00
.gitignore
acct.c
bsdacct: fix uid/gid misreporting
2009-12-15 08:53:10 -08:00
async.c
audit.c
Audit: send signal info if selinux is disabled
2009-09-24 03:50:26 -04:00
audit.h
Fix rule eviction order for AUDIT_DIR
2009-06-24 00:02:38 -04:00
audit_tree.c
Fix rule eviction order for AUDIT_DIR
2009-06-24 00:02:38 -04:00
audit_watch.c
Audit: reorganize struct audit_watch to save 8 bytes
2009-09-24 03:50:25 -04:00
auditfilter.c
Audit: clean up all op= output to include string quoting
2009-06-24 00:00:52 -04:00
auditsc.c
Audit: rearrange audit_context to save 16 bytes per struct
2009-09-24 03:50:26 -04:00
backtracetest.c
bounds.c
capability.c
remove CONFIG_SECURITY_FILE_CAPABILITIES compile option
2009-11-24 15:06:47 +11:00
cgroup.c
cgroup: fix strstrip() misuse
2009-10-29 07:39:25 -07:00
cgroup_freezer.c
cgroups: let ss->can_attach and ss->attach do whole threadgroups at a time
2009-09-24 07:20:58 -07:00
compat.c
configs.c
cpu.c
sched: Fix set_cpu_active() in cpu_down()
2009-12-16 19:01:53 +01:00
cpuset.c
sched: Fix balance vs hotplug race
2009-12-06 21:10:56 +01:00
cred-internals.h
cred.c
creds_are_invalid() needs to be exported for use by modules:
2009-09-23 11:02:26 -07:00
delayacct.c
headers: taskstats_kern.h trim
2009-09-18 09:48:52 -07:00
dma.c
exec_domain.c
exit.c
sched: Convert pi_lock to raw_spinlock
2009-12-14 23:55:33 +01:00
extable.c
fork.c
sched: Convert pi_lock to raw_spinlock
2009-12-14 23:55:33 +01:00
freezer.c
sched: fix nr_uninterruptible accounting of frozen tasks really
2009-07-18 14:19:53 +02:00
futex.c
rtmutes: Convert rtmutex.lock to raw_spinlock
2009-12-14 23:55:33 +01:00
futex_compat.c
futex: Fix compat_futex to be same as futex for REQUEUE_PI
2009-08-10 15:41:12 +02:00
groups.c
groups: move code to kernel/groups.c
2009-06-16 19:47:48 -07:00
hrtimer.c
hrtimers: Convert to raw_spinlocks
2009-12-14 23:55:34 +01:00
hung_task.c
softlockup: Fix hung_task_check_count sysctl
2009-11-27 06:21:57 +01:00
hw_breakpoint.c
perf_event: Convert to raw_spinlock
2009-12-14 23:55:34 +01:00
itimer.c
itimers: Fix racy writes to cpu_itimer fields
2009-11-18 16:32:12 +01:00
kallsyms.c
hw-breakpoints: Fix broken hw-breakpoint sample module
2009-11-10 11:23:29 +01:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks
mutex: Better control mutex adaptive spinning config
2009-12-03 11:50:11 +01:00
Kconfig.preempt
kexec.c
kexec: fix omitting offset in extended crashkernel syntax
2009-07-29 19:10:34 -07:00
kfifo.c
kfifo: Use "const" definitions
2009-09-19 13:13:17 -07:00
kgdb.c
kgdb: Always process the whole breakpoint list on activate or deactivate
2009-12-11 08:43:20 -06:00
kmod.c
security: report the module name to security_module_request
2009-11-10 09:33:46 +11:00
kprobes.c
Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-12-05 15:30:21 -08:00
ksysfs.c
kthread.c
sched: Move kthread_bind() back to kthread.c
2009-12-16 19:01:57 +01:00
latencytop.c
lockdep.c
locking: Convert __raw_spin* functions to arch_spin*
2009-12-14 23:55:32 +01:00
lockdep_internals.h
lockdep: BFS cleanup
2009-07-24 10:53:29 +02:00
lockdep_proc.c
seq_file: constify seq_operations
2009-09-23 07:39:29 -07:00
lockdep_states.h
Makefile
Merge branch 'kvm-updates/2.6.33' of git://git.kernel.org/pub/scm/virt/kvm/kvm
2009-12-08 08:02:38 -08:00
module.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu
2009-12-14 09:58:24 -08:00
mutex-debug.c
headers: remove sched.h from interrupt.h
2009-10-11 11:20:58 -07:00
mutex-debug.h
locking: Implement new raw_spinlock
2009-12-14 23:55:32 +01:00
mutex.c
mutex: Better control mutex adaptive spinning config
2009-12-03 11:50:11 +01:00
mutex.h
notifier.c
kprobes: Fix to add __kprobes to notify_die
2009-08-30 03:08:26 +02:00
ns_cgroup.c
cgroups: let ss->can_attach and ss->attach do whole threadgroups at a time
2009-09-24 07:20:58 -07:00
nsproxy.c
nsproxy: extract create_nsproxy()
2009-06-18 13:03:56 -07:00
panic.c
Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-10-08 12:16:35 -07:00
params.c
tree-wide: convert open calls to remove spaces to skip_spaces() lib function
2009-12-15 08:53:32 -08:00
perf_event.c
perf_event: Convert to raw_spinlock
2009-12-14 23:55:34 +01:00
pid.c
mm: also use alloc_large_system_hash() for the PID hash table
2009-09-22 07:17:38 -07:00
pid_namespace.c
pidns: deny CLONE_PARENT|CLONE_NEWPID combination
2009-09-24 07:21:04 -07:00
pm_qos_params.c
pm_qos: clean up racy global "name" variable
2009-10-14 15:31:10 +02:00
posix-cpu-timers.c
posix-cpu-timers: optimize and document timer_create callback
2009-11-18 12:36:05 +01:00
posix-timers.c
time: Introduce CLOCK_REALTIME_COARSE
2009-08-21 21:43:46 +02:00
printk.c
Merge branch 'core-printk-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-12-05 09:50:22 -08:00
profile.c
kernel/profile.c: Switch /proc/irq/prof_cpu_mask to seq_file
2009-09-20 20:15:40 +02:00
ptrace.c
ptrace: __ptrace_detach: do __wake_up_parent() if we reap the tracee
2009-09-24 07:20:59 -07:00
rcupdate.c
rcu: Re-arrange code to reduce #ifdef pain
2009-11-22 18:58:16 +01:00
rcutiny.c
rcu: Eliminate unneeded function wrapping
2009-11-22 18:58:16 +01:00
rcutorture.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu
2009-12-14 09:58:24 -08:00
rcutree.c
rcu: Add expedited grace-period support for preemptible RCU
2009-12-03 11:35:25 +01:00
rcutree.h
rcu: Add expedited grace-period support for preemptible RCU
2009-12-03 11:35:25 +01:00
rcutree_plugin.h
rcu: Add expedited grace-period support for preemptible RCU
2009-12-03 11:35:25 +01:00
rcutree_trace.c
rcu: Add expedited grace-period support for preemptible RCU
2009-12-03 11:35:25 +01:00
relay.c
const: mark struct vm_struct_operations
2009-09-27 11:39:25 -07:00
res_counter.c
memcg: some modification to softlimit under hierarchical memory reclaim.
2009-10-01 16:11:13 -07:00
resource.c
resources: when allocate_resource() fails, leave resource untouched
2009-11-04 13:06:46 -08:00
rtmutex-debug.c
sched: Convert pi_lock to raw_spinlock
2009-12-14 23:55:33 +01:00
rtmutex-debug.h
rtmutex-tester.c
rtmutex.c
rtmutes: Convert rtmutex.lock to raw_spinlock
2009-12-14 23:55:33 +01:00
rtmutex.h
rtmutex_common.h
rwsem.c
sched.c
sched: Remove the cfs_rq dependency from set_task_cpu()
2009-12-16 19:01:58 +01:00
sched_clock.c
sched: Fix cpu_clock() in NMIs, on !CONFIG_HAVE_UNSTABLE_SCHED_CLOCK
2009-12-15 09:04:36 +01:00
sched_cpupri.c
sched: Convert cpupri lock to raw_spinlock
2009-12-14 23:55:33 +01:00
sched_cpupri.h
sched: Convert cpupri lock to raw_spinlock
2009-12-14 23:55:33 +01:00
sched_debug.c
sched: Convert rq->lock to raw_spinlock
2009-12-14 23:55:33 +01:00
sched_fair.c
sched: Remove the cfs_rq dependency from set_task_cpu()
2009-12-16 19:01:58 +01:00
sched_features.h
sched: Discard some old bits
2009-12-09 10:03:07 +01:00
sched_idletask.c
Merge branch 'linus' into sched/urgent
2009-12-16 18:33:49 +01:00
sched_rt.c
sched: Add pre and post wakeup hooks
2009-12-16 19:01:58 +01:00
sched_stats.h
seccomp.c
semaphore.c
signal.c
Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-12-05 15:30:21 -08:00
slow-work-debugfs.c
SLOW_WORK: Move slow_work's proc file to debugfs
2009-12-01 08:20:31 -08:00
slow-work.c
Merge git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/sysctl-2.6
2009-12-08 07:38:50 -08:00
slow-work.h
SLOW_WORK: Move slow_work's proc file to debugfs
2009-12-01 08:20:31 -08:00
smp.c
Merge branch 'core-locking-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-12-15 09:02:01 -08:00
softirq.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu
2009-12-14 09:58:24 -08:00
softlockup.c
percpu: make percpu symbols under kernel/ and mm/ unique
2009-10-29 22:34:13 +09:00
spinlock.c
locking: Cleanup the name space completely
2009-12-14 23:55:33 +01:00
srcu.c
rcu: Add synchronize_srcu_expedited()
2009-10-26 09:40:30 +01:00
stacktrace.c
stop_machine.c
sys.c
kernel/sys.c: fix "warning: do-while statement is not a compound statement" noise
2009-12-15 08:53:26 -08:00
sys_ni.c
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next-2.6
2009-12-08 07:55:01 -08:00
sysctl.c
'sysctl_max_map_count' should be non-negative
2009-12-15 08:53:23 -08:00
sysctl_binary.c
sysctl binary: Reorder the tests to process wild card entries first.
2009-11-12 01:42:31 -08:00
sysctl_check.c
ipv4 05/05: add sysctl to accept packets with local source addresses
2009-12-03 12:14:38 -08:00
taskstats.c
genetlink: make netns aware
2009-07-12 14:03:27 -07:00
test_kprobes.c
time.c
Merge branch 'timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-12-08 19:27:08 -08:00
timeconst.pl
timer.c
Merge branch 'timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-09-23 09:46:15 -07:00
tracepoint.c
trivial: fix typo "to to" in multiple files
2009-09-21 15:14:55 +02:00
tsacct.c
uid16.c
headers: utsname.h redux
2009-09-23 18:13:10 -07:00
up.c
user-return-notifier.c
core: Clean up user return notifers use of per_cpu
2009-12-02 10:22:59 +01:00
user.c
uids: Prevent tear down race
2009-11-02 16:02:39 +01:00
user_namespace.c
utsname.c
utsns: extract creeate_uts_ns()
2009-06-18 13:03:55 -07:00
utsname_sysctl.c
sysctl kernel: Remove binary sysctl logic
2009-11-12 02:04:55 -08:00
wait.c
locking, sched: Give waitqueue spinlocks their own lockdep classes
2009-08-10 14:43:09 +02:00
workqueue.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq
2009-12-10 09:35:44 -08:00