Patrick Bellasi
69842cba9a
sched/uclamp: Add CPU's clamp buckets refcounting
...
Utilization clamping allows to clamp the CPU's utilization within a
[util_min, util_max] range, depending on the set of RUNNABLE tasks on
that CPU. Each task references two "clamp buckets" defining its minimum
and maximum (util_{min,max}) utilization "clamp values". A CPU's clamp
bucket is active if there is at least one RUNNABLE tasks enqueued on
that CPU and refcounting that bucket.
When a task is {en,de}queued {on,from} a rq, the set of active clamp
buckets on that CPU can change. If the set of active clamp buckets
changes for a CPU a new "aggregated" clamp value is computed for that
CPU. This is because each clamp bucket enforces a different utilization
clamp value.
Clamp values are always MAX aggregated for both util_min and util_max.
This ensures that no task can affect the performance of other
co-scheduled tasks which are more boosted (i.e. with higher util_min
clamp) or less capped (i.e. with higher util_max clamp).
A task has:
task_struct::uclamp[clamp_id]::bucket_id
to track the "bucket index" of the CPU's clamp bucket it refcounts while
enqueued, for each clamp index (clamp_id).
A runqueue has:
rq::uclamp[clamp_id]::bucket[bucket_id].tasks
to track how many RUNNABLE tasks on that CPU refcount each
clamp bucket (bucket_id) of a clamp index (clamp_id).
It also has a:
rq::uclamp[clamp_id]::bucket[bucket_id].value
to track the clamp value of each clamp bucket (bucket_id) of a clamp
index (clamp_id).
The rq::uclamp::bucket[clamp_id][] array is scanned every time it's
needed to find a new MAX aggregated clamp value for a clamp_id. This
operation is required only when it's dequeued the last task of a clamp
bucket tracking the current MAX aggregated clamp value. In this case,
the CPU is either entering IDLE or going to schedule a less boosted or
more clamped task.
The expected number of different clamp values configured at build time
is small enough to fit the full unordered array into a single cache
line, for configurations of up to 7 buckets.
Add to struct rq the basic data structures required to refcount the
number of RUNNABLE tasks for each clamp bucket. Add also the max
aggregation required to update the rq's clamp value at each
enqueue/dequeue event.
Use a simple linear mapping of clamp values into clamp buckets.
Pre-compute and cache bucket_id to avoid integer divisions at
enqueue/dequeue time.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Alessio Balsini <balsini@android.com>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Joel Fernandes <joelaf@google.com>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Quentin Perret <quentin.perret@arm.com>
Cc: Rafael J . Wysocki <rafael.j.wysocki@intel.com>
Cc: Steve Muckle <smuckle@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Todd Kjos <tkjos@google.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Link: https://lkml.kernel.org/r/20190621084217.8167-2-patrick.bellasi@arm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2019-06-24 19:23:44 +02:00
..
bpf
SPDX update for 5.2-rc6
2019-06-21 09:58:42 -07:00
cgroup
Linux 5.2-rc6
2019-06-24 19:19:53 +02:00
configs
debug
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
dma
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 333
2019-06-05 17:37:06 +02:00
events
perf/ring-buffer: Use regular variables for nesting
2019-05-24 09:00:11 +02:00
gcov
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
irq
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
livepatch
This includes the following fixes:
2019-06-15 07:24:11 -10:00
locking
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 436
2019-06-05 17:37:17 +02:00
power
sched/topology: Remove unused 'sd' parameter from arch_scale_cpu_capacity()
2019-06-24 19:23:39 +02:00
printk
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 13
2019-05-21 11:28:45 +02:00
rcu
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
sched
sched/uclamp: Add CPU's clamp buckets refcounting
2019-06-24 19:23:44 +02:00
time
Linux 5.2-rc5
2019-06-17 12:12:27 +02:00
trace
Linux 5.2-rc6
2019-06-24 19:19:53 +02:00
.gitignore
Provide in-kernel headers to make extending kernel easier
2019-04-29 16:48:03 +02:00
acct.c
async.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
audit.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
audit.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
audit_fsnotify.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157
2019-05-30 11:26:37 -07:00
audit_tree.c
fsnotify: switch send_to_group() and ->handle_event to const struct qstr *
2019-04-26 13:51:03 -04:00
audit_watch.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
auditfilter.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
auditsc.c
Merge branch 'work.dcache' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2019-05-07 20:03:32 -07:00
backtracetest.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
bounds.c
capability.c
compat.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500
2019-06-19 17:09:55 +02:00
configs.c
context_tracking.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
cpu.c
x86/power: Fix 'nosmt' vs hibernation triple fault during resume
2019-06-03 12:02:03 +02:00
cpu_pm.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 282
2019-06-05 17:36:37 +02:00
crash_core.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230
2019-06-19 17:09:06 +02:00
crash_dump.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
cred.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2019-06-11 15:44:45 -10:00
delayacct.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 25
2019-05-21 11:52:39 +02:00
dma.c
elfcore.c
exec_domain.c
exit.c
cgroup: Call cgroup_release() before __exit_signal()
2019-05-31 10:38:57 -07:00
extable.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
fail_function.c
treewide: Switch printk users from %pf and %pF to %ps and %pS, respectively
2019-04-09 14:19:06 +02:00
fork.c
sched/core: Provide a pointer to the valid CPU mask
2019-06-03 11:49:37 +02:00
freezer.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
futex.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
gen_kheaders.sh
kheaders: Do not regenerate archive if config is not changed
2019-05-24 20:16:01 +02:00
groups.c
hung_task.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
iomem.c
mm/resource: Use resource_overlaps() to simplify region_intersects()
2019-04-19 12:59:36 +02:00
irq_work.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
jump_label.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
kallsyms.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
kcmp.c
Kconfig.freezer
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
Kconfig.hz
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
Kconfig.locks
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
Kconfig.preempt
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
kcov.c
kexec.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230
2019-06-19 17:09:06 +02:00
kexec_core.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230
2019-06-19 17:09:06 +02:00
kexec_file.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230
2019-06-19 17:09:06 +02:00
kexec_internal.h
kheaders.c
kheaders: Move from proc to sysfs
2019-05-24 20:16:01 +02:00
kmod.c
kprobes.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
ksysfs.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 170
2019-05-30 11:26:39 -07:00
kthread.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
latencytop.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
Makefile
kheaders: Move from proc to sysfs
2019-05-24 20:16:01 +02:00
memremap.c
mm/devm_memremap_pages: fix final page put race
2019-06-13 17:34:56 -10:00
module-internal.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36
2019-05-24 17:27:11 +02:00
module.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
module_signing.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36
2019-05-24 17:27:11 +02:00
notifier.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
nsproxy.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
padata.c
padata: Replace padata_attr_type default_attrs field with groups
2019-04-25 22:06:11 +02:00
panic.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
params.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
pid.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
pid_namespace.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
profile.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
ptrace.c
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2019-06-11 15:44:45 -10:00
range.c
reboot.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
relay.c
resource.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
rseq.c
rseq: Remove superfluous rseq_len from task_struct
2019-04-19 12:39:32 +02:00
seccomp.c
audit/stable-5.2 PR 20190507
2019-05-07 19:06:04 -07:00
signal.c
signal: improve comments
2019-06-05 15:06:07 +02:00
smp.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
smpboot.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
smpboot.h
softirq.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 430
2019-06-05 17:37:16 +02:00
stackleak.c
stacktrace.c
Merge branch 'core-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-06-02 11:04:42 -07:00
stop_machine.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 38
2019-05-24 17:27:11 +02:00
sys.c
prctl_set_mm: downgrade mmap_sem to read lock
2019-06-01 15:51:31 -07:00
sys_ni.c
signal: support CLONE_PIDFD with pidfd_send_signal
2019-05-07 14:31:03 +02:00
sysctl.c
sysctl: define proc_do_static_key()
2019-06-14 20:18:27 -07:00
sysctl_binary.c
task_work.c
taskstats.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157
2019-05-30 11:26:37 -07:00
test_kprobes.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 25
2019-05-21 11:52:39 +02:00
torture.c
tracepoint.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
tsacct.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157
2019-05-30 11:26:37 -07:00
ucount.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
uid16.c
uid16.h
umh.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
up.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
user-return-notifier.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
user.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
user_namespace.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
utsname.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
utsname_sysctl.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
watchdog.c
watchdog: Fix typo in comment
2019-04-18 14:05:51 +02:00
watchdog_hld.c
kernel/watchdog_hld.c: hard lockup message should end with a newline
2019-04-19 09:46:05 -07:00
workqueue.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
workqueue_internal.h
sched/core, workqueues: Distangle worker accounting from rq lock
2019-04-16 16:55:15 +02:00