Johannes Weiner
eb414681d5
psi: pressure stall information for CPU, memory, and IO ...
When systems are overcommitted and resources become contended, it's hard
to tell exactly the impact this has on workload productivity, or how close
the system is to lockups and OOM kills. In particular, when machines work
multiple jobs concurrently, the impact of overcommit in terms of latency
and throughput on the individual job can be enormous.
In order to maximize hardware utilization without sacrificing individual
job health or risk complete machine lockups, this patch implements a way
to quantify resource pressure in the system.
A kernel built with CONFIG_PSI=y creates files in /proc/pressure/ that
expose the percentage of time the system is stalled on CPU, memory, or IO,
respectively. Stall states are aggregate versions of the per-task delay
accounting delays:
cpu: some tasks are runnable but not executing on a CPU
memory: tasks are reclaiming, or waiting for swapin or thrashing cache
io: tasks are waiting for io completions
These percentages of walltime can be thought of as pressure percentages,
and they give a general sense of system health and productivity loss
incurred by resource overcommit. They can also indicate when the system
is approaching lockup scenarios and OOMs.
To do this, psi keeps track of the task states associated with each CPU
and samples the time they spend in stall states. Every 2 seconds, the
samples are averaged across CPUs - weighted by the CPUs' non-idle time to
eliminate artifacts from unused CPUs - and translated into percentages of
walltime. A running average of those percentages is maintained over 10s,
1m, and 5m periods (similar to the loadaverage).
[hannes@cmpxchg.org: doc fixlet, per Randy]
Link: http://lkml.kernel.org/r/20180828205625.GA14030@cmpxchg.org
[hannes@cmpxchg.org: code optimization]
Link: http://lkml.kernel.org/r/20180907175015.GA8479@cmpxchg.org
[hannes@cmpxchg.org: rename psi_clock() to psi_update_work(), per Peter]
Link: http://lkml.kernel.org/r/20180907145404.GB11088@cmpxchg.org
[hannes@cmpxchg.org: fix build]
Link: http://lkml.kernel.org/r/20180913014222.GA2370@cmpxchg.org
Link: http://lkml.kernel.org/r/20180828172258.3185-9-hannes@cmpxchg.org
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Daniel Drake <drake@endlessm.com>
Tested-by: Suren Baghdasaryan <surenb@google.com>
Cc: Christopher Lameter <cl@linux.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Johannes Weiner <jweiner@fb.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Enderborg <peter.enderborg@sony.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Vinayak Menon <vinmenon@codeaurora.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-10-26 16:26:32 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
2018-10-21 21:11:46 -07:00
for-4.20/block-20181021
2018-10-22 17:46:08 +01:00
kconfig: tinyconfig: remove stale stack protector fixups
2018-06-15 07:15:28 +09:00
sched: loadavg: consolidate LOAD_INT, LOAD_FRAC, CALC_LOAD
2018-10-26 16:26:32 -07:00
dma-direct: respect DMA_ATTR_NO_WARN
2018-10-09 15:08:46 +02:00
Merge branch 'siginfo-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2018-10-24 11:22:39 +01:00
gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT
2018-06-08 18:56:02 +09:00
Merge branch 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:43:47 -07:00
Merge branch 'for-4.19/upstream' into for-linus
2018-08-20 18:33:50 +02:00
Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-23 13:08:53 +01:00
Merge branches 'acpi-pm' and 'pm-sleep'
2018-10-18 12:27:30 +02:00
printk: fix integer overflow in setup_log_buf()
2018-10-12 10:44:17 +02:00
Merge branches 'doc.2018.08.30a', 'dynticks.2018.08.30b', 'srcu.2018.08.30b' and 'torture.2018.08.29a' into HEAD
2018-08-30 16:12:53 -07:00
psi: pressure stall information for CPU, memory, and IO
2018-10-26 16:26:32 -07:00
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:14:36 -07:00
for-4.20/block-20181021
2018-10-22 17:46:08 +01:00
kernel/async.c: revert "async: simplify lowest_in_progress()"
2018-02-06 18:32:44 -08:00
audit: use ktime_get_coarse_real_ts64() for timestamps
2018-07-17 14:45:08 -04:00
audit: track the owner of the command mutex ourselves
2018-02-23 11:22:22 -05:00
fsnotify: add fsnotify_add_inode_mark() wrappers
2018-05-18 14:58:22 +02:00
\n
2018-08-17 09:41:28 -07:00
audit: fix use-after-free in audit_add_watch
2018-07-18 11:43:36 -04:00
audit: rename FILTER_TYPE to FILTER_EXCLUDE
2018-06-19 10:39:54 -04:00
audit/stable-4.18 PR 20180814
2018-08-15 10:46:54 -07:00
y2038: globally rename compat_time to old_time32
2018-08-27 14:48:48 +02:00
Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-23 18:43:04 +01:00
kernel/crash_core.c: print timestamp using time64_t
2018-08-22 10:52:47 -07:00
delayacct: track delays from thrashing cache pages
2018-10-26 16:26:32 -07:00
proc: introduce proc_create_single{,_data}
2018-05-16 07:23:35 +02:00
proc: introduce proc_create_single{,_data}
2018-05-16 07:23:35 +02:00
signal: Pass pid type into group_send_sig_info
2018-07-21 12:57:35 -05:00
extable: Make init_kernel_text() global
2018-02-21 16:54:06 +01:00
bpf/error-inject/kprobes: Clear current_kprobe and enable preempt in kprobe
2018-06-21 12:33:19 +02:00
psi: pressure stall information for CPU, memory, and IO
2018-10-26 16:26:32 -07:00
PM / reboot: Eliminate race between reboot and suspend
2018-08-06 12:35:20 +02:00
futex: Replace spin_is_locked() with lockdep
2018-10-09 13:19:28 +02:00
y2038: globally rename compat_time to old_time32
2018-08-27 14:48:48 +02:00
kernel/hung_task.c: allow to set checking interval separately from timeout
2018-08-22 10:52:47 -07:00
memremap: split devm_memremap_pages() and memremap() infrastructure
2018-05-15 23:08:33 -07:00
Merge branch 'x86/build' into locking/core, to pick up dependent patches and unify jump-label work
2018-10-16 17:30:11 +02:00
kallsyms, x86: Export addresses of PTI entry trampolines
2018-08-14 19:12:29 -03:00
kconfig: include kernel/Kconfig.preempt from init/Kconfig
2018-08-02 08:06:54 +09:00
sched/core / kcov: avoid kcov_area during task switch
2018-06-15 07:55:24 +09:00
kexec: add call to LSM hook in original kexec_load syscall
2018-07-16 12:31:57 -07:00
kexec: Allocate decrypted control pages for kdump if SME is enabled
2018-10-06 12:01:51 +02:00
treewide: Use array_size() in vzalloc()
2018-06-12 16:19:22 -07:00
kprobes: Don't call BUG_ON() if there is a kprobe in use on free list
2018-09-12 08:01:16 +02:00
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-08-13 11:25:07 -07:00
kbuild: move bin2c back to scripts/ from scripts/basic/
2018-07-18 01:18:05 +09:00
libnvdimm-for-4.19_dax-memory-failure
2018-08-25 18:43:59 -07:00
modsign: log module name in the event of an error
2018-07-02 11:36:17 +02:00
jump_table: Move entries into ro_after_init region
2018-09-27 17:56:49 +02:00
modsign: log module name in the event of an error
2018-07-02 11:36:17 +02:00
Kbuild: rename CC_STACKPROTECTOR[_STRONG] config variables
2018-06-14 12:21:18 +09:00
kernel/params.c: downgrade warning for unsafe parameters
2018-04-11 10:28:37 -07:00
fork: report pid exhaustion correctly
2018-09-20 22:01:11 +02:00
signal: Use group_send_sig_info to kill all processes in a pid namespace
2018-09-16 16:08:25 +02:00
Merge branch 'siginfo-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2018-10-24 11:22:39 +01:00
kernel/reboot.c: export pm_power_off_prepare
2018-09-11 16:13:24 +01:00
kernel/relay.c: change return type to vm_fault_t
2018-06-15 07:55:24 +09:00
resource: Clean it up a bit
2018-10-09 17:25:58 +02:00
rseq: uapi: Declare rseq_cs field as union, update includes
2018-07-10 22:18:52 +02:00
Merge branch 'next-general' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2018-10-24 11:49:35 +01:00
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:14:36 -07:00
smp,cpumask: introduce on_each_cpu_cond_mask
2018-10-09 16:51:11 +02:00
smpboot: Remove cpumask from the API
2018-07-03 09:20:44 +02:00
Merge branch 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:43:47 -07:00
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-08-13 11:25:07 -07:00
kernel/sys.c: remove duplicated include
2018-09-20 22:01:11 +02:00
Merge branch 'core-rseq-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-06-10 10:17:09 -07:00
namei: allow restricted O_CREAT of FIFOs and regular files
2018-08-23 18:48:43 -07:00
staging: irda: remove remaining remants of irda code removal
2018-04-16 11:26:49 +02:00
pids: introduce find_get_task_by_vpid() helper
2018-02-06 18:32:46 -08:00
kprobes: Remove jprobe API implementation
2018-06-21 12:33:05 +02:00
rcutorture: Check GP completion at stutter end
2018-08-29 09:20:48 -07:00
tracepoint: Fix tracepoint array element size mismatch
2018-10-17 15:35:29 -04:00
headers: untangle kmemleak.h from mm.h
2018-04-05 21:36:27 -07:00
fs: add do_fchownat(), ksys_fchown() helpers and ksys_{,l}chown() wrappers
2018-04-02 20:15:59 +02:00
kernel: provide ksys_*() wrappers for syscalls called by kernel/uid16.c
2018-04-02 20:15:30 +02:00
umh: Add command line to user mode helpers
2018-10-22 19:37:36 -07:00
smp,cpumask: introduce on_each_cpu_cond_mask
2018-10-09 16:51:11 +02:00
userns: use irqsave variant of refcount_dec_and_lock()
2018-08-22 10:52:47 -07:00
userns: move user access out of the mutex
2018-08-11 02:05:53 -05:00
uts: create "struct uts_namespace" from kmem_cache
2018-04-11 10:28:35 -07:00
sys: don't hold uts_sem while accessing userspace memory
2018-08-11 02:05:53 -05:00
watchdog: Mark watchdog touch functions as notrace
2018-08-30 12:56:40 +02:00
watchdog: Mark watchdog touch functions as notrace
2018-08-30 12:56:40 +02:00
watchdog: Mark watchdog touch functions as notrace
2018-08-30 12:56:40 +02:00
workqueue: Set worker->desc to workqueue name by default
2018-05-18 08:47:13 -07:00
eb414681d5