Johannes Weiner
eb414681d5
psi: pressure stall information for CPU, memory, and IO ...
When systems are overcommitted and resources become contended, it's hard
to tell exactly the impact this has on workload productivity, or how close
the system is to lockups and OOM kills. In particular, when machines work
multiple jobs concurrently, the impact of overcommit in terms of latency
and throughput on the individual job can be enormous.
In order to maximize hardware utilization without sacrificing individual
job health or risk complete machine lockups, this patch implements a way
to quantify resource pressure in the system.
A kernel built with CONFIG_PSI=y creates files in /proc/pressure/ that
expose the percentage of time the system is stalled on CPU, memory, or IO,
respectively. Stall states are aggregate versions of the per-task delay
accounting delays:
cpu: some tasks are runnable but not executing on a CPU
memory: tasks are reclaiming, or waiting for swapin or thrashing cache
io: tasks are waiting for io completions
These percentages of walltime can be thought of as pressure percentages,
and they give a general sense of system health and productivity loss
incurred by resource overcommit. They can also indicate when the system
is approaching lockup scenarios and OOMs.
To do this, psi keeps track of the task states associated with each CPU
and samples the time they spend in stall states. Every 2 seconds, the
samples are averaged across CPUs - weighted by the CPUs' non-idle time to
eliminate artifacts from unused CPUs - and translated into percentages of
walltime. A running average of those percentages is maintained over 10s,
1m, and 5m periods (similar to the loadaverage).
[hannes@cmpxchg.org: doc fixlet, per Randy]
Link: http://lkml.kernel.org/r/20180828205625.GA14030@cmpxchg.org
[hannes@cmpxchg.org: code optimization]
Link: http://lkml.kernel.org/r/20180907175015.GA8479@cmpxchg.org
[hannes@cmpxchg.org: rename psi_clock() to psi_update_work(), per Peter]
Link: http://lkml.kernel.org/r/20180907145404.GB11088@cmpxchg.org
[hannes@cmpxchg.org: fix build]
Link: http://lkml.kernel.org/r/20180913014222.GA2370@cmpxchg.org
Link: http://lkml.kernel.org/r/20180828172258.3185-9-hannes@cmpxchg.org
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Daniel Drake <drake@endlessm.com>
Tested-by: Suren Baghdasaryan <surenb@google.com>
Cc: Christopher Lameter <cl@linux.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Johannes Weiner <jweiner@fb.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Enderborg <peter.enderborg@sony.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Vinayak Menon <vinmenon@codeaurora.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-10-26 16:26:32 -07:00
mmc: mmci: Drop support for pdata GPIO numbers
2018-10-08 11:40:43 +02:00
virtchnl: Added support to exchange additional speed values
2018-10-03 07:42:29 -07:00
dmaengine: sprd: Support DMA link-list mode
2018-09-03 16:58:50 +05:30
firmware /meson
fpga: add devm_fpga_region_create
2018-10-16 11:13:50 +02:00
bus: fsl-mc: add support for dpseci device type
2018-09-21 13:24:50 +08:00
This is the bulk of GPIO changes for the v4.20 series:
2018-10-23 08:45:05 +01:00
KVM updates for v4.20
2018-10-25 17:57:35 -07:00
power supply and reset changes for the v4.20 series
2018-10-25 06:28:08 -07:00
First merge window pull request
2018-10-26 07:38:19 -07:00
mmc: core: add helper to see if a host is doing a retune
2018-10-08 11:40:43 +02:00
SPI NOR changes:
2018-10-23 01:09:22 +01:00
netfilter: nft_osf: Add ttl option support
2018-10-16 10:01:48 +02:00
arm64: perf: Reject stand-alone CHAIN events for PMUv3
2018-10-12 15:25:17 +01:00
USB/PHY patches for 4.20-rc1
2018-10-26 08:14:13 -07:00
USB/PHY patches for 4.20-rc1
2018-10-26 08:14:13 -07:00
power: supply: bq27xxx: Add support for BQ27411
2018-09-20 11:35:40 +02:00
First merge window pull request
2018-10-26 07:38:19 -07:00
regulator/mfd: Support for the ROHM BD71847
2018-09-28 15:07:30 +01:00
sched: loadavg: make calc_load_n() public
2018-10-26 16:26:32 -07:00
soundwire: keep track of Masters in a stream
2018-08-27 09:49:48 +05:30
Merge remote-tracking branch 'spi/topic/of' into spi-next
2018-10-21 17:00:17 +01:00
gss_krb5: Remove VLA usage of skcipher
2018-09-28 12:46:07 +08:00
USB/PHY patches for 4.20-rc1
2018-10-26 08:14:13 -07:00
pci-v4.20-changes
2018-10-25 06:50:48 -07:00
ACPI/ADXL: Add address translation interface using an ACPI DSM
2018-10-16 10:03:00 +02:00
sched/topology, drivers/base/arch_topology: Rebuild the sched_domain hierarchy when capacities change
2018-09-10 11:05:47 +02:00
arm/arm64: smccc-1.1: Handle function result as parameters
2018-08-30 14:18:03 +01:00
signal: Distinguish between kernel_siginfo and siginfo
2018-10-03 16:47:43 +02:00
blkcg: reassociate bios when make_request() is called recursively
2018-10-20 15:39:55 -06:00
blkcg: fix edge case for blk_get_rl() under memory pressure
2018-10-20 15:39:54 -06:00
blk-mq: provide helper for setting up an SQ queue and tag set
2018-10-15 20:05:18 -06:00
block: Move power management code into a new source file
2018-09-26 15:11:28 -06:00
blkcg: remove bio->bi_css and instead use bio->bi_blkg
2018-09-21 20:29:13 -06:00
pci-v4.20-changes
2018-10-25 06:50:48 -07:00
bpf: introduce per-cpu cgroup local storage
2018-10-01 16:18:32 +02:00
bpf: add queue and stack maps
2018-10-19 13:24:31 -07:00
bpf: add queue and stack maps
2018-10-19 13:24:31 -07:00
bpf: add verifier callback to get stack usage info for offloaded progs
2018-10-08 10:24:12 +02:00
ext4: convert fault handler to use vm_fault_t type
2018-10-02 22:20:50 -04:00
block: remove bio_rewind_iter()
2018-09-06 15:12:24 -06:00
cgroup: Fix dom_cgrp propagation when enabling threaded mode
2018-10-04 13:28:08 -07:00
Merge branch 'for-4.20' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
2018-10-25 17:15:46 -07:00
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:14:36 -07:00
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:14:36 -07:00
s390 updates for the 4.20 merge window
2018-10-23 11:14:47 +01:00
x86/objtool: Use asm macros to work around GCC inlining bugs
2018-10-04 11:24:58 +02:00
Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
2018-10-25 16:43:35 -07:00
signal: Distinguish between kernel_siginfo and siginfo
2018-10-03 16:47:43 +02:00
coresight: Add support for CLAIM tag protocol
2018-09-25 20:09:19 +02:00
cpufeature: avoid warning when compiling with clang
2018-09-21 13:24:52 +08:00
cpuidle: menu: Fix wakeup statistics updates for polling state
2018-10-04 10:23:37 +02:00
kdump, proc/vmcore: Enable kdumping encrypted memory with SME enabled
2018-10-06 12:09:26 +02:00
crc-t10dif: Pick better transform if one becomes available
2018-09-04 11:37:05 +08:00
crypto: user - Implement a generic crypto statistics
2018-09-28 12:46:25 +08:00
locking/lockdep: Make global debug_locks* variables read-mostly
2018-10-19 07:53:18 +02:00
delayacct: track delays from thrashing cache pages
2018-10-26 16:26:32 -07:00
PM / devfreq: Make update_devfreq() public
2018-10-02 10:16:41 +09:00
Driver core patches for 4.20-rc1
2018-10-26 08:42:25 -07:00
dma-debug: Check for drivers mapping invalid addresses in dma_map_single()
2018-10-08 09:44:17 +02:00
dma-direct: implement complete bus_dma_mask handling
2018-10-01 07:28:03 -07:00
dma-mapping: translate __GFP_NOFAIL to DMA_ATTR_NO_WARN
2018-10-09 15:08:26 +02:00
dma-mapping: consolidate the dma mmap implementations
2018-09-20 09:01:16 +02:00
dns: Allow the dns resolver to retrieve a server set
2018-10-04 09:40:52 -07:00
EDAC: Raise the maximum number of memory controllers
2018-09-27 07:52:05 +02:00
efi/x86: Handle page faults occurring while running EFI runtime services
2018-09-26 12:14:55 +02:00
block: move call of scheduler's ->completed_request() hook
2018-09-27 17:34:52 -06:00
y2038: globally rename compat_time to old_time32
2018-08-27 14:48:48 +02:00
ethtool: drop get_settings and set_settings callbacks
2018-08-29 19:46:10 -07:00
Kbuild updates for v4.19 (2nd)
2018-08-25 13:40:38 -07:00
f2fs: guarantee journalled quota data by checkpoint
2018-10-22 17:54:47 -07:00
bpf: add cg_skb_is_valid_access for BPF_PROG_TYPE_CGROUP_SKB
2018-10-19 13:49:34 -07:00
vfs: swap names of {do,vfs}_clone_file_range()
2018-09-24 10:54:01 +02:00
mtd: rawnand: fsl_ifc: fixup SRAM init for newer ctrl versions
2018-09-04 23:37:38 +02:00
This is the 4.19-rc6 release
2018-10-01 08:58:57 -06:00
docs/mm: make GFP flags descriptions usable as kernel-doc
2018-08-23 18:48:43 -07:00
Merge branch 'for-4.20/logitech-highres' into for-linus
2018-10-23 13:35:22 +02:00
mm: remove references to vm_insert_pfn()
2018-10-26 16:25:20 -07:00
mremap: properly flush TLB before releasing the page
2018-10-18 11:30:52 +02:00
mm: migration: fix migration of huge PMD shared pages
2018-10-05 16:32:04 -07:00
hwrng: core - document the quality field
2018-10-05 10:16:56 +08:00
hwmon: (core) Add hwmon_in_enable attribute
2018-10-10 20:37:13 -07:00
vmbus: split ring buffer allocation from open
2018-09-25 20:33:24 +02:00
i2c: refactor function to release a DMA safe buffer
2018-08-30 23:13:15 +02:00
Merge branch 'ida-4.19' of git://git.infradead.org/users/willy/linux-dax
2018-08-26 11:48:42 -07:00
mac80211: minstrel: Enable STBC and LDPC for VHT Rates
2018-10-11 16:01:00 +02:00
tun: switch to new type of msg_control
2018-09-13 09:25:40 -07:00
LSM: Convert from initcall to struct lsm_info
2018-10-10 20:40:21 -07:00
genirq: Fix grammar s/an /a /
2018-10-09 07:50:41 +02:00
fs/iomap.c: change return type to vm_fault_t
2018-10-26 16:25:18 -07:00
ipmi: Finally get rid of ipmi_user_t and ipmi_smi_t
2018-09-18 16:15:33 -05:00
ipmi: Finally get rid of ipmi_user_t and ipmi_smi_t
2018-09-18 16:15:33 -05:00
ipv6: Add sockopt IPV6_MULTICAST_ALL analogue to IP_MULTICAST_ALL
2018-09-13 08:17:27 -07:00
irq/matrix: Spread managed interrupts on allocation
2018-09-18 18:27:24 +02:00
irqchip/irq-mvebu-sei: Add new driver for Marvell SEI
2018-10-02 12:00:08 +01:00
jump_label: Annotate entries that operate on __init code earlier
2018-09-27 17:56:48 +02:00
Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-08-26 11:25:21 -07:00
kernfs: update comment about kernfs_path() return value
2018-09-16 22:37:16 +02:00
kvm: x86: make kvm_{load|put}_guest_fpu() static
2018-09-20 00:51:43 +02:00
leds: core: Introduce LED pattern trigger
2018-10-11 21:55:58 +02:00
Merge branch 'for-4.19' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/libata
2018-08-24 13:20:33 -07:00
lightnvm: pblk: fix mapping issue on failed writes
2018-10-09 08:25:07 -06:00
include/linux/linkage.h: align weak symbols
2018-10-26 16:25:18 -07:00
net: phy: Replace phy driver features u32 with link_mode bitmap
2018-10-01 22:55:36 -07:00
locking/lockdep: Make class->ops a percpu counter and move it under CONFIG_DEBUG_LOCKDEP=y
2018-10-09 09:56:33 +02:00
Merge branch 'next-general' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2018-10-24 11:49:35 +01:00
mm: don't miss the last page because of round-off error
2018-10-26 16:25:19 -07:00
mm: rework memcg kernel stack accounting
2018-10-26 16:25:19 -07:00
PCI/P2PDMA: Add PCI p2pmem DMA mappings to adjust the bus offset
2018-10-17 12:18:16 -05:00
net: phy: micrel: add Microchip KSZ9131 initial driver
2018-10-19 17:02:23 -07:00
net: phy: Add limkmode equivalents to some of the MII ethtool helpers
2018-10-01 22:55:36 -07:00
mm: remove vm_insert_pfn()
2018-10-26 16:25:20 -07:00
mm: get rid of vmacache_flush_all() entirely
2018-09-13 15:18:04 -10:00
mm: get rid of vmacache_flush_all() entirely
2018-09-13 15:18:04 -10:00
Revert "mm, mmu_notifier: annotate mmu notifiers with blockable invalidate callbacks"
2018-10-26 16:25:19 -07:00
mm: workingset: tell cache transitions from workingset thrashing
2018-10-26 16:26:32 -07:00
linux/mod_devicetable.h: fix kernel-doc missing notation for typec_device_id
2018-09-05 14:36:53 +02:00
Merge branch 'next-general' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2018-10-24 11:49:35 +01:00
net: Plumb support for filtering ipv4 and ipv6 multicast route dumps
2018-10-16 00:13:39 -07:00
genirq/msi: Allow creation of a tree-based irqdomain for platform-msi
2018-10-02 11:16:38 +01:00
UAPI: ndctl: Remove use of PAGE_SIZE
2018-10-17 13:56:58 -07:00
FDDI: defza: Support capturing outgoing SMT traffic
2018-10-15 21:46:06 -07:00
netfilter: avoid erronous array bounds warning
2018-09-28 14:47:40 +02:00
netlink: Add answer_flags to netlink_callback
2018-10-16 00:13:12 -07:00
netpoll: allow cleanup to be synchronous
2018-10-19 17:01:43 -07:00
nvme: call nvme_complete_rq when nvmf_check_ready fails for mpath I/O
2018-10-01 14:16:13 -07:00
nvmem: use EOPNOTSUPP instead of ENOSYS
2018-09-28 15:14:55 +02:00
nvmem: use EOPNOTSUPP instead of ENOSYS
2018-09-28 15:14:55 +02:00
of: Add device_type access helper functions
2018-08-31 08:30:42 -04:00
dma-mapping: remove dma_deconfigure
2018-09-08 11:19:28 +02:00
mm: Change return type int to vm_fault_t for fault handlers
2018-08-23 18:48:44 -07:00
mm: workingset: tell cache transitions from workingset thrashing
2018-10-26 16:26:32 -07:00
PCI: Remove pci_set_dma_max_seg_size()
2018-10-10 15:47:00 -05:00
PCI/P2PDMA: Introduce configfs/sysfs enable attribute helpers
2018-10-17 12:18:17 -05:00
pci-v4.20-changes
2018-10-25 06:50:48 -07:00
PCI: hotplug: Embed hotplug_slot
2018-09-18 17:52:15 -05:00
pci-v4.20-changes
2018-10-25 06:50:48 -07:00
percpu-refcount: Introduce percpu_ref_resurrect()
2018-09-26 15:11:29 -06:00
include/linux/pfn_t.h: force '~' to be parsed as an unary operator
2018-10-26 16:25:18 -07:00
net: phy: improve handling delayed work
2018-10-01 23:14:10 -07:00
ACPI / glue: Split dev_is_platform() out of module for wide use
2018-09-10 12:48:50 +02:00
PM / Domains: Document flags for genpd
2018-10-18 12:25:10 +02:00
OPP: Pass index to _of_init_opp_table()
2018-09-19 14:56:41 -07:00
signal: Distinguish between kernel_siginfo and siginfo
2018-10-03 16:47:43 +02:00
psi: pressure stall information for CPU, memory, and IO
2018-10-26 16:26:32 -07:00
psi: pressure stall information for CPU, memory, and IO
2018-10-26 16:26:32 -07:00
pstore/ram: Clarify resource reservation labels
2018-10-22 07:11:58 -07:00
Merge branch 'siginfo-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2018-10-24 11:22:39 +01:00
spi: pxa2xx: Add devicetree support
2018-10-11 15:28:04 +01:00
soc: qcom: geni: Make version macros simpler
2018-09-23 21:28:00 -07:00
firmware: qcom: scm: Add WLAN VMID for Qualcomm SCM interface
2018-10-13 20:31:37 +03:00
rcu: Clean up flavor-related definitions and comments in rculist.h
2018-08-30 16:03:31 -07:00
rcu: Provide functions for determining if call_rcu() has been invoked
2018-08-30 16:03:42 -07:00
rcu: Clean up flavor-related definitions and comments in rcupdate_wait.h
2018-08-30 16:03:32 -07:00
Merge branches 'doc.2018.08.30a', 'dynticks.2018.08.30b', 'srcu.2018.08.30b' and 'torture.2018.08.29a' into HEAD
2018-08-30 16:12:53 -07:00
rcu: Define rcu_all_qs() only in !PREEMPT builds
2018-08-30 16:03:37 -07:00
Merge remote-tracking branches 'regmap/topic/noinc' and 'regmap/topic/single-rw' into regmap-next
2018-10-21 12:07:26 +01:00
y2038: globally rename compat_time to old_time32
2018-08-27 14:48:48 +02:00
net: sched: extend Qdisc with rcu
2018-09-25 20:17:35 -07:00
locking/rwsem: Make owner store task pointer of last owning reader
2018-09-10 12:04:07 +02:00
psi: pressure stall information for CPU, memory, and IO
2018-10-26 16:26:32 -07:00
signal: Distinguish between kernel_siginfo and siginfo
2018-10-03 16:47:43 +02:00
Revert "serial: sh-sci: Remove SCIx_RZ_SCIFA_REGTYPE"
2018-10-02 14:38:02 -07:00
signal: Guard against negative signal numbers in copy_siginfo_from_user32
2018-10-10 20:34:14 -05:00
signal: Use a smaller struct siginfo in the kernel
2018-10-03 16:50:39 +02:00
net: skbuff.h: Mark expected switch fall-throughs
2018-10-17 22:31:30 -07:00
bpf: sk_msg program helper bpf_msg_push_data
2018-10-20 21:37:11 +02:00
mm, slab/slub: introduce kmalloc-reclaimable caches
2018-10-26 16:26:31 -07:00
smp,cpumask: introduce on_each_cpu_cond_mask
2018-10-09 16:51:11 +02:00
y2038: socket: Change recvmmsg to use __kernel_timespec
2018-08-29 15:42:24 +02:00
srcu: Make early-boot call_srcu() reuse workqueue lists
2018-08-30 16:10:49 -07:00
srcu: Make early-boot call_srcu() reuse workqueue lists
2018-08-30 16:10:49 -07:00
init: add arch_call_rest_init to allow stack switching
2018-10-09 11:20:55 +02:00
net: stmmac: Rework coalesce timer and fix multi-queue races
2018-09-18 19:48:08 -07:00
lib: Add memcat_p(): paste 2 pointer arrays together
2018-10-11 12:12:55 +02:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input
2018-10-12 12:35:02 +02:00
mm: workingset: tell cache transitions from workingset thrashing
2018-10-26 16:26:32 -07:00
mm: Change return type int to vm_fault_t for fault handlers
2018-08-23 18:48:44 -07:00
y2038: signal: Change rt_sigtimedwait to use __kernel_timespec
2018-08-29 15:42:25 +02:00
tcp: do not change tcp_wstamp_ns in tcp_mstamp_refresh
2018-10-15 22:56:41 -07:00
thunderbolt: Convert rest of the driver files to use SPDX identifier
2018-10-02 15:52:08 -07:00
y2038: utimes: Rework #ifdef guards for compat syscalls
2018-08-29 15:42:23 +02:00
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:14:36 -07:00
y2038: make do_gettimeofday() and get_seconds() inline
2018-08-27 14:45:58 +02:00
rcutorture: Check GP completion at stutter end
2018-08-29 09:20:48 -07:00
signal: Simplify tracehook_report_syscall_exit
2018-09-19 15:45:42 +02:00
tracepoint: Fix tracepoint array element size mismatch
2018-10-17 15:35:29 -04:00
tracepoint: Fix tracepoint array element size mismatch
2018-10-17 15:35:29 -04:00
move compat handling of tty ioctls to tty_compat_ioctl()
2018-09-14 11:12:17 -04:00
tty_ioctl(): start taking TIOC[SG]SERIAL into separate methods
2018-09-14 11:19:30 -04:00
change semantics of ldisc ->compat_ioctl()
2018-10-13 00:50:53 -04:00
uaccess: Fix is_source param for check_copy_size() in copy_to_iter_mcsafe()
2018-09-12 14:58:47 -07:00
uio: introduce UIO_MEM_IOVA
2018-09-25 20:33:24 +02:00
umh: Add command line to user mode helpers
2018-10-22 19:37:36 -07:00
mm: Change return type int to vm_fault_t for fault handlers
2018-08-23 18:48:44 -07:00
ALSA: hda - Enable runtime PM only for discrete GPU
2018-09-13 17:58:30 +02:00
net/packet: fix packet drop as of virtio gso
2018-10-04 22:23:15 -07:00
mm: get rid of vmacache_flush_all() entirely
2018-09-13 15:18:04 -10:00
mm: get rid of vmacache_flush_all() entirely
2018-09-13 15:18:04 -10:00
scsi: sched/wait: Add wait_event_lock_irq_timeout for TASK_UNINTERRUPTIBLE usage
2018-10-16 00:11:13 -04:00
blkcg: associate writeback bios with a blkg
2018-09-21 20:29:11 -06:00
