Johannes Weiner
42a3003535
mm: memcontrol: fix recursive statistics correctness & scalabilty ...
Right now, when somebody needs to know the recursive memory statistics
and events of a cgroup subtree, they need to walk the entire subtree and
sum up the counters manually.
There are two issues with this:
1. When a cgroup gets deleted, its stats are lost. The state counters
should all be 0 at that point, of course, but the events are not.
When this happens, the event counters, which are supposed to be
monotonic, can go backwards in the parent cgroups.
2. During regular operation, we always have a certain number of lazily
freed cgroups sitting around that have been deleted, have no tasks,
but have a few cache pages remaining. These groups' statistics do not
change until we eventually hit memory pressure, but somebody
watching, say, memory.stat on an ancestor has to iterate those every
time.
This patch addresses both issues by introducing recursive counters at
each level that are propagated from the write side when stats change.
Upward propagation happens when the per-cpu caches spill over into the
local atomic counter. This is the same thing we do during charge and
uncharge, except that the latter uses atomic RMWs, which are more
expensive; stat changes happen at around the same rate. In a sparse
file test (page faults and reclaim at maximum CPU speed) with 5 cgroup
nesting levels, perf shows __mod_memcg_page state at ~1%.
Link: http://lkml.kernel.org/r/20190412151507.2769-4-hannes@cmpxchg.org
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Shakeel Butt <shakeelb@google.com>
Reviewed-by: Roman Gushchin <guro@fb.com>
Cc: Michal Hocko <mhocko@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-05-14 19:52:53 -07:00
Merge branch 'clk-ti' into clk-next
2019-05-07 11:46:02 -07:00
dmaengine: idma64: Move driver name to the header
2019-04-26 16:55:23 +05:30
net: dsa: sja1105: Add support for traffic through standalone ports
2019-05-05 21:52:42 -07:00
This is the bulk of the GPIO changes for the v5.2 kernel cycle:
2019-05-11 10:54:43 -04:00
lockd: Store the lockd client credential in struct nlm_host
2019-04-26 17:51:23 -04:00
- Core Frameworks
2019-05-14 10:39:08 -07:00
5.2 Merge Window pull request
2019-05-09 09:02:46 -07:00
ipset: drop ipset_nest_start() and ipset_nest_end()
2019-04-27 17:03:44 -04:00
- Fix-ups
2019-05-14 10:45:03 -07:00
include/linux/sched/signal.h: replace tsk' with task'
2019-05-14 19:52:52 -07:00
soundwire: remove multiple blank lines
2019-05-02 17:17:52 +02:00
spi: expand mode support
2019-05-02 10:37:55 +09:00
SUNRPC: Cache cred of process creating the rpc_client
2019-04-26 16:00:48 -04:00
USB: changes for v5.2 merge window
2019-05-03 18:05:27 +02:00
pci-v5.2-changes
2019-05-14 10:30:10 -07:00
mmc: alcor: work with multiple-entry sglists
2019-05-06 11:55:39 +02:00
mailbox: Add support for Armada 37xx rWTM mailbox
2019-05-09 00:41:00 -05:00
include/linux/balloon_compaction.h: drop unused function stubs
2019-05-14 09:47:48 -07:00
exec: move struct linux_binprm::buf
2019-05-14 19:52:50 -07:00
block: switch all files cleared marked as GPLv2 to SPDX tags
2019-04-30 16:11:57 -06:00
include/linux/bitops.h: sanitize rotate primitives
2019-05-14 19:52:49 -07:00
block: add a SPDX tag to blk-mq-rdma.h
2019-04-30 16:12:02 -06:00
blk-mq: always free hctx after request queue is freed
2019-05-04 07:24:08 -06:00
blk-mq: always free hctx after request queue is freed
2019-05-04 07:24:08 -06:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
2019-05-02 22:14:21 -04:00
bpf: Introduce bpf sk local storage
2019-04-27 09:07:04 -07:00
block: switch all files cleared marked as GPLv2 or later to SPDX tags
2019-04-30 16:11:59 -06:00
block: fix mismerge in bvec_advance
2019-05-07 08:39:02 -06:00
cgroup: get rid of cgroup_freezer_frozen_exit()
2019-05-06 08:39:11 -07:00
Merge branch 'clk-parent-rewrite-1' into clk-next
2019-05-07 11:46:13 -07:00
compiler: allow all arches to enable CONFIG_OPTIMIZE_INLINING
2019-05-14 19:52:48 -07:00
coresight: Communicate perf event to sink buffer allocation functions
2019-04-25 22:00:17 +02:00
Merge branch 'x86-mds-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-05-14 07:57:29 -07:00
powerpc updates for 5.2
2019-05-10 05:29:27 -07:00
include/linux/cpumask.h: fix double string traverse in cpumask_parse
2019-05-14 19:52:50 -07:00
Clean up fscrypt's dcache revalidation support, and other
2019-05-07 21:28:04 -07:00
We have a couple new features and changes in the core clk framework this time
2019-05-09 14:50:09 -07:00
dma-mapping: add a Kconfig symbol to indicate arch_dma_prep_coherent presence
2019-05-06 15:04:40 +02:00
RDMA/core: Introduce RDMA subsystem ibdev_* print functions
2019-05-01 12:29:28 -04:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
2019-04-28 08:42:41 -04:00
f2fs: allow unfixed f2fs_checkpoint.checksum_offset
2019-05-08 21:23:11 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2019-05-07 22:03:58 -07:00
Add as a feature case-insensitive directories (the casefold feature)
2019-05-07 21:12:44 -07:00
This pull request contains the following changes for UBI/UBIFS
2019-05-12 18:16:31 -04:00
\n
2019-05-13 15:08:16 -07:00
\n
2019-05-13 15:08:16 -07:00
tracing: Cleanup stack trace code
2019-04-29 12:37:46 +02:00
hugetlb: allow to free gigantic pages regardless of the configuration
2019-05-14 09:47:47 -07:00
Merge branches 'for-5.1/upstream-fixes', 'for-5.2/core', 'for-5.2/ish', 'for-5.2/logitech', 'for-5.2/macally', 'for-5.2/picolcd', 'for-5.2/sensor' and 'for-5.2/u2fzero' into for-linus
2019-05-06 15:45:18 +02:00
mm/hmm: convert various hmm_pfn_* to device_entry which is a better name
2019-05-14 09:47:48 -07:00
mm/huge_memory: fix vmf_insert_pfn_{pmd, pud}() crash, handle unaligned addresses
2019-05-14 09:47:44 -07:00
hugetlb: use same fault hash key for shared and private mappings
2019-05-14 09:47:48 -07:00
i2c: core: ratelimit 'transfer when suspended' errors
2019-05-03 15:25:00 +02:00
ieee80211: update HE IEs to D4.0 spec
2019-04-26 13:02:11 +02:00
s390/kexec_file: Disable kexec_load when IPLed secure
2019-04-29 10:44:03 +02:00
iomap: Add a page_prepare callback
2019-05-01 07:47:37 -07:00
Merge branch 'api-features' into arm/smmu
2019-04-26 17:11:46 +02:00
ipc: conserve sequence numbers in ipcmni_extend mode
2019-05-14 19:52:52 -07:00
locking/static_key: Add support for deferred static branches
2019-04-29 08:29:20 +02:00
lib/math: move int_pow() from pwm_bl.c for wider use
2019-05-14 19:52:49 -07:00
selinux/stable-5.2 PR 20190507
2019-05-07 18:48:09 -07:00
kobject: Add support for default attribute groups to kobj_type
2019-04-25 22:06:10 +02:00
include/: refactor headers to allow kthread.h inclusion in psi_types.h
2019-05-14 19:52:48 -07:00
kernel/latencytop.c: rename clear_all_latency_tracing to clear_tsk_latency_tracing
2019-05-14 19:52:49 -07:00
mm: shuffle initial free memory to improve memory-side-cache utilization
2019-05-14 19:52:48 -07:00
lib/list_sort: simplify and remove MAX_LIST_LENGTH_BITS
2019-05-14 19:52:49 -07:00
livepatch: Remove custom kobject state handling
2019-05-03 21:11:22 +02:00
Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-05-06 13:50:15 -07:00
Merge branch 'work.mount-syscalls' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2019-05-07 20:17:51 -07:00
IOMMU Updates for Linux v5.2
2019-05-13 09:23:18 -04:00
mm: memblock: make keeping memblock memory opt-in rather than opt-out
2019-05-14 09:47:50 -07:00
mm: memcontrol: fix recursive statistics correctness & scalabilty
2019-05-14 19:52:53 -07:00
mm/memory_hotplug: make unregister_memory_section() never fail
2019-05-14 09:47:49 -07:00
mm/memory_hotplug: make __remove_pages() and arch_remove_memory() never fail
2019-05-14 09:47:50 -07:00
mm: move buddy list manipulations into helpers
2019-05-14 19:52:48 -07:00
mm: memcontrol: track LRU counts in the vmstats array
2019-05-14 09:47:46 -07:00
mm: move buddy list manipulations into helpers
2019-05-14 19:52:48 -07:00
mm/mmu_notifier: mmu_notifier_range_update_to_read_only() helper
2019-05-14 09:47:49 -07:00
mm: maintain randomization of page free lists
2019-05-14 19:52:48 -07:00
Merge branch 'next-integrity' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-05-09 12:54:40 -07:00
moduleparam: Save information about built-in modules in separate file
2019-05-07 21:50:24 +09:00
net/tls: move definition of tls ops into net/tls.h
2019-04-27 16:52:21 -04:00
netfilter: slightly optimize nf_inet_addr_mask
2019-05-06 01:18:58 +02:00
NFS: Store the credential of the mount process in the nfs_server
2019-04-26 16:11:54 -04:00
of: fix clang -Wunsequenced for be32_to_cpu()
2019-05-01 14:26:36 -05:00
lib: Add support for generic packing operations
2019-05-03 10:49:17 -04:00
mm: delete find_get_entries_tag
2019-05-14 09:47:51 -07:00
PCI: al: Add Amazon Annapurna Labs PCIe host controller driver
2019-04-25 16:33:07 -05:00
pci-v5.2-changes
2019-05-14 10:30:10 -07:00
Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-05-06 14:16:36 -07:00
net: phy: improve resuming from hibernation
2019-05-04 00:50:58 -04:00
clone: add CLONE_PIDFD
2019-05-07 14:31:03 +02:00
There tracing fixes:
2019-04-26 11:09:55 -07:00
lib/plist: rename DEBUG_PI_LIST to DEBUG_PLIST
2019-05-14 19:52:49 -07:00
PM / Domains: Allow to attach a CPU via genpd_dev_pm_attach_by_id|name()
2019-04-25 23:18:03 +02:00
fs/select: avoid clang stack usage warning
2019-05-14 19:52:48 -07:00
pps: pps-gpio PPS ECHO implementation
2019-05-14 19:52:51 -07:00
panic: avoid the extra noise dmesg
2019-05-14 19:52:51 -07:00
kernel/sched/psi.c: expose pressure metrics on root cgroup
2019-05-14 19:52:48 -07:00
psi: introduce psi monitor
2019-05-14 19:52:48 -07:00
treewide: remove SPDX "WITH Linux-syscall-note" from kernel-space headers
2019-05-14 19:52:48 -07:00
panic/reboot: allow specifying reboot_mode for panic only
2019-05-14 19:52:51 -07:00
rtc: drop set_mms and set_mmss64
2019-05-08 22:14:36 +02:00
lib/scatterlist: Remove leftover from sg_page_iter comment
2019-05-07 12:47:47 -03:00
include/: refactor headers to allow kthread.h inclusion in psi_types.h
2019-05-14 19:52:48 -07:00
Merge branch 'work.mount-syscalls' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2019-05-07 20:17:51 -07:00
block: switch all files cleared marked as GPLv2 to SPDX tags
2019-04-30 16:11:57 -06:00
x86/mm/cpa: Add set_direct_map_*() functions
2019-04-30 12:37:56 +02:00
bpf: support BPF_PROG_QUERY for BPF_FLOW_DISSECTOR attach_type
2019-04-25 23:49:06 +02:00
lib/stackdepot: Remove obsolete functions
2019-04-29 12:37:57 +02:00
stacktrace: Provide common infrastructure
2019-04-29 12:37:57 +02:00
Power management updates for 5.2-rc1
2019-05-06 19:40:31 -07:00
bpf: add writable context for raw tracepoints
2019-04-26 19:04:19 -07:00
iov_iter: fix iov_iter_type
2019-05-01 08:38:47 -06:00
uprobes: Initialize uprobes earlier
2019-04-30 12:37:51 +02:00
USB/PHY patches for 5.2-rc1
2019-05-08 10:03:52 -07:00
userfaultfd/sysctl: add vm.unprivileged_userfaultfd
2019-05-14 09:47:45 -07:00
mm/vmalloc: Add flag for freeing of special permsissions
2019-04-30 12:37:58 +02:00
mm: move recent_rotated pages calculation to shrink_inactive_list()
2019-05-14 09:47:45 -07:00
