Chris Down
1bc63fb127
mm, memcg: make scan aggression always exclude protection ...
This patch is an incremental improvement on the existing
memory.{low,min} relative reclaim work to base its scan pressure
calculations on how much protection is available compared to the current
usage, rather than how much the current usage is over some protection
threshold.
This change doesn't change the experience for the user in the normal
case too much. One benefit is that it replaces the (somewhat arbitrary)
100% cutoff with an indefinite slope, which makes it easier to ballpark
a memory.low value.
As well as this, the old methodology doesn't quite apply generically to
machines with varying amounts of physical memory. Let's say we have a
top level cgroup, workload.slice, and another top level cgroup,
system-management.slice. We want to roughly give 12G to
system-management.slice, so on a 32GB machine we set memory.low to 20GB
in workload.slice, and on a 64GB machine we set memory.low to 52GB.
However, because these are relative amounts to the total machine size,
while the amount of memory we want to generally be willing to yield to
system.slice is absolute (12G), we end up putting more pressure on
system.slice just because we have a larger machine and a larger workload
to fill it, which seems fairly unintuitive. With this new behaviour, we
don't end up with this unintended side effect.
Previously the way that memory.low protection works is that if you are
50% over a certain baseline, you get 50% of your normal scan pressure.
This is certainly better than the previous cliff-edge behaviour, but it
can be improved even further by always considering memory under the
currently enforced protection threshold to be out of bounds. This means
that we can set relatively low memory.low thresholds for variable or
bursty workloads while still getting a reasonable level of protection,
whereas with the previous version we may still trivially hit the 100%
clamp. The previous 100% clamp is also somewhat arbitrary, whereas this
one is more concretely based on the currently enforced protection
threshold, which is likely easier to reason about.
There is also a subtle issue with the way that proportional reclaim
worked previously -- it promotes having no memory.low, since it makes
pressure higher during low reclaim. This happens because we base our
scan pressure modulation on how far memory.current is between memory.min
and memory.low, but if memory.low is unset, we only use the overage
method. In most cromulent configurations, this then means that we end
up with *more* pressure than with no memory.low at all when we're in low
reclaim, which is not really very usable or expected.
With this patch, memory.low and memory.min affect reclaim pressure in a
more understandable and composable way. For example, from a user
standpoint, "protected" memory now remains untouchable from a reclaim
aggression standpoint, and users can also have more confidence that
bursty workloads will still receive some amount of guaranteed
protection.
Link: http://lkml.kernel.org/r/20190322160307.GA3316@chrisdown.name
Signed-off-by: Chris Down <chris@chrisdown.name>
Reviewed-by: Roman Gushchin <guro@fb.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Michal Hocko <mhocko@kernel.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Dennis Zhou <dennis@kernel.org>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-10-07 15:47:20 -07:00
linux-can-next-for-5.4-20190904
2019-09-05 12:17:50 +02:00
libceph: add function that clears osd client's abort_err
2019-09-16 12:06:23 +02:00
clk: add include guard to clk-conf.h
2019-09-17 10:27:46 -07:00
Char/Misc driver patches for 5.4-rc1
2019-09-18 11:14:31 -07:00
gpio: Initialize the irqchip valid_mask with a callback
2019-09-11 01:09:37 +01:00
staging: greybus: fix more header declarations
2019-08-28 17:24:22 +02:00
chrome platform changes for v5.4
2019-09-19 14:14:28 -07:00
Merge branch 'next' into for-linus
2019-09-16 09:56:27 -07:00
mailbox: mediatek: cmdq: clear the event in cmdq initial flow
2019-09-17 00:40:05 -05:00
- New Drivers
2019-09-23 19:37:49 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net
2019-09-28 17:47:33 -07:00
dma-mapping updates for 5.4:
2019-09-19 13:27:23 -07:00
This pull-request contains the following changes for MTD:
2019-09-21 10:59:54 -07:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
pinctrl/gpio: Take MUX usage into account
2019-08-23 11:09:41 +02:00
Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/evalenti/linux-soc-thermal
2019-09-29 10:24:23 -07:00
RDMA subsystem updates for 5.4
2019-09-21 10:26:24 -07:00
regulator: add missing 'static inline' to a helper's stub
2019-09-03 12:37:16 +01:00
Merge branch 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-09-28 12:39:07 -07:00
drm main pull for 5.4-rc1
2019-09-19 16:24:24 -07:00
soundwire: stream: make stream name a const pointer
2019-09-04 14:57:28 +05:30
Highlights:
2019-09-27 17:00:27 -07:00
USB changes for 5.4-rc1
2019-09-18 10:33:46 -07:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
kill LOOKUP_NO_EVAL, don't bother including namei.h from audit.h
2019-08-30 21:29:32 -04:00
writeback, memcg: Implement foreign dirty flushing
2019-08-27 09:22:38 -06:00
writeback: Separate out wb_get_lookup() from wb_get_create()
2019-08-27 09:22:38 -06:00
backlight: Expose brightness curve type through sysfs
2019-09-02 15:55:03 +01:00
lib: introduce copy_struct_from_user() helper
2019-10-01 15:45:03 +02:00
blkcg: separate blkcg_conf_get_disk() out of blkg_conf_prep()
2019-08-28 21:17:04 -06:00
block: Delay default elevator initialization
2019-09-05 19:52:34 -06:00
blkcg: implement blk-iocost
2019-08-28 21:17:12 -06:00
for-5.4/post-2019-09-24
2019-09-24 16:31:50 -07:00
bpf: introduce verifier internal test flag
2019-08-28 00:30:11 +02:00
kprobes: Prohibit probing on BUG() and WARN() address
2019-09-05 10:15:16 +02:00
Merge branches 'clk-init-destroy', 'clk-doc', 'clk-imx' and 'clk-allwinner' into clk-next
2019-09-19 15:31:27 -07:00
We have a small collection of core framework updates this time, mostly around
2019-09-20 15:45:07 -07:00
mm, compaction: raise compaction priority after it withdrawns
2019-09-24 15:54:10 -07:00
include/linux/compiler.h: fix Oops for Clang-compiled kernels
2019-09-08 14:53:58 +02:00
Make use of gcc 9's "asm inline()" (Rasmus Villemoes):
2019-09-21 09:47:19 -07:00
driver-core: add include guard to linux/container.h
2019-09-04 13:38:26 +02:00
cpu/SMT: create and export cpu_smt_possible()
2019-09-24 13:37:28 +02:00
cpufreq: Remove CPUFREQ_ADJUST and CPUFREQ_NOTIFY policy notifier events
2019-09-02 22:44:04 +02:00
Power management updates for 5.4-rc1
2019-09-17 19:15:14 -07:00
cpuidle-haltpoll: vcpu hotplug support
2019-09-03 09:36:36 +02:00
cpumask: nicer for_each_cpumask_and() signature
2019-09-25 17:51:40 -07:00
selinux/stable-5.4 PR 20190917
2019-09-23 11:21:04 -07:00
PM / devfreq: events: extend events by type of counted data
2019-08-25 12:44:38 +09:00
USB changes for 5.4-rc1
2019-09-18 10:33:46 -07:00
dma-mapping updates for 5.4:
2019-09-19 13:27:23 -07:00
dma-mapping: remove arch_dma_mmap_pgprot
2019-08-29 16:43:22 +02:00
EDAC: Prefer 'unsigned int' to bare use of 'unsigned'
2019-09-03 19:21:19 +02:00
block: Introduce elevator features
2019-09-05 19:52:33 -06:00
Modules updates for v5.4
2019-09-22 10:34:46 -07:00
f2fs: include charset encoding information in the superblock
2019-08-23 07:57:13 -07:00
bpf: fix accessing bpf_sysctl.file_pos on s390
2019-09-16 11:44:05 +02:00
Highlights:
2019-09-27 17:00:27 -07:00
fuse update for 5.4
2019-09-25 09:55:59 -07:00
device property: Remove duplicate test for NULL
2019-08-26 11:40:24 +02:00
Revert "Revert "Revert "mm, thp: consolidate THP gfp handling into alloc_hugepage_direct_gfpmask""
2019-09-28 14:05:38 -07:00
Linux 5.3-rc7
2019-09-05 11:40:54 +02:00
staging: greybus: move core include files to include/linux/greybus/
2019-08-27 19:02:59 +02:00
HID: core: Add printk_once variants to hid_warn() etc
2019-09-18 15:58:13 +02:00
hrtimer: Add kernel doc annotation for HRTIMER_MODE_HARD
2019-08-28 13:01:25 +02:00
libnvdimm fixes v5.4-rc1
2019-09-29 10:33:41 -07:00
mm: introduce page_size()
2019-09-24 15:54:08 -07:00
- First round of vmbus hibernation support from Dexuan Cui.
2019-09-24 12:36:31 -07:00
Merge branch 'i2c/for-5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux
2019-09-24 16:48:02 -07:00
powercap: idle_inject: Use higher resolution for idle injection
2019-09-03 11:33:29 +02:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
posix-cpu-timers: Create a container struct
2019-08-28 11:50:33 +02:00
Merge branches 'arm/omap', 'arm/exynos', 'arm/smmu', 'arm/mediatek', 'arm/qcom', 'arm/renesas', 'x86/amd', 'x86/vt-d' and 'core' into next
2019-09-11 12:39:19 +02:00
Power management updates for 5.4-rc1
2019-09-17 19:15:14 -07:00
augmented rbtree: add new RB_DECLARE_CALLBACKS_MAX macro
2019-09-25 17:51:39 -07:00
Merge branch 'arm/smmu' into arm/mediatek
2019-08-30 16:12:10 +02:00
iomap: move the iomap_dio_rw ->end_io callback into a structure
2019-09-19 15:32:45 -07:00
iommu: Add helpers to set/get default domain type
2019-08-23 10:09:58 +02:00
jbd2: remove jbd2_journal_inode_add_[write|wait]
2019-09-24 15:54:07 -07:00
kernel.h: Add non_block_start/end()
2019-09-07 04:28:05 -03:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
afs: Provide an RCU-capable key lookup
2019-09-02 11:43:54 +01:00
kgdb: don't use a notifier to enter kgdb at panic; call directly
2019-09-25 17:51:40 -07:00
khugepaged: enable collapse pmd for pte-mapped THP
2019-09-24 15:54:11 -07:00
kvm: x86, powerpc: do not allow clearing largepages debugfs entry
2019-09-30 18:52:00 +02:00
leds: Replace {devm_}led_classdev_register() macros with inlines
2019-09-01 13:33:16 +02:00
libnvdimm/security: Introduce a 'frozen' attribute
2019-08-29 13:49:13 -07:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
include: mdio: Add driver data helpers
2019-09-05 12:32:05 +02:00
mm, memcg: make scan aggression always exclude protection
2019-10-07 15:47:20 -07:00
drivers/base/memory.c: don't store end_section_nr in memory blocks
2019-09-24 15:54:09 -07:00
Revert "Revert "mm, thp: restore node-local hugepage allocations""
2019-09-28 14:05:38 -07:00
libnvdimm fixes v5.4-rc1
2019-09-29 10:33:41 -07:00
mm: treewide: clarify pgtable_page_{ctor,dtor}() naming
2019-09-26 10:10:44 -07:00
Merge branch 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-09-28 12:39:07 -07:00
fork: improve error message for corrupted page tables
2019-09-25 17:51:40 -07:00
mm/mmu_notifiers: annotate with might_sleep()
2019-09-07 04:28:04 -03:00
mm: thp: extract split_queue_* into a struct
2019-09-24 15:54:11 -07:00
Merge branch 'next-integrity' of git://git.kernel.org/pub/scm/linux/kernel/git/zohar/linux-integrity
2019-09-27 19:37:27 -07:00
fs/namei.c: keep track of nd->root refcount status
2019-09-03 09:30:45 -04:00
taprio: Add support for hardware offloading
2019-09-16 21:32:57 +02:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
netfilter: move nf_bridge_frag_data struct definition to a more appropriate header.
2019-09-13 12:35:33 +02:00
netfilter: move nf_bridge_frag_data struct definition to a more appropriate header.
2019-09-13 12:35:33 +02:00
NFS: Refactor nfs_instantiate() for dentry referencing callers
2019-09-20 15:15:24 -04:00
nvme-pci: Add support for variable IO SQ element size
2019-08-29 12:55:02 -07:00
gpio: of: Make of_gpio_simple_xlate() private
2019-09-11 14:46:02 +01:00
padata: remove cpu_index from the parallel_queue
2019-09-13 21:15:41 +10:00
mm, page_owner: keep owner info when freeing the page
2019-09-24 15:54:08 -07:00
mm: page cache: store only head pages in i_pages
2019-09-24 15:54:08 -07:00
pagewalk: separate function pointers from iterator data
2019-09-07 04:28:04 -03:00
pci-v5.4-changes
2019-09-23 19:16:01 -07:00
PCI/ACPI: Move _HPP & _HPX functions to pci-acpi.c
2019-08-28 15:25:53 -05:00
pci-v5.4-changes
2019-09-23 19:16:01 -07:00
perf: Allow normal events to output AUX data
2019-08-28 11:29:38 +02:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net
2019-09-02 11:20:17 -07:00
net: fixed_phy: Add forward declaration for struct gpio_desc;
2019-09-05 11:54:14 +02:00
USB changes for 5.4-rc1
2019-09-18 10:33:46 -07:00
posix-cpu-timers: Always clear head pointer on dequeue
2019-09-05 21:16:22 +02:00
lib/hexdump: make print_hex_dump_bytes() a nop on !DEBUG builds
2019-09-25 17:51:39 -07:00
software node: Add software_node_find_by_name()
2019-08-26 11:35:13 +02:00
pwm: Ensure pwm_apply_state() doesn't modify the state argument
2019-09-21 03:25:10 +02:00
quota: fix wrong condition in is_quota_modification()
2019-09-12 12:09:16 +02:00
vfs: Convert ramfs, shmem, tmpfs, devtmpfs, rootfs to use the new mount API
2019-09-12 21:05:34 -04:00
fdt: add support for rng-seed
2019-08-23 16:39:26 +01:00
augmented rbtree: rework the RB_DECLARE_CALLBACKS macro definition
2019-09-25 17:51:39 -07:00
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-09-16 17:25:49 -07:00
rcu: Don't include <linux/ktime.h> in rcutiny.h
2019-08-26 16:27:01 -07:00
tasks, sched/core: With a grace period after finish_task_switch(), remove unnecessary code
2019-09-25 17:42:29 +02:00
cifs: Add support for root file systems
2019-09-16 11:43:38 -05:00
Merge branch 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-09-28 12:39:07 -07:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
vfs: Convert ramfs, shmem, tmpfs, devtmpfs, rootfs to use the new mount API
2019-09-12 21:05:34 -04:00
mm: shrinker: make shrinker not depend on memcg kmem
2019-09-24 15:54:11 -07:00
sk_buff: drop all skb extensions on free and skb scrubbing
2019-09-27 20:40:19 +02:00
mm, slab: move memcg_cache_params structure to mm/slab.h
2019-09-24 15:54:07 -07:00
net: stmmac: implement support for passive mode converters via dt
2019-09-11 15:27:09 +01:00
kernel-doc: core-api: include string.h into core-api
2019-09-25 17:51:39 -07:00
mm: introduce MADV_PAGEOUT
2019-09-25 17:51:41 -07:00
swiotlb: Split size parameter to map/unmap APIs
2019-09-11 12:34:29 +02:00
ipc: fix sparc64 ipc() wrapper
2019-09-07 21:42:25 +02:00
sysfs: add BIN_ATTR_WO() macro
2019-09-04 13:37:17 +02:00
block: centralize PI remapping logic to the block layer
2019-09-17 20:03:49 -06:00
tcp: Add TCP_INFO counter for packets received out-of-order
2019-09-16 16:26:11 +02:00
uaccess: add missing __must_check attributes
2019-09-25 17:51:40 -07:00
thunderbolt: Add support for Intel Ice Lake
2019-08-26 12:15:06 +03:00
vfs: Add file timestamp range support
2019-08-30 07:27:17 -07:00
timekeeping/vsyscall: Prevent math overflow in BOOTTIME update
2019-08-23 02:12:11 +02:00
posix-cpu-timers: Utilize timerqueue for storage
2019-08-28 11:50:43 +02:00
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-09-16 17:25:49 -07:00
tracing: Make exported ftrace_set_clr_event non-static
2019-08-31 06:51:49 -04:00
lib: introduce copy_struct_from_user() helper
2019-10-01 15:45:03 +02:00
mm/vmalloc: modify struct vmap_area to reduce its size
2019-09-24 15:54:10 -07:00
w1: add 1-wire master driver for IP block found in SGI ASICs
2019-09-04 14:34:31 +02:00
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-09-17 12:35:15 -07:00
workqueue: unconfine alloc/apply/free_workqueue_attrs()
2019-09-13 21:15:39 +10:00
writeback, memcg: Implement cgroup_writeback_by_id()
2019-08-27 09:22:38 -06:00
zpool: add malloc_support_movable to zpool_driver
2019-09-24 15:54:12 -07:00
