Roman Gushchin
9b6f7e163c
mm: rework memcg kernel stack accounting
...
If CONFIG_VMAP_STACK is set, kernel stacks are allocated using
__vmalloc_node_range() with __GFP_ACCOUNT. So kernel stack pages are
charged against corresponding memory cgroups on allocation and uncharged
on releasing them.
The problem is that we do cache kernel stacks in small per-cpu caches and
do reuse them for new tasks, which can belong to different memory cgroups.
Each stack page still holds a reference to the original cgroup, so the
cgroup can't be released until the vmap area is released.
To make this happen we need more than two subsequent exits without forks
in between on the current cpu, which makes it very unlikely to happen. As
a result, I saw a significant number of dying cgroups (in theory, up to 2
* number_of_cpu + number_of_tasks), which can't be released even by
significant memory pressure.
As a cgroup structure can take a significant amount of memory (first of
all, per-cpu data like memcg statistics), it leads to a noticeable waste
of memory.
Link: http://lkml.kernel.org/r/20180827162621.30187-1-guro@fb.com
Fixes: ac496bf48d ("fork: Optimize task creation by caching two thread stacks per CPU if CONFIG_VMAP_STACK=y")
Signed-off-by: Roman Gushchin <guro@fb.com>
Reviewed-by: Shakeel Butt <shakeelb@google.com>
Acked-by: Michal Hocko <mhocko@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: Tejun Heo <tj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-10-26 16:25:19 -07:00
..
bpf
Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
2018-10-21 21:11:46 -07:00
cgroup
for-4.20/block-20181021
2018-10-22 17:46:08 +01:00
configs
kconfig: tinyconfig: remove stale stack protector fixups
2018-06-15 07:15:28 +09:00
debug
treewide: kzalloc() -> kcalloc()
2018-06-12 16:19:22 -07:00
dma
dma-direct: respect DMA_ATTR_NO_WARN
2018-10-09 15:08:46 +02:00
events
Merge branch 'siginfo-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2018-10-24 11:22:39 +01:00
gcov
gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT
2018-06-08 18:56:02 +09:00
irq
Merge branch 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:43:47 -07:00
livepatch
Merge branch 'for-4.19/upstream' into for-linus
2018-08-20 18:33:50 +02:00
locking
Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-23 13:08:53 +01:00
power
Merge branches 'acpi-pm' and 'pm-sleep'
2018-10-18 12:27:30 +02:00
printk
printk: fix integer overflow in setup_log_buf()
2018-10-12 10:44:17 +02:00
rcu
Merge branches 'doc.2018.08.30a', 'dynticks.2018.08.30b', 'srcu.2018.08.30b' and 'torture.2018.08.29a' into HEAD
2018-08-30 16:12:53 -07:00
sched
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:14:36 -07:00
time
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:14:36 -07:00
trace
for-4.20/block-20181021
2018-10-22 17:46:08 +01:00
.gitignore
acct.c
async.c
audit.c
audit: use ktime_get_coarse_real_ts64() for timestamps
2018-07-17 14:45:08 -04:00
audit.h
audit: track the owner of the command mutex ourselves
2018-02-23 11:22:22 -05:00
audit_fsnotify.c
fsnotify: add fsnotify_add_inode_mark() wrappers
2018-05-18 14:58:22 +02:00
audit_tree.c
\n
2018-08-17 09:41:28 -07:00
audit_watch.c
audit: fix use-after-free in audit_add_watch
2018-07-18 11:43:36 -04:00
auditfilter.c
audit: rename FILTER_TYPE to FILTER_EXCLUDE
2018-06-19 10:39:54 -04:00
auditsc.c
audit/stable-4.18 PR 20180814
2018-08-15 10:46:54 -07:00
backtracetest.c
bounds.c
capability.c
compat.c
y2038: globally rename compat_time to old_time32
2018-08-27 14:48:48 +02:00
configs.c
context_tracking.c
cpu.c
Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-23 18:43:04 +01:00
cpu_pm.c
crash_core.c
kernel/crash_core.c: print timestamp using time64_t
2018-08-22 10:52:47 -07:00
crash_dump.c
cred.c
delayacct.c
delayacct: Use raw_spinlocks
2018-04-27 14:34:51 +02:00
dma.c
proc: introduce proc_create_single{,_data}
2018-05-16 07:23:35 +02:00
elfcore.c
exec_domain.c
proc: introduce proc_create_single{,_data}
2018-05-16 07:23:35 +02:00
exit.c
signal: Pass pid type into group_send_sig_info
2018-07-21 12:57:35 -05:00
extable.c
extable: Make init_kernel_text() global
2018-02-21 16:54:06 +01:00
fail_function.c
bpf/error-inject/kprobes: Clear current_kprobe and enable preempt in kprobe
2018-06-21 12:33:19 +02:00
fork.c
mm: rework memcg kernel stack accounting
2018-10-26 16:25:19 -07:00
freezer.c
PM / reboot: Eliminate race between reboot and suspend
2018-08-06 12:35:20 +02:00
futex.c
futex: Replace spin_is_locked() with lockdep
2018-10-09 13:19:28 +02:00
futex_compat.c
y2038: globally rename compat_time to old_time32
2018-08-27 14:48:48 +02:00
groups.c
hung_task.c
kernel/hung_task.c: allow to set checking interval separately from timeout
2018-08-22 10:52:47 -07:00
iomem.c
memremap: split devm_memremap_pages() and memremap() infrastructure
2018-05-15 23:08:33 -07:00
irq_work.c
jump_label.c
Merge branch 'x86/build' into locking/core, to pick up dependent patches and unify jump-label work
2018-10-16 17:30:11 +02:00
kallsyms.c
kallsyms, x86: Export addresses of PTI entry trampolines
2018-08-14 19:12:29 -03:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kconfig: include kernel/Kconfig.preempt from init/Kconfig
2018-08-02 08:06:54 +09:00
kcov.c
sched/core / kcov: avoid kcov_area during task switch
2018-06-15 07:55:24 +09:00
kexec.c
kexec: add call to LSM hook in original kexec_load syscall
2018-07-16 12:31:57 -07:00
kexec_core.c
kexec: Allocate decrypted control pages for kdump if SME is enabled
2018-10-06 12:01:51 +02:00
kexec_file.c
treewide: Use array_size() in vzalloc()
2018-06-12 16:19:22 -07:00
kexec_internal.h
kmod.c
kprobes.c
kprobes: Don't call BUG_ON() if there is a kprobe in use on free list
2018-09-12 08:01:16 +02:00
ksysfs.c
kthread.c
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-08-13 11:25:07 -07:00
latencytop.c
Makefile
kbuild: move bin2c back to scripts/ from scripts/basic/
2018-07-18 01:18:05 +09:00
memremap.c
libnvdimm-for-4.19_dax-memory-failure
2018-08-25 18:43:59 -07:00
module-internal.h
modsign: log module name in the event of an error
2018-07-02 11:36:17 +02:00
module.c
jump_table: Move entries into ro_after_init region
2018-09-27 17:56:49 +02:00
module_signing.c
modsign: log module name in the event of an error
2018-07-02 11:36:17 +02:00
notifier.c
nsproxy.c
padata.c
panic.c
Kbuild: rename CC_STACKPROTECTOR[_STRONG] config variables
2018-06-14 12:21:18 +09:00
params.c
kernel/params.c: downgrade warning for unsafe parameters
2018-04-11 10:28:37 -07:00
pid.c
fork: report pid exhaustion correctly
2018-09-20 22:01:11 +02:00
pid_namespace.c
signal: Use group_send_sig_info to kill all processes in a pid namespace
2018-09-16 16:08:25 +02:00
profile.c
ptrace.c
Merge branch 'siginfo-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2018-10-24 11:22:39 +01:00
range.c
reboot.c
kernel/reboot.c: export pm_power_off_prepare
2018-09-11 16:13:24 +01:00
relay.c
kernel/relay.c: change return type to vm_fault_t
2018-06-15 07:55:24 +09:00
resource.c
resource: Clean it up a bit
2018-10-09 17:25:58 +02:00
rseq.c
rseq: uapi: Declare rseq_cs field as union, update includes
2018-07-10 22:18:52 +02:00
seccomp.c
Merge branch 'next-general' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2018-10-24 11:49:35 +01:00
signal.c
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:14:36 -07:00
smp.c
smp,cpumask: introduce on_each_cpu_cond_mask
2018-10-09 16:51:11 +02:00
smpboot.c
smpboot: Remove cpumask from the API
2018-07-03 09:20:44 +02:00
smpboot.h
softirq.c
Merge branch 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-10-25 11:43:47 -07:00
stacktrace.c
stop_machine.c
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-08-13 11:25:07 -07:00
sys.c
kernel/sys.c: remove duplicated include
2018-09-20 22:01:11 +02:00
sys_ni.c
Merge branch 'core-rseq-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2018-06-10 10:17:09 -07:00
sysctl.c
namei: allow restricted O_CREAT of FIFOs and regular files
2018-08-23 18:48:43 -07:00
sysctl_binary.c
staging: irda: remove remaining remants of irda code removal
2018-04-16 11:26:49 +02:00
task_work.c
taskstats.c
test_kprobes.c
kprobes: Remove jprobe API implementation
2018-06-21 12:33:05 +02:00
torture.c
rcutorture: Check GP completion at stutter end
2018-08-29 09:20:48 -07:00
tracepoint.c
tracepoint: Fix tracepoint array element size mismatch
2018-10-17 15:35:29 -04:00
tsacct.c
ucount.c
headers: untangle kmemleak.h from mm.h
2018-04-05 21:36:27 -07:00
uid16.c
fs: add do_fchownat(), ksys_fchown() helpers and ksys_{,l}chown() wrappers
2018-04-02 20:15:59 +02:00
uid16.h
kernel: provide ksys_*() wrappers for syscalls called by kernel/uid16.c
2018-04-02 20:15:30 +02:00
umh.c
umh: Add command line to user mode helpers
2018-10-22 19:37:36 -07:00
up.c
smp,cpumask: introduce on_each_cpu_cond_mask
2018-10-09 16:51:11 +02:00
user-return-notifier.c
user.c
userns: use irqsave variant of refcount_dec_and_lock()
2018-08-22 10:52:47 -07:00
user_namespace.c
userns: move user access out of the mutex
2018-08-11 02:05:53 -05:00
utsname.c
uts: create "struct uts_namespace" from kmem_cache
2018-04-11 10:28:35 -07:00
utsname_sysctl.c
sys: don't hold uts_sem while accessing userspace memory
2018-08-11 02:05:53 -05:00
watchdog.c
watchdog: Mark watchdog touch functions as notrace
2018-08-30 12:56:40 +02:00
watchdog_hld.c
watchdog: Mark watchdog touch functions as notrace
2018-08-30 12:56:40 +02:00
workqueue.c
watchdog: Mark watchdog touch functions as notrace
2018-08-30 12:56:40 +02:00
workqueue_internal.h
workqueue: Set worker->desc to workqueue name by default
2018-05-18 08:47:13 -07:00