Johannes Weiner
db2ba40c27
mm: memcontrol: make per-cpu charge cache IRQ-safe for socket accounting
...
During cgroup2 rollout into production, we started encountering css
refcount underflows and css access crashes in the memory controller.
Splitting the heavily shared css reference counter into logical users
narrowed the imbalance down to the cgroup2 socket memory accounting.
The problem turns out to be the per-cpu charge cache. Cgroup1 had a
separate socket counter, but the new cgroup2 socket accounting goes
through the common charge path that uses a shared per-cpu cache for all
memory that is being tracked. Those caches are safe against scheduling
preemption, but not against interrupts - such as the newly added packet
receive path. When cache draining is interrupted by network RX taking
pages out of the cache, the resuming drain operation will put references
of in-use pages, thus causing the imbalance.
Disable IRQs during all per-cpu charge cache operations.
Fixes: f7e1cb6ec5 ("mm: memcontrol: account socket memory in unified hierarchy memory controller")
Link: http://lkml.kernel.org/r/20160914194846.11153-1-hannes@cmpxchg.org
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Tejun Heo <tj@kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: <stable@vger.kernel.org> [4.5+]
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-09-19 15:36:17 -07:00
..
kasan
kasan: remove the unnecessary WARN_ONCE from quarantine.c
2016-08-11 16:58:14 -07:00
backing-dev.c
block: fix bdi vs gendisk lifetime mismatch
2016-08-04 14:19:16 -06:00
balloon_compaction.c
mm: balloon: use general non-lru movable page feature
2016-07-26 16:19:19 -07:00
bootmem.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
cleancache.c
cleancache: constify cleancache_ops structure
2016-01-27 09:09:57 -05:00
cma.c
mm/cma: silence warnings due to max() usage
2016-05-27 14:49:37 -07:00
cma.h
cma_debug.c
compaction.c
mm, compaction: simplify contended compaction handling
2016-07-28 16:07:41 -07:00
debug.c
mm: avoid endless recursion in dump_page()
2016-09-19 15:36:16 -07:00
debug_page_ref.c
mm/page_ref: add tracepoint to track down page reference manipulation
2016-03-17 15:09:34 -07:00
dmapool.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
early_ioremap.c
fadvise.c
mm/fadvise.c: do not discard partial pages with POSIX_FADV_DONTNEED
2016-06-09 14:23:11 -07:00
failslab.c
mm: fault-inject take over bootstrap kmem_cache check
2016-03-15 16:55:16 -07:00
filemap.c
block/mm: make bdev_ops->rw_page() take a bool for read/write
2016-08-07 14:41:02 -06:00
frame_vector.c
mm/gup: Switch all callers of get_user_pages() to not pass tsk/mm
2016-02-16 10:11:12 +01:00
frontswap.c
mm, frontswap: convert frontswap_enabled to static key
2016-07-26 16:19:19 -07:00
gup.c
- ARM: GICv3 ITS emulation and various fixes. Removal of the old
2016-08-02 16:11:27 -04:00
highmem.c
mm/highmem: make nr_free_highpages() handles all highmem zones by itself
2016-05-19 19:12:14 -07:00
huge_memory.c
mm: fix show_smap() for zone_device-pmd ranges
2016-09-09 17:34:45 -07:00
hugetlb.c
mm/hugetlb: fix incorrect hugepages count during mem hotplug
2016-08-11 16:58:13 -07:00
hugetlb_cgroup.c
mm, hugetlb_cgroup: round limit_in_bytes down to hugepage size
2016-05-20 17:58:30 -07:00
hwpoison-inject.c
init-mm.c
internal.h
mm, compaction: simplify contended compaction handling
2016-07-28 16:07:41 -07:00
interval_tree.c
Kconfig
mm: clarify COMPACTION Kconfig text
2016-08-26 17:39:35 -07:00
Kconfig.debug
mm/page_ref: add tracepoint to track down page reference manipulation
2016-03-17 15:09:34 -07:00
khugepaged.c
mm, thp: fix leaking mapped pte in __collapse_huge_page_swapin()
2016-09-19 15:36:16 -07:00
kmemcheck.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
kmemleak-test.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
kmemleak.c
kmemleak: don't hang if user disables scanning early
2016-07-28 16:07:41 -07:00
ksm.c
mm: do not pass mm_struct into handle_mm_fault
2016-07-26 16:19:19 -07:00
list_lru.c
mm: memcontrol: move kmem accounting code to CONFIG_MEMCG
2016-01-20 17:09:18 -08:00
maccess.c
x86: remove more uaccess_32.h complexity
2016-05-22 17:21:27 -07:00
madvise.c
mm: make mmap_sem for write waits killable for mm syscalls
2016-05-23 17:04:14 -07:00
Makefile
Implements HARDENED_USERCOPY verification of copy_to_user/copy_from_user
2016-08-08 14:48:14 -07:00
memblock.c
mm/memblock.c: fix NULL dereference error
2016-08-04 20:02:09 -04:00
memcontrol.c
mm: memcontrol: make per-cpu charge cache IRQ-safe for socket accounting
2016-09-19 15:36:17 -07:00
memory-failure.c
mm: hwpoison: remove incorrect comments
2016-07-28 16:07:41 -07:00
memory.c
mm: move swap-in anonymous page into active list
2016-08-02 17:31:41 -04:00
memory_hotplug.c
mem-hotplug: don't clear the only node in new_node_page()
2016-09-19 15:36:16 -07:00
mempolicy.c
mm, mempolicy: task->mempolicy must be NULL before dropping final reference
2016-09-01 17:52:01 -07:00
mempool.c
Revert "mm, mempool: only set __GFP_NOMEMALLOC if there are free elements"
2016-07-28 16:07:41 -07:00
memtest.c
migrate.c
mm, thp: remove __GFP_NORETRY from khugepaged and madvised allocations
2016-07-28 16:07:41 -07:00
mincore.c
mm, fs: remove remaining PAGE_CACHE_* and page_cache_{get,release} usage
2016-04-04 10:41:08 -07:00
mlock.c
mm, vmscan: move LRU lists to node
2016-07-28 16:07:41 -07:00
mm_init.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
mmap.c
mm: refuse wrapped vm_brk requests
2016-08-02 19:35:15 -04:00
mmu_context.c
mm/mmu_context, sched/core: Fix mmu_context.h assumption
2016-04-28 11:44:19 +02:00
mmu_notifier.c
fix Christoph's email addresses
2016-03-17 15:09:34 -07:00
mmzone.c
mm, page_alloc: inline the fast path of the zonelist iterator
2016-05-19 19:12:14 -07:00
mprotect.c
mm: thp: check pmd_trans_unstable() after split_huge_pmd()
2016-07-26 16:19:19 -07:00
mremap.c
mm: thp: check pmd_trans_unstable() after split_huge_pmd()
2016-07-26 16:19:19 -07:00
msync.c
nobootmem.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
nommu.c
mm: introduce fault_env
2016-07-26 16:19:19 -07:00
oom_kill.c
mm, oom: fix uninitialized ret in task_will_free_mem()
2016-08-11 16:58:14 -07:00
page-writeback.c
mm: remove reclaim and compaction retry approximations
2016-07-28 16:07:41 -07:00
page_alloc.c
mm, vmscan: only allocate and reclaim from zones with pages managed by the buddy allocator
2016-09-01 17:52:01 -07:00
page_counter.c
page_ext.c
mm: use early_pfn_to_nid in page_ext_init
2016-05-27 14:49:37 -07:00
page_idle.c
mm, vmscan: move lru_lock to the node
2016-07-28 16:07:41 -07:00
page_io.c
mm: fix the page_swap_info() BUG_ON check
2016-09-19 15:36:17 -07:00
page_isolation.c
mm/page_isolation: clean up confused code
2016-07-26 16:19:19 -07:00
page_owner.c
mm/page_owner: use stackdepot to store stacktrace
2016-07-26 16:19:19 -07:00
page_poison.c
mm: check the return value of lookup_page_ext for all call sites
2016-06-03 15:06:22 -07:00
pagewalk.c
thp: rename split_huge_page_pmd() to split_huge_pmd()
2016-01-15 17:56:32 -08:00
percpu-km.c
mm: percpu: use pr_fmt to prefix output
2016-03-17 15:09:34 -07:00
percpu-vm.c
percpu.c
percpu: fix synchronization between synchronous map extension and chunk destruction
2016-05-25 11:48:25 -04:00
pgtable-generic.c
mm/thp/migration: switch from flush_tlb_range to flush_pmd_tlb_range
2016-03-17 15:09:34 -07:00
process_vm_access.c
mm/gup: Introduce get_user_pages_remote()
2016-02-16 10:04:09 +01:00
quicklist.c
fix Christoph's email addresses
2016-03-17 15:09:34 -07:00
readahead.c
mm: silently skip readahead for DAX inodes
2016-08-26 17:39:35 -07:00
rmap.c
rmap: fix compound check logic in page_remove_file_rmap
2016-08-10 16:40:56 -07:00
shmem.c
thp: move shmem_huge_enabled() outside of SYSFS ifdef
2016-08-10 16:40:56 -07:00
slab.c
Implements HARDENED_USERCOPY verification of copy_to_user/copy_from_user
2016-08-08 14:48:14 -07:00
slab.h
mm, kasan: switch SLUB to stackdepot, enable memory quarantine for SLUB
2016-07-28 16:07:41 -07:00
slab_common.c
mm: charge/uncharge kmemcg from generic page allocator paths
2016-07-26 16:19:19 -07:00
slob.c
mm: slab: free kmem_cache_node after destroy sysfs file
2016-02-18 16:23:24 -08:00
slub.c
mm/slub.c: run free_partial() outside of the kmem_cache_node->list_lock
2016-08-10 16:40:56 -07:00
sparse-vmemmap.c
treewide: replace obsolete _refok by __ref
2016-08-02 17:31:41 -04:00
sparse.c
treewide: replace obsolete _refok by __ref
2016-08-02 17:31:41 -04:00
swap.c
mm, pagevec: release/reacquire lru_lock on pgdat change
2016-07-28 16:07:41 -07:00
swap_cgroup.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
swap_state.c
mm: move most file-based accounting to the node
2016-07-28 16:07:41 -07:00
swapfile.c
mm: fix the page_swap_info() BUG_ON check
2016-09-19 15:36:17 -07:00
truncate.c
truncate: handle file thp
2016-07-26 16:19:19 -07:00
usercopy.c
usercopy: remove page-spanning test for now
2016-09-07 11:33:26 -07:00
userfaultfd.c
mm, fs: get rid of PAGE_CACHE_* and page_cache_{get,release} macros
2016-04-04 10:41:08 -07:00
util.c
mm: move most file-based accounting to the node
2016-07-28 16:07:41 -07:00
vmacache.c
vmalloc.c
mm: charge/uncharge kmemcg from generic page allocator paths
2016-07-26 16:19:19 -07:00
vmpressure.c
mm/vmpressure.c: fix subtree pressure detection
2016-02-03 08:28:43 -08:00
vmscan.c
mm, vmscan: only allocate and reclaim from zones with pages managed by the buddy allocator
2016-09-01 17:52:01 -07:00
vmstat.c
mm: remove reclaim and compaction retry approximations
2016-07-28 16:07:41 -07:00
workingset.c
mm, workingset: make working set detection node-aware
2016-07-28 16:07:41 -07:00
z3fold.c
mm/z3fold.c: avoid modifying HEADLESS page and minor cleanup
2016-06-03 16:02:55 -07:00
zbud.c
mm/zbud.c: use list_last_entry() instead of list_tail_entry()
2016-01-15 11:40:52 -08:00
zpool.c
zsmalloc.c
zsmalloc: Delete an unnecessary check before the function call "iput"
2016-07-28 16:07:41 -07:00
zswap.c
mm/zswap: use workqueue to destroy pool
2016-05-20 17:58:30 -07:00