Michal Hocko
864b9a393d
mm: consider memblock reservations for deferred memory initialization sizing
...
We have seen an early OOM killer invocation on ppc64 systems with
crashkernel=4096M:
kthreadd invoked oom-killer: gfp_mask=0x16040c0(GFP_KERNEL|__GFP_COMP|__GFP_NOTRACK), nodemask=7, order=0, oom_score_adj=0
kthreadd cpuset=/ mems_allowed=7
CPU: 0 PID: 2 Comm: kthreadd Not tainted 4.4.68-1.gd7fe927-default #1
Call Trace:
dump_stack+0xb0/0xf0 (unreliable)
dump_header+0xb0/0x258
out_of_memory+0x5f0/0x640
__alloc_pages_nodemask+0xa8c/0xc80
kmem_getpages+0x84/0x1a0
fallback_alloc+0x2a4/0x320
kmem_cache_alloc_node+0xc0/0x2e0
copy_process.isra.25+0x260/0x1b30
_do_fork+0x94/0x470
kernel_thread+0x48/0x60
kthreadd+0x264/0x330
ret_from_kernel_thread+0x5c/0xa4
Mem-Info:
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:0 dirty:0 writeback:0 unstable:0
slab_reclaimable:5 slab_unreclaimable:73
mapped:0 shmem:0 pagetables:0 bounce:0
free:0 free_pcp:0 free_cma:0
Node 7 DMA free:0kB min:0kB low:0kB high:0kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:52428800kB managed:110016kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:320kB slab_unreclaimable:4672kB kernel_stack:1152kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 7 DMA: 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 0kB
0 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap = 0kB
Total swap = 0kB
819200 pages RAM
0 pages HighMem/MovableOnly
817481 pages reserved
0 pages cma reserved
0 pages hwpoisoned
the reason is that the managed memory is too low (only 110MB) while the
rest of the the 50GB is still waiting for the deferred intialization to
be done. update_defer_init estimates the initial memoty to initialize
to 2GB at least but it doesn't consider any memory allocated in that
range. In this particular case we've had
Reserving 4096MB of memory at 128MB for crashkernel (System RAM: 51200MB)
so the low 2GB is mostly depleted.
Fix this by considering memblock allocations in the initial static
initialization estimation. Move the max_initialise to
reset_deferred_meminit and implement a simple memblock_reserved_memory
helper which iterates all reserved blocks and sums the size of all that
start below the given address. The cumulative size is than added on top
of the initial estimation. This is still not ideal because
reset_deferred_meminit doesn't consider holes and so reservation might
be above the initial estimation whihch we ignore but let's make the
logic simpler until we really need to handle more complicated cases.
Fixes: 3a80a7fa79 ("mm: meminit: initialise a subset of struct pages if CONFIG_DEFERRED_STRUCT_PAGE_INIT is set")
Link: http://lkml.kernel.org/r/20170531104010.GI27783@dhcp22.suse.cz
Signed-off-by: Michal Hocko <mhocko@suse.com>
Acked-by: Mel Gorman <mgorman@suse.de>
Tested-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: <stable@vger.kernel.org> [4.2+]
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-06-02 15:07:38 -07:00
..
kasan
Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2017-05-10 10:30:46 -07:00
backing-dev.c
bdi: Drop 'parent' argument from bdi_register[_va]()
2017-04-20 12:09:55 -06:00
balloon_compaction.c
mm: balloon: use general non-lru movable page feature
2016-07-26 16:19:19 -07:00
bootmem.c
mm/bootmem.c: cosmetic improvement of code readability
2017-02-22 16:41:29 -08:00
cleancache.c
cma.c
cma: Introduce cma_for_each_area
2017-04-18 20:41:12 +02:00
cma.h
cma: Store a name in the cma structure
2017-04-18 20:41:12 +02:00
cma_debug.c
cma: Store a name in the cma structure
2017-04-18 20:41:12 +02:00
compaction.c
mm, compaction: finish whole pageblock to reduce fragmentation
2017-05-08 17:15:10 -07:00
debug.c
mm, debug: print raw struct page data in __dump_page()
2016-12-12 18:55:08 -08:00
debug_page_ref.c
mm/page_ref: add tracepoint to track down page reference manipulation
2016-03-17 15:09:34 -07:00
dmapool.c
lib/vsprintf.c: remove %Z support
2017-02-27 18:43:47 -08:00
early_ioremap.c
fadvise.c
mm: fadvise: avoid expensive remote LRU cache draining after FADV_DONTNEED
2016-12-20 09:48:46 -08:00
failslab.c
mm: fault-inject take over bootstrap kmem_cache check
2016-03-15 16:55:16 -07:00
filemap.c
Merge branch 'work.iov_iter' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2017-05-09 09:01:21 -07:00
frame_vector.c
treewide: use kv[mz]alloc* rather than opencoded variants
2017-05-08 17:15:13 -07:00
frontswap.c
mm, frontswap: convert frontswap_enabled to static key
2016-07-26 16:19:19 -07:00
gup.c
mm/hugetlb: report -EHWPOISON not -EFAULT when FOLL_HWPOISON is specified
2017-06-02 15:07:38 -07:00
highmem.c
mm/highmem: make nr_free_highpages() handles all highmem zones by itself
2016-05-19 19:12:14 -07:00
huge_memory.c
mm/huge_memory.c: deposit a pgtable for DAX PMD faults when required
2017-05-08 17:15:15 -07:00
hugetlb.c
mm/hugetlb: report -EHWPOISON not -EFAULT when FOLL_HWPOISON is specified
2017-06-02 15:07:38 -07:00
hugetlb_cgroup.c
mm, hugetlb_cgroup: round limit_in_bytes down to hugepage size
2016-05-20 17:58:30 -07:00
hwpoison-inject.c
mm: hwpoison: call shake_page() unconditionally
2017-05-03 15:52:12 -07:00
init-mm.c
mm: Add a user_ns owner to mm_struct and fix ptrace permission checks
2016-11-22 11:49:48 -06:00
internal.h
mm, compaction: finish whole pageblock to reduce fragmentation
2017-05-08 17:15:10 -07:00
interval_tree.c
Kconfig
mm: remove AVR32 arch special handling in mm/Kconfig
2017-05-01 09:36:31 +02:00
Kconfig.debug
mm: enable page poisoning early at boot
2017-05-03 15:52:10 -07:00
khugepaged.c
mm, thp: copying user pages must schedule on collapse
2017-05-12 15:57:16 -07:00
kmemcheck.c
mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU
2017-04-18 11:42:36 -07:00
kmemleak-test.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
kmemleak.c
mm: fix section name for .data..ro_after_init
2017-03-31 17:13:30 -07:00
ksm.c
ksm: prevent crash after write_protect_page fails
2017-06-02 15:07:37 -07:00
list_lru.c
mm/list_lru.c: avoid error-path NULL pointer deref
2016-10-27 18:43:42 -07:00
maccess.c
x86: remove more uaccess_32.h complexity
2016-05-22 17:21:27 -07:00
madvise.c
mm/madvise: move up the behavior parameter validation
2017-05-03 15:52:11 -07:00
Makefile
mm: add arch-independent testcases for RODATA
2017-02-27 18:43:48 -08:00
memblock.c
mm: consider memblock reservations for deferred memory initialization sizing
2017-06-02 15:07:38 -07:00
memcontrol.c
hwpoison, memcg: forcibly uncharge LRU pages
2017-05-12 15:57:15 -07:00
memory-failure.c
mm/migrate: fix refcount handling when !hugepage_migration_supported()
2017-06-02 15:07:38 -07:00
memory.c
mm: avoid spurious 'bad pmd' warning messages
2017-06-02 15:07:37 -07:00
memory_hotplug.c
mm, vmscan: prevent kswapd sleeping prematurely due to mismatched classzone_idx
2017-05-03 15:52:09 -07:00
mempolicy.c
mm/mempolicy.c: fix error handling in set_mempolicy and mbind.
2017-04-08 10:57:55 -07:00
mempool.c
Revert "mm, mempool: only set __GFP_NOMEMALLOC if there are free elements"
2016-07-28 16:07:41 -07:00
memtest.c
migrate.c
mm: make rmap_one boolean function
2017-05-03 15:52:10 -07:00
mincore.c
mm: remove shmem_mapping() shmem_zero_setup() duplicates
2017-02-24 17:46:56 -08:00
mlock.c
mlock: fix mlock count can not decrease in race condition
2017-06-02 15:07:38 -07:00
mm_init.c
mm: convert printk(KERN_<LEVEL> to pr_<level>
2016-03-17 15:09:34 -07:00
mmap.c
mm/mmap: replace SHM_HUGE_MASK with MAP_HUGE_MASK inside mmap_pgoff
2017-05-03 15:52:10 -07:00
mmu_context.c
sched/headers: Prepare to move the task_lock()/unlock() APIs to <linux/sched/task.h>
2017-03-02 08:42:38 +01:00
mmu_notifier.c
mm: Use static initialization for "srcu"
2017-04-18 11:38:22 -07:00
mmzone.c
mm/mmzone.c: swap likely to unlikely as code logic is different for next_zones_zonelist()
2017-02-22 16:41:29 -08:00
mprotect.c
mm: convert generic code to 5-level paging
2017-03-09 11:48:47 -08:00
mremap.c
mm: convert generic code to 5-level paging
2017-03-09 11:48:47 -08:00
msync.c
nobootmem.c
mm: kmemleak: avoid using __va() on addresses that don't have a lowmem mapping
2016-10-11 15:06:33 -07:00
nommu.c
mm, vmalloc: use __GFP_HIGHMEM implicitly
2017-05-08 17:15:13 -07:00
oom_kill.c
oom: improve oom disable handling
2017-05-03 15:52:10 -07:00
page-writeback.c
Add GETFSMAP support; some performance improvements for very large
2017-05-08 11:30:05 -07:00
page_alloc.c
mm: consider memblock reservations for deferred memory initialization sizing
2017-06-02 15:07:38 -07:00
page_counter.c
page_ext.c
mm: enable page poisoning early at boot
2017-05-03 15:52:10 -07:00
page_idle.c
mm: make rmap_one boolean function
2017-05-03 15:52:10 -07:00
page_io.c
writeback: add wbc_to_write_flags()
2016-11-02 10:24:03 -06:00
page_isolation.c
mm, page_alloc: count movable pages when stealing from pageblock
2017-05-08 17:15:10 -07:00
page_owner.c
mm/page_owner: don't define fields on struct page_ext by hard-coding
2016-10-07 18:46:27 -07:00
page_poison.c
mm: enable page poisoning early at boot
2017-05-03 15:52:10 -07:00
page_vma_mapped.c
mm: fix page_vma_mapped_walk() for ksm pages
2017-04-08 00:47:48 -07:00
pagewalk.c
mm: convert generic code to 5-level paging
2017-03-09 11:48:47 -08:00
percpu-km.c
mm: percpu: use pr_fmt to prefix output
2016-03-17 15:09:34 -07:00
percpu-vm.c
percpu: remove unused chunk_alloc parameter from pcpu_get_pages()
2017-03-06 15:56:55 -05:00
percpu.c
Merge branch 'sched/core' into locking/core
2017-04-04 11:31:12 +02:00
pgtable-generic.c
mm: convert generic code to 5-level paging
2017-03-09 11:48:47 -08:00
process_vm_access.c
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/mm.h>
2017-03-02 08:42:28 +01:00
quicklist.c
fix Christoph's email addresses
2016-03-17 15:09:34 -07:00
readahead.c
mm: don't cap request size based on read-ahead setting
2016-12-12 18:55:08 -08:00
rmap.c
Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2017-05-10 10:30:46 -07:00
rodata_test.c
mm: remove rodata_test_data export, add pr_fmt
2017-05-03 15:52:09 -07:00
shmem.c
Merge branch 'rebased-statx' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2017-03-03 11:38:56 -08:00
slab.c
Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2017-05-10 10:30:46 -07:00
slab.h
mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU
2017-04-18 11:42:36 -07:00
slab_common.c
mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU
2017-04-18 11:42:36 -07:00
slob.c
mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU
2017-04-18 11:42:36 -07:00
slub.c
slub/memcg: cure the brainless abuse of sysfs attributes
2017-06-02 15:07:37 -07:00
sparse-vmemmap.c
mm: convert generic code to 5-level paging
2017-03-09 11:48:47 -08:00
sparse.c
mm/sparse: refine usemap_size() a little
2017-05-03 15:52:09 -07:00
swap.c
mm: move MADV_FREE pages into LRU_INACTIVE_FILE list
2017-05-03 15:52:08 -07:00
swap_cgroup.c
mm, swap_cgroup: reschedule when neeed in swap_cgroup_swapoff()
2017-04-08 00:47:49 -07:00
swap_slots.c
mm, swap: use kvzalloc to allocate some swap data structures
2017-05-08 17:15:13 -07:00
swap_state.c
mm, swap: use kvzalloc to allocate some swap data structures
2017-05-08 17:15:13 -07:00
swapfile.c
mm, swap: use kvzalloc to allocate some swap data structures
2017-05-08 17:15:13 -07:00
truncate.c
mm: fix data corruption due to stale mmap reads
2017-05-12 15:57:15 -07:00
usercopy.c
mm/usercopy: Drop extra is_vmalloc_or_module() check
2017-04-05 12:30:18 -07:00
userfaultfd.c
mm: convert generic code to 5-level paging
2017-03-09 11:48:47 -08:00
util.c
mm: clarify why we want kmalloc before falling backto vmallock
2017-06-02 15:07:37 -07:00
vmacache.c
sched/headers: Prepare to move 'init_task' and 'init_thread_union' from <linux/sched.h> to <linux/sched/task.h>
2017-03-02 08:42:38 +01:00
vmalloc.c
mm, vmalloc: fix vmalloc users tracking properly
2017-05-12 15:57:15 -07:00
vmpressure.c
mm: vmpressure: fix sending wrong events on underflow
2017-02-24 17:46:56 -08:00
vmscan.c
mm: vmscan: scan until it finds eligible pages
2017-05-12 15:57:16 -07:00
vmstat.c
mm, vmstat: Remove spurious WARN() during zoneinfo print
2017-05-12 15:57:15 -07:00
workingset.c
mm: memcontrol: use node page state naming scheme for memcg
2017-05-03 15:52:11 -07:00
z3fold.c
z3fold: fix page locking in z3fold_alloc()
2017-04-13 18:24:20 -07:00
zbud.c
zpool.c
zsmalloc.c
zsmalloc: expand class bit
2017-04-13 18:24:21 -07:00
zswap.c
zswap: don't param_set_charp while holding spinlock
2017-02-27 18:43:45 -08:00