Mel Gorman
0aaa29a56e
mm, page_alloc: reserve pageblocks for high-order atomic allocations on demand ...
High-order watermark checking exists for two reasons -- kswapd high-order
awareness and protection for high-order atomic requests. Historically the
kernel depended on MIGRATE_RESERVE to preserve min_free_kbytes as
high-order free pages for as long as possible. This patch introduces
MIGRATE_HIGHATOMIC that reserves pageblocks for high-order atomic
allocations on demand and avoids using those blocks for order-0
allocations. This is more flexible and reliable than MIGRATE_RESERVE was.
A MIGRATE_HIGHORDER pageblock is created when an atomic high-order
allocation request steals a pageblock but limits the total number to 1% of
the zone. Callers that speculatively abuse atomic allocations for
long-lived high-order allocations to access the reserve will quickly fail.
Note that SLUB is currently not such an abuser as it reclaims at least
once. It is possible that the pageblock stolen has few suitable
high-order pages and will need to steal again in the near future but there
would need to be strong justification to search all pageblocks for an
ideal candidate.
The pageblocks are unreserved if an allocation fails after a direct
reclaim attempt.
The watermark checks account for the reserved pageblocks when the
allocation request is not a high-order atomic allocation.
The reserved pageblocks can not be used for order-0 allocations. This may
allow temporary wastage until a failed reclaim reassigns the pageblock.
This is deliberate as the intent of the reservation is to satisfy a
limited number of atomic high-order short-lived requests if the system
requires them.
The stutter benchmark was used to evaluate this but while it was running
there was a systemtap script that randomly allocated between 1 high-order
page and 12.5% of memory's worth of order-3 pages using GFP_ATOMIC. This
is much larger than the potential reserve and it does not attempt to be
realistic. It is intended to stress random high-order allocations from an
unknown source, show that there is a reduction in failures without
introducing an anomaly where atomic allocations are more reliable than
regular allocations. The amount of memory reserved varied throughout the
workload as reserves were created and reclaimed under memory pressure.
The allocation failures once the workload warmed up were as follows;
4.2-rc5-vanilla 70%
4.2-rc5-atomic-reserve 56%
The failure rate was also measured while building multiple kernels. The
failure rate was 14% but is 6% with this patch applied.
Overall, this is a small reduction but the reserves are small relative to
the number of allocation requests. In early versions of the patch, the
failure rate reduced by a much larger amount but that required much larger
reserves and perversely made atomic allocations seem more reliable than
regular allocations.
[yalin.wang2010@gmail.com: fix redundant check and a memory leak]
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Vitaly Wool <vitalywool@gmail.com>
Cc: Rik van Riel <riel@redhat.com>
Signed-off-by: yalin wang <yalin.wang2010@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-11-06 17:50:42 -08:00
ARM: 8446/1: amba: Remove unused callbacks for legacy system PM
2015-10-19 17:08:32 +01:00
dmaengine: hsu: remove platform data
2015-10-17 21:22:08 -07:00
powerpc/fsl: Move fsl_guts.h out of arch/powerpc
2015-10-21 18:05:50 -05:00
gpiolib: provide generic request/free implementations
2015-10-16 22:11:16 +02:00
Power management and ACPI updates for v4.4-rc1
2015-11-04 18:10:13 -08:00
- New Device Support
2015-11-06 10:23:50 -08:00
net/mlx4_core: Replace VF zero mac with random mac in mlx4_core
2015-10-14 19:14:44 -07:00
net/mlx5_core: Wait for FW readiness on startup
2015-10-14 19:14:43 -07:00
mmc: mmc: extend the mmc_send_tuning()
2015-10-27 10:38:41 +01:00
drivers/pinctrl: Add the concept of an "init" state
2015-10-27 11:24:23 +01:00
TTY/Serial driver patches for 4.4-rc1
2015-11-04 21:35:12 -08:00
qed: Add statistics support
2015-10-27 19:34:54 -07:00
regulator: introduce min_dropout_uV
2015-10-16 17:57:33 +01:00
Merge remote-tracking branches 'spi/topic/omap-100k', 'spi/topic/omap-uwire', 'spi/topic/owner', 'spi/topic/pxa' and 'spi/topic/pxa2xx' into spi-next
2015-11-04 11:02:12 +00:00
USB patches for 4.4-rc1
2015-11-04 21:26:27 -08:00
Power management and ACPI updates for v4.4-rc1
2015-11-04 18:10:13 -08:00
atomic: remove all traces of READ_ONCE_CTRL() and atomic*_read_ctrl()
2015-11-03 17:22:17 -08:00
audit: audit_string_contains_control can be boolean
2015-11-04 08:23:51 -05:00
Merge branch 'for-4.4' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
2015-11-05 14:51:32 -08:00
blkcg: fix incorrect read/write sync/async stat accounting
2015-10-27 15:04:56 +09:00
block: generic request_queue reference counting
2015-10-21 14:43:41 -06:00
Merge branch 'for-4.4/reservations' of git://git.kernel.dk/linux-block
2015-11-04 21:01:27 -08:00
bpf: add support for persistent maps/progs
2015-11-02 22:48:39 -05:00
cgroup: add cgroup_subsys->free() method and use it to fix pids controller
2015-10-15 16:41:53 -04:00
cgroup: keep zombies associated with their original cgroups
2015-10-15 16:41:53 -04:00
clk: Add clk_hw_is_enabled() for use by clk providers
2015-10-26 12:36:55 -07:00
mm: cma: fix incorrect type conversion for size during dma allocation
2015-10-23 17:55:10 +09:00
mm, compaction: distinguish contended status in tracepoints
2015-11-05 19:34:48 -08:00
include/linux/compiler-gcc.h: hide assume_aligned attribute from sparse
2015-11-05 19:34:48 -08:00
compiler.h: add support for function attribute assume_aligned
2015-11-05 19:34:48 -08:00
cpufreq: create cpu/cpufreq/policyX directories
2015-10-28 09:21:12 +01:00
mm, page_alloc: remove unnecessary taking of a seqlock when cpusets are disabled
2015-11-06 17:50:42 -08:00
debugfs: Add debugfs_create_ulong()
2015-10-18 10:14:39 -07:00
dm: refactor ioctl handling
2015-10-31 19:05:59 -04:00
mm: cma: fix incorrect type conversion for size during dma allocation
2015-10-23 17:55:10 +09:00
iommu: Implement common IOMMU ops for DMA mapping
2015-10-15 16:41:04 +02:00
iommu/vt-d: Implement deferred invalidate for SVM
2015-10-15 13:22:35 +01:00
driver core update for 4.4-rc1
2015-11-04 21:50:37 -08:00
efi: Use correct type for struct efi_memory_map::phys_map
2015-10-28 12:28:06 +01:00
extcon: Modify the id and name of external connector
2015-10-16 08:30:09 +09:00
fb.h: Provide alternate screen_base pointer
2015-10-13 10:51:30 -07:00
vfs: Fix pathological performance case for __alloc_fd()
2015-10-31 16:12:10 -07:00
Merge branch 'akpm' (patches from Andrew)
2015-11-05 23:10:54 -08:00
Power management and ACPI updates for v4.4-rc1
2015-11-04 18:10:13 -08:00
block: move blk_integrity to request_queue
2015-10-21 14:43:42 -06:00
mm, page_alloc: rename __GFP_WAIT to __GFP_RECLAIM
2015-11-06 17:50:42 -08:00
mm: hugetlb: proc: add HugetlbPages field to /proc/PID/status
2015-11-05 19:34:48 -08:00
drivers/hv: share Hyper-V SynIC constants with userspace
2015-11-04 16:24:33 +01:00
wireless: add WNM action frame categories
2015-10-13 10:34:46 +02:00
if_link: Add control trust VF
2015-10-23 05:44:28 -07:00
Merge branch 'for-4.4' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
2015-11-05 14:51:32 -08:00
iommu/vt-d: Implement SVM_FLAG_PRIVATE_PASID to allocate unique PASIDs
2015-10-15 15:35:32 +01:00
iommu/vt-d: Expose struct svm_dev_ops without CONFIG_INTEL_IOMMU_SVM
2015-10-27 08:36:08 +09:00
iommu-common: Fix error code used in iommu_tbl_range_{alloc,free}().
2015-11-04 11:30:57 -08:00
iommu: Add generic_device_group() function
2015-10-22 00:00:49 +02:00
ACPI/PCI: Enhance ACPI core to support sparse IO space
2015-10-16 22:18:51 +02:00
irqdomain: Documentation updates
2015-10-13 19:01:25 +02:00
Document that IRQ_NONE should be returned when IRQ not actually handled
2015-10-30 10:13:26 +01:00
Merge branch 'for-4.4' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
2015-11-05 14:51:32 -08:00
KEYS: Merge the type-specific data with the payload data
2015-10-21 15:18:36 +01:00
KEYS: Merge the type-specific data with the payload data
2015-10-21 15:18:36 +01:00
KVM: x86: move kvm_set_irq_inatomic to legacy device assignment
2015-11-04 16:24:36 +01:00
arcnet: com20020-pci: add led trigger support
2015-10-26 09:10:56 +01:00
lightnvm: refactor phys addrs type to u64
2015-11-03 09:53:24 -07:00
mm/memblock: make memblock_remove_range() static
2015-11-05 19:34:48 -08:00
Merge branch 'akpm' (patches from Andrew)
2015-11-05 23:10:54 -08:00
mm: memory hotplug with an existing resource
2015-10-23 14:19:58 +01:00
mm: introduce VM_LOCKONFAULT
2015-11-05 19:34:48 -08:00
mm: hugetlb: proc: add HugetlbPages field to /proc/PID/status
2015-11-05 19:34:48 -08:00
mm, page_alloc: reserve pageblocks for high-order atomic allocations on demand
2015-11-06 17:50:42 -08:00
lib/mpi: clean unused SHA1_DIGEST_LENGTH
2015-10-20 22:10:45 +08:00
PCI/MSI: Allow the MSI domain to be device-specific
2015-10-16 13:07:16 +01:00
tty: r3964: Replace/remove bogus tty lock use
2015-10-17 21:11:29 -07:00
net/core: fix for_each_netdev_feature
2015-11-03 11:29:57 -05:00
Merge branch 'parisc-4.3-2' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux
2015-11-04 11:30:22 -08:00
netfilter: turn NF_HOOK into an inline function
2015-10-16 18:45:36 +02:00
kernel/watchdog.c: perform all-CPU backtrace in case of hard lockup
2015-11-05 19:34:48 -08:00
gpiolib: Add and use OF_GPIO_SINGLE_ENDED flag
2015-10-16 22:49:26 +02:00
of/irq: Use the msi-map property to provide device-specific MSI domain
2015-10-16 13:07:15 +01:00
ARM: OMAP1: fix incorrect INT_DMA_LCD
2015-10-28 10:05:58 -07:00
mm: optimize PageHighMem() check
2015-11-05 19:34:48 -08:00
mm: page_counter: let page_counter_try_charge() return bool
2015-11-05 19:34:48 -08:00
PCI/ACPI: Add interface acpi_pci_root_create()
2015-10-16 22:18:51 +02:00
net: phy: Add nested variants of mdiobus read/write
2015-10-23 05:57:43 -07:00
PM / PCI / ACPI: Kick devices that might have been reset by firmware
2015-10-14 02:17:34 +02:00
PM / Domains: Merge measurements for PM QoS device latencies
2015-10-28 04:33:04 +01:00
block: add an API for Persistent Reservations
2015-10-21 14:46:56 -06:00
pstore: add pstore unregister
2015-10-22 08:59:18 -07:00
ptp: Change ptp_class to a proper bitmask
2015-11-03 11:08:22 -05:00
spi: pxa2xx: Add support for Intel Broxton
2015-10-30 11:18:05 +09:00
Merge remote-tracking branches 'regmap/topic/atomic', 'regmap/topic/debugfs' and 'regmap/topic/irq-hdr' into regmap-next
2015-10-12 19:25:07 +01:00
Merge branch 'akpm' (patches from Andrew)
2015-11-05 23:10:54 -08:00
seccomp, ptrace: add support for dumping seccomp filters
2015-10-27 19:55:13 -07:00
mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd
2015-11-06 17:50:42 -08:00
slab: convert slab_is_available() to boolean
2015-11-05 19:34:48 -08:00
stop_machine: Kill smp_hotplug_thread->pre_unpark, introduce stop_machine_unpark()
2015-10-20 10:23:55 +02:00
stop_machine: Kill smp_hotplug_thread->pre_unpark, introduce stop_machine_unpark()
2015-10-20 10:23:55 +02:00
PM / sleep: Add flags to indicate platform firmware involvement
2015-10-14 02:17:33 +02:00
mm: mlock: add new mlock system call
2015-11-05 19:34:48 -08:00
sysfs: added __compat_only_sysfs_link_entry_to_kobj()
2015-10-19 01:01:19 +02:00
block: Consolidate static integrity profile properties
2015-10-21 14:42:38 -06:00
tcp: fastopen: limit max_qlen
2015-10-22 06:22:13 -07:00
tpm: seal/unseal for TPM 2.0
2015-10-19 01:01:21 +02:00
memcg: punt high overage reclaim to return-to-userland path
2015-11-05 19:34:48 -08:00
tty: Abstract tty buffer work
2015-10-17 21:32:21 -07:00
rcu: force alignment on struct callback_head/rcu_head
2015-11-05 19:34:48 -08:00
uaccess: reimplement probe_kernel_address() using probe_kernel_read()
2015-11-05 19:34:48 -08:00
include/linux/vm_event_item.h: change HIGHMEM_ZONE macro definition
2015-11-05 19:34:48 -08:00
mm: get rid of 'vmalloc_info' from /proc/meminfo
2015-11-01 17:09:15 -08:00
vme: 8-bit status/id takes 256 values, not 255
2015-10-17 22:03:00 -07:00
mm: remove refresh_cpu_vm_stats() definition for !SMP kernel
2015-11-05 19:34:48 -08:00
