Johannes Weiner
81c0a2bb51
mm: page_alloc: fair zone allocator policy ...
Each zone that holds userspace pages of one workload must be aged at a
speed proportional to the zone size. Otherwise, the time an individual
page gets to stay in memory depends on the zone it happened to be
allocated in. Asymmetry in the zone aging creates rather unpredictable
aging behavior and results in the wrong pages being reclaimed, activated
etc.
But exactly this happens right now because of the way the page allocator
and kswapd interact. The page allocator uses per-node lists of all zones
in the system, ordered by preference, when allocating a new page. When
the first iteration does not yield any results, kswapd is woken up and the
allocator retries. Due to the way kswapd reclaims zones below the high
watermark while a zone can be allocated from when it is above the low
watermark, the allocator may keep kswapd running while kswapd reclaim
ensures that the page allocator can keep allocating from the first zone in
the zonelist for extended periods of time. Meanwhile the other zones
rarely see new allocations and thus get aged much slower in comparison.
The result is that the occasional page placed in lower zones gets
relatively more time in memory, even gets promoted to the active list
after its peers have long been evicted. Meanwhile, the bulk of the
working set may be thrashing on the preferred zone even though there may
be significant amounts of memory available in the lower zones.
Even the most basic test -- repeatedly reading a file slightly bigger than
memory -- shows how broken the zone aging is. In this scenario, no single
page should be able stay in memory long enough to get referenced twice and
activated, but activation happens in spades:
$ grep active_file /proc/zoneinfo
nr_inactive_file 0
nr_active_file 0
nr_inactive_file 0
nr_active_file 8
nr_inactive_file 1582
nr_active_file 11994
$ cat data data data data >/dev/null
$ grep active_file /proc/zoneinfo
nr_inactive_file 0
nr_active_file 70
nr_inactive_file 258753
nr_active_file 443214
nr_inactive_file 149793
nr_active_file 12021
Fix this with a very simple round robin allocator. Each zone is allowed a
batch of allocations that is proportional to the zone's size, after which
it is treated as full. The batch counters are reset when all zones have
been tried and the allocator enters the slowpath and kicks off kswapd
reclaim. Allocation and reclaim is now fairly spread out to all
available/allowable zones:
$ grep active_file /proc/zoneinfo
nr_inactive_file 0
nr_active_file 0
nr_inactive_file 174
nr_active_file 4865
nr_inactive_file 53
nr_active_file 860
$ cat data data data data >/dev/null
$ grep active_file /proc/zoneinfo
nr_inactive_file 0
nr_active_file 0
nr_inactive_file 666622
nr_active_file 4988
nr_inactive_file 190969
nr_active_file 937
When zone_reclaim_mode is enabled, allocations will now spread out to all
zones on the local node, not just the first preferred zone (which on a 4G
node might be a tiny Normal zone).
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Mel Gorman <mgorman@suse.de>
Reviewed-by: Rik van Riel <riel@redhat.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Paul Bolle <paul.bollee@gmail.com>
Cc: Zlatko Calusic <zcalusic@bitsync.net>
Tested-by: Kevin Hilman <khilman@linaro.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-09-11 15:57:23 -07:00
dmaengine: PL08x: Add support for PL080S variant
2013-09-02 11:49:56 +05:30
bcma: add bcma_core_pci_power_save()
2013-08-26 14:09:03 -04:00
can: mcp251x: Eliminate irq_flags from driver platform_data
2013-08-21 09:28:29 +02:00
fsl /bestcomm
dma: mxs-dma: remove code left from generic DMA binding conversion
2013-08-19 14:20:21 +05:30
ARM: SoC cleanups for 3.12
2013-09-06 13:21:16 -07:00
iio: trigger: implement devm_iio_trigger_alloc/devm_iio_triger_free
2013-08-17 19:34:52 +01:00
ARM: SoC late changes for v3.12
2013-09-09 16:35:29 -07:00
MMC highlights for 3.12:
2013-09-10 13:33:09 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2013-09-05 14:54:29 -07:00
net/mlx5_core: Support MANAGE_PAGES and QUERY_PAGES firmware command changes
2013-08-15 15:42:57 -07:00
mmc: sdhci: get voltage from sdhc host
2013-08-26 11:32:07 -04:00
mtd: add a new ecc_step_size field to mtd_info{}
2013-08-30 21:37:01 +01:00
pinctrl: Pass all configs to driver on pin_config_set()
2013-08-28 13:34:41 +02:00
Merge branch 'for-linus' of git://git.infradead.org/users/vkoul/slave-dma
2013-09-10 13:37:36 -07:00
power: Add twl4030_madc battery driver.
2013-08-30 17:34:52 -07:00
md update for v3.12
2013-09-10 13:03:41 -07:00
Merge remote-tracking branch 'regulator/topic/ramp' into regulator-next
2013-09-01 13:50:20 +01:00
MMC highlights for 3.12:
2013-09-10 13:33:09 -07:00
Merge branch 'nfsd-next' of git://linux-nfs.org/~bfields/linux
2013-09-10 20:04:59 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2013-09-05 14:54:29 -07:00
libata: Add support for SEND/RECEIVE FPDMA QUEUED
2013-08-25 09:39:12 -04:00
direct-io: Implement generic deferred AIO completions
2013-09-04 09:23:46 -04:00
userns: Kill nsown_capable it makes the wrong thing easy
2013-08-30 23:44:11 -07:00
cgroup: implement CFTYPE_NO_PREFIX
2013-08-26 18:40:56 -04:00
clk: add support for clock reparent on set_rate
2013-08-19 12:27:17 -07:00
clk: wrap I/O access for improved portability
2013-08-27 17:50:38 -07:00
block: support embedded device command line partition
2013-09-11 15:56:57 -07:00
Introduce [compat_]save_altstack_ex() to unbreak x86 SMAP
2013-09-01 14:16:33 -07:00
Merge branch 'pm-cpufreq'
2013-08-27 01:44:40 +02:00
Reinstate "crypto: crct10dif - Wrap crc_t10dif function all to use crypto transform framework"
2013-09-07 12:56:26 +10:00
vfs: reorganize dput() memory accesses
2013-09-08 13:26:18 -07:00
debugfs: provide debugfs_create_x64() when disabled
2013-08-27 22:09:12 -07:00
debugobjects: Make debug_object_activate() return status
2013-08-18 17:39:55 -07:00
dm: add statistics support
2013-09-05 20:46:06 -04:00
Merge branch 'for-v3.12' of git://git.linaro.org/people/mszyprowski/linux-dma-mapping
2013-09-09 10:26:33 -07:00
drivers: dma-contiguous: clean source code and prepare for device tree
2013-08-27 09:18:29 +02:00
drivers:net: Convert dma_alloc_coherent(...__GFP_ZERO) to dma_zalloc_coherent
2013-08-29 21:55:23 -04:00
Merge branch 'for-linus' of git://git.infradead.org/users/vkoul/slave-dma
2013-09-10 13:37:36 -07:00
net: etherdevice: add address inherit helper
2013-09-03 20:57:49 -04:00
Kill indirect include of file.h from eventfd.h, use fdget() in cgroup.c
2013-09-07 19:54:57 -04:00
f2fs: support the inline xattrs
2013-08-26 20:15:23 +09:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2013-09-07 14:35:32 -07:00
fs_enet: cleanup clock API use
2013-08-22 22:13:54 -07:00
FS-Cache: Add interface to check consistency of a cached object
2013-09-06 09:17:30 +01:00
fscache: Netfs function for cleanup post readpages
2013-09-06 09:17:30 +01:00
Merge branch 'rcu/next' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcu
2013-09-03 07:41:11 +02:00
video/hdmi: Rename HDMI_IDENTIFIER to HDMI_IEEE_OUI
2013-08-30 08:42:01 +10:00
HID: hid-sensor-hub: fix style of comments
2013-08-26 13:46:11 +02:00
HID: hid-sensor-hub: fix style of comments
2013-08-26 13:46:11 +02:00
Merge branches 'for-3.12/devm', 'for-3.12/i2c-hid', 'for-3.12/i2c-hid-dt', 'for-3.12/logitech', 'for-3.12/multitouch-win8', 'for-3.12/trasnport-driver-cleanup', 'for-3.12/uhid', 'for-3.12/upstream' and 'for-3.12/wiimote' into for-linus
2013-09-06 11:58:37 +02:00
HID: hidraw: Add spinlock in struct hidraw to protect list
2013-08-26 21:40:24 +02:00
i2c: move ACPI helpers into the core
2013-08-23 10:22:29 +02:00
Input: add driver for slidebar on Lenovo IdeaPad laptops
2013-08-26 09:30:03 -07:00
Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-next into for-davem
2013-08-29 14:08:24 -04:00
ipv4: expose IPV4_DEVCONF
2013-08-22 20:30:15 -07:00
ipv6: drop fragmented ndisc packets by default (RFC 6980)
2013-08-29 15:32:08 -04:00
jiffies: Avoid undefined behavior from signed overflow
2013-08-20 11:46:54 -07:00
sysfs: Restrict mounting sysfs
2013-08-28 21:35:14 -07:00
libata: Add support for queued DSM TRIM
2013-08-25 09:39:12 -04:00
lockref: add ability to mark lockrefs "dead"
2013-09-07 15:49:18 -07:00
math64: New separate div64_u64_rem helper
2013-08-23 09:02:14 -04:00
drivers: base: use standard device online/offline for state change
2013-08-21 11:52:20 -07:00
mm: mempolicy: turn vma_set_policy() into vma_dup_policy()
2013-09-11 15:57:00 -07:00
phy: micrel: Add definitions for common Micrel PHY registers
2013-08-22 23:20:54 +08:00
x86 get_unmapped_area: Access mmap_legacy_base through mm_struct member
2013-08-22 10:19:35 -07:00
mm: page_alloc: fair zone allocator policy
2013-09-11 15:57:23 -07:00
Revert "Input: introduce BTN/ABS bits for drums and guitars"
2013-09-07 09:48:41 -07:00
module: Fix mod->mkobj.kobj potentially freed too early
2013-09-03 16:35:47 +09:30
module: Add flag to allow mod params to have no arguments
2013-08-20 15:37:42 +09:30
introduce kern_path_mountpoint()
2013-09-08 20:20:23 -04:00
net: add documentation for BQL helpers
2013-09-06 14:43:49 -04:00
netfilter: nf_conntrack: make sequence number adjustments usuable without NAT
2013-08-28 00:26:48 +02:00
NFS: Ensure that rmdir() waits for sillyrenames to complete
2013-09-03 15:26:29 -04:00
NFSv4.1 Use MDS auth flavor for data server connection
2013-09-06 14:49:16 -04:00
nfs4.1: Minimal SP4_MACH_CRED implementation
2013-09-05 10:40:45 -04:00
Rename nsproxy.pid_ns to nsproxy.pid_ns_for_children
2013-08-27 13:52:52 -04:00
NVMe: Use normal shutdown
2013-09-03 16:40:32 -04:00
Device tree core updates for v3.12
2013-09-10 13:53:52 -07:00
of/device: add helper to get cpu device node from logical cpu index
2013-08-21 10:29:51 +01:00
Device tree core updates for v3.12
2013-09-10 13:53:52 -07:00
of: Make of_get_phy_mode() return int i.s.o. const int
2013-08-28 20:50:19 +01:00
drivers: of: add initialization code for dma reserved memory
2013-08-27 10:53:44 +02:00
Staging: olpc_dcon: replace some magic numbers
2013-08-15 17:22:27 -07:00
oprofile: get rid of pointless forward declarations of struct super_block
2013-09-03 22:52:48 -04:00
ARM: SoC platform changes for 3.12
2013-09-06 13:30:06 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2013-09-05 14:54:29 -07:00
Merge branch 'x86-ras-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2013-09-04 11:07:04 -07:00
percpu: Make __verify_pcu_ptr handle per cpu pointers to arrays
2013-08-23 16:08:27 -04:00
Merge branch 'x86-spinlocks-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2013-09-04 11:55:10 -07:00
power_supply: Prevent suspend until power supply events are processed
2013-08-27 18:42:48 -07:00
PPS: convert class code to use dev_groups
2013-08-19 21:22:34 -07:00
pstore: Introduce new argument 'compressed' in the read callback
2013-08-19 10:18:11 -07:00
quota: Add a new quotactl command Q_XGETQSTATV
2013-08-20 16:53:58 -05:00
quota: provide interface for readding allocated space into reserved space
2013-08-17 09:32:32 -04:00
rculist: list_first_or_null_rcu() should use list_entry_rcu()
2013-08-18 17:40:16 -07:00
nohz_full: Add full-system-idle state machine
2013-08-31 14:43:50 -07:00
regmap: Updates for v3.12
2013-09-03 10:07:40 -07:00
include/linux/sched.h: don't use task->pid/tgid in same_thread_group/has_group_leader_pid
2013-09-11 15:56:56 -07:00
DMA: shdma: remove private and unused defines from a global header
2013-08-27 14:25:27 +05:30
sh_eth: remove 'register_type' field from 'struct sh_eth_plat_data'
2013-08-20 17:10:41 -07:00
DMA: shdma: switch DT mode to use configuration data from a match table
2013-08-27 14:25:09 +05:30
Introduce [compat_]save_altstack_ex() to unbreak x86 SMAP
2013-09-01 14:16:33 -07:00
skb: allow skb_scrub_packet() to be used by tunnels
2013-09-04 00:27:25 -04:00
include/linux/smp.h:on_each_cpu(): switch back to a C function
2013-09-11 15:56:19 -07:00
driver:net:stmmac: Disable DMA store and forward mode if platform data force_thresh_dma_mode is set.
2013-08-30 17:26:09 -04:00
swap: make cluster allocation per-cpu
2013-09-11 15:57:17 -07:00
sysfs: sysfs_create_groups returns a value.
2013-08-28 09:51:41 -07:00
tegra-cpuidle: provide stub when !CONFIG_CPU_IDLE
2013-08-29 13:54:34 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2013-09-07 14:35:32 -07:00
gpu/vga_switcheroo: add driver control power feature. (v3)
2013-08-29 13:30:21 +10:00
i915: Update VGA arbiter support for newer devices
2013-09-03 19:17:59 +02:00
mm: vmstats: track TLB flush stats on UP too
2013-09-11 15:57:09 -07:00
Merge branch 'for-3.12' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
2013-09-03 18:25:03 -07:00
[SCSI] zfcp: fix lock imbalance by reworking request queue locking
2013-08-22 08:53:30 -07:00
