Minchan Kim
8912db2538
BACKPORT: mm: don't be stuck to rmap lock on reclaim path ...
The rmap locks(i_mmap_rwsem and anon_vma->root->rwsem) could be contended
under memory pressure if processes keep working on their vmas(e.g., fork,
mmap, munmap). It makes reclaim path stuck. In our real workload traces,
we see kswapd is waiting the lock for 300ms+(worst case, a sec) and it
makes other processes entering direct reclaim, which were also stuck on
the lock.
This patch makes lru aging path try_lock mode like shink_page_list so the
reclaim context will keep working with next lru pages without being stuck.
if it found the rmap lock contended, it rotates the page back to head of
lru in both active/inactive lrus to make them consistent behavior, which
is basic starting point rather than adding more heristic.
Since this patch introduces a new "contended" field as out-param along
with try_lock in-param in rmap_walk_control, it's not immutable any longer
if the try_lock is set so remove const keywords on rmap related functions.
Since rmap walking is already expensive operation, I doubt the const
would help sizable benefit( And we didn't have it until 5.17).
In a heavy app workload in Android, trace shows following statistics. It
almost removes rmap lock contention from reclaim path.
Martin Liu reported:
Before:
max_dur(ms) min_dur(ms) max-min(dur)ms avg_dur(ms) sum_dur(ms) count blocked_function
1632 0 1631 151.542173 31672 209 page_lock_anon_vma_read
601 0 601 145.544681 28817 198 rmap_walk_file
After:
max_dur(ms) min_dur(ms) max-min(dur)ms avg_dur(ms) sum_dur(ms) count blocked_function
NaN NaN NaN NaN NaN 0.0 NaN
0 0 0 0.127645 1 12 rmap_walk_file
[minchan@kernel.org: add comment, per Matthew]
Link: https://lkml.kernel.org/r/YnNqeB5tUf6LZ57b@google.com
Link: https://lkml.kernel.org/r/20220510215423.164547-1-minchan@kernel.org
Signed-off-by: Minchan Kim <minchan@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: John Dias <joaodias@google.com>
Cc: Tim Murray <timmurray@google.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Martin Liu <liumartin@google.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Conflicts:
folio->page
(cherry picked from commit 6d4675e601357834dadd2ba1d803f6484596015c)
Bug: 239681156
Bug: 252333201
Bug: 265247508
Signed-off-by: Minchan Kim <minchan@google.com>
Change-Id: I0c63e0291120c8a1b5f2d83b8a7b210cb56c27a2
Signed-off-by: chenxin <chenxinxin@xiaomi.corp-partner.google.com>
Signed-off-by: guchongchong <guchongchong@xiaomi.corp-partner.google.com>
(cherry picked from commit a0eae55f26a0cbdd828db226592093ad2142889c)
2023-01-13 08:56:32 +00:00
iio: st_sensors: Add a local lock for protecting odr
2022-06-14 18:11:59 +02:00
This is the 5.4.211 stable release
2022-08-25 12:12:16 +02:00
net/mlx5: Fix size field in bufferx_reg struct
2022-03-16 13:21:46 +01:00
mtd: cfi_cmdset_0002: Use chip_ready() for write on S29GL064N
2022-06-14 18:12:03 +02:00
netfilter: ebtables: reject blobs that don't provide all entry points
2022-09-05 10:27:41 +02:00
platform/x86: pmc_atom: Fix SLP_TYPx bitfield mask
2022-09-15 12:04:50 +02:00
This is the 5.4.146 stable release
2021-09-15 14:01:16 +02:00
This is the 5.4.215 stable release
2022-09-28 12:48:08 +02:00
SUNRPC: Don't call connect() more than once on a TCP socket
2022-05-25 09:14:34 +02:00
This is the 5.4.213 stable release
2022-09-21 15:55:20 +02:00
ANDROID: GKI: rework the ANDROID_KABI_USE() macro to not use __UNIQUE()
2021-11-09 18:38:36 +01:00
arm64: entry: Add vectors that have the bhb mitigation sequences
2022-03-19 13:40:15 +01:00
bitfield.h: Fix "type of reg too small for mask" test
2022-07-29 17:14:18 +02:00
ANDROID: GKI: fix crc issue with commit 3f91687e6e ("block: don't merge across cgroup boundaries if blkcg is enabled")
2022-04-19 16:30:10 +02:00
This is the 5.4.196 stable release
2022-05-25 10:40:14 +02:00
This is the 5.4.198 stable release
2022-06-21 14:58:56 +02:00
bpf: Fix pointer arithmetic mask tightening under state pruning
2021-09-15 09:47:39 +02:00
fs: only do a memory barrier for the first set_buffer_uptodate()
2022-09-15 12:04:49 +02:00
drivers: base: cacheinfo: Get rid of DEFINE_SMP_CALL_CACHE_FUNCTION()
2021-09-26 14:07:10 +02:00
Revert "cgroup: Use separate src/dst nodes when preloading css_sets for migration"
2022-07-23 15:30:57 +02:00
compiler.h: Introduce absolute_pointer macro
2021-09-30 10:09:25 +02:00
Revert "serial: core: Fix initializing and restoring termios speed"
2021-11-19 13:42:39 +01:00
x86/bugs: Report AMD retbleed vulnerability
2022-10-07 09:16:54 +02:00
ANDROID: cpu/hotplug: avoid breaking Android ABI by fusing cpuhp steps
2022-06-23 15:38:47 +02:00
debugfs: add debugfs_lookup_and_remove()
2022-09-15 12:04:54 +02:00
This is the 5.4.201 stable release
2022-06-25 16:13:44 +02:00
Revert "PCI/MSI: Protect msi_desc::masked for multi-MSI"
2021-08-27 16:29:28 +02:00
linux/dim: Fix divide by 0 in RDMA DIM
2022-07-07 17:36:49 +02:00
This is the 5.4.196 stable release
2022-05-25 10:40:14 +02:00
efi: Add missing prototype for efi_capsule_setup_info
2022-06-14 18:11:33 +02:00
elfcore: correct reference to CONFIG_UML
2021-10-27 09:54:27 +02:00
PM: EM: Increase energy calculation precision
2021-09-15 09:47:33 +02:00
etherdevice: Adjust ether_addr* prototypes to silence -Wstringop-overead
2022-04-27 13:50:45 +02:00
BACKPORT: f2fs: introduce FI_COMPRESS_RELEASED instead of using IMMUTABLE bit
2022-08-04 09:05:59 +00:00
fbcon: Prevent that screen size is smaller than font size
2022-07-12 16:30:47 +02:00
This is the 5.4.160 stable release
2021-11-17 10:19:21 +01:00
BACKPORT: mm: don't be stuck to rmap lock on reclaim path
2023-01-13 08:56:32 +00:00
devtmpfs regression fix: reconfigure on each mount
2022-01-20 09:19:17 +01:00
This is the 5.4.145 stable release
2021-09-12 09:54:40 +02:00
This is the 5.4.176 stable release
2022-02-01 17:48:27 +01:00
module/ftrace: handle patchable-function-entry
2022-02-23 11:59:56 +01:00
This is the 5.4.174 stable release
2022-01-31 15:14:13 +01:00
This is the 5.4.146 stable release
2021-09-15 14:01:16 +02:00
mm/hugetlb: initialize hugetlb_usage in mm_init
2021-09-22 12:26:37 +02:00
random: pull add_hwgenerator_randomness() declaration into random.h
2022-06-22 14:11:12 +02:00
net: handle ARPHRD_PIMREG in dev_is_mac_header_xmit()
2022-03-23 09:12:07 +01:00
Revert "net: igmp: increase size of mr_ifc_count"
2021-08-27 17:58:47 +02:00
This is the 5.4.215 stable release
2022-09-28 12:48:08 +02:00
shm: extend forced shm destroy to support objects from several IPC nses
2021-12-01 09:23:35 +01:00
This is the 5.4.189 stable release
2022-04-19 16:29:31 +02:00
This is the 5.4.142 stable release
2021-08-27 16:27:13 +02:00
ANDROID: GKI: update the abi .xml file
2022-05-12 18:25:29 +02:00
kexec_file: drop weak attribute from arch_kexec_apply_relocations[_add]
2022-07-02 16:28:50 +02:00
kfifo: fix kfifo_to_user() return type
2022-08-25 11:18:04 +02:00
kprobes: Limit max data_size of the kretprobe instances
2021-12-08 09:01:10 +01:00
UPSTREAM: mm/ksm: Remove reuse_ksm_page()
2022-07-06 18:36:53 -07:00
x86/kvm/vmx: Make noinstr clean
2022-10-07 09:16:54 +02:00
UPSTREAM: leds: flash: Fix multicolor no-ops registration by return 0
2022-12-21 16:28:02 +00:00
libata: fix read log timeout value
2021-11-17 09:48:17 +01:00
include/linux/list.h: add a macro to test if entry is pointing to the head
2021-09-22 12:26:20 +02:00
BACKPORT: binder: use cred instead of task for selinux checks
2021-11-19 16:58:43 -08:00
Revert "mailbox: forward the hrtimer if not queued and under a lock"
2022-06-22 16:59:16 +02:00
rtc: mc146818-lib: Fix the AltCentury for AMD platforms
2022-05-25 09:14:32 +02:00
net: mdio: introduce a shutdown method to mdio device drivers
2021-10-09 14:39:49 +02:00
memblock: fix memblock_phys_alloc() section mismatch error
2022-01-05 12:37:44 +01:00
Linux 5.4.143
2021-08-27 22:38:57 +02:00
mm/memory_hotplug: use "unsigned long" for PFN in zone_for_pfn_range()
2021-09-22 12:26:43 +02:00
This is the 5.4.208 stable release
2022-07-30 14:33:25 +02:00
mmap locking API: initial implementation as rwsem wrappers
2022-07-29 17:14:18 +02:00
Merge 5.4.194 into android11-5.4-lts
2022-05-18 16:49:42 +02:00
This is the 5.4.217 stable release
2022-10-08 08:39:36 +02:00
Revert "PCI/MSI: Protect msi_desc::masked for multi-MSI"
2021-08-27 16:29:28 +02:00
Merge 5.4.195 into android11-5.4-lts
2022-05-18 16:50:18 +02:00
This is the 5.4.189 stable release
2022-04-19 16:29:31 +02:00
NFS: swap IO handling is slightly different for O_DIRECT IO
2022-04-15 14:18:35 +02:00
watchdog: export lockup_detector_reconfigure
2022-08-25 11:18:37 +02:00
nodemask: Fix return values to be unsigned
2022-06-14 18:12:02 +02:00
of: clk: Make <linux/of_clk.h> self-contained
2021-12-08 09:01:08 +01:00
Revert "once: Fix panic when module unload"
2021-09-07 14:06:59 +02:00
mm, oom: make the calculation of oom badness more accurate
2021-09-03 10:08:12 +02:00
PCI/ACPI: Add Ampere Altra SOC MCFG quirk
2021-09-26 14:07:05 +02:00
Revert "PCI: Reduce warnings on possible RW1C corruption"
2022-04-19 16:30:08 +02:00
This is the 5.4.211 stable release
2022-08-25 12:12:16 +02:00
This is the 5.4.173 stable release
2022-01-20 09:36:50 +01:00
siphash: use one source of truth for siphash permutations
2022-06-22 14:11:16 +02:00
psi: Fix uaf issue when psi trigger is destroyed while being polled
2022-02-05 12:35:36 +01:00
psi: Fix uaf issue when psi trigger is destroyed while being polled
2022-02-05 12:35:36 +01:00
ptrace/xtensa: Replace PT_SINGLESTEP with TIF_SINGLESTEP
2022-06-14 18:11:24 +02:00
ANDROID: change function signatures for some random functions.
2022-06-23 15:37:58 +02:00
random: quiet urandom warning ratelimit suppression message
2022-06-29 08:58:45 +02:00
locking/refcount: Consolidate implementations of refcount_t
2022-07-29 17:14:17 +02:00
BACKPORT: mm: don't be stuck to rmap lock on reclaim path
2023-01-13 08:56:32 +00:00
This is the 5.4.160 stable release
2021-11-17 10:19:21 +01:00
misc: rtsx_usb: use separate command and response buffers
2022-07-12 16:30:49 +02:00
Revert "sched/deadline: Fix priority inheritance with multiple scheduling classes"
2022-09-21 15:44:54 +02:00
This is the 5.4.197 stable release
2022-06-06 11:06:30 +02:00
serial: Create uart_xmit_advance()
2022-09-28 11:04:08 +02:00
siphash: use one source of truth for siphash permutations
2022-06-22 14:11:16 +02:00
This is the 5.4.212 stable release
2022-09-21 12:34:35 +02:00
net: stmmac: disable Split Header (SPH) for Intel platforms
2022-05-25 09:14:38 +02:00
This is the 5.4.180 stable release
2022-02-16 17:17:22 +01:00
tee: export teedev_open() and teedev_close_context()
2022-03-02 11:41:04 +01:00
thermal/core: Fix thermal_cooling_device_register() prototype
2021-09-26 14:07:10 +02:00
This is the 5.4.147 stable release
2021-09-16 15:57:48 +02:00
timekeeping: Add raw clock fallback for random_get_entropy()
2022-06-22 14:11:15 +02:00
sched/topology: Make sched_init_numa() use a set for the deduplicating sort
2022-03-08 19:07:48 +01:00
tpm: eventlog: Fix section mismatch for DEBUG_SECTION_MISMATCH
2022-08-25 11:18:11 +02:00
ANDROID: add tty_schedule_flip() back to the kernel
2022-07-30 14:34:42 +02:00
UPSTREAM: udp: never accept GSO_FRAGLIST packets
2021-10-01 11:32:55 -07:00
Revert "USB: core: Prevent nested device-reset calls"
2022-09-21 16:24:14 +02:00
This is the 5.4.185 stable release
2022-03-16 14:40:31 +01:00
virtio: acknowledge all features before access
2022-03-16 13:21:48 +01:00
net: skip virtio_net_hdr_set_proto if protocol already set
2021-12-29 12:23:34 +01:00
wait: Fix __wait_event_hrtimeout for RT/DL tasks
2022-08-25 11:17:29 +02:00
