Marco Elver
423c5febd4
FROMGIT: kfence: await for allocation using wait_event ...
Patch series "kfence: optimize timer scheduling", v2.
We have observed that mostly-idle systems with KFENCE enabled wake up
otherwise idle CPUs, preventing such to enter a lower power state.
Debugging revealed that KFENCE spends too much active time in
toggle_allocation_gate().
While the first version of KFENCE was using all the right bits to be
scheduling optimal, and thus power efficient, by simply using wait_event()
+ wake_up(), that code was unfortunately removed.
As KFENCE was exposed to various different configs and tests, the
scheduling optimal code slowly disappeared. First because of hung task
warnings, and finally because of deadlocks when an allocation is made by
timer code with debug objects enabled. Clearly, the "fixes" were not too
friendly for devices that want to be power efficient.
Therefore, let's try a little harder to fix the hung task and deadlock
problems that we have with wait_event() + wake_up(), while remaining as
scheduling friendly and power efficient as possible.
Crucially, we need to defer the wake_up() to an irq_work, avoiding any
potential for deadlock.
The result with this series is that on the devices where we observed a
power regression, power usage returns back to baseline levels.
This patch (of 3):
On mostly-idle systems, we have observed that toggle_allocation_gate() is
a cause of frequent wake-ups, preventing an otherwise idle CPU to go into
a lower power state.
A late change in KFENCE's development, due to a potential deadlock [1],
required changing the scheduling-friendly wait_event_timeout() and
wake_up() to an open-coded wait-loop using schedule_timeout(). [1]
https://lkml.kernel.org/r/000000000000c0645805b7f982e4@google.com
To avoid unnecessary wake-ups, switch to using wait_event_timeout().
Unfortunately, we still cannot use a version with direct wake_up() in
__kfence_alloc() due to the same potential for deadlock as in [1].
Instead, add a level of indirection via an irq_work that is scheduled if
we determine that the kfence_timer requires a wake_up().
Link: https://lkml.kernel.org/r/20210421105132.3965998-1-elver@google.com
Link: https://lkml.kernel.org/r/20210421105132.3965998-2-elver@google.com
Fixes: 0ce20dd84089 ("mm: add Kernel Electric-Fence infrastructure")
Signed-off-by: Marco Elver <elver@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Jann Horn <jannh@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Hillf Danton <hdanton@sina.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Bug: 172317151
Bug: 185280916
Test: power team confirmed there's no regression
(cherry picked from commit 0ac66fbbadde5547db190e9d87ff2e77d245f9a2
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Idba39683f0b76eb3f70df1113e43d94845fab5bf
2026-05-07 17:24:08 +03:00
lib: 842: Improve error handling in sw842_compress()
2025-04-10 14:29:40 +02:00
UPSTREAM: crypto: poly1305 - fix poly1305_core_setkey() declaration
2025-09-24 12:16:39 +02:00
linux/dim: Do nothing if no time delta between samples
2023-05-30 12:44:01 +01:00
lib/fonts: fix undefined behavior in bit shift for get_default_font
2023-01-18 11:40:54 +01:00
livepatch: Proper error handling in the shadow variables selftest
2019-02-06 11:01:57 +01:00
lz4: fix kernel decompression speed
2024-04-16 03:34:44 +03:00
lib/lzo: fix ambiguous encoding bug in lzo-rle
2020-06-17 16:40:28 +02:00
bitmap: introduce generic optimized bitmap_size()
2024-09-04 13:14:50 +02:00
crypto: lib/mpi - avoid null pointer deref in mpi_cmp_ui()
2023-09-23 11:00:03 +02:00
lib/raid6/test/Makefile: Use $(pound) instead of \# for Make 4.3
2022-04-15 14:18:23 +02:00
This is the 5.4.226 stable release
2022-12-08 16:50:20 +00:00
xz: cleanup CRC32 edits from 2018
2024-11-08 16:20:30 +01:00
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 120
2019-05-24 17:39:02 +02:00
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 120
2019-05-24 17:39:02 +02:00
assoc_array: Fix BUG_ON during garbage collect
2022-06-06 08:33:50 +02:00
locking/atomic: Use s64 for atomic64
2019-06-03 12:32:56 +02:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
lib/bch: fix possible stack overrun
2018-10-12 09:17:46 +02:00
cpumask: Implement cpumask_or_equal()
2019-07-25 15:47:37 +02:00
treewide: Add SPDX license identifier for more missed files
2019-05-21 10:50:45 +02:00
Merge remote-tracking branch 'remotes/origin/tmp-f686d9f' into msm-lahaina
2019-07-01 17:41:24 -07:00
bug: Remove redundant condition check in report_bug
2021-05-14 09:44:27 +02:00
Merge remote-tracking branch 'remotes/origin/tmp-f686d9f' into msm-lahaina
2019-07-01 17:41:24 -07:00
lib/clz_ctz.c: Fix __clzdi2() and __ctzdi2() for 32-bit kernels
2023-08-30 16:27:25 +02:00
lib: cpu_rmap: Fix potential use-after-free in irq_cpu_rmap_release()
2023-06-14 10:59:56 +02:00
treewide: add checks for the return value of memblock_alloc*()
2019-03-12 10:04:02 -07:00
Merge remote-tracking branch 'remotes/origin/tmp-f686d9f' into msm-lahaina
2019-07-01 17:41:24 -07:00
Merge remote-tracking branch 'remotes/origin/tmp-f686d9f' into msm-lahaina
2019-07-01 17:41:24 -07:00
Merge remote-tracking branch 'remotes/origin/tmp-f686d9f' into msm-lahaina
2019-07-01 17:41:24 -07:00
Merge remote-tracking branch 'remotes/origin/tmp-f686d9f' into msm-lahaina
2019-07-01 17:41:24 -07:00
lib/crc32.c: fix trivial typo in preprocessor condition
2020-10-29 09:57:52 +01:00
lib/crc: Move polynomial definition to separate header
2018-07-27 19:04:33 +08:00
lib/crc32test: remove extra local_irq_disable/enable
2020-11-10 12:37:27 +01:00
debugobjects: Fix conditions in fill_pool()
2024-11-08 16:20:34 +01:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
decompress_bunzip2: fix rare decompression failure
2024-08-19 05:33:38 +02:00
lib/decompress_unlz4.c: correctly handle zero-padding around initrds.
2021-07-20 16:10:46 +02:00
lib/xz: Avoid overlapping memcpy() with invalid input with in-place decompression
2021-11-17 09:48:31 +01:00
devres: keep both device name and resource name in pretty name
2020-08-21 13:05:32 +02:00
Revert "Merge tag 'keys-acl-20190703' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs"
2019-07-10 18:43:43 -07:00
This is the 5.4.276 stable release
2024-05-17 15:29:56 +00:00
dql: Fix dql->limit value when reset.
2025-06-04 14:32:30 +02:00
Merge remote-tracking branch 'remotes/origin/tmp-f686d9f' into msm-lahaina
2019-07-01 17:41:24 -07:00
errseq: Always report a writeback error once
2018-04-27 08:51:26 -04:00
fault-inject: clean up debugfs file creation logic
2019-06-18 13:47:24 +02:00
of/lib: Allow scripts/dtc/libfdt to be used from kernel code
2012-07-23 13:54:52 +01:00
lib/genalloc: fix device leak in of_gen_pool_get()
2025-10-29 13:59:53 +01:00
lib/generic-radix-tree.c: Fix rare race in __genradix_ptr_alloc()
2024-09-12 11:03:55 +02:00
lib: add module support to glob tests
2017-02-24 17:46:57 -08:00
lib: add module support to glob tests
2017-02-24 17:46:57 -08:00
ANDROID: GKI: update the abi .xml file
2022-05-12 18:25:29 +02:00
ida: Fix crash in ida_free when the bitmap is empty
2024-01-25 14:34:21 -08:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
iomap: introduce io{read|write}64_{lo_hi|hi_lo}
2019-01-22 13:39:59 +01:00
mm/highmem: Lift memcpy_[to|from]_page to core
2023-01-18 11:41:55 +01:00
Merge tag 'ASB-2022-11-01_11-5.4' of https://android.googlesource.com/kernel/common into android13-5.4-lahaina
2022-11-09 19:53:28 +02:00
BACKPORT: mm: add Kernel Electric-Fence infrastructure
2026-05-07 17:24:01 +03:00
kasan: add memory corruption identification for software tag-based mode
2019-09-24 15:54:07 -07:00
FROMGIT: kfence: await for allocation using wait_event
2026-05-07 17:24:08 +03:00
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
kobject: Fix slab-out-of-bounds in fill_kobj_path()
2023-11-08 11:23:38 +01:00
kobject_uevent: Fix OOB access within zap_modalias_env()
2024-08-19 05:33:38 +02:00
lib: vsprintf: Fix handling of number field widths in vsscanf
2021-07-14 16:53:16 +02:00
lib: vsprintf: Fix handling of number field widths in vsscanf
2021-07-14 16:53:16 +02:00
Merge remote-tracking branch 'remotes/origin/tmp-f686d9f' into msm-lahaina
2019-07-01 17:41:24 -07:00
lib/list_debug.c: Detect uninitialized lists
2022-08-25 11:18:36 +02:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
[PATCH] lockdep: locking API self tests
2006-07-03 15:27:03 -07:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
[PATCH] lockdep: locking API self tests
2006-07-03 15:27:03 -07:00
[PATCH] lockdep: locking API self tests
2006-07-03 15:27:03 -07:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
[PATCH] lockdep: locking API self tests
2006-07-03 15:27:03 -07:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
locking/lockdep: Make it easy to detect whether or not inside a selftest
2019-02-28 07:55:43 +01:00
lockref: stop doing cpu_relax in the cmpxchg loop
2023-02-06 07:52:42 +01:00
PCI: Fix pci_register_io_range() memory leak
2021-03-17 17:03:44 +01:00
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 120
2019-05-24 17:39:02 +02:00
UPSTREAM: crypto: chacha - move existing library code into lib/crypto
2025-09-24 12:16:29 +02:00
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
2017-11-02 11:10:55 +01:00
netlink: prevent potential spectre v1 gadgets
2023-02-06 07:52:45 +01:00
nodemask: Fix return values to be unsigned
2022-06-14 18:12:02 +02:00
lib/notifier-error-inject: fix error when writing -errno to debugfs file
2023-01-18 11:40:55 +01:00
mlxsw: spectrum_acl_erp: Fix object nesting warning
2024-08-19 05:33:28 +02:00
ANDROID: fix up 131287ff83 ("once: add DO_ONCE_SLOW() for sleepable contexts")
2022-10-30 11:07:55 +01:00
lib: Add support for generic packing operations
2019-05-03 10:49:17 -04:00
lib: Introduce priority array area manager
2017-02-03 16:35:42 -05:00
Merge remote-tracking branch 'remotes/origin/tmp-f686d9f' into msm-lahaina
2019-07-01 17:41:24 -07:00
Merge branch 'for-5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/dennis/percpu
2019-07-14 16:17:18 -07:00
radix tree: remove unused variable
2023-08-30 16:27:26 +02:00
random: replace custom notifier chain with standard one
2022-06-22 14:11:13 +02:00
ratelimit: Fix data-races in ___ratelimit().
2022-09-05 10:27:42 +02:00
lib/rbtree: avoid generating code twice for the cached versions
2019-07-16 19:23:22 -07:00
ANDROID: restore some removed refcount functions
2022-07-30 15:19:41 +02:00
Merge remote-tracking branch 'remotes/origin/tmp-f686d9f' into msm-lahaina
2019-07-01 17:41:24 -07:00
sgl_alloc_order: fix memory leak
2020-11-05 11:43:22 +01:00
seq_buf: Fix overflow in seq_buf_putmem_hex()
2021-07-19 08:53:16 +02:00
lib: scatterlist: fix sg_split_phys to preserve original scatterlist offsets
2025-05-02 07:39:15 +02:00
BACKPORT: crypto: lib/sha1 - rename "sha" to "sha1"
2025-12-23 13:36:17 -08:00
siphash: use one source of truth for siphash permutations
2022-06-22 14:11:16 +02:00
Merge tag 'ASB-2024-07-05_11-5.4' of https://android.googlesource.com/kernel/common into android13-5.4-lahaina
2024-08-05 21:38:21 +03:00
lib/string.c: implement stpcpy
2020-10-01 13:18:23 +02:00
lib: string_helpers: silence snprintf() output truncation warning
2024-12-14 19:44:39 +01:00
UPSTREAM: lib/strncpy_from_user.c: Mask out bytes after NUL terminator.
2026-02-01 20:44:43 -08:00
lib/syscall: fix syscall registers retrieval on 32-bit platforms
2020-12-11 13:23:32 +01:00
kstrto*: accept "-0" for signed conversion
2015-09-10 13:29:01 -07:00
UPSTREAM: lib: use KSTM_MODULE_GLOBALS macro in kselftest drivers
2026-05-07 17:24:06 +03:00
net: blackhole_dev: fix build warning for ethh set but not used
2024-03-26 18:22:15 -04:00
UPSTREAM: bpf/tests: Use migrate disable instead of preempt disable
2025-12-23 13:35:58 -08:00
test_firmware: return ENOMEM instead of ENOSPC on failed memory allocation
2023-08-11 11:54:00 +02:00
ida: Fix crash in ida_free when the bitmap is empty
2024-01-25 14:34:21 -08:00
lib/test: use after free in register_test_dev_kmod()
2022-04-15 14:18:21 +02:00
lib/test_meminit: fix off-by-one error in test_pages()
2023-10-25 11:53:17 +02:00
lib: test_objagg: Set error message in check_expect_hints_stats()
2025-07-17 18:24:56 +02:00
overflow: Implement size_t saturating arithmetic helpers
2024-09-04 13:14:51 +02:00
UPSTREAM: lib/vsprintf: no_hash_pointers prints all addresses as unhashed
2026-05-07 17:24:06 +03:00
This is the 5.4.148 stable release
2021-09-25 14:41:58 +02:00
XArray: Fix xas_create_range() when multi-order entry present
2022-04-15 14:18:28 +02:00
lib: textsearch: fix escapes in example code
2019-10-03 12:12:23 -04:00
lib/timerqueue: Rely on rbtree semantics for next timer
2019-07-24 17:38:01 +02:00
lib/ts_bm: reset initial match offset for every block of text
2023-07-27 08:37:12 +02:00
ubsan: Remove vla bound checks.
2019-05-06 11:12:09 -07:00
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 120
2019-05-24 17:39:02 +02:00
uaccess: Add speculation barrier to copy_from_user()
2023-02-25 11:53:26 +01:00
UPSTREAM: lib/vsprintf: no_hash_pointers prints all addresses as unhashed
2026-05-07 17:24:06 +03:00
XArray: Update the LRU list in xas_split()
2022-04-15 14:18:28 +02:00
lib: Add xxhash module
2017-08-15 09:02:07 -07:00
423c5febd4