Marco Elver
423c5febd4
FROMGIT: kfence: await for allocation using wait_event ...
Patch series "kfence: optimize timer scheduling", v2.
We have observed that mostly-idle systems with KFENCE enabled wake up
otherwise idle CPUs, preventing such to enter a lower power state.
Debugging revealed that KFENCE spends too much active time in
toggle_allocation_gate().
While the first version of KFENCE was using all the right bits to be
scheduling optimal, and thus power efficient, by simply using wait_event()
+ wake_up(), that code was unfortunately removed.
As KFENCE was exposed to various different configs and tests, the
scheduling optimal code slowly disappeared. First because of hung task
warnings, and finally because of deadlocks when an allocation is made by
timer code with debug objects enabled. Clearly, the "fixes" were not too
friendly for devices that want to be power efficient.
Therefore, let's try a little harder to fix the hung task and deadlock
problems that we have with wait_event() + wake_up(), while remaining as
scheduling friendly and power efficient as possible.
Crucially, we need to defer the wake_up() to an irq_work, avoiding any
potential for deadlock.
The result with this series is that on the devices where we observed a
power regression, power usage returns back to baseline levels.
This patch (of 3):
On mostly-idle systems, we have observed that toggle_allocation_gate() is
a cause of frequent wake-ups, preventing an otherwise idle CPU to go into
a lower power state.
A late change in KFENCE's development, due to a potential deadlock [1],
required changing the scheduling-friendly wait_event_timeout() and
wake_up() to an open-coded wait-loop using schedule_timeout(). [1]
https://lkml.kernel.org/r/000000000000c0645805b7f982e4@google.com
To avoid unnecessary wake-ups, switch to using wait_event_timeout().
Unfortunately, we still cannot use a version with direct wake_up() in
__kfence_alloc() due to the same potential for deadlock as in [1].
Instead, add a level of indirection via an irq_work that is scheduled if
we determine that the kfence_timer requires a wake_up().
Link: https://lkml.kernel.org/r/20210421105132.3965998-1-elver@google.com
Link: https://lkml.kernel.org/r/20210421105132.3965998-2-elver@google.com
Fixes: 0ce20dd84089 ("mm: add Kernel Electric-Fence infrastructure")
Signed-off-by: Marco Elver <elver@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Jann Horn <jannh@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Hillf Danton <hdanton@sina.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Bug: 172317151
Bug: 185280916
Test: power team confirmed there's no regression
(cherry picked from commit 0ac66fbbadde5547db190e9d87ff2e77d245f9a2
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Idba39683f0b76eb3f70df1113e43d94845fab5bf
2026-05-07 17:24:08 +03:00
lib: 842: Improve error handling in sw842_compress()
2025-04-10 14:29:40 +02:00
UPSTREAM: crypto: poly1305 - fix poly1305_core_setkey() declaration
2025-09-24 12:16:39 +02:00
linux/dim: Do nothing if no time delta between samples
2023-05-30 12:44:01 +01:00
lib/fonts: fix undefined behavior in bit shift for get_default_font
2023-01-18 11:40:54 +01:00
lz4: fix kernel decompression speed
2024-04-16 03:34:44 +03:00
lib/lzo: fix ambiguous encoding bug in lzo-rle
2020-06-17 16:40:28 +02:00
bitmap: introduce generic optimized bitmap_size()
2024-09-04 13:14:50 +02:00
crypto: lib/mpi - avoid null pointer deref in mpi_cmp_ui()
2023-09-23 11:00:03 +02:00
lib/raid6/test/Makefile: Use $(pound) instead of \# for Make 4.3
2022-04-15 14:18:23 +02:00
This is the 5.4.226 stable release
2022-12-08 16:50:20 +00:00
xz: cleanup CRC32 edits from 2018
2024-11-08 16:20:30 +01:00
lib/zlib: remove outdated and incorrect pre-increment optimization
2020-06-24 17:50:39 +02:00
assoc_array: Fix BUG_ON during garbage collect
2022-06-06 08:33:50 +02:00
bug: Remove redundant condition check in report_bug
2021-05-14 09:44:27 +02:00
lib/clz_ctz.c: Fix __clzdi2() and __ctzdi2() for 32-bit kernels
2023-08-30 16:27:25 +02:00
lib: cpu_rmap: Fix potential use-after-free in irq_cpu_rmap_release()
2023-06-14 10:59:56 +02:00
crc-t10dif: Fix potential crypto notify dead-lock
2020-08-19 08:15:54 +02:00
lib/crc32.c: fix trivial typo in preprocessor condition
2020-10-29 09:57:52 +01:00
lib/crc32test: remove extra local_irq_disable/enable
2020-11-10 12:37:27 +01:00
debugobjects: Fix conditions in fill_pool()
2024-11-08 16:20:34 +01:00
decompress_bunzip2: fix rare decompression failure
2024-08-19 05:33:38 +02:00
lib/decompress_unlz4.c: correctly handle zero-padding around initrds.
2021-07-20 16:10:46 +02:00
lib/xz: Avoid overlapping memcpy() with invalid input with in-place decompression
2021-11-17 09:48:31 +01:00
Revert "ANDROID: Revert: Merge 5.4.60 into android11-5.4"
2020-08-23 13:12:51 +02:00
This is the 5.4.276 stable release
2024-05-17 15:29:56 +00:00
dql: Fix dql->limit value when reset.
2025-06-04 14:32:30 +02:00
uapi: rename ext2_swab() to swab() and share globally in swab.h
2020-04-13 10:48:07 +02:00
lib/genalloc: fix device leak in of_gen_pool_get()
2025-10-29 13:59:53 +01:00
lib/generic-radix-tree.c: Fix rare race in __genradix_ptr_alloc()
2024-09-12 11:03:55 +02:00
ANDROID: GKI: update the abi .xml file
2022-05-12 18:25:29 +02:00
ida: Fix crash in ida_free when the bitmap is empty
2024-01-25 14:34:21 -08:00
mm/highmem: Lift memcpy_[to|from]_page to core
2023-01-18 11:41:55 +01:00
Merge tag 'ASB-2022-11-01_11-5.4' of https://android.googlesource.com/kernel/common into android13-5.4-lahaina
2022-11-09 19:53:28 +02:00
BACKPORT: mm: add Kernel Electric-Fence infrastructure
2026-05-07 17:24:01 +03:00
FROMGIT: kfence: await for allocation using wait_event
2026-05-07 17:24:08 +03:00
kobject: Fix slab-out-of-bounds in fill_kobj_path()
2023-11-08 11:23:38 +01:00
kobject_uevent: Fix OOB access within zap_modalias_env()
2024-08-19 05:33:38 +02:00
lib: vsprintf: Fix handling of number field widths in vsscanf
2021-07-14 16:53:16 +02:00
lib: vsprintf: Fix handling of number field widths in vsscanf
2021-07-14 16:53:16 +02:00
lib/list_debug.c: Detect uninitialized lists
2022-08-25 11:18:36 +02:00
lib/list_sort: fix function type mismatches
2020-02-11 13:25:58 -08:00
lockref: stop doing cpu_relax in the cmpxchg loop
2023-02-06 07:52:42 +01:00
PCI: Fix pci_register_io_range() memory leak
2021-03-17 17:03:44 +01:00
UPSTREAM: crypto: chacha - move existing library code into lib/crypto
2025-09-24 12:16:29 +02:00
netlink: prevent potential spectre v1 gadgets
2023-02-06 07:52:45 +01:00
nodemask: Fix return values to be unsigned
2022-06-14 18:12:02 +02:00
lib/notifier-error-inject: fix error when writing -errno to debugfs file
2023-01-18 11:40:55 +01:00
mlxsw: spectrum_acl_erp: Fix object nesting warning
2024-08-19 05:33:28 +02:00
ANDROID: fix up 131287ff83 ("once: add DO_ONCE_SLOW() for sleepable contexts")
2022-10-30 11:07:55 +01:00
radix tree: remove unused variable
2023-08-30 16:27:26 +02:00
random: replace custom notifier chain with standard one
2022-06-22 14:11:13 +02:00
ratelimit: Fix data-races in ___ratelimit().
2022-09-05 10:27:42 +02:00
ANDROID: restore some removed refcount functions
2022-07-30 15:19:41 +02:00
sgl_alloc_order: fix memory leak
2020-11-05 11:43:22 +01:00
seq_buf: Fix overflow in seq_buf_putmem_hex()
2021-07-19 08:53:16 +02:00
lib: scatterlist: fix sg_split_phys to preserve original scatterlist offsets
2025-05-02 07:39:15 +02:00
BACKPORT: crypto: lib/sha1 - rename "sha" to "sha1"
2025-12-23 13:36:17 -08:00
siphash: use one source of truth for siphash permutations
2022-06-22 14:11:16 +02:00
Merge tag 'ASB-2024-07-05_11-5.4' of https://android.googlesource.com/kernel/common into android13-5.4-lahaina
2024-08-05 21:38:21 +03:00
lib/string.c: implement stpcpy
2020-10-01 13:18:23 +02:00
lib: string_helpers: silence snprintf() output truncation warning
2024-12-14 19:44:39 +01:00
UPSTREAM: lib/strncpy_from_user.c: Mask out bytes after NUL terminator.
2026-02-01 20:44:43 -08:00
lib/syscall: fix syscall registers retrieval on 32-bit platforms
2020-12-11 13:23:32 +01:00
UPSTREAM: lib: use KSTM_MODULE_GLOBALS macro in kselftest drivers
2026-05-07 17:24:06 +03:00
net: blackhole_dev: fix build warning for ethh set but not used
2024-03-26 18:22:15 -04:00
UPSTREAM: bpf/tests: Use migrate disable instead of preempt disable
2025-12-23 13:35:58 -08:00
test_firmware: return ENOMEM instead of ENOSPC on failed memory allocation
2023-08-11 11:54:00 +02:00
ida: Fix crash in ida_free when the bitmap is empty
2024-01-25 14:34:21 -08:00
kasan: stop tests being eliminated as dead code with FORTIFY_SOURCE
2020-06-22 09:31:12 +02:00
lib/test: use after free in register_test_dev_kmod()
2022-04-15 14:18:21 +02:00
lib/test_meminit: fix off-by-one error in test_pages()
2023-10-25 11:53:17 +02:00
lib: test_objagg: Set error message in check_expect_hints_stats()
2025-07-17 18:24:56 +02:00
overflow: Implement size_t saturating arithmetic helpers
2024-09-04 13:14:51 +02:00
UPSTREAM: lib/vsprintf: no_hash_pointers prints all addresses as unhashed
2026-05-07 17:24:06 +03:00
This is the 5.4.148 stable release
2021-09-25 14:41:58 +02:00
XArray: Fix xas_create_range() when multi-order entry present
2022-04-15 14:18:28 +02:00
lib/ts_bm: reset initial match offset for every block of text
2023-07-27 08:37:12 +02:00
uaccess: Add speculation barrier to copy_from_user()
2023-02-25 11:53:26 +01:00
UPSTREAM: lib/vsprintf: no_hash_pointers prints all addresses as unhashed
2026-05-07 17:24:06 +03:00
XArray: Update the LRU list in xas_split()
2022-04-15 14:18:28 +02:00
423c5febd4