Peter Zijlstra
0b9ccc0a9b
x86/percpu: Differentiate this_cpu_{}() and __this_cpu_{}()
...
Nadav Amit reported that commit:
b59167ac7b ("x86/percpu: Fix this_cpu_read()")
added a bunch of constraints to all sorts of code; and while some of
that was correct and desired, some of that seems superfluous.
The thing is, the this_cpu_*() operations are defined IRQ-safe, this
means the values are subject to change from IRQs, and thus must be
reloaded.
Also, the generic form:
local_irq_save()
__this_cpu_read()
local_irq_restore()
would not allow the re-use of previous values; if by nothing else,
then the barrier()s implied by local_irq_*().
Which raises the point that percpu_from_op() and the others also need
that volatile.
OTOH __this_cpu_*() operations are not IRQ-safe and assume external
preempt/IRQ disabling and could thus be allowed more room for
optimization.
This makes the this_cpu_*() vs __this_cpu_*() behaviour more
consistent with other architectures.
$ ./compare.sh defconfig-build defconfig-build1 vmlinux.o
x86_pmu_cancel_txn 80 71 -9,+0
__text_poke 919 964 +45,+0
do_user_addr_fault 1082 1058 -24,+0
__do_page_fault 1194 1178 -16,+0
do_exit 2995 3027 -43,+75
process_one_work 1008 989 -67,+48
finish_task_switch 524 505 -19,+0
__schedule_bug 103 98 -59,+54
__schedule_bug 103 98 -59,+54
__sched_setscheduler 2015 2030 +15,+0
freeze_processes 203 230 +31,-4
rcu_gp_kthread_wake 106 99 -7,+0
rcu_core 1841 1834 -7,+0
call_timer_fn 298 286 -12,+0
can_stop_idle_tick 146 139 -31,+24
perf_pending_event 253 239 -14,+0
shmem_alloc_page 209 213 +4,+0
__alloc_pages_slowpath 3284 3269 -15,+0
umount_tree 671 694 +23,+0
advance_transaction 803 798 -5,+0
con_put_char 71 51 -20,+0
xhci_urb_enqueue 1302 1295 -7,+0
xhci_urb_enqueue 1302 1295 -7,+0
tcp_sacktag_write_queue 2130 2075 -55,+0
tcp_try_undo_loss 229 208 -21,+0
tcp_v4_inbound_md5_hash 438 411 -31,+4
tcp_v4_inbound_md5_hash 438 411 -31,+4
tcp_v6_inbound_md5_hash 469 411 -33,-25
tcp_v6_inbound_md5_hash 469 411 -33,-25
restricted_pointer 434 420 -14,+0
irq_exit 162 154 -8,+0
get_perf_callchain 638 624 -14,+0
rt_mutex_trylock 169 156 -13,+0
avc_has_extended_perms 1092 1089 -3,+0
avc_has_perm_noaudit 309 306 -3,+0
__perf_sw_event 138 122 -16,+0
perf_swevent_get_recursion_context 116 102 -14,+0
__local_bh_enable_ip 93 72 -21,+0
xfrm_input 4175 4161 -14,+0
avc_has_perm 446 443 -3,+0
vm_events_fold_cpu 57 56 -1,+0
vfree 68 61 -7,+0
freeze_processes 203 230 +31,-4
_local_bh_enable 44 30 -14,+0
ip_do_fragment 1982 1944 -38,+0
do_exit 2995 3027 -43,+75
__do_softirq 742 724 -18,+0
cpu_init 1510 1489 -21,+0
account_system_time 80 79 -1,+0
total 12985281 12984819 -742,+280
Reported-by: Nadav Amit <nadav.amit@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: https://lkml.kernel.org/r/20181206112433.GB13675@hirez.programming.kicks-ass.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2019-06-17 12:43:40 +02:00
arch
x86/percpu: Differentiate this_cpu_{}() and __this_cpu_{}()
2019-06-17 12:43:40 +02:00
block
blk-mq: remove WARN_ON(!q->elevator) from blk_mq_sched_free_requests
2019-06-13 03:05:58 -06:00
certs
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36
2019-05-24 17:27:11 +02:00
crypto
SPDX update for 5.2-rc4
2019-06-08 12:52:42 -07:00
Documentation
x86/atomic: Fix smp_mb__{before,after}_atomic()
2019-06-17 12:09:59 +02:00
drivers
locking/lockdep: Rename lockdep_assert_held_exclusive() -> lockdep_assert_held_write()
2019-06-17 12:09:24 +02:00
fs
locking/lockdep: Rename lockdep_assert_held_exclusive() -> lockdep_assert_held_write()
2019-06-17 12:09:24 +02:00
include
locking/rwsem: Make rwsem->owner an atomic_long_t
2019-06-17 12:28:06 +02:00
init
Linux 5.2-rc5
2019-06-17 12:06:34 +02:00
ipc
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
kernel
locking/rwsem: Guard against making count negative
2019-06-17 12:28:11 +02:00
lib
locking/rwsem: Make owner available even if !CONFIG_RWSEM_SPIN_ON_OWNER
2019-06-17 12:27:54 +02:00
LICENSES
LICENSES: Rename other to deprecated
2019-05-03 06:34:32 -06:00
mm
Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-06-16 07:28:14 -10:00
net
SPDX update for 5.2-rc4
2019-06-08 12:52:42 -07:00
samples
SPDX update for 5.2-rc4
2019-06-08 12:52:42 -07:00
scripts
scripts/decode_stacktrace.sh: prefix addr2line with $CROSS_COMPILE
2019-06-13 17:34:56 -10:00
security
locking/lockdep: Rename lockdep_assert_held_exclusive() -> lockdep_assert_held_write()
2019-06-17 12:09:24 +02:00
sound
sound fixes for 5.2-rc5
2019-06-14 05:37:06 -10:00
tools
mm/devm_memremap_pages: fix final page put race
2019-06-13 17:34:56 -10:00
usr
virt
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 333
2019-06-05 17:37:06 +02:00
.clang-format
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
2019-04-17 11:26:25 -07:00
.cocciconfig
.get_maintainer.ignore
Opt out of scripts/get_maintainer.pl
2019-05-16 10:53:40 -07:00
.gitattributes
.gitignore
.gitignore: exclude .get_maintainer.ignore and .gitattributes
2019-05-18 11:49:54 +09:00
.mailmap
A reasonably busy cycle for docs, including:
2019-05-08 12:42:50 -07:00
COPYING
CREDITS
MAINTAINERS: Farewell Martin Schwidefsky
2019-05-31 10:14:11 +02:00
Kbuild
Kbuild updates for v5.1
2019-03-10 17:48:21 -07:00
Kconfig
MAINTAINERS
Merge branch 'i2c/for-current' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux
2019-06-08 12:48:49 -07:00
Makefile
Linux 5.2-rc5
2019-06-16 08:49:45 -10:00
README