Daniel Jordan
bfde23ce20
padata: unbind parallel jobs from specific CPUs
...
Padata binds the parallel part of a job to a single CPU and round-robins
over all CPUs in the system for each successive job. Though the serial
parts rely on per-CPU queues for correct ordering, they're not necessary
for parallel work, and it improves performance to run the job locally on
NUMA machines and let the scheduler pick the CPU within a node on a busy
system.
So, make the parallel workqueue unbound.
Update the parallel workqueue's cpumask when the instance's parallel
cpumask changes.
Now that parallel jobs no longer run on max_active=1 workqueues, two or
more parallel works that hash to the same CPU may run simultaneously,
finish out of order, and so be serialized out of order. Prevent this by
keeping the works sorted on the reorder list by sequence number and
checking that in the reordering logic.
padata_get_next becomes padata_find_next so it can be reused for the end
of padata_reorder, where it's used to avoid uselessly queueing work when
the next job by sequence number isn't finished yet but a later job that
hashed to the same CPU has.
The ENODATA case in padata_find_next no longer makes sense because
parallel jobs aren't bound to specific CPUs. The EINPROGRESS case takes
care of the scenario where a parallel job is potentially running on the
same CPU as padata_find_next, and with only one error code left, just
use NULL instead.
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-09-13 21:15:40 +10:00
..
bpf
Merge branch 'core-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-07-20 10:45:15 -07:00
cgroup
Merge branch 'work.mount0' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2019-07-19 10:42:02 -07:00
configs
debug
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
dma
dma-mapping fixes for 5.3-rc1
2019-07-20 12:09:52 -07:00
events
Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2019-07-19 11:35:08 -07:00
gcov
gcov: no need to check return value of debugfs_create functions
2019-06-03 16:18:12 +02:00
irq
Merge branch 'x86-apic-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-07-08 11:22:57 -07:00
livepatch
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/livepatching/livepatching
2019-07-11 15:30:05 -07:00
locking
docs: locking: convert docs to ReST and rename to *.rst
2019-07-15 08:53:27 -03:00
power
pci-v5.3-changes
2019-07-15 20:44:49 -07:00
printk
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 13
2019-05-21 11:28:45 +02:00
rcu
Merge branch 'for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcu
2019-06-28 19:46:47 +02:00
sched
Mostly bugfixes, but also:
2019-07-20 10:20:27 -07:00
time
timekeeping/vsyscall: Use __iter_div_u64_rem()
2019-07-10 20:37:49 +02:00
trace
Eiichi Tsukata found a small bug from the fixup of the stack code
2019-07-19 12:18:46 -07:00
.gitignore
Provide in-kernel headers to make extending kernel easier
2019-04-29 16:48:03 +02:00
acct.c
acct_on(): don't mess with freeze protection
2019-04-04 21:04:13 -04:00
async.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
audit.c
audit/stable-5.3 PR 20190702
2019-07-08 18:55:42 -07:00
audit.h
audit/stable-5.3 PR 20190702
2019-07-08 18:55:42 -07:00
audit_fsnotify.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157
2019-05-30 11:26:37 -07:00
audit_tree.c
fsnotify: switch send_to_group() and ->handle_event to const struct qstr *
2019-04-26 13:51:03 -04:00
audit_watch.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
auditfilter.c
audit/stable-5.3 PR 20190702
2019-07-08 18:55:42 -07:00
auditsc.c
audit: enforce op for string fields
2019-05-28 17:46:43 -04:00
backtracetest.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
bounds.c
capability.c
compat.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500
2019-06-19 17:09:55 +02:00
configs.c
kernel/configs: use .incbin directive to embed config_data.gz
2019-03-07 18:32:02 -08:00
context_tracking.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
cpu.c
Merge branch 'smp-hotplug-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-07-08 10:39:56 -07:00
cpu_pm.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 282
2019-06-05 17:36:37 +02:00
crash_core.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230
2019-06-19 17:09:06 +02:00
crash_dump.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
cred.c
request_key improvements
2019-07-08 19:19:37 -07:00
delayacct.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 25
2019-05-21 11:52:39 +02:00
dma.c
elfcore.c
exec_domain.c
exit.c
cgroup: Call cgroup_release() before __exit_signal()
2019-05-31 10:38:57 -07:00
extable.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
fail_function.c
fail_function: no need to check return value of debugfs_create functions
2019-06-03 15:49:06 +02:00
fork.c
for-linus-20190715
2019-07-16 11:30:07 -07:00
freezer.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
futex.c
Linux 5.2-rc3
2019-06-03 11:50:18 +02:00
gen_kheaders.sh
kheaders: include only headers into kheaders_data.tar.xz
2019-07-09 10:10:52 +09:00
groups.c
hung_task.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
iomem.c
mm/nvdimm: add is_ioremap_addr and use that to check ioremap address
2019-07-12 11:05:40 -07:00
irq_work.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
jump_label.c
jump_label: Batch updates if arch supports it
2019-06-17 12:09:22 +02:00
kallsyms.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
kcmp.c
Kconfig.freezer
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
Kconfig.hz
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
Kconfig.locks
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
Kconfig.preempt
sched/rt, Kconfig: Introduce CONFIG_PREEMPT_RT
2019-07-18 23:10:57 +02:00
kcov.c
kcov: convert kcov.refcount to refcount_t
2019-03-07 18:32:02 -08:00
kexec.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230
2019-06-19 17:09:06 +02:00
kexec_core.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230
2019-06-19 17:09:06 +02:00
kexec_file.c
Merge branch 'next-integrity' of git://git.kernel.org/pub/scm/linux/kernel/git/zohar/linux-integrity
2019-07-08 20:28:59 -07:00
kexec_internal.h
kheaders.c
kheaders: Move from proc to sysfs
2019-05-24 20:16:01 +02:00
kmod.c
kprobes.c
The main changes in this release include:
2019-07-18 11:51:00 -07:00
ksysfs.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 170
2019-05-30 11:26:39 -07:00
kthread.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
latencytop.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
Makefile
kheaders: Move from proc to sysfs
2019-05-24 20:16:01 +02:00
memremap.c
mm/devm_memremap_pages: enable sub-section remap
2019-07-18 17:08:07 -07:00
module-internal.h
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36
2019-05-24 17:27:11 +02:00
module.c
Modules updates for v5.3
2019-07-18 12:06:57 -07:00
module_signing.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36
2019-05-24 17:27:11 +02:00
notifier.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
nsproxy.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
padata.c
padata: unbind parallel jobs from specific CPUs
2019-09-13 21:15:40 +10:00
panic.c
docs: admin-guide: move sysctl directory to it
2019-07-15 11:03:01 -03:00
params.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156
2019-05-30 11:26:35 -07:00
pid.c
kernel/pid.c: convert struct pid count to refcount_t
2019-07-16 19:23:24 -07:00
pid_namespace.c
proc/sysctl: add shared variables for range check
2019-07-18 17:08:07 -07:00
profile.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
ptrace.c
ptrace: add PTRACE_GET_SYSCALL_INFO request
2019-07-16 19:23:24 -07:00
range.c
reboot.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
relay.c
Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2019-03-12 13:27:20 -07:00
resource.c
resource: avoid unnecessary lookups in find_next_iomem_res()
2019-07-18 17:08:06 -07:00
rseq.c
signal: Remove task parameter from force_sig
2019-05-27 09:36:28 -05:00
seccomp.c
signal: Remove the signal number and task parameters from force_sig_info
2019-05-29 09:31:44 -05:00
signal.c
signal: simplify set_user_sigmask/restore_user_sigmask
2019-07-16 19:23:24 -07:00
smp.c
smp: Warn on function calls from softirq context
2019-07-20 11:27:16 +02:00
smpboot.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
smpboot.h
softirq.c
Merge branch 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-07-08 11:01:13 -07:00
stackleak.c
stacktrace.c
stacktrace: Force USER_DS for stack_trace_save_user()
2019-07-18 16:47:24 +02:00
stop_machine.c
processor: get rid of cpu_relax_yield
2019-06-15 12:25:55 +02:00
sys.c
prctl_set_mm: downgrade mmap_sem to read lock
2019-06-01 15:51:31 -07:00
sys_ni.c
arch: handle arches who do not yet define clone3
2019-06-21 01:54:53 +02:00
sysctl.c
proc/sysctl: add shared variables for range check
2019-07-18 17:08:07 -07:00
sysctl_binary.c
task_work.c
taskstats.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157
2019-05-30 11:26:37 -07:00
test_kprobes.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 25
2019-05-21 11:52:39 +02:00
torture.c
torture: Allow inter-stutter interval to be specified
2019-05-28 09:06:09 -07:00
tracepoint.c
The main changes in this release include:
2019-07-18 11:51:00 -07:00
tsacct.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157
2019-05-30 11:26:37 -07:00
ucount.c
proc/sysctl: add shared variables for range check
2019-07-18 17:08:07 -07:00
uid16.c
uid16.h
umh.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
up.c
smp: Remove smp_call_function() and on_each_cpu() return values
2019-06-23 14:26:26 +02:00
user-return-notifier.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
user.c
Keyrings namespacing
2019-07-08 19:36:47 -07:00
user_namespace.c
Keyrings namespacing
2019-07-08 19:36:47 -07:00
utsname.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
utsname_sysctl.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 441
2019-06-05 17:37:17 +02:00
watchdog.c
watchdog: Fix typo in comment
2019-04-18 14:05:51 +02:00
watchdog_hld.c
kernel/watchdog_hld.c: hard lockup message should end with a newline
2019-04-19 09:46:05 -07:00
workqueue.c
workqueue: require CPU hotplug read exclusion for apply_workqueue_attrs
2019-09-13 21:15:40 +10:00
workqueue_internal.h
sched/core, workqueues: Distangle worker accounting from rq lock
2019-04-16 16:55:15 +02:00