Alexei Starovoitov
dc2a4ebc0b
bpf: convert explored_states to hash table
...
All prune points inside a callee bpf function most likely will have
different callsites. For example, if function foo() is called from
two callsites the half of explored states in all prune points in foo()
will be useless for subsequent walking of one of those callsites.
Fortunately explored_states pruning heuristics keeps the number of states
per prune point small, but walking these states is still a waste of cpu
time when the callsite of the current state is different from the callsite
of the explored state.
To improve pruning logic convert explored_states into hash table and
use simple insn_idx ^ callsite hash to select hash bucket.
This optimization has no effect on programs without bpf2bpf calls
and drastically improves programs with calls.
In the later case it reduces total memory consumption in 1M scale tests
by almost 3 times (peak_states drops from 5752 to 2016).
Care should be taken when comparing the states for equivalency.
Since the same hash bucket can now contain states with different indices
the insn_idx has to be part of verifier_state and compared.
Different hash table sizes and different hash functions were explored,
but the results were not significantly better vs this patch.
They can be improved in the future.
Hit/miss heuristic is not counting index miscompare as a miss.
Otherwise verifier stats become unstable when experimenting
with different hash functions.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-05-24 01:46:22 +02:00
..
bpf
bpf: convert explored_states to hash table
2019-05-24 01:46:22 +02:00
cgroup
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
configs
debug
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
dma
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
events
mm/mmu_notifier: use correct mmu_notifier events for each invalidation
2019-05-14 09:47:49 -07:00
gcov
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
irq
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
livepatch
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 13
2019-05-21 11:28:45 +02:00
locking
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 9
2019-05-21 11:28:40 +02:00
power
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
printk
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 13
2019-05-21 11:28:45 +02:00
rcu
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
sched
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
time
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
trace
SPDX update for 5.2-rc2, round 1
2019-05-21 12:33:38 -07:00
.gitignore
Provide in-kernel headers to make extending kernel easier
2019-04-29 16:48:03 +02:00
acct.c
acct_on(): don't mess with freeze protection
2019-04-04 21:04:13 -04:00
async.c
treewide: Switch printk users from %pf and %pF to %ps and %pS, respectively
2019-04-09 14:19:06 +02:00
audit.c
audit: connect LOGIN record to its syscall record
2019-03-20 20:57:48 -04:00
audit.h
audit_compare_dname_path(): switch to const struct qstr *
2019-04-28 20:33:43 -04:00
audit_fsnotify.c
audit_compare_dname_path(): switch to const struct qstr *
2019-04-28 20:33:43 -04:00
audit_tree.c
fsnotify: switch send_to_group() and ->handle_event to const struct qstr *
2019-04-26 13:51:03 -04:00
audit_watch.c
audit_compare_dname_path(): switch to const struct qstr *
2019-04-28 20:33:43 -04:00
auditfilter.c
Merge branch 'work.dcache' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2019-05-07 20:03:32 -07:00
auditsc.c
Merge branch 'work.dcache' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2019-05-07 20:03:32 -07:00
backtracetest.c
backtrace-test: Simplify stack trace handling
2019-04-29 12:37:47 +02:00
bounds.c
capability.c
compat.c
kernel/compat.c: mark expected switch fall-throughs
2019-05-15 08:16:14 -07:00
configs.c
kernel/configs: use .incbin directive to embed config_data.gz
2019-03-07 18:32:02 -08:00
context_tracking.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
cpu.c
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-05-06 14:50:46 -07:00
cpu_pm.c
crash_core.c
kexec: export PG_offline to VMCOREINFO
2019-03-05 21:07:14 -08:00
crash_dump.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
cred.c
delayacct.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 25
2019-05-21 11:52:39 +02:00
dma.c
elfcore.c
exec_domain.c
exit.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
extable.c
fail_function.c
treewide: Switch printk users from %pf and %pF to %ps and %pS, respectively
2019-04-09 14:19:06 +02:00
fork.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
freezer.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
futex.c
mm/gup: change GUP fast to use flags rather than a write 'bool'
2019-05-14 09:47:46 -07:00
gen_ikh_data.sh
Provide in-kernel headers to make extending kernel easier
2019-04-29 16:48:03 +02:00
groups.c
hung_task.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
iomem.c
mm/resource: Use resource_overlaps() to simplify region_intersects()
2019-04-19 12:59:36 +02:00
irq_work.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
jump_label.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
kallsyms.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
kcmp.c
Kconfig.freezer
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
Kconfig.hz
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
Kconfig.locks
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
Kconfig.preempt
treewide: Add SPDX license identifier - Makefile/Kconfig
2019-05-21 10:50:46 +02:00
kcov.c
kcov: convert kcov.refcount to refcount_t
2019-03-07 18:32:02 -08:00
kexec.c
kexec_core.c
power/suspend: Add function to disable secondaries for suspend
2019-05-03 19:42:41 +02:00
kexec_file.c
mm: memblock: make keeping memblock memory opt-in rather than opt-out
2019-05-14 09:47:50 -07:00
kexec_internal.h
kheaders.c
Provide in-kernel headers to make extending kernel easier
2019-04-29 16:48:03 +02:00
kmod.c
kprobes.c
kprobes: Fix error check when reusing optimized probes
2019-04-16 09:38:16 +02:00
ksysfs.c
kthread.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
latencytop.c
kernel/latencytop.c: rename clear_all_latency_tracing to clear_tsk_latency_tracing
2019-05-14 19:52:49 -07:00
Makefile
kernel/Makefile: don't assume that kernel/gen_ikh_data.sh is executable
2019-05-14 19:52:47 -07:00
memremap.c
kernel/memremap.c: remove the unused device_private_entry_fault() export
2019-05-14 09:47:51 -07:00
module-internal.h
kallsyms: store type information in its own array
2019-03-28 15:00:37 +01:00
module.c
Modules updates for v5.2
2019-05-14 10:55:54 -07:00
module_signing.c
notifier.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
nsproxy.c
padata.c
padata: Replace padata_attr_type default_attrs field with groups
2019-04-25 22:06:11 +02:00
panic.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
params.c
pid.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
pid_namespace.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
profile.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
ptrace.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
range.c
reboot.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
relay.c
Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2019-03-12 13:27:20 -07:00
resource.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
rseq.c
rseq: Remove superfluous rseq_len from task_struct
2019-04-19 12:39:32 +02:00
seccomp.c
audit/stable-5.2 PR 20190507
2019-05-07 19:06:04 -07:00
signal.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
smp.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
smpboot.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
smpboot.h
softirq.c
softirq: Remove tasklet_hrtimer
2019-03-22 14:36:02 +01:00
stackleak.c
stacktrace.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
stop_machine.c
treewide: Switch printk users from %pf and %pF to %ps and %pS, respectively
2019-04-09 14:19:06 +02:00
sys.c
kernel/sys.c: prctl: fix false positive in validate_prctl_map()
2019-05-14 09:47:44 -07:00
sys_ni.c
signal: support CLONE_PIDFD with pidfd_send_signal
2019-05-07 14:31:03 +02:00
sysctl.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
sysctl_binary.c
task_work.c
taskstats.c
genetlink: optionally validate strictly/dumps
2019-04-27 17:07:22 -04:00
test_kprobes.c
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 25
2019-05-21 11:52:39 +02:00
torture.c
torture: Don't try to offline the last CPU
2019-03-26 14:42:53 -07:00
tracepoint.c
tsacct.c
ucount.c
uid16.c
uid16.h
umh.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
up.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
user-return-notifier.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
user.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
user_namespace.c
utsname.c
utsname_sysctl.c
watchdog.c
watchdog: Fix typo in comment
2019-04-18 14:05:51 +02:00
watchdog_hld.c
kernel/watchdog_hld.c: hard lockup message should end with a newline
2019-04-19 09:46:05 -07:00
workqueue.c
treewide: Add SPDX license identifier for missed files
2019-05-21 10:50:45 +02:00
workqueue_internal.h
sched/core, workqueues: Distangle worker accounting from rq lock
2019-04-16 16:55:15 +02:00