No description
  • C 98.2%
  • Assembly 1%
  • Makefile 0.3%
  • Shell 0.2%
  • Python 0.1%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Willem de Bruijn 86137342fd net/packet: tpacket_rcv: avoid a producer race condition
[ Upstream commit 61fad6816fc10fb8793a925d5c1256d1c3db0cd2 ]

PACKET_RX_RING can cause multiple writers to access the same slot if a
fast writer wraps the ring while a slow writer is still copying. This
is particularly likely with few, large, slots (e.g., GSO packets).

Synchronize kernel thread ownership of rx ring slots with a bitmap.

Writers acquire a slot race-free by testing tp_status TP_STATUS_KERNEL
while holding the sk receive queue lock. They release this lock before
copying and set tp_status to TP_STATUS_USER to release to userspace
when done. During copying, another writer may take the lock, also see
TP_STATUS_KERNEL, and start writing to the same slot.

Introduce a new rx_owner_map bitmap with a bit per slot. To acquire a
slot, test and set with the lock held. To release race-free, update
tp_status and owner bit as a transaction, so take the lock again.

This is the one of a variety of discussed options (see Link below):

* instead of a shadow ring, embed the data in the slot itself, such as
in tp_padding. But any test for this field may match a value left by
userspace, causing deadlock.

* avoid the lock on release. This leaves a small race if releasing the
shadow slot before setting TP_STATUS_USER. The below reproducer showed
that this race is not academic. If releasing the slot after tp_status,
the race is more subtle. See the first link for details.

* add a new tp_status TP_KERNEL_OWNED to avoid the transactional store
of two fields. But, legacy applications may interpret all non-zero
tp_status as owned by the user. As libpcap does. So this is possible
only opt-in by newer processes. It can be added as an optional mode.

* embed the struct at the tail of pg_vec to avoid extra allocation.
The implementation proved no less complex than a separate field.

The additional locking cost on release adds contention, no different
than scaling on multicore or multiqueue h/w. In practice, below
reproducer nor small packet tcpdump showed a noticeable change in
perf report in cycles spent in spinlock. Where contention is
problematic, packet sockets support mitigation through PACKET_FANOUT.
And we can consider adding opt-in state TP_KERNEL_OWNED.

Easy to reproduce by running multiple netperf or similar TCP_STREAM
flows concurrently with `tcpdump -B 129 -n greater 60000`.

Based on an earlier patchset by Jon Rosen. See links below.

I believe this issue goes back to the introduction of tpacket_rcv,
which predates git history.

Link: https://www.mail-archive.com/netdev@vger.kernel.org/msg237222.html
Suggested-by: Jon Rosen <jrosen@cisco.com>
Signed-off-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jon Rosen <jrosen@cisco.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-04-01 11:01:35 +02:00
arch arm64: smp: fix crash_smp_send_stop() behaviour 2020-03-25 08:25:59 +01:00
block block, bfq: fix overwrite of bfq_group pointer in bfq_find_set_group() 2020-03-25 08:25:49 +01:00
certs
crypto
Documentation modpost: move the namespace field in Module.symvers last 2020-03-25 08:25:55 +01:00
drivers net: mvneta: Fix the case where the last poll did not process all rx 2020-04-01 11:01:34 +02:00
fs futex: Fix inode life-time issue 2020-03-25 08:25:58 +01:00
include net: dsa: tag_8021q: replace dsa_8021q_remove_header with __skb_vlan_pop 2020-04-01 11:01:34 +02:00
init
ipc
kernel futex: Unbreak futex hashing 2020-03-25 08:25:58 +01:00
lib
LICENSES
mm x86/mm: split vmalloc_sync_all() 2020-03-25 08:25:58 +01:00
net net/packet: tpacket_rcv: avoid a producer race condition 2020-04-01 11:01:35 +02:00
samples
scripts modpost: move the namespace field in Module.symvers last 2020-03-25 08:25:55 +01:00
security efi: Only print errors about failing to get certs if EFI vars are found 2020-03-12 13:00:14 +01:00
sound ALSA: hda/realtek: Fix pop noise on ALC225 2020-03-25 08:25:59 +01:00
tools selftests/rseq: Fix out-of-tree compilation 2020-03-21 08:11:49 +01:00
usr kbuild: fix 'No such file or directory' warning when cleaning 2020-03-12 13:00:09 +01:00
virt
.clang-format
.cocciconfig
.get_maintainer.ignore
.gitattributes
.gitignore
.mailmap
COPYING
CREDITS
Kbuild
Kconfig
MAINTAINERS
Makefile Linux 5.4.28 2020-03-25 08:26:00 +01:00
README

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.