Eric Dumazet
d3836f21b0
net: allow skb->head to be a page fragment ...
skb->head is currently allocated from kmalloc(). This is convenient but
has the drawback the data cannot be converted to a page fragment if
needed.
We have three spots were it hurts :
1) GRO aggregation
When a linear skb must be appended to another skb, GRO uses the
frag_list fallback, very inefficient since we keep all struct sk_buff
around. So drivers enabling GRO but delivering linear skbs to network
stack aren't enabling full GRO power.
2) splice(socket -> pipe).
We must copy the linear part to a page fragment.
This kind of defeats splice() purpose (zero copy claim)
3) TCP coalescing.
Recently introduced, this permits to group several contiguous segments
into a single skb. This shortens queue lengths and save kernel memory,
and greatly reduce probabilities of TCP collapses. This coalescing
doesnt work on linear skbs (or we would need to copy data, this would be
too slow)
Given all these issues, the following patch introduces the possibility
of having skb->head be a fragment in itself. We use a new skb flag,
skb->head_frag to carry this information.
build_skb() is changed to accept a frag_size argument. Drivers willing
to provide a page fragment instead of kmalloc() data will set a non zero
value, set to the fragment size.
Then, on situations we need to convert the skb head to a frag in itself,
we can check if skb->head_frag is set and avoid the copies or various
fallbacks we have.
This means drivers currently using frags could be updated to avoid the
current skb->head allocation and reduce their memory footprint (aka skb
truesize). (thats 512 or 1024 bytes saved per skb). This also makes
bpf/netfilter faster since the 'first frag' will be part of skb linear
part, no need to copy data.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Maciej Żenczykowski <maze@google.com>
Cc: Neal Cardwell <ncardwell@google.com>
Cc: Tom Herbert <therbert@google.com>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Cc: Ben Hutchings <bhutchings@solarflare.com>
Cc: Matt Carlson <mcarlson@broadcom.com>
Cc: Michael Chan <mchan@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2012-04-30 21:35:11 -04:00
Merge branch 'fixes' of git://git.linaro.org/people/rmk/linux-arm
2012-04-15 17:35:19 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client
2012-03-28 10:01:29 -07:00
mxs-dma : move the mxs dma.h to a more common place
2012-03-27 00:37:23 +01:00
mfd: Convert twl6040 to i2c driver, and separate it from twl core
2012-04-16 16:45:34 +02:00
mfd: Fix modular builds of rc5t583 regulator support
2012-04-16 21:27:07 +02:00
net/mlx4_core: set port QoS attributes
2012-04-05 05:08:03 -04:00
mmc: remove MMC bus legacy suspend/resume method
2012-04-20 20:30:19 -04:00
MTD merge for 3.4
2012-03-30 17:31:56 -07:00
net: cleanup unsigned to unsigned int
2012-04-15 12:44:40 -04:00
netfilter: ip6_tables: ip6t_ext_hdr is now static inline
2012-04-09 16:29:34 +02:00
nfsd: include cld.h in the headers_install target
2012-04-16 15:19:12 -04:00
pinctrl: include <linux/bug.h> to prevent compile errors
2012-04-11 09:31:01 +02:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
2012-04-10 14:30:45 -04:00
Add I2C driver for Summit Microelectronics SMB347 Battery Charger.
2012-03-26 20:41:01 +04:00
regulator: Fixes for -rc1
2012-04-04 10:09:30 -07:00
ARM: More device tree support updates
2012-03-28 12:34:33 -07:00
The following text was taken from the original review request:
2012-03-24 10:08:39 -07:00
Merge nfs containerization work from Trond's tree
2012-03-26 11:48:54 -04:00
usb: fixes for v3.4-rc cycle
2012-04-16 08:35:33 -07:00
Merge branch 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux
2012-03-30 16:45:39 -07:00
apple_bl: Add register/unregister functions
2012-03-22 09:31:55 -04:00
The following text was taken from the original review request:
2012-03-24 10:41:37 -07:00
constify path argument of audit_log_d_path()
2012-03-20 21:29:40 -04:00
__register_binfmt() made void
2012-03-20 21:29:46 -04:00
The following text was taken from the original review request:
2012-03-24 10:08:39 -07:00
bitops: introduce for_each_clear_bit()
2012-03-23 16:58:34 -07:00
block: use lockdep_assert_held for queue locking
2012-03-30 12:33:28 +02:00
The following text was taken from the original review request:
2012-03-24 10:41:37 -07:00
cgroup: revert ss_id_lock to spinlock
2012-03-21 17:55:01 -07:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
vmscan: only defer compaction for failed order and higher
2012-03-21 17:54:56 -07:00
Merge branch 'x86-x32-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2012-03-29 18:12:23 -07:00
include/ and checkpatch: prefer __scanf to __attribute__((format(scanf,...)
2012-03-23 16:58:36 -07:00
The following text was taken from the original review request:
2012-03-24 10:41:37 -07:00
The following text was taken from the original review request:
2012-03-24 10:41:37 -07:00
cpuidle: power_usage should be declared signed integer
2012-03-30 03:23:30 -04:00
cpumask: remove old cpu_*_map.
2012-03-29 15:38:31 +10:30
Merge branch 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2012-03-29 14:46:05 -07:00
crc32: bolt on crc32c
2012-03-23 16:58:38 -07:00
The following text was taken from the original review request:
2012-03-24 10:08:39 -07:00
crypto: user - Fix size of netlink dump message
2012-03-29 19:52:48 +08:00
vfs: d_alloc_root() gone
2012-03-20 21:29:37 -04:00
net/dcb: Add an optional max rate attribute
2012-04-05 05:08:04 -04:00
inet: makes syn_ack_timeout mandatory
2012-04-14 15:24:26 -04:00
Disintegrate and delete asm/system.h
2012-03-28 15:58:21 -07:00
debugfs-related mode_t whack-a-mole
2012-03-20 21:29:53 -04:00
Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/benh/powerpc
2012-03-21 18:55:10 -07:00
common: DMA-mapping: add NON-CONSISTENT attribute
2012-03-28 16:36:44 +02:00
Merge branch 'for-linus-3.4' of git://git.linaro.org/people/sumitsemwal/linux-dma-buf
2012-03-28 15:02:41 -07:00
Merge branch 'for-linus' of git://git.linaro.org/people/mszyprowski/linux-dma-mapping
2012-04-04 17:13:43 -07:00
dmaengine-fixes for 3.4-rc3
2012-04-10 15:30:16 -07:00
Merge branch 'linux_next' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-edac
2012-03-28 14:24:40 -07:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
2012-04-10 14:30:45 -04:00
migrate ext2_fs.h guts to fs/ext2/ext2.h
2012-03-31 16:03:16 -04:00
The following text was taken from the original review request:
2012-03-24 10:41:37 -07:00
vfs: drop_file_write_access() made static
2012-03-20 21:29:32 -04:00
filter: add XOR operation
2012-04-03 18:36:20 -04:00
firewire: restore the device.h include in linux/firewire.h
2012-04-02 14:41:27 -07:00
Merge branch 'akpm' (Andrew's patch-bomb)
2012-04-05 15:30:34 -07:00
fuse: use flexible array in fuse.h
2012-04-12 12:57:08 +02:00
GPIO changes for v3.4
2012-03-28 14:08:46 -07:00
The following text was taken from the original review request:
2012-03-24 10:08:39 -07:00
thp: add HPAGE_PMD_* definitions for !CONFIG_TRANSPARENT_HUGEPAGE
2012-03-21 17:55:02 -07:00
hugetlbfs: fix alignment of huge page requests
2012-03-21 17:54:59 -07:00
net/hyperv: Add flow control based on hi/low watermark
2012-04-03 17:47:15 -04:00
i2c: Update the FSF address
2012-03-26 21:47:19 +02:00
i2c: Update the FSF address
2012-03-26 21:47:19 +02:00
i2c: Update the FSF address
2012-03-26 21:47:19 +02:00
i2c: Update the FSF address
2012-03-26 21:47:19 +02:00
i2c: Update the FSF address
2012-03-26 21:47:19 +02:00
i2c: Update the FSF address
2012-03-26 21:47:19 +02:00
Disintegrate and delete asm/system.h
2012-03-28 15:58:21 -07:00
mac80211: Set the correct values for hwmp (1) and airtimeLinkMetric (1)
2012-04-16 14:19:29 -04:00
eql: dont rely on HZ=100
2012-03-27 22:45:26 -04:00
macvlan: add FDB bridge ops and macvlan flags
2012-04-15 13:06:05 -04:00
macvlan: add FDB bridge ops and macvlan flags
2012-04-15 13:06:05 -04:00
net/l2tp: add support for L2TP over IPv6 UDP
2012-04-28 22:21:51 -04:00
net/l2tp: add support for L2TP over IPv6 UDP
2012-04-28 22:21:51 -04:00
team: allow to enable/disable ports
2012-04-21 16:26:33 -04:00
The following text was taken from the original review request:
2012-03-24 10:08:39 -07:00
cpuset: mm: reduce large amounts of memory barrier related damage v3
2012-03-21 17:54:59 -07:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
net: cleanup unsigned to unsigned int
2012-04-15 12:44:40 -04:00
irq: Add IRQ_TYPE_DEFAULT for use by PIC drivers
2012-04-23 11:04:29 +10:00
irq_domain: Move irq_virq_count into NOMAP revmap
2012-04-12 00:37:48 -06:00
Merge branch 'for-next' of git://gitorious.org/kernel-hsi/kernel-hsi
2012-04-02 09:50:40 -07:00
kconfig: fix IS_ENABLED to not require all options to be defined
2012-04-12 18:35:58 -07:00
pagemap: export KPF_THP
2012-03-21 17:54:57 -07:00
Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2012-03-31 13:34:04 -07:00
NFS client updates for Linux 3.4
2012-03-23 08:53:47 -07:00
x86,kgdb: Fix DEBUG_RODATA limitation using text_poke()
2012-03-29 17:41:25 -05:00
PM / Sleep: Mitigate race between the freezer and request_firmware()
2012-03-28 23:30:28 +02:00
KVM: unmap pages from the iommu when slots are removed
2012-04-11 22:55:25 -03:00
leds-lm3530: support pwm input mode
2012-03-23 16:58:35 -07:00
drivers/leds/leds-lp5521.c: support led pattern data
2012-03-23 16:58:34 -07:00
Add #includes needed to permit the removal of asm/system.h
2012-03-28 18:30:03 +01:00
backlight: new backlight driver for LP855x devices
2012-03-23 16:58:33 -07:00
lp8727_charger: Correct typos on the comment
2012-03-26 20:40:59 +04:00
lsm_audit: don't specify the audit pre/post callbacks in 'struct common_audit_data'
2012-04-03 09:49:59 -07:00
magic.h: move some FS magic numbers into magic.h
2012-03-23 16:58:31 -07:00
memcg: fix performance of mem_cgroup_begin_update_page_stat()
2012-03-21 17:55:02 -07:00
mm: replace PAGE_MIGRATION with IS_ENABLED(CONFIG_MIGRATION)
2012-03-21 17:54:57 -07:00
kill mm argument of vm_munmap()
2012-04-21 01:58:20 -04:00
vmscan: only defer compaction for failed order and higher
2012-03-21 17:54:56 -07:00
remoteproc/rpmsg: new subsystem
2012-03-27 16:30:09 -07:00
module: move __module_get and try_module_get() out of line.
2012-03-26 12:50:52 +10:30
params: <level>_initcall-like kernel parameters
2012-03-26 12:50:51 +10:30
[SCSI] st: expand ability to write immediate filemarks
2012-03-27 08:26:34 +01:00
net: add generic PF_BRIDGE:RTM_ FDB hooks
2012-04-15 13:06:04 -04:00
net: addr_list: add exclusive dev_uc_add and dev_mc_add
2012-04-15 13:06:04 -04:00
net: Delete all remaining instances of ctl_path
2012-04-20 21:22:30 -04:00
NFC: Add a target lost netlink event
2012-04-12 15:10:33 -04:00
Merge nfs containerization work from Trond's tree
2012-03-26 11:48:54 -04:00
NFS: Fix more NFS debug related build warnings
2012-03-21 09:31:44 -04:00
NFS: Remove nfs4_setup_sequence from generic rename code
2012-03-21 09:31:46 -04:00
cfg80211: add channel switch notify event
2012-04-11 16:23:59 -04:00
nmi watchdog: do not use cpp symbol in Kconfig
2012-03-23 16:58:31 -07:00
Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/benh/powerpc
2012-03-28 14:41:36 -07:00
The following text was taken from the original review request:
2012-03-24 10:41:37 -07:00
mm, oom: force oom kill on sysrq+f
2012-03-21 17:54:58 -07:00
The following text was taken from the original review request:
2012-03-24 10:08:39 -07:00
memcg: remove PCG_FILE_MAPPED
2012-03-21 17:55:01 -07:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
Merge branch 'linux-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci
2012-03-23 14:02:12 -07:00
perf: Move mmap page data_head offset assertion out of header
2012-03-24 08:46:59 +01:00
ethtool: Introduce a method for getting time stamping capabilities.
2012-04-04 05:28:45 -04:00
pidns: add reboot_pid_ns() to handle the reboot syscall
2012-03-28 17:14:36 -07:00
magic.h: move some FS magic numbers into magic.h
2012-03-23 16:58:31 -07:00
The following text was taken from the original review request:
2012-03-24 10:41:37 -07:00
PM / QoS: add pm_qos_update_request_timeout() API
2012-03-28 23:31:24 +02:00
poll: add poll_requested_events() and poll_does_not_wait() functions
2012-03-23 16:58:38 -07:00
prctl: add PR_{SET,GET}_CHILD_SUBREAPER to allow simple process supervision
2012-03-23 16:58:32 -07:00
ptp: Add a method for obtaining the device index.
2012-04-04 05:28:45 -04:00
The following text was taken from the original review request:
2012-03-24 10:08:39 -07:00
fs: initial qnx6fs addition
2012-03-20 21:29:38 -04:00
radix-tree: introduce bit-optimized iterator
2012-03-28 17:14:37 -07:00
The following text was taken from the original review request:
2012-03-24 10:08:39 -07:00
The following text was taken from the original review request:
2012-03-24 10:41:37 -07:00
The following text was taken from the original review request:
2012-03-24 10:08:39 -07:00
move private bits of reiserfs_fs.h to fs/reiserfs/reiserfs.h
2012-03-20 21:29:43 -04:00
take private bits of reiserfs_xattr.h to fs/reiserfs/xattr.h
2012-03-20 21:29:42 -04:00
rmap: remove __anon_vma_link() declaration
2012-03-21 17:54:57 -07:00
net: add generic PF_BRIDGE:RTM_ FDB hooks
2012-04-15 13:06:04 -04:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
Disintegrate and delete asm/system.h
2012-03-28 15:58:21 -07:00
Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2012-03-21 13:25:04 -07:00
The following text was taken from the original review request:
2012-03-24 10:08:39 -07:00
serial/8250_pci: add a "force background timer" flag and use it for the "kt" serial port
2012-04-09 10:38:30 -07:00
net: allow skb->head to be a page fragment
2012-04-30 21:35:11 -04:00
Merge branch 'slab/for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/linux
2012-03-28 15:04:26 -07:00
smp: add func to IPI cpus based on parameter func
2012-03-28 17:14:35 -07:00
net: sock_diag_handler structs can be const
2012-04-25 20:46:59 -04:00
net: cleanup unsigned to unsigned int
2012-04-15 12:44:40 -04:00
Disintegrate and delete asm/system.h
2012-03-28 15:58:21 -07:00
locking/kconfig: Simplify INLINE_SPIN_UNLOCK usage
2012-03-23 13:18:57 +01:00
do not export kernel's NULL #define to userspace
2012-04-14 07:47:49 -07:00
stmmac: use custom init/exit functions in pm ops
2012-04-19 20:08:44 -04:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
memcg swap: use mem_cgroup_uncharge_swap fix
2012-04-05 15:25:51 -07:00
tcp repair: Fix unaligned access when repairing options (v2)
2012-04-26 06:13:51 -04:00
Merge branch 'x86-x32-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2012-03-29 18:12:23 -07:00
ntp: Fix leap-second hrtimer livelock
2012-03-22 19:43:43 -07:00
constify path argument of trace_seq_path()
2012-03-20 21:29:40 -04:00
ptrace: the killed tracee should not enter the syscall
2012-03-23 16:58:40 -07:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
irq: Kill pointless irqd_to_hw export
2012-04-10 22:39:17 -06:00
vgaarb.h: fix build warnings
2012-04-11 09:36:00 +01:00
virtio: drop thaw PM operation
2012-03-31 08:09:50 +05:30
remoteproc/rpmsg: new subsystem
2012-03-27 16:30:09 -07:00
virtio-net: send gratuitous packets when needed
2012-04-15 03:23:31 -04:00
Remove all #inclusions of asm/system.h
2012-03-28 18:30:03 +01:00
watchdog: Add support for WDIOC_GETTIMELEFT IOCTL in watchdog core
2012-03-27 20:15:37 +02:00
