Commit graph

633,587 commits

Author SHA1 Message Date
Neal Cardwell
556c6b46d1 tcp: export tcp_mss_to_mtu() for congestion control modules
Export tcp_mss_to_mtu(), so that congestion control modules can use
this to help calculate a pacing rate.

Signed-off-by: Van Jacobson <vanj@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Nandita Dukkipati <nanditad@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:23:01 -04:00
Neal Cardwell
1b3878ca15 tcp: export tcp_tso_autosize() and parameterize minimum number of TSO segments
To allow congestion control modules to use the default TSO auto-sizing
algorithm as one of the ingredients in their own decision about TSO sizing:

1) Export tcp_tso_autosize() so that CC modules can use it.

2) Change tcp_tso_autosize() to allow callers to specify a minimum
   number of segments per TSO skb, in case the congestion control
   module has a different notion of the best floor for TSO skbs for
   the connection right now. For very low-rate paths or policed
   connections it can be appropriate to use smaller TSO skbs.

Signed-off-by: Van Jacobson <vanj@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Nandita Dukkipati <nanditad@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:23:00 -04:00
Neal Cardwell
ed6e7268b9 tcp: allow congestion control module to request TSO skb segment count
Add the tso_segs_goal() function in tcp_congestion_ops to allow the
congestion control module to specify the number of segments that
should be in a TSO skb sent by tcp_write_xmit() and
tcp_xmit_retransmit_queue(). The congestion control module can either
request a particular number of segments in TSO skb that we transmit,
or return 0 if it doesn't care.

This allows the upcoming BBR congestion control module to select small
TSO skb sizes if the module detects that the bottleneck bandwidth is
very low, or that the connection is policed to a low rate.

Signed-off-by: Van Jacobson <vanj@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Nandita Dukkipati <nanditad@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:23:00 -04:00
Yuchung Cheng
eb8329e0a0 tcp: export data delivery rate
This commit export two new fields in struct tcp_info:

  tcpi_delivery_rate: The most recent goodput, as measured by
    tcp_rate_gen(). If the socket is limited by the sending
    application (e.g., no data to send), it reports the highest
    measurement instead of the most recent. The unit is bytes per
    second (like other rate fields in tcp_info).

  tcpi_delivery_rate_app_limited: A boolean indicating if the goodput
    was measured when the socket's throughput was limited by the
    sending application.

This delivery rate information can be useful for applications that
want to know the current throughput the TCP connection is seeing,
e.g. adaptive bitrate video streaming. It can also be very useful for
debugging or troubleshooting.

Signed-off-by: Van Jacobson <vanj@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Nandita Dukkipati <nanditad@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:23:00 -04:00
Soheil Hassas Yeganeh
d7722e8570 tcp: track application-limited rate samples
This commit adds code to track whether the delivery rate represented
by each rate_sample was limited by the application.

Upon each transmit, we store in the is_app_limited field in the skb a
boolean bit indicating whether there is a known "bubble in the pipe":
a point in the rate sample interval where the sender was
application-limited, and did not transmit even though the cwnd and
pacing rate allowed it.

This logic marks the flow app-limited on a write if *all* of the
following are true:

  1) There is less than 1 MSS of unsent data in the write queue
     available to transmit.

  2) There is no packet in the sender's queues (e.g. in fq or the NIC
     tx queue).

  3) The connection is not limited by cwnd.

  4) There are no lost packets to retransmit.

The tcp_rate_check_app_limited() code in tcp_rate.c determines whether
the connection is application-limited at the moment. If the flow is
application-limited, it sets the tp->app_limited field. If the flow is
application-limited then that means there is effectively a "bubble" of
silence in the pipe now, and this silence will be reflected in a lower
bandwidth sample for any rate samples from now until we get an ACK
indicating this bubble has exited the pipe: specifically, until we get
an ACK for the next packet we transmit.

When we send every skb we record in scb->tx.is_app_limited whether the
resulting rate sample will be application-limited.

The code in tcp_rate_gen() checks to see when it is safe to mark all
known application-limited bubbles of silence as having exited the
pipe. It does this by checking to see when the delivered count moves
past the tp->app_limited marker. At this point it zeroes the
tp->app_limited marker, as all known bubbles are out of the pipe.

We make room for the tx.is_app_limited bit in the skb by borrowing a
bit from the in_flight field used by NV to record the number of bytes
in flight. The receive window in the TCP header is 16 bits, and the
max receive window scaling shift factor is 14 (RFC 1323). So the max
receive window offered by the TCP protocol is 2^(16+14) = 2^30. So we
only need 30 bits for the tx.in_flight used by NV.

Signed-off-by: Van Jacobson <vanj@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Nandita Dukkipati <nanditad@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:23:00 -04:00
Yuchung Cheng
b9f64820fb tcp: track data delivery rate for a TCP connection
This patch generates data delivery rate (throughput) samples on a
per-ACK basis. These rate samples can be used by congestion control
modules, and specifically will be used by TCP BBR in later patches in
this series.

Key state:

tp->delivered: Tracks the total number of data packets (original or not)
	       delivered so far. This is an already-existing field.

tp->delivered_mstamp: the last time tp->delivered was updated.

Algorithm:

A rate sample is calculated as (d1 - d0)/(t1 - t0) on a per-ACK basis:

  d1: the current tp->delivered after processing the ACK
  t1: the current time after processing the ACK

  d0: the prior tp->delivered when the acked skb was transmitted
  t0: the prior tp->delivered_mstamp when the acked skb was transmitted

When an skb is transmitted, we snapshot d0 and t0 in its control
block in tcp_rate_skb_sent().

When an ACK arrives, it may SACK and ACK some skbs. For each SACKed
or ACKed skb, tcp_rate_skb_delivered() updates the rate_sample struct
to reflect the latest (d0, t0).

Finally, tcp_rate_gen() generates a rate sample by storing
(d1 - d0) in rs->delivered and (t1 - t0) in rs->interval_us.

One caveat: if an skb was sent with no packets in flight, then
tp->delivered_mstamp may be either invalid (if the connection is
starting) or outdated (if the connection was idle). In that case,
we'll re-stamp tp->delivered_mstamp.

At first glance it seems t0 should always be the time when an skb was
transmitted, but actually this could over-estimate the rate due to
phase mismatch between transmit and ACK events. To track the delivery
rate, we ensure that if packets are in flight then t0 and and t1 are
times at which packets were marked delivered.

If the initial and final RTTs are different then one may be corrupted
by some sort of noise. The noise we see most often is sending gaps
caused by delayed, compressed, or stretched acks. This either affects
both RTTs equally or artificially reduces the final RTT. We approach
this by recording the info we need to compute the initial RTT
(duration of the "send phase" of the window) when we recorded the
associated inflight. Then, for a filter to avoid bandwidth
overestimates, we generalize the per-sample bandwidth computation
from:

    bw = delivered / ack_phase_rtt

to the following:

    bw = delivered / max(send_phase_rtt, ack_phase_rtt)

In large-scale experiments, this filtering approach incorporating
send_phase_rtt is effective at avoiding bandwidth overestimates due to
ACK compression or stretched ACKs.

Signed-off-by: Van Jacobson <vanj@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Nandita Dukkipati <nanditad@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:23:00 -04:00
Neal Cardwell
0682e6902a tcp: count packets marked lost for a TCP connection
Count the number of packets that a TCP connection marks lost.

Congestion control modules can use this loss rate information for more
intelligent decisions about how fast to send.

Specifically, this is used in TCP BBR policer detection. BBR uses a
high packet loss rate as one signal in its policer detection and
policer bandwidth estimation algorithm.

The BBR policer detection algorithm cannot simply track retransmits,
because a retransmit can be (and often is) an indicator of packets
lost long, long ago. This is particularly true in a long CA_Loss
period that repairs the initial massive losses when a policer kicks
in.

Signed-off-by: Van Jacobson <vanj@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Nandita Dukkipati <nanditad@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:23:00 -04:00
Eric Dumazet
b2d3ea4a73 tcp: switch back to proper tcp_skb_cb size check in tcp_init()
Revert to the tcp_skb_cb size check that tcp_init() had before commit
b4772ef879 ("net: use common macro for assering skb->cb[] available
size in protocol families"). As related commit 744d5a3e9f ("net:
move skb->dropcount to skb->cb[]") explains, the
sock_skb_cb_check_size() mechanism was added to ensure that there is
space for dropcount, "for protocol families using it". But TCP is not
a protocol using dropcount, so tcp_init() doesn't need to provision
space for dropcount in the skb->cb[], and thus we can revert to the
older form of the tcp_skb_cb size check. Doing so allows TCP to use 4
more bytes of the skb->cb[] space.

Fixes: b4772ef879 ("net: use common macro for assering skb->cb[] available size in protocol families")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:23:00 -04:00
Eric Dumazet
77879147a3 net_sched: sch_fq: add low_rate_threshold parameter
This commit adds to the fq module a low_rate_threshold parameter to
insert a delay after all packets if the socket requests a pacing rate
below the threshold.

This helps achieve more precise control of the sending rate with
low-rate paths, especially policers. The basic issue is that if a
congestion control module detects a policer at a certain rate, it may
want fq to be able to shape to that policed rate. That way the sender
can avoid policer drops by having the packets arrive at the policer at
or just under the policed rate.

The default threshold of 550Kbps was chosen analytically so that for
policers or links at 500Kbps or 512Kbps fq would very likely invoke
this mechanism, even if the pacing rate was briefly slightly above the
available bandwidth. This value was then empirically validated with
two years of production testing on YouTube video servers.

Signed-off-by: Van Jacobson <vanj@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Nandita Dukkipati <nanditad@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:23:00 -04:00
Neal Cardwell
6403389211 tcp: use windowed min filter library for TCP min_rtt estimation
Refactor the TCP min_rtt code to reuse the new win_minmax library in
lib/win_minmax.c to simplify the TCP code.

This is a pure refactor: the functionality is exactly the same. We
just moved the windowed min code to make TCP easier to read and
maintain, and to allow other parts of the kernel to use the windowed
min/max filter code.

Signed-off-by: Van Jacobson <vanj@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Nandita Dukkipati <nanditad@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:22:59 -04:00
Neal Cardwell
a4f1f9ac81 lib/win_minmax: windowed min or max estimator
This commit introduces a generic library to estimate either the min or
max value of a time-varying variable over a recent time window. This
is code originally from Kathleen Nichols. The current form of the code
is from Van Jacobson.

A single struct minmax_sample will track the estimated windowed-max
value of the series if you call minmax_running_max() or the estimated
windowed-min value of the series if you call minmax_running_min().

Nearly equivalent code is already in place for minimum RTT estimation
in the TCP stack. This commit extracts that code and generalizes it to
handle both min and max. Moving the code here reduces the footprint
and complexity of the TCP code base and makes the filter generally
available for other parts of the codebase, including an upcoming TCP
congestion control module.

This library works well for time series where the measurements are
smoothly increasing or decreasing.

Signed-off-by: Van Jacobson <vanj@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Nandita Dukkipati <nanditad@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:22:59 -04:00
Soheil Hassas Yeganeh
f78e73e27f tcp: cdg: rename struct minmax in tcp_cdg.c to avoid a naming conflict
The upcoming change "lib/win_minmax: windowed min or max estimator"
introduces a struct called minmax, which is then included in
include/linux/tcp.h in the upcoming change "tcp: use windowed min
filter library for TCP min_rtt estimation". This would create a
compilation error for tcp_cdg.c, which defines its own minmax
struct. To avoid this naming conflict (and potentially others in the
future), this commit renames the version used in tcp_cdg.c to
cdg_minmax.

Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Kenneth Klette Jonassen <kennetkl@ifi.uio.no>
Acked-by: Kenneth Klette Jonassen <kennetkl@ifi.uio.no>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:22:59 -04:00
Sean Wang
94d308d060 net: ethernet: mediatek: enhance with avoiding superfluous assignment inside mtk_get_ethtool_stats
data_src is unchanged inside the loop, so this patch moves
the assignment to outside the loop to avoid unnecessarily
assignment

Signed-off-by: Sean Wang <sean.wang@mediatek.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:06:32 -04:00
Vivien Didelot
8847293992 net: dsa: mv88e6xxx: handle multiple ports in ATU
An address can be loaded in the ATU with multiple ports, for instance
when adding multiple ports to a Multicast group with "bridge mdb".

The current code doesn't allow that. Add an helper to get a single entry
from the ATU, then set or clear the requested port, before loading the
entry back in the ATU.

Note that the required _mv88e6xxx_atu_getnext function is defined below
mv88e6xxx_port_db_load_purge, so forward-declare it for the moment. The
ATU code will be isolated in future patches.

Fixes: 83dabd1fa8 ("net: dsa: mv88e6xxx: make switchdev DB ops generic")
Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-21 00:04:50 -04:00
Russell Currey
b79331a5eb powerpc/powernv/pci: Fix m64 checks for SR-IOV and window alignment
Commit 5958d19a14 checks for prefetchable m64 BARs by comparing the
addresses instead of using resource flags.  This broke SR-IOV as the m64
check in pnv_pci_ioda_fixup_iov_resources() fails.

The condition in pnv_pci_window_alignment() also changed to checking
only IORESOURCE_MEM_64 instead of both IORESOURCE_MEM_64 and
IORESOURCE_PREFETCH.

Revert these cases to the previous behaviour, adding a new helper function
to do so.  This is named pnv_pci_is_m64_flags() to make it clear this
function is only looking at resource flags and should not be relied on for
non-SRIOV resources.

Fixes: 5958d19a14 ("Fix incorrect PE reservation attempt on some 64-bit BARs")
Reported-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Russell Currey <ruscur@russell.cc>
Tested-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2016-09-21 14:04:13 +10:00
Max Filippov
5b8b3c860b xtensa: add default memmap and mmio32native options to defconfigs
Now that memory initialization doesn't add default memory region
specify it explicitly in the memmap command line option in case somebody
wants to boot in non-DT-enabled configuration.

While at it update earlycon access mode to mmio32native to support both
LE and BE cores transparently.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2016-09-20 20:43:24 -07:00
Max Filippov
6edbbd1675 xtensa: add default memmap option to common_defconfig
Now that memory initialization doesn't add default memory region specify
it explicitly in the memmap command line option.

Save common_defconfig as defconfig so that it doesn't have all option
settings in it, only those that are different from the Kconfig defaults.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2016-09-20 20:43:23 -07:00
Max Filippov
81da313f2c xtensa: add default memmap option to iss_defconfig
Now that memory initialization doesn't add default memory region specify
it explicitly in the memmap command line option.

Save iss_defconfig as defconfig so that it doesn't have all option
settings in it, only those that are different from the Kconfig defaults.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2016-09-20 20:43:23 -07:00
Max Filippov
549409b4b5 xtensa: ISS: allow simdisk to use high memory buffers
ISS kernel by default has only low memory. But it may be configured to
support high memory and started in a simulator with more than 128M of
RAM. Simdisk driver in such configuration can get IO request with a
high memory page. There may be no TLB entry for that page, only page
table entry. However simulators don't do pagewalking, so such IO request
will fail. Touch IO buffer in the buffer read/write loop so that a TLB
entry is likely there when read or write simcall is invoked.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2016-09-20 20:43:22 -07:00
Jamal Hadi Salim
aecc5cefc3 net sched actions: fix GETing actions
With the batch changes that translated transient actions into
a temporary list lost in the translation was the fact that
tcf_action_destroy() will eventually delete the action from
the permanent location if the refcount is zero.

Example of what broke:
...add a gact action to drop
sudo $TC actions add action drop index 10
...now retrieve it, looks good
sudo $TC actions get action gact index 10
...retrieve it again and find it is gone!
sudo $TC actions get action gact index 10

Fixes: 22dc13c837 ("net_sched: convert tcf_exts from list to pointer array"),
Fixes: 824a7e8863 ("net_sched: remove an unnecessary list_del()")
Fixes: f07fed82ad ("net_sched: remove the leftover cleanup_a()")

Acked-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 23:34:55 -04:00
David S. Miller
1d9423ae0d Merge branch 'bpf-direct-packet-access-improvements'
Daniel Borkmann says:

====================
BPF direct packet access improvements

This set adds write support to the currently available read support
for {cls,act}_bpf programs. First one is a fix for affected commit
sitting in net-next and prerequisite for the second one, last patch
adds a number of test cases against the verifier. For details, please
see individual patches.
====================

Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 23:32:19 -04:00
Daniel Borkmann
7d95b0ab5b bpf: add test cases for direct packet access
Add couple of test cases for direct write and the negative size issue, and
also adjust the direct packet access test4 since it asserts that writes are
not possible, but since we've just added support for writes, we need to
invert the verdict to ACCEPT, of course. Summary: 133 PASSED, 0 FAILED.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 23:32:11 -04:00
Daniel Borkmann
36bbef52c7 bpf: direct packet write and access for helpers for clsact progs
This work implements direct packet access for helpers and direct packet
write in a similar fashion as already available for XDP types via commits
4acf6c0b84 ("bpf: enable direct packet data write for xdp progs") and
6841de8b0d ("bpf: allow helpers access the packet directly"), and as a
complementary feature to the already available direct packet read for tc
(cls/act) programs.

For enabling this, we need to introduce two helpers, bpf_skb_pull_data()
and bpf_csum_update(). The first is generally needed for both, read and
write, because they would otherwise only be limited to the current linear
skb head. Usually, when the data_end test fails, programs just bail out,
or, in the direct read case, use bpf_skb_load_bytes() as an alternative
to overcome this limitation. If such data sits in non-linear parts, we
can just pull them in once with the new helper, retest and eventually
access them.

At the same time, this also makes sure the skb is uncloned, which is, of
course, a necessary condition for direct write. As this needs to be an
invariant for the write part only, the verifier detects writes and adds
a prologue that is calling bpf_skb_pull_data() to effectively unclone the
skb from the very beginning in case it is indeed cloned. The heuristic
makes use of a similar trick that was done in 233577a220 ("net: filter:
constify detection of pkt_type_offset"). This comes at zero cost for other
programs that do not use the direct write feature. Should a program use
this feature only sparsely and has read access for the most parts with,
for example, drop return codes, then such write action can be delegated
to a tail called program for mitigating this cost of potential uncloning
to a late point in time where it would have been paid similarly with the
bpf_skb_store_bytes() as well. Advantage of direct write is that the
writes are inlined whereas the helper cannot make any length assumptions
and thus needs to generate a call to memcpy() also for small sizes, as well
as cost of helper call itself with sanity checks are avoided. Plus, when
direct read is already used, we don't need to cache or perform rechecks
on the data boundaries (due to verifier invalidating previous checks for
helpers that change skb->data), so more complex programs using rewrites
can benefit from switching to direct read plus write.

For direct packet access to helpers, we save the otherwise needed copy into
a temp struct sitting on stack memory when use-case allows. Both facilities
are enabled via may_access_direct_pkt_data() in verifier. For now, we limit
this to map helpers and csum_diff, and can successively enable other helpers
where we find it makes sense. Helpers that definitely cannot be allowed for
this are those part of bpf_helper_changes_skb_data() since they can change
underlying data, and those that write into memory as this could happen for
packet typed args when still cloned. bpf_csum_update() helper accommodates
for the fact that we need to fixup checksum_complete when using direct write
instead of bpf_skb_store_bytes(), meaning the programs can use available
helpers like bpf_csum_diff(), and implement csum_add(), csum_sub(),
csum_block_add(), csum_block_sub() equivalents in eBPF together with the
new helper. A usage example will be provided for iproute2's examples/bpf/
directory.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 23:32:11 -04:00
Daniel Borkmann
b399cf64e3 bpf, verifier: enforce larger zero range for pkt on overloading stack buffs
Current contract for the following two helper argument types is:

  * ARG_CONST_STACK_SIZE: passed argument pair must be (ptr, >0).
  * ARG_CONST_STACK_SIZE_OR_ZERO: passed argument pair can be either
    (NULL, 0) or (ptr, >0).

With 6841de8b0d ("bpf: allow helpers access the packet directly"), we can
pass also raw packet data to helpers, so depending on the argument type
being PTR_TO_PACKET, we now either assert memory via check_packet_access()
or check_stack_boundary(). As a result, the tests in check_packet_access()
currently allow more than intended with regards to reg->imm.

Back in 969bf05eb3 ("bpf: direct packet access"), check_packet_access()
was fine to ignore size argument since in check_mem_access() size was
bpf_size_to_bytes() derived and prior to the call to check_packet_access()
guaranteed to be larger than zero.

However, for the above two argument types, it currently means, we can have
a <= 0 size and thus breaking current guarantees for helpers. Enforce a
check for size <= 0 and bail out if so.

check_stack_boundary() doesn't have such an issue since it already tests
for access_size <= 0 and bails out, resp. access_size == 0 in case of NULL
pointer passed when allowed.

Fixes: 6841de8b0d ("bpf: allow helpers access the packet directly")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 23:32:11 -04:00
Mahesh Bandewar
cf714ac147 ipvlan: Fix dependency issue
kbuild-build-bot reported that if NETFILTER is not selected, the
build fails pointing to netfilter symbols.

Fixes: 4fbae7d83c ("ipvlan: Introduce l3s mode")

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 22:55:23 -04:00
pravin shelar
2279994d07 openvswitch: avoid resetting flow key while installing new flow.
since commit commit db74a3335e ("openvswitch: use percpu
flow stats") flow alloc resets flow-key. So there is no need
to reset the flow-key again if OVS is using newly allocated
flow-key.

Signed-off-by: Pravin B Shelar <pshelar@ovn.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 22:54:35 -04:00
pravin shelar
190aa3e778 openvswitch: Fix Frame-size larger than 1024 bytes warning.
There is no need to declare separate key on stack,
we can just use sw_flow->key to store the key directly.

This commit fixes following warning:

net/openvswitch/datapath.c: In function ‘ovs_flow_cmd_new’:
net/openvswitch/datapath.c:1080:1: warning: the frame size of 1040 bytes
is larger than 1024 bytes [-Wframe-larger-than=]

Signed-off-by: Pravin B Shelar <pshelar@ovn.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 22:54:35 -04:00
David S. Miller
204dfe1798 Merge branch 'for-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/bluetooth/bluetooth-next
Johan Hedberg says:

====================
pull request: bluetooth-next 2016-09-19

Here's the main bluetooth-next pull request for the 4.9 kernel.

 - Added new messages for monitor sockets for better mgmt tracing
 - Added local name and appearance support in scan response
 - Added new Qualcomm WCNSS SMD based HCI driver
 - Minor fixes & cleanup to 802.15.4 code
 - New USB ID to btusb driver
 - Added Marvell support to HCI UART driver
 - Add combined LED trigger for controller power
 - Other minor fixes here and there

Please let me know if there are any issues pulling. Thanks.
====================

Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 22:52:50 -04:00
Alan Cox
ad9798967d 6pack: fix buffer length mishandling
Dmitry Vyukov wrote:
> different runs). Looking at code, the following looks suspicious -- we
> limit copy by 512 bytes, but use the original count which can be
> larger than 512:
>
> static void sixpack_receive_buf(struct tty_struct *tty,
>     const unsigned char *cp, char *fp, int count)
> {
>     unsigned char buf[512];
>     ....
>     memcpy(buf, cp, count < sizeof(buf) ? count : sizeof(buf));
>     ....
>     sixpack_decode(sp, buf, count1);

With the sane tty locking we now have I believe the following is safe as
we consume the bytes and move them into the decoded buffer before
returning.

Signed-off-by: Alan Cox <alan@linux.intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 22:51:30 -04:00
David S. Miller
ceb16a9013 linux-can-fixes-for-4.8-20160919
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABCgAGBQJX3/GcAAoJED07qiWsqSVq9PgH/jucrHh2H7IAnJi6ZasZmu1t
 +beOkzg954rJfz2cGulhpNBAl9xJYRhPuJ+Jw5JhDEPXVisgeRVm6lT+8YJaCsWL
 utTotzyRYI3HUrXIl6V2rjn1ms7TAlZsHrAGPj1ufJKEZg8/SF5C15NuY9YTlh4d
 JZT0V13Yxj0CV3d2jqZshFXrF+CiAsGyQF1aGA9Wo2xQeZIt2vY/1DfNtC9eXlXW
 +t7tBCgLka7cwTBC513BKjl9fGwzYO0eYU993N9+Ijir6HLM9N3igZwxjH1hEl/S
 ow6n5T1O89JEnghSN5Xq+2ThkkXR7zZwi+6l74eFICkIwHK/pekZ8dPr9Ct9ZZ0=
 =JcjG
 -----END PGP SIGNATURE-----

Merge tag 'linux-can-fixes-for-4.8-20160919' of git://git.kernel.org/pub/scm/linux/kernel/git/mkl/linux-can

Marc Kleine-Budde says:

====================
pull-request: can 2016-09-19

this is a pull request of one patch for the upcoming linux-4.8 release.

The patch by Fabio Estevam fixes the pm handling in the flexcan driver.
====================

Signed-off-by: David S. Miller <davem@davemloft.net>
2016-09-20 22:46:14 -04:00
James Morris
8a17ef9d85 Merge branch 'stable-4.9' of git://git.infradead.org/users/pcmoore/selinux into next 2016-09-21 11:54:19 +10:00
Max Filippov
feec273a2b xtensa: ISS: define simc_exit and use it instead of inline asm
A number of ISS platform functions use inline assembly to invoke
simulator exit, not all correctly. Define simc_exit(exit_code) and use
it instead of inline assembly.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2016-09-20 18:53:00 -07:00
Max Filippov
70feca7199 xtensa: xtfpga: group platform_* functions together
Group platform_* functions together and turn two separate #ifdef/#ifndef
blocks into single #ifdef/#else. No functional changes.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2016-09-20 18:52:59 -07:00
Max Filippov
205ad548a7 xtensa: rearrange CCOUNT calibration
DT-enabled kernel should have a CPU node connected to a clock. This clock
is the CCOUNT clock. Use old platform_calibrate_ccount call as a fallback
when CPU node cannot be found or has no clock and in non-DT-enabled
configurations.

Drop no longer needed code that updates CPU clock-frequency property in
the DT; drop DT-related code from the platform_calibrate_ccount too.

Move of_clk_init to the top of time_init, so that clocks are initialized
before CCOUNT calibration is attempted.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2016-09-20 18:52:59 -07:00
Max Filippov
58c3e3ac7a xtensa: xtfpga: use clock provider, don't update DT
Instead of querying hardcoded FPGA frequency register and then updating
clock-frequency property in specificly named DT nodes in machine setup
code register a clock provider that returns fixed-rate clock, configured
by register specified in DT. This way we have less magic/hardcoded names
and use more existing common clock framework code.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
2016-09-20 18:52:51 -07:00
Mauro Carvalho Chehab
cc68fd957f Documentation/kernel-docs.txt: reorder based on timestamp
Reorder the on-line documents based on their timestamp or
copyright notes. More updated documents come first.

While here, add the number of pages for POSIX4 document.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-09-20 18:54:42 -06:00
Mauro Carvalho Chehab
57b2e1c831 Documentation/kernel-docs.txt: Add dates for online docs
It is a way better to have a timestamp to help identifying
when something is too old.

So, retrieve the dates marked on the existing documents.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-09-20 18:54:11 -06:00
Mauro Carvalho Chehab
98cadc165d Documentation/kernel-docs.txt: get rid of broken docs
There are still some broken docs: the URLs point to somewhere,
however, the texts are not there anymore. I was able to
find the texts on other URLs for some of those, but they're all
too old. So, just get rid of them.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-20 18:53:47 -06:00
Mauro Carvalho Chehab
608ee2ff56 Documentation/kernel-docs.txt: move in-kernel docs
There are three places where it mentions in-kernel docs.

Move them to a separate topic.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-20 18:53:43 -06:00
Mauro Carvalho Chehab
7b5f2bd74c Documentation/kernel-docs.txt: remove more legacy references
The Linux Kernel - This book is for Kernel 2.0.33

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-20 18:53:39 -06:00
Mauro Carvalho Chehab
8a24bd1a69 Documentation/kernel-docs.txt: add two published books
Add two books from my own bookshelf. I found them useful by
the time I bought; so it could be useful to others ;)

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-20 18:53:35 -06:00
Mauro Carvalho Chehab
cefd1f725b Documentation/kernel-docs.txt: sort books per publication date
Instead of using a random order, place the books on publication
date, from the newest to the oldest.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-09-20 18:53:26 -06:00
Mauro Carvalho Chehab
d8b7165f2c Documentation/kernel-docs.txt: adjust LDD references
- remove LDD versions 1 and 2, as there's already an entry for
  LDD3;
- add a link between LDD online and published entries.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-20 18:53:01 -06:00
Mauro Carvalho Chehab
be948b6579 Documentation/kernel-docs.txt: some improvements on the ReST output
- Use lower case for sections, as this is the standard used on
  the other ReST files;
- The latest version of this document is at the Kernel source, and
  not at the listed URL. So, move it to the end of the doc.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-20 18:52:54 -06:00
Richard Sailer
a8332a07af Documentation/kernel-docs.txt: Consistent indenting: 4 spaces
This introduces a consistent indenting of 4 spaces for all
lists.

[mchehab@s-opensource.com: rebased to apply before rename]

Signed-off-by: Richard Sailer <richard@weltraumpflege.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-09-20 18:52:39 -06:00
Richard Sailer
c3e84d1ce5 Documentation/kernel-docs.txt: Add 4 paper/book references
Background/Reasoning:

Books:
------
 * Linux Kernel Networking by Rami Rosen
   While some parts are quite short and could be
   more carefully explained it's still a good recomendation
   for understanding linux kernel networking, (IMHO)

* Linux Treiber entwickeln:
  It sure is a drawback that this is a german book.
  But it's quite recent, well structured and there are also
  other non-english (spanish) books/papers in this list.

Papers:
-------

  * On Submitting kernel Patches
    Contains 2 case studies of bigger patch sets and how (or how not)
    they were merged. I found it helpful

  * Tracing the Way of Data in a TCP Connection through the Linux Kernel
    Since this was written by me this inclusion may be a bit biased :p
    Neitherless I think this gives a good introduction on
    understanding/exploring linux internals using ftrace and an overview
    of Linux TCP internals.

[mchehab@s-opensource.com: rebased to apply before rename]

Signed-off-by: Richard Sailer <richard@weltraumpflege.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-09-20 18:52:06 -06:00
Richard Sailer
83d4d3c976 Documentation/kernel-docs.txt: Improve layouting of book list
The dots at the ends of the list elements introduced
unnecesarry newlines in the "compiled" document.

While this was not "mission critical" it's not nice to look at
either.

[mchehab@s-opensource.com: rebased to apply before rename]

Signed-off-by: Richard Sailer <richard@weltraumpflege.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-20 18:51:37 -06:00
Richard Sailer
249ad66404 Documentation/kernel-docs.txt: Remove offline or outdated entries
This removes all dead links to online docs which
are dead according to Jon and Mauro in
https://lkml.kernel.org/r/20160916182849.2a7101ea () vento ! lan

Additionally some references to very old articles refering to
linux 2.2 and 2.0 are deleted.

[mchehab@s-opensource.com: rebased to apply before rename]

Signed-off-by: Richard Sailer <richard@weltraumpflege.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-09-20 18:51:25 -06:00
Jonathan Corbet
1b49ecf2f3 docs: Clean up bare :: lines
Mauro's patch set introduced some bare :: lines; these can be represented
by a double colon at the end of the preceding text line.  The result looks
a little less weird and is less verbose.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-20 18:46:36 -06:00
Mauro Carvalho Chehab
7a71a8095b Documentation/SubmitChecklist: convert it to ReST markup
- use ``foo`` to markup inline literal stuff, effectively making it
  to be presented as a monospaced font when parsed by Sphinx;

- the markup below the title should have the same length as the
  title;

- Fix the list markups, from "1:" to "1)";

- Split item 2 into a separate list for the build options, in order
  to be presented as a list on Sphinx;

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-20 18:41:50 -06:00