Repository files (latest commit first)

Filename	Latest commit message	Latest commit date
Daniel Borkmann 677fbf13cd UPSTREAM: bpf, net: Rework cookie generator as per-cpu one ... With its use in BPF, the cookie generator can be called very frequently in particular when used out of cgroup v2 hooks (e.g. connect / sendmsg) and attached to the root cgroup, for example, when used in v1/v2 mixed environments. In particular, when there's a high churn on sockets in the system there can be many parallel requests to the bpf_get_socket_cookie() and bpf_get_netns_cookie() helpers which then cause contention on the atomic counter. As similarly done in f991bd2e14 ("fs: introduce a per-cpu last_ino allocator"), add a small helper library that both can use for the 64 bit counters. Given this can be called from different contexts, we also need to deal with potential nested calls even though in practice they are considered extremely rare. One idea as suggested by Eric Dumazet was to use a reverse counter for this situation since we don't expect 64 bit overflows anyways; that way, we can avoid bigger gaps in the 64 bit counter space compared to just batch-wise increase. Even on machines with small number of cores (e.g. 4) the cookie generation shrinks from min/max/med/avg (ns) of 22/50/40/38.9 down to 10/35/14/17.3 when run in parallel from multiple CPUs. Change-Id: Ia5898f5a9f89532101b059cb82183697bfabe770 Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Reviewed-by: Eric Dumazet <edumazet@google.com> Acked-by: Martin KaFai Lau <kafai@fb.com> Cc: Eric Dumazet <eric.dumazet@gmail.com> Link: https://lore.kernel.org/bpf/8a80b8d27d3c49f9a14e1d5213c19d8be87d1dc8.1601477936.git.daniel@iogearbox.net		2026-01-14 18:12:39 -08:00
..
9p
bluetooth	This is the 5.4.279 stable release	2024-07-08 13:55:50 +00:00
caif
iucv
netfilter	netfilter: nf_tables: do not defer rule destruction via call_rcu	2025-06-04 14:32:36 +02:00
netns	UPSTREAM: bpf, netns: Keep a list of attached bpf_link's	2026-01-14 17:53:12 -08:00
nfc	net: nfc: nci: Increase NCI_DATA_TIMEOUT to 3000 ms	2025-12-03 12:45:11 +01:00
phonet
sctp	sctp: detect and prevent references to a freed transport in sendmsg	2025-05-02 07:39:16 +02:00
tc_act	Merge android11-5.4.197+ (3970bc6) into msm-5.4	2022-08-16 16:52:01 +05:30
6lowpan.h
act_api.h	net: sched: extract qstats update code into functions	2025-08-28 16:21:36 +02:00
addrconf.h	This is the 5.4.275 stable release	2024-05-15 16:00:38 +00:00
af_ieee802154.h
af_rxrpc.h
af_unix.h	UPSTREAM: af_unix: Do not use atomic ops for unix_sk(sk)->inflight.	2024-07-02 13:43:53 +03:00
af_vsock.h
ah.h
arp.h
atmclip.h
ax25.h
ax88796.h
bond_3ad.h
bond_alb.h	bonding (gcc13): synchronize bond_{a,t}lb_xmit() types	2023-06-14 10:59:58 +02:00
bond_options.h
bonding.h	bonding: fix macvlan over alb bond support	2023-08-30 16:27:24 +02:00
bpf_sk_storage.h	UPSTREAM: bpf: Change bpf_sk_storage_*() to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON	2026-01-14 18:12:34 -08:00
busy_poll.h	net: busy-poll: use ktime_get_ns() instead of local_clock()	2024-09-04 13:15:03 +02:00
calipso.h
cfg80211-wext.h
cfg80211.h	Merge tag 'ASB-2025-10-06_11-5.4' of https://android.googlesource.com/kernel/common into android13-5.4-lahaina	2025-10-08 15:17:54 +03:00
cfg802154.h
checksum.h
cipso_ipv4.h
cls_cgroup.h	UPSTREAM: bpf: Allow to retrieve cgroup v1 classid from v2 hooks	2025-12-23 13:36:09 -08:00
cnss.h
cnss2.h	cnss2: Add API to send WFC mode to WLAN FW	2023-01-17 14:37:14 +05:30
cnss_logger.h
cnss_nl.h
cnss_prealloc.h
cnss_utils.h
codel.h
codel_impl.h
codel_qdisc.h
compat.h	BACKPORT: net: simplify cBPF setsockopt compat handling	2026-01-14 18:12:33 -08:00
datalink.h
dcbevent.h
dcbnl.h
devlink.h
drop_monitor.h
dsa.h
dsfield.h
dst.h	Merge 5.4.248 into android11-5.4-lts	2023-06-22 16:16:24 +00:00
dst_cache.h	UPSTREAM: wireguard: device: reset peer src endpoint when netns exits	2025-09-24 12:16:39 +02:00
dst_metadata.h
dst_ops.h	net: fix __dst_negative_advice() race	2024-06-16 13:28:52 +02:00
erspan.h	erspan: Add type I version 0 support.	2024-04-13 12:51:36 +02:00
esp.h	esp: limit skb_page_frag_refill use to a single page	2022-07-12 16:30:45 +02:00
ethoc.h
failover.h
fib_notifier.h
fib_rules.h
firewire.h
flow.h	Revert "inet: shrink struct flowi_common"	2023-11-27 17:25:07 +00:00
flow_dissector.h	UPSTREAM: flow_dissector: Pull BPF program assignment up to bpf-netns	2026-01-14 17:53:12 -08:00
flow_offload.h	net: extract port range fields from fl_flow_key	2025-03-13 12:43:23 +01:00
fou.h
fq.h
fq_impl.h
garp.h
gen_stats.h
genetlink.h	Revert "genetlink: hold RCU in genlmsg_mcast()"	2024-11-09 16:39:42 +00:00
geneve.h
gre.h
gro_cells.h
gtp.h
gue.h
hwbm.h
icmp.h
ieee80211_radiotap.h
ieee802154_netdev.h	net: ieee802154: return -EINVAL for unknown addr type	2022-10-26 13:22:59 +02:00
if_inet6.h	net: ipv6: support reporting otherwise unknown prefix flags in RTM_NEWPREFIX	2023-12-20 15:41:13 +01:00
ife.h
ila.h
inet6_connection_sock.h
inet6_hashtables.h	UPSTREAM: net: Track socket refcounts in skb_steal_sock()	2025-12-23 13:36:11 -08:00
inet_common.h	UPSTREAM: bpf: Allow any port in bpf_bind helper	2025-12-23 13:36:18 -08:00
inet_connection_sock.h	UPSTREAM: tcp: bpf: Add TCP_BPF_RTO_MIN for bpf_setsockopt	2026-01-14 18:12:17 -08:00
inet_ecn.h
inet_frag.h
inet_hashtables.h	UPSTREAM: net: Track socket refcounts in skb_steal_sock()	2025-12-23 13:36:11 -08:00
inet_sock.h	tcp/dccp: Fix a data-race around sysctl_tcp_fwmark_accept.	2022-07-29 17:14:11 +02:00
inet_timewait_sock.h
inetpeer.h
ip.h	Merge 5.4.245 into android11-5.4-lts	2023-06-20 16:54:26 +00:00
ip6_checksum.h
ip6_fib.h	UPSTREAM: bpf: Enable bpf_iter targets registering ctx argument types	2026-01-14 17:41:40 -08:00
ip6_route.h
ip6_tunnel.h	ip_gre, ip6_gre: Fix race condition on o_seqno in collect_md mode	2023-05-30 12:44:05 +01:00
ip_fib.h
ip_tunnels.h	Merge tag 'ASB-2025-12-01_11-5.4' of https://android.googlesource.com/kernel/common into android13-5.4-lahaina	2025-12-04 19:21:35 +02:00
ip_vs.h
ipcomp.h
ipconfig.h
ipv6.h	tcp: Reduce chance of collisions in inet6_hashfn().	2023-08-11 11:53:47 +02:00
ipv6_frag.h
ipv6_stubs.h	UPSTREAM: net: Refactor arguments of inet{,6}_bind	2025-12-23 13:36:18 -08:00
ipx.h
iw_handler.h
kcm.h	kcm: Serialise kcm_sendmsg() for the same socket.	2024-09-04 13:14:59 +02:00
l3mdev.h	vrf: use RCU protection in l3mdev_l3_out()	2025-03-13 12:43:12 +01:00
lag.h
lapb.h	net: lapb: increase LAPB_HEADER_LEN	2024-12-19 18:05:03 +01:00
lib80211.h
llc.h
llc_c_ac.h
llc_c_ev.h
llc_c_st.h
llc_conn.h
llc_if.h
llc_pdu.h	llc: Drop support for ETH_P_TR_802_2.	2024-02-23 08:24:50 +01:00
llc_s_ac.h
llc_s_ev.h
llc_s_st.h
llc_sap.h
lwtunnel.h	lwt: Check LWTUNNEL_XMIT_CONTINUE strictly	2023-09-23 10:59:42 +02:00
mac80211.h	mac80211: Add support to trigger sta disconnect on hardware restart	2024-11-08 16:20:52 +01:00
mac802154.h
macsec.h
mip6.h
mld.h
mpls.h
mpls_iptunnel.h
mrp.h	mrp: introduce active flags to prevent UAF when applicant uninit	2023-01-18 11:41:37 +01:00
ncsi.h
ndisc.h
neighbour.h	Merge android11-5.4.249+ (d57e792) into msm-5.4	2023-09-28 16:45:28 +05:30
net_failover.h
net_namespace.h	UPSTREAM: bpf, net: Rework cookie generator as per-cpu one	2026-01-14 18:12:39 -08:00
net_ratelimit.h
netevent.h
netlabel.h
netlink.h
netprio_cgroup.h
netrom.h
nexthop.h
nl802154.h
nsh.h
p8022.h
page_pool.h
ping.h
pkt_cls.h
pkt_sched.h	net/sched: sch_qfq: Fix null-deref in agg_dequeue	2025-12-03 12:45:04 +01:00
pptp.h
protocol.h	tcp/udp: Make early_demux back namespacified.	2022-11-10 17:57:55 +01:00
psample.h
psnap.h
raw.h	raw: Fix a data-race around sysctl_raw_l3mdev_accept.	2022-07-21 20:59:22 +02:00
rawv6.h
red.h
regulatory.h
request_sock.h	BACKPORT: tcp: bpf: Optionally store mac header in TCP_SAVE_SYN	2026-01-14 18:12:18 -08:00
rmnet_config.h
rose.h
route.h
rsi_91x.h
rtnetlink.h	Revert "net: rtnetlink: add msg kind names"	2025-10-31 07:58:56 +00:00
rtnh.h
sch_generic.h	This is the 5.4.297 stable release	2025-09-02 10:52:40 +00:00
scm.h	scm: fix MSG_CTRUNC setting condition for SO_PASSSEC	2023-05-17 11:35:41 +02:00
secure_seq.h
seg6.h	UPSTREAM: seg6: fix seg6_validate_srh() to avoid slab-out-of-bounds	2026-01-14 17:48:10 -08:00
seg6_hmac.h
seg6_local.h
slhc_vj.h
smc.h
snmp.h
sock.h	BACKPORT: bpf: Renames in preparation for bpf_local_storage	2026-01-14 18:12:19 -08:00
sock_reuseport.h	UPSTREAM: net: Generate reuseport group ID on group creation	2025-12-23 13:35:57 -08:00
Space.h
stp.h
strparser.h
switchdev.h
tcp.h	UPSTREAM: tcp: simplify tcp_set_congestion_control(): Always reinitialize	2026-01-14 18:12:28 -08:00
tcp_states.h
TEST_MAPPING	ANDROID: add TEST_MAPPING for net/, include/net	2023-01-23 17:54:01 +00:00
timewait_sock.h
tipc.h
tls.h
transp_v6.h
tso.h
tun_proto.h
udp.h	UPSTREAM: net: bpf: Add bpf_seq_afinfo in udp_iter_state	2026-01-14 18:12:06 -08:00
udp_tunnel.h	BACKPORT: udp_tunnel: add central NIC RX port offload infrastructure	2026-01-14 18:12:10 -08:00
udplite.h	tcp/udp: Call inet6_destroy_sock() in IPv6 sk->sk_destruct().	2023-04-26 11:24:05 +02:00
virt_wifi.h
vsock_addr.h
vxlan.h	vxlan: calculate correct header length for GPE	2023-08-11 11:53:47 +02:00
wext.h
wimax.h
x25.h
x25device.h
xdp.h	UPSTREAM: bpf: cpumap: Add the possibility to attach an eBPF program to cpumap	2026-01-14 18:12:04 -08:00
xdp_priv.h
xdp_sock.h	UPSTREAM: xsk: Move xskmap.c to net/xdp/	2026-01-14 17:48:04 -08:00
xfrm.h	This is the 5.4.276 stable release	2024-05-17 15:29:56 +00:00