phy_port is aimed at representing the various physical interfaces of a
net_device. They can be controlled by various components in the link,
such as the Ethernet PHY, the Ethernet MAC, and SFP module, etc.
Let's therefore make so we keep track of all the ports connected to a
netdev in phy_link_topology. The only ports added for now are phy-driven
ports.
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Tested-by: Aleksei Sviridkin <f@lex.la>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Tested-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Signed-off-by: Maxime Chevallier <maxime.chevallier@bootlin.com>
Link: https://patch.msgid.link/20261016133107.54054-3-maxime.chevallier@bootlin.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
An skb which exceeds an egress device limit loses its GSO feature mask and
is segmented into individual packets, although the device can still offload
smaller TCP GSO skbs. A BIG TCP hop which feeds a non-BIG TCP one pays
that on every flow.
An unencapsulated TCP GSO skb which exceeds gso_max_size or gso_max_segs is
re-segmented with a max_segs which keeps each output inside that limit. An
encapsulated or frag-list skb, a GSO type the device cannot offload, a
max_segs which leaves room for a single MSS and an skb whose transport header
is unset or stale all keep today's segmentation.
This path emits a plain GSO skb, not a BIG TCP one: inet_gso_segment() and
ipv6_gso_segment() write each output's whole length into the 16-bit L3
length field, so the size limit is also capped at what that field can
express.
The helper runs on the skb handed to the driver and only from the
netif_needs_gso() branch: an skb the device takes as it is pays nothing, and
an over-limit one pays the TCP header read and the features recomputation.
Measured on a veth -> bridge -> TAP -> guest virtio-net path, with BIG TCP
enabled on the veth endpoints and left off in the guest, so the skbs which
the veth hop accepts have to be segmented before the TAP device. A single
iperf3 TCP flow, six alternating runs per state (`-t 15 -O 5`, fixed CPU
affinity and port tuple). The middle column is the same tree with the
re-segmentation disabled:
protocol no BIG TCP mixed, no reseg mixed, resegmented
TCP/IPv4 51.550 Gbps 15.850 Gbps 52.617 Gbps
TCP/IPv6 52.050 Gbps 15.783 Gbps 51.933 Gbps
Coefficient of variation for the two mixed columns was 0.48% and 0.82% for
IPv4 and 0.44% and 0.44% for IPv6. A BIG TCP hop which feeds a 64 KiB hop
loses 69% of the throughput of a path which never enables BIG TCP at all;
re-segmentation recovers it, 3.3x over the existing segmentation path and
within noise of the no BIG TCP baseline.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Assisted-by: LLM
Signed-off-by: Wang Zhan <wang.zhan@smartx.com>
Link: https://patch.msgid.link/20261008100651.2534957-6-wang.zhan@smartx.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
gso_features_check() decides whether an egress device can offload a GSO
skb as a single TSO frame by comparing the segment count and the frame
length against the device limits. The resegmentation path added by a
later patch caps the MSS segments per output skb and needs the same
features computed without those limits, so put the two tests in a helper
and gate them with one flag.
__netif_skb_features() takes that flag and takes the place of
netif_skb_features(), which becomes an inline wrapper in netdevice.h and
passes the flag set. Existing callers keep their call, and the
resegmentation path asks for the features without the limit checks.
No functional changes.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Assisted-by: LLM
Signed-off-by: Wang Zhan <wang.zhan@smartx.com>
Link: https://patch.msgid.link/20261008100651.2534957-3-wang.zhan@smartx.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
gso_features_check() compares the frame length against
netif_get_gso_max_size(), which picks the IPv4 or the IPv6 limit from
skb->protocol. A tag which is already inside the frame replaces that field
with the VLAN ethertype, as skb_vlan_push() does, and an IPv6 packet is
then measured against the IPv4 limit and segmented although the device
could send it as one TSO frame.
The lookup moves from netdevice.h into dev.c: vlan_get_protocol() reads the
L3 protocol from behind an in-frame tag, while if_vlan.h includes
netdevice.h, so an inline in that header cannot call it. dev.c is the only
file which calls the lookup, so it becomes a static helper there.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Assisted-by: LLM
Signed-off-by: Wang Zhan <wang.zhan@smartx.com>
Link: https://patch.msgid.link/20261008100651.2534957-2-wang.zhan@smartx.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Cross-merge networking fixes after downstream PR (net-7.3-rc7).
Conflicts:
include/net/strparser.h
0984ebc631 ("strparser: make sure __strp_recv isn't running before tearing down the parser")
02fd0a2111 ("net: strparser: removed the aborted bit and aborts counter")
https://lore.kernel.org/asYfmLsfy-SCxXjr@sirena.co.uk
Adjacent changes:
Documentation/networking/strparser.rst
5f193cdabf ("docs: networking: strparser: remove general mode")
0984ebc631 ("strparser: make sure __strp_recv isn't running before tearing down the parser")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
IPv6 uncached routes are linked to the global per-cpu lists,
rt6_uncached_list.
When unregistering a netdev, rt6_uncached_list_flush_dev() iterates
over the potentially long lists to find uncached routes tied to
the device and swap it with blackhole_netdev.
Since it is called for every device in dying netns under RTNL,
it adds O(N_dev x (N_cpu + N_route)) costs to any batched device
unregistration.
Let's call it once per batched device unregistration without RTNL.
Because it now runs without RTNL, rt->dst.dev and rt->rt6i_idev->dev
may transition to NETREG_UNREGISTERED at different times. Thus, we
must check that both devices are blackhole_netdev before unlinking
the route; otherwise, the remaining device reference would not be
flushed when the other device is unregistered.
In addition, since netdev_run_todo() runs even when IPv6 is disabled,
rt6_uncached_list_flush_dev() must check ipv6_mod_enabled() before
touching rt6_uncached_list.
Note also that netdev_run_todo() cannot be called before
rt6_uncached_list is initialised because:
1. unregister_netdevice_many_notify() panics when
dev_boot_phase == 1 (net_dev_init() is subsys_initcall()).
2. the only device registered before fs_initcall_sync() is
loopback and loopback_net_init() panics in case of failure.
Reported-by: Chris J Arges <carges@cloudflare.com>
Closes: https://lore.kernel.org/netdev/20260917-hash-bucket-route-lists-v3-0-30493a37b6eb@cloudflare.com/
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Tested-by: Chris J Arges <carges@cloudflare.com>
Link: https://patch.msgid.link/20261001204752.2572265-7-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
IPv4 uncached routes are linked to the global per-cpu lists,
rt_uncached_list.
When unregistering a netdev, rt_flush_dev() iterates over the
potentially long lists to find uncached routes tied to the device
and swap it with blackhole_netdev.
Since it is called for every device in dying netns under RTNL,
it adds O(N_dev x (N_cpu + N_route)) costs to any batched device
unregistration.
Let's call it once per batched device unregistration without RTNL.
Note that rt_flush_dev() must be called after setting dev->reg_state
to NETREG_UNREGISTERED. Otherwise, because rt_flush_dev(NULL) runs
without RTNL, it could race with unregister_netdevice_many_notify()
and prematurely purge routes for NETREG_UNREGISTERING dev, for
which flush_all_backlogs() has not been called yet.
Reported-by: Chris J Arges <carges@cloudflare.com>
Closes: https://lore.kernel.org/netdev/20260917-hash-bucket-route-lists-v3-0-30493a37b6eb@cloudflare.com/
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Tested-by: Chris J Arges <carges@cloudflare.com>
Link: https://patch.msgid.link/20261001204752.2572265-6-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ndo_hwtstamp_get and ndo_hwtstamp_set only make sense as a pair. A
driver providing just one gets no complaint today, and the mismatch
turns up in user space, where a set can succeed while the matching get
fails.
Catch it at registration instead.
Suggested-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Nicolai Buchwitz <nb@tipi-net.de>
Link: https://patch.msgid.link/20261002094211.2350702-4-nb@tipi-net.de
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ndo_set_rx_mode cannot sleep and runs with netif_addr_lock_bh held, so an
ops-locked driver cannot take the instance lock from it.
ndo_set_rx_mode_async is the replacement and runs in process context.
Registration has only warned about this since v7.1, long enough to find
the offenders. Make it an error.
Suggested-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Nicolai Buchwitz <nb@tipi-net.de>
Link: https://patch.msgid.link/20261002094211.2350702-3-nb@tipi-net.de
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ethtool refuses to enable tcp-data-split on a device running
a single-buffer XDP program (which can't deal with the payload
landing in a separate buffer). dev_xdp_sb_prog_count() only looks
at xdp_state[].prog though, ignoring bpf_link integration.
Get the prog pointer from dev_xdp_prog(), which will consult both.
bpf_xdp_link_update() needs a bit of a touch-up, too, since
ethtool calls dev_xdp_sb_prog_count() with just the instance
lock (for ops-locked devices) - we now have to make sure the
link updates happen under that lock.
Spotted by AI while reviewing the XDP propagation series,
verified and tested on netdevsim with a C test along these lines:
link = bpf_program__attach_xdp(prog, ifindex);
system("ethtool -G eth0 tcp-data-split on")
Fixes: 197258f0ef ("net: ethtool: add hds_config member in ethtool_netdev_state")
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20261001012559.297751-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
skb_attempt_defer_free() only queues skbs on the current CPU's node
(numa_node_id() of a running CPU), which is always online, so the
flush loop never needs to visit nodes that are merely possible.
for_each_node() walks node_possible_map. On machines where the
possible map is much larger than the online map -- e.g. a POWER10
LPAR with 32 possible but 1 online node -- the flush loop touches 31
cold, always-empty per-node lists on every softirq pass.
Use for_each_online_node() to iterate only node_online_map. Nodes
that go offline are drained by dev_cpu_dead() ("net: flush
skb_defer_nodes in dev_cpu_dead()"), so the flush no longer needs to
visit offline nodes.
Loopback UDP throughput in a QEMU guest with 32 possible / 1 online
nodes (bench_udp, 8 senders, interleaved runs) improves by ~5%.
Fixes: 5628f3fe3b ("net: add NUMA awareness to skb_attempt_defer_free()")
Reported-by: kernel test robot <oliver.sang@intel.com>
Suggested-by: Adrian Tomasov <atomasov@redhat.com>
Closes: https://lore.kernel.org/oe-lkp/202512112119.5b9829a-lkp@intel.com
Signed-off-by: Kris Pan <kris.pan@intel.com>
Link: https://patch.msgid.link/20260928014708.3903028-1-kris.pan@intel.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Commit b6f74dff6d ("net: use two lockdep classes for the netdev
instance lock") put every device without a parent in the virtual class.
It also restricted the locking order for the virtual class, because
queue leasing has hard requirements on the exact order.
This bites us back on bond, which is "virtual" and needs to be taken
before taking the locks of the lowers. NIPA hit the following on the
new test I recently posted for XDP+bond:
WARNING: possible circular locking dependency detected
------------------------------------------------------
python3/18635 is trying to acquire lock:
ff11000120b8ce30 (&dev->lock){+.+.}-{4:4}, at:
netdev_put_lock+0x2d/0x1a0
but task is already holding lock:
ff110001ef77ae30 (&netdev_virt_instance_lock_key){+.+.}-{4:4}, at:
netdev_put_lock+0x2d/0x1a0
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #1 (&netdev_virt_instance_lock_key){+.+.}-{4:4}:
__mutex_lock+0x1ae/0x1f10
xdp_set_features_flag+0x2b/0x50
bond_xdp_set_features+0x1eb/0x360
bond_netdev_event+0x13f/0x300
notifier_call_chain+0xae/0x300
call_netdevice_notifiers+0x70/0xa0
bnxt_xdp_set+0x2f6/0x620
netif_xdp_propagate+0x503/0xc60
dev_xdp_propagate+0xa1/0x230
bond_xdp_set+0x234/0x700
dev_xdp_install+0x592/0xd70
dev_xdp_attach+0x355/0xf50
dev_change_xdp_fd+0x176/0x210
do_setlink.isra.0+0x220d/0x2b20
rtnl_newlink+0x9f1/0x11b0
-> #0 (&dev->lock){+.+.}-{4:4}:
__mutex_lock+0x1ae/0x1f10
netdev_put_lock+0x2d/0x1a0
netdev_nl_queue_create_doit+0x801/0x1a70
genl_family_rcv_msg_doit+0x206/0x300
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&netdev_virt_instance_lock_key);
lock(&dev->lock);
lock(&netdev_virt_instance_lock_key);
lock(&dev->lock);
Let's narrow down the "virtual" class to only the devices which
can actually create a queue. More LoC and complexity, but that
is what we actually care about here. The rest needs to nest
under rtnl_lock, which bond does (famous last words?)
Take the instance locks in two passes, first the netkits then
the rest (matching the queue leasing order).
An alternative would be to make sure the close list is sorted
correctly from the start (queue head/tail appropriately in
unregister_netdevice_queue()). I think it works but feels
a little more fragile. Happy to change, tho.
netdev_can_create_queue() will now be used on paths where we
genuinely handle non-netkit, so we can't always set the extack.
Unfortunately, the (recently) added tracepoint in extack fires
even when extack is NULL.
Fixes: b6f74dff6d ("net: use two lockdep classes for the netdev instance lock")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Nikolay Aleksandrov <razor@blackwall.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://patch.msgid.link/20260929191543.3295633-1-kuba@kernel.org
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Commit c4336a07eb ("net: correctly handle tunneled traffic on IPV6_CSUM
GSO fallback") split skb_gso_has_extension_hdr() into mutually exclusive
branches on skb->encapsulation. This did not yet address all paths:
1. With skb->encapsulation set, the outer header is not checked. IPv6
tunnels such as ip6_gre and ip6_tunnel add an outer Destination
Options header by default (encap_limit). Their GSO packets skip
software GSO, then skb_csum_hwoffload_help() sees the outer extension
header and calls skb_checksum_help() on the GSO skb, which warns and
drops it.
2. Tunnels over IPv6 without an outer transport header, such as
ip6_tunnel, leave skb->transport_header at the inner transport
header. skb_network_header_len() then spans the outer IPv6, tunnel
and inner IP headers, a false positive.
3. UDP tunnels without an inner network header, such as SCTP-in-UDP or
PSP, have no inner IPv6 header to check. Decide on the outer header
alone.
4. Directly dereferencing inner_ip_hdr(skb)->version without
skb_header_pointer() is unsafe.
Instead, check ipv6_ext_hdr(nexthdr) on the outer IPv6 header and, if
set, on the inner IPv6 header. Read the headers with skb_header_pointer().
Keep the skb_network_header_len() check when !skb->encapsulation to also
catch encapsulated packets without skb->encapsulation (e.g., virtio).
Use the same helper in skb_csum_hwoffload_help(). Its open coded test
has the false positive of (2) and ignores the inner header.
Background: checksum offload of tunneled packets invariants:
Non-GSO skb:
- If the inner packet is CHECKSUM_PARTIAL, Local Checksum Offload computes
the outer checksum in software and the device offloads only the inner L4
checksum.
- If the inner packet is CHECKSUM_NONE (e.g., SCTP-in-UDP, ESP-in-UDP,
Remote Checksum Offload), the device offloads the outer UDP or GRE
checksum instead.
GSO skb:
- A device with NETIF_F_GSO_UDP_TUNNEL_CSUM or NETIF_F_GSO_GRE_CSUM
computes both inner and outer checksums per segment.
The outer checksum is seeded with only the pseudo-header checksum.
A NETIF_F_IPV6_CSUM device must parse through the outer headers to
reach the inner ones.
Fixes: c4336a07eb ("net: correctly handle tunneled traffic on IPV6_CSUM GSO fallback")
Cc: stable@vger.kernel.org
Signed-off-by: Willem de Bruijn <willemb@google.com>
Link: https://patch.msgid.link/20260926140506.2335137-1-willemdebruijn.kernel@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
skbedit can set skb->queue_mapping and raise the per-CPU skip_txqueue
flag so __dev_queue_xmit() honours the mapping. __dev_queue_xmit()
cleared the flag before sch_handle_egress() and only read it afterwards,
so the flag was not confined to the xmit that set it: a nested xmit
(mirred redirect or mirror, or a drop after skbedit) could set the flag
and the outer xmit would consume it for an skb that never went through
skbedit.
A forwarded packet still carries the ingress NIC's rx_queue + 1 in
skb->queue_mapping, so the outer device then indexes its tx queue state
with that stale value. Taprio's child array q->qdiscs[] is sized to the
device's queue count, so taprio_enqueue() indexes past its allocation
and dereferences the result as a struct Qdisc *.
We (ab)use the skb->nf_skip_egress which means "skip netfilter egress
for this packet" to tag to "am I in tc egress?". Despite the overload
I dont see it as a conflict since the marker is set only around the
single sch_handle_egress() call and ingress path is guarded by
tc_at_ingress.
I will send a followup(net-next) patch once this hits net-next to
rename the skb->nf_skip_egress bit/flag to skb->skip_egress
Arm the flag only from the egress classifier that can use it: raise
skip_txqueue from tcf_skbedit_act() only when it runs inside
sch_handle_egress(), thanks to skb->nf_skip_egress. An egress qdisc
classifier runs in q->enqueue(), after the tx queue has been picked,
so a mapping it sets cannot affect the current packet; arming the flag
there only pollutes it for a later xmit. Then own the flag for the xmit
frame the egress hook runs in: save the incoming value and clear it just
before sch_handle_egress(), and restore it after the hook - on the
consumed (drop) path, or, in the same call that reads it, on the
surviving path. The save and the restores stay inside the
egress_needed_key static branch, so a packet pays for them only when
egress hooks are active (2f1e85b1ae).
Store the value netdev_cap_txqueue() selected back into skb->queue_mapping
in netdev_tx_queue_mapping(), as netdev_core_pick_tx() already does, so
the skip_txqueue path never hands a later reader on the xmit path a
mapping the device cannot serve. A store made still later in the same
frame, by a tc BPF program attached to a transmit qdisc, is outside this
path and is not re-capped; a separate followup will resolve that path.
netdev_xmit_skip_txqueue() returns the previous flag value so the
save-and-clear is one call, and a no-op stub is provided when
CONFIG_NET_EGRESS is disabled. skb->nf_skip_egress is compiled under
CONFIG_NET_EGRESS rather than CONFIG_NETFILTER_SKIP_EGRESS, so
skb_at_tc_egress() is valid whenever the egress path is built.
A local user in a network namespace can redirect a packet from a device
with more TX queues to one with fewer after setting a mapping valid only
on the larger device. That reaches these reads and, under KASAN, faults
with "slab-out-of-bounds in taprio_enqueue".
Conditions to recreate the bug: the report's own trigger is a local user
with CAP_NET_ADMIN in a network namespace, so no eBPF program is needed.
With CONFIG_NET_SCH_TAPRIO=y, CONFIG_NET_ACT_SKBEDIT=y,
CONFIG_NET_ACT_MIRRED=y, CONFIG_NET_CLS_MATCHALL=y,
CONFIG_NET_SCH_PRIO=y and KASAN enabled, create qa (3 queues), qb
(2 queues) and qc (1 queue) as dummy devices; put a taprio root on qb
(num_tc 1, queues 2@0) and clsact on all three; then add an egress
matchall filter on every device. On qa: "action skbedit queue_mapping 2
pipe action mirred egress redirect dev qb". On qb: "action mirred egress
mirror dev qc". On qc: "action skbedit queue_mapping 0 pipe". Send one
packet out qa. qc's skbedit sets the flag while qb's outer xmit is in
flight; without the fix qb consumes it and reads its two-entry taprio
child array with the forwarded packet's stale mapping. A qc whose
skbedit is instead installed in a transmit-qdisc classifier (a matchall
filter on the qc root qdisc) reaches the same code path the same way
without the fix.
Testing: on a KASAN build with panic_on_warn=1 the unfixed kernel panics
with "BUG: KASAN: slab-out-of-bounds in taprio_enqueue", a read 0 bytes
past a 16-byte taprio_init() allocation, for the clsact-setter and the
transmit-qdisc-classifier reproducers and for a clsact skbedit-then-tc-BPF
store; the fixed kernel runs all three with no report, and the BPF store
variant additionally shows the expected "selects TX queue" clamp notice
from the write-back.
Fixes: 2f1e85b1ae ("net: sched: use queue_mapping to pick tx queue")
Reported-by: Zero Day Initiative <zdi-disclosures@trendmicro.com>
Link: https://lore.kernel.org/netdev/CANn89iLwYx8nCVf0pCEk_MmEiyC6kQaMwCQT9WkQVeeNzNQHqQ@mail.gmail.com/
Link: https://lore.kernel.org/netdev/179008581937.2160803.7117814290574262942@kernel.org/
Link: https://lore.kernel.org/netdev/179033713973.2160803.4914570693994398206@kernel.org/
Link: https://lore.kernel.org/netdev/20260925180407.63647514@kernel.org/
Link: https://lore.kernel.org/netdev/CANn89i+k-mZKDQVtvws_MEXeuMTAdaCcOXFZE-RfhcGTu90sjA@mail.gmail.com/
Suggested-by: Eric Dumazet <edumazet@google.com>
Suggested-by: Jakub Kicinski <kuba@kernel.org>
Tested-by: hybris <hybris@mojatatu.ai>
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/QDISC-9R8V.v4.20260928081529@mojatatu.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
netdev_lockdep_set_classes() puts dev->lock in a separate lockdep class,
by type (netkit vs dummy etc), with the intent of keeping the instance
locks of individual devices as independent from each other as possible.
In practice it does the opposite. lockdep only calls the cmp_fn for locks
of the same class, so netdev_lock_cmp_fn() never gets a say when devices
from different classes are nested. Instead lockdep records a dependency
between the classes, and reports a circular locking problem as soon as
the nesting happens the other way round, e.g. when devices are unregistered
in a batch.
======================================================
WARNING: possible circular locking dependency detected
7.3.0-rc3+ #26 Not tainted
------------------------------------------------------
kworker/u256:1/326 is trying to acquire lock:
ff110000104fce28 (&dev_instance_lock_key#6){+.+.}-{4:4}, at:
unregister_netdevice_many_notify+0x1141/0x1c30
but task is already holding lock:
ff110000127f2e28 (&dev_instance_lock_key#7){+.+.}-{4:4}, at:
unregister_netdevice_many_notify+0x1141/0x1c30
-> #1 (&dev_instance_lock_key#7){+.+.}-{4:4}:
__lock_acquire+0x767/0xd60
lock_acquire.part.0+0xd0/0x260
__mutex_lock+0x17d/0x1f20
unregister_netdevice_many_notify+0x1141/0x1c30
default_device_exit_batch+0x3ee/0x520
ops_undo_list+0x2cc/0x8a0
cleanup_net+0x442/0x9c0
process_one_work+0x951/0x1ab0
worker_thread+0x5a6/0xd10
kthread+0x339/0x430
ret_from_fork+0x4a4/0x6f0
ret_from_fork_asm+0x1a/0x30
-> #0 (&dev_instance_lock_key#6){+.+.}-{4:4}:
check_prev_add+0xeb/0xe60
validate_chain+0x598/0x900
__lock_acquire+0x767/0xd60
lock_acquire.part.0+0xd0/0x260
__mutex_lock+0x17d/0x1f20
unregister_netdevice_many_notify+0x1141/0x1c30
default_device_exit_batch+0x3ee/0x520
ops_undo_list+0x2cc/0x8a0
cleanup_net+0x442/0x9c0
process_one_work+0x951/0x1ab0
worker_thread+0x5a6/0xd10
kthread+0x339/0x430
ret_from_fork+0x4a4/0x6f0
ret_from_fork_asm+0x1a/0x30
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&dev_instance_lock_key#7);
lock(&dev_instance_lock_key#6);
lock(&dev_instance_lock_key#7);
lock(&dev_instance_lock_key#6);
*** DEADLOCK ***
locks held by kworker/u256:1/326: 6, last CPU#6:
#0: ff11000001c2b540 ((wq_completion)netns){+.+.}-{0:0}, at:
process_one_work+0x117c/0x1ab0
#1: ffa0000001a3fd18 (net_cleanup_work){+.+.}-{0:0}, at:
process_one_work+0x8ce/0x1ab0
#2: ffffffff98f53288 (pernet_ops_rwsem){++++}-{4:4}, at:
cleanup_net+0xc1/0x9c0
#3: ffffffff98f6ede0 (rtnl_mutex){+.+.}-{4:4}, at:
default_device_exit_batch+0x92/0x520
#4: ff1100001321ae28 (&dev_instance_lock_key#7){+.+.}-{4:4}, at:
unregister_netdevice_many_notify+0x1141/0x1c30
We can't put all devices in the same class, that'd be too permissive.
netdev_nl_queue_create_doit() and netdev_nl_bind_tx_doit() lock
a virtual device (netkit) before a physical device, without rtnl_lock.
We can never allow locking in the opposite order even under rtnl_lock.
cmp_fn cannot enforce this sort of rule: lockdep keys its chain cache on
the sequence of lock classes, so with a single class both orders hash
to the same chain and only whichever happens first is validated.
netdev_lock_cmp_fn() itself has another source of false-negatives.
lockdep calls it from within __lock_acquire(), with the recursion counter
already raised, so lock_is_held_type() always returns LOCK_STATE_UNKNOWN,
which means lockdep_rtnl_is_held() always returns true / held.
Use rtnl_is_locked(), which looks at the mutex directly and does
work from that context. We may still miss a bad case if some other
process is holding the lock, not us, but that's better than the
100% false negative rate of lockdep_rtnl_is_held().
One last thing, lockdep compares the address of the cmp function,
so it can't be a static inline - that would work similarly to having
separate classes, again. Move it to a source file.
Tested by nesting two instance locks from a module, one scenario per boot
since the first splat turns debug_locks off:
first second rtnl reported
-----------------------------------------------------------
virt->virt no recursive locking
virt->virt yes -
virt->phys no -
virt->phys yes -
phys->virt no - (records the edge)
phys->virt yes - (records the edge)
phys->phys no recursive locking
phys->phys yes -
virt->phys phys->virt no, no circular dependency
virt->phys phys->virt no, yes circular dependency
virt->phys phys->virt yes, no circular dependency
virt->phys phys->virt yes, yes circular dependency
phys->virt virt->phys no, no circular dependency
phys->virt virt->phys yes, yes circular dependency
virt->virt virt->virt yes, no recursive locking
phys->phys phys->phys yes, no recursive locking
(netkit and dummy stood in for the virtual devices, virtio_net and
netdevsim for the physical ones.)
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20260926041832.1649675-1-kuba@kernel.org
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
When a CPU goes offline, dev_cpu_dead() drains its softnet queues
(completion_queue, output_queue, poll_list, process_queue, and
input_pkt_queue), but leaves net_hotdata.skb_defer_nodes untouched.
If oldcpu goes offline while holding pending skbs in its
skb_defer_nodes lists (e.g. below the sysctl_skb_defer_max >> 1 IPI
threshold, or if the IPI races with CPU teardown), those skbs remain
stranded until oldcpu is brought back online. If any of these skbs
hold page_pool fragments, page_pool_destroy() will stall indefinitely
waiting for inflight pages to be returned when a netdev or driver is
torn down while oldcpu is offline.
Additionally, if smp_call_function_single_async() fails in
kick_defer_list_purge() because the target CPU went offline, reset
defer_ipi_scheduled to 0 so future IPI kicks are not blocked when the
CPU comes back online.
Also, if oldcpu was the last online CPU on its NUMA node, drain that
node's slot across all CPUs so no skbs deferred from that node remain
stranded on idle remote CPUs (or if the node itself is subsequently
offlined).
Finally, in skb_attempt_defer_free(), re-check cpu_online(cpu) and
whether the caller migrated CPUs after llist_add(), flushing the node
list if so, to close the preemption TOCTOU race against CPU/node
teardown.
Fixes: 68822bdf76 ("net: generalize skb freeing deferral to per-cpu lists")
Fixes: 5628f3fe3b ("net: add NUMA awareness to skb_attempt_defer_free()")
Closes: https://lore.kernel.org/netdev/20260916003430.3612956-1-kris.pan@intel.com/
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Kris Pan <kris.pan@intel.com>
Link: https://patch.msgid.link/20260923130318.607255-1-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Only the entries below dev->num_tc are valid in dev->tc_to_txq[], and
dev->prio_tc_map[] may only name classes below it. netdev_set_num_tc()
lowers dev->num_tc without touching either array.
netdev_txq_to_tc() walks all TC_MAX_QUEUE slots and
netdev_get_prio_tc_map() returns the entry as it stands, so a leftover
entry is handed out as a traffic class >= dev->num_tc. Taking that
class from netdev_txq_to_tc(), __netif_set_xps_queue() rejects only a
negative one and indexes an XPS map sized for dev->num_tc classes:
tci = j * num_tc + tc;
RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map);
attr_map[] holds nr_ids * num_tc entries and j runs over the ids named
in the mask, so a class that is not below num_tc pushes tci past the end
of the map for the last ids and the store overruns it.
Any caller that lowers num_tc leaves such entries behind, and
mqprio_destroy() tears down with netdev_set_num_tc(dev, 0) rather than
netdev_reset_tc(). After mqprio with 8 classes then 1, tc_to_txq[1..7]
still describe txq 1..7. The splat is from an XPS write to txq 2 on a
veth with 8 rx queues: attr_map[] has 8 * 1 entries, tci = j + 2, and
j == 6 stores one past the end of the 88-byte map:
BUG: KASAN: slab-out-of-bounds in __netif_set_xps_queue (net/core/dev.c:2954)
Write of size 8 at addr ffff88813016bc58 by task xps_oob/634
__netif_set_xps_queue (net/core/dev.c:2954)
xps_rxqs_store (net/core/net-sysfs.c:1880)
netdev_queue_attr_store (net/core/net-sysfs.c:1390)
Allocated by task 634:
__kmalloc_noprof (mm/slub.c:5439)
__netif_set_xps_queue (net/core/dev.c:2937)
The buggy address is located 0 bytes to the right of
allocated 88-byte region [ffff88813016bc00, ffff88813016bc58)
Reject a class the map has no room for.
Fixes: 184c449f91 ("net: Add support for XPS with QoS via traffic classes")
Signed-off-by: Norbert Szetei <norbert@doyensec.com>
Link: https://patch.msgid.link/162DD16F-54C6-444A-9E09-0B8CB3D591F2@doyensec.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ipip_fill_forward_path() and ip6_tnl_fill_forward_path() look up the
route to the tunnel's remote endpoint and set ctx->dev to its device,
which is the tunnel itself when that route resolves back to the tunnel.
dev_fill_forward_path() then makes no progress and trips
WARN_ON_ONCE(last_dev == ctx->dev) as soon as a flowtable tries to
offload a flow through the tunnel. That routing loop is a configuration
any CAP_NET_ADMIN user can set up, and ip_tunnel_xmit() and
ip6_tnl_xmit() already treat it as a tx error, so remove the warning and
just fail the walk, as commit 008e7a7c29 ("net: remove WARN_ON_ONCE
when accessing forward path array") did for the path stack overflow.
Fixes: ab427db178 ("netfilter: flowtable: Add IPIP rx sw acceleration")
Fixes: d98103575d ("netfilter: flowtable: Add IP6IP6 rx sw acceleration")
Closes: https://lore.kernel.org/all/CA+0ovCgaRvbd0Udj70b2xxG8Cx3CaCpNhnf1V4RWQuDveZYZhA@mail.gmail.com/
Suggested-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Farhad Alemi <farhad.alemi@berkeley.edu>
Reviewed-by: Xuanqiang Luo <luoxuanqiang@kylinos.cn>
Link: https://patch.msgid.link/CA+0ovCgKDOk+Bg6Gh5Lwx94u_jJjQ30-vY1JcY2BYfhnWJJbPA@mail.gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Several subsystems allocate ring buffers sized by dev->tx_queue_len
with no upper bound. An unprivileged user (via unshare -Urn) can set a
huge tx_queue_len and exhaust global memory with ring allocations:
- pfifo_fast: pfifo_fast_init() and pfifo_fast_change_tx_queue_len()
allocate 3 skb_array rings of tx_queue_len entries each.
- tun: tun_queue_resize() and the queue-attach path resize ptr_rings
to tx_queue_len on the NETDEV_CHANGE_TX_QUEUE_LEN notifier.
- tap (macvtap/ipvtap): tap_queue_resize() and tap_init() resize/init
ptr_rings to tx_queue_len on the same notifier.
netif_change_tx_queue_len() is the single entry point for IFLA_TXQLEN,
sysfs, and the SIOCSIFTXQLEN ioctl. Cap new_len at S16_MAX (32767)
there so the oversized value is rejected at set time. This takes
effect whether the device is up or down, before dev->tx_queue_len is
written, before any notifier fires, and before any ring is allocated.
The "> S16_MAX" check also subsumes the previous unsigned-long
truncation test, and a negative ifr_qlen from the ioctl lands far
above the cap after conversion, so both old failure modes are covered
by the one comparison.
tx_queue_len is ambigious: both a per-ring sizing multiplier and a
default queue-length/limit knob for consumers that allocate
nothing at set time (pfifo/bfifo/gred/plug/sfb limits, htb
direct_qlen, qfq max_classes, teql). 32767 is chosen as the largest
value NLA_POLICY_FULL_RANGE can express for the u32 IFLA_TXQLEN
policy in patch 2/3 while staying a legitimate queue length on
high-BDP paths; the ring-memory trade-off of a shared knob is
disclosed below.
Conditions to recreate the bug:
- CONFIG_NET_SCHED=y, CONFIG_VETH=y, CONFIG_USER_NS=y, CONFIG_NET_NS=y.
- Unprivileged user in a fresh user+net namespace (unshare -Urn).
- pfifo_fast: create veth pairs, set tx_queue_len to 500000, attach
mq+pfifo_fast. ~28 iterations OOMs a 2GB guest.
- tun: create 50 tun devices with IFF_MULTI_QUEUE, set tx_queue_len to
500000, open 8 queues each. ~1.6GB of ptr_ring allocations OOMs a
512MB guest.
- tap: same as tun with IFF_TAP. ~960MB OOMs a 512MB guest.
- On the fixed kernel the oversized tx_queue_len is rejected with
-ERANGE at set time (all four paths: RTM_SETLINK, RTM_NEWLINK
create, sysfs, ioctl - the latter two via this check, the former
two via this check and the 2/3 parse policy respectively).
Fixes: 6a643ddb56 ("net: introduce helper dev_change_tx_queue_len()")
Reported-by: Vega <vega@nebusec.ai>
Closes: https://lore.kernel.org/netdev/20260828121902.66837-1-jhs@mojatatu.com/
Tested-by: Victor Nogueira <victor@mojatatu.com>
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
Link: https://patch.msgid.link/QDISC-2899.v2.20260901233641@mojatatu.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Concurrent fast-path readers access dev->prio_tc_map (e.g. via
skb_tx_hash(), netdev_get_prio_tc_map(), and qdiscs) while writers
update entries in dev->prio_tc_map or reset/clear the map via
netdev_reset_tc() and netdev_unbind_sb_channel().
Furthermore, memset() in netdev_reset_tc() and
netdev_unbind_sb_channel() provides no guarantee of performing
atomic word/byte stores.
Add READ_ONCE() and WRITE_ONCE() annotations to netdev_get_prio_tc_map()
and netdev_set_prio_tc_map(), replace memset() in dev.c with explicit
WRITE_ONCE() loops, and update direct array accesses in qdiscs to use
netdev_get_prio_tc_map().
Signed-off-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260812085440.3917924-4-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Several fast-path and control-path lockless readers access dev->num_tc
(e.g., skb_tx_hash(), netdev_txq_to_tc(), netdev_get_num_tc(), and
qdisc/driver lookups) while concurrent writers update dev->num_tc
during TC setup, device reset, or channel configuration.
Add READ_ONCE() and WRITE_ONCE() annotations to prevent compiler
reordering and load/store tearing when accessing dev->num_tc.
Update inline helpers in netdevice.h (netdev_get_num_tc(),
netdev_set_prio_tc_map(), and netdev_get_sb_channel()) as well as
writers and lockless readers in core networking code and drivers.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260812085440.3917924-3-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
netdev_set_tc_queue() (and related helpers/drivers such as
netdev_bind_sb_channel_queue(), netdev_reset_tc(), and
netdev_unbind_sb_channel()) perform separate 16-bit writes to
dev->tc_to_txq[tc].count and dev->tc_to_txq[tc].offset.
Furthermore, memset() in netdev_reset_tc() and
netdev_unbind_sb_channel() provides no guarantee of performing
full 32-bit word stores.
Concurrent lockless readers (e.g. skb_tx_hash(), netdev_txq_to_tc(),
ixgbe_select_queue(), taprio, mqprio, FPE drivers) can observe torn
values where offset and count belong to inconsistent configurations.
Redefine struct netdev_tc_txq to embed count and offset inside a union
with a u32 combined field, allowing atomic manipulation via
READ_ONCE() and WRITE_ONCE().
Update all lockless readers and writers across the kernel to use
READ_ONCE() and WRITE_ONCE() on the combined field.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260812085440.3917924-2-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
bpf_xdp_link_update() calls dev_xdp_install() directly and skips
dev_xdp_attach(), so the checks in dev_xdp_attach() do not run. A user can
make an XDP link with a normal program and then swap in an offloaded or
device-bound program with BPF_LINK_UPDATE, which puts it on the software
path.
dev_xdp_install() is the one place all three paths go through:
"ip link set xdp" and BPF_LINK_CREATE reach it via dev_xdp_attach(), and
BPF_LINK_UPDATE calls it directly. So move the program checks (offloaded,
bound to another device, device-bound in generic mode, native vs generic,
DEVMAP and CPUMAP) there, and keep only the netlink-flag check
(XDP_FLAGS_UPDATE_IF_NOEXIST) in dev_xdp_attach().
Fixes: 026a4c28e1 ("bpf, xdp: Implement LINK_UPDATE for BPF XDP link")
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Pablo Neira Ayuso says:
====================
Netfilter updates for net
This includes an enhancement to detect ct memleaks easier via
DEBUG_NET and flowtable preparation patches for IPv4 over IPV6
and vice-versa. This also includes a fix for the nft_ct custom
expectation support.
1) Add DEBUG_NET_WARN_ON_ONCE to nf_ct_set() to spot ct memleaks.
2) Pass struct net_device_path_ctx to dev_fill_forward_path() to
make it easier to pass more parameters to this function.
From Lorenzo Bianconi.
3) Add ether_type field to net_device_path context structucture.
4) Rename tun.l3_proto field to tun.inner_proto.
5) Rename ctx.tun.proto to ctx.tun.inner_proto.
6) Store ether_type in flowtable context.
7) Move IPv4 and IPv6 xmit path to a helper function.
8) Move encapsulation header parser out of the flowtable lookup
function.
9) Rework nft_ct custom expectation support to address a possible
reallocation of ct extension area while expectation list also
contains expectations. Move datapath to a ct helper to fix it.
10) Ensure timeout is always lowered for the non-closing RST case
in the TCP connection tracking.
11) Bail out when inserting already dead expectation, this should
not ever happen, hence report it via DEBUG_NET.
12) Comestic updates for improving the conntrack selftest dump and
flush userspace program, from Qingshuang Fu.
* tag 'nf-next-26-08-10' of git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf-next:
selftests: netfilter: conntrack_dump_flush: remove unused variables and fix typo
netfilter: nf_conntrack_expect: bail out on insert dead expectations
netfilter: conntrack: always lower timeout for non-closing RST packets
netfilter: nft_ct: move custom expectation support to helper
netfilter: flowtable: detach layer 2 encapsulation parser from lookup
netfilter: flowtable: move ipv4 and ipv6 xmit path to function
netfilter: flowtable: store ethertype in flowtable context
netfilter: flowtable: rename ctx.tun.proto to ctx.tun.inner_proto
netfilter: flowtable: rename tun.l3_proto to tun.inner_proto
net: netfilter: add ether_type to net_device_path_ctx and use it
net: pass net_device_path_ctx to dev_fill_forward_path()
netfilter: add DEBUG_NET_WARN_ON_ONCE to skb_set_nfct()
====================
Link: https://patch.msgid.link/20260810194015.932627-1-pablo@netfilter.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Refactor dev_fill_forward_path() to take a struct net_device_path_ctx
pointer instead of a (dev, daddr) pair, so the caller can build and
populate the context up front and keep it after the forward path walk.
This allows additional fields (e.g. vlan and ether_type) to be carried
in the context and shared with ndo_fill_forward_path implementations,
instead of being reconstructed on the stack inside the core helper.
Update the mtk_ppe_offload, airoha_ppe and nf_flow_table_path callers to
allocate and fill the context before invoking dev_fill_forward_path().
The network topology resolution behaviour is unchanged.
This is a preliminary patch to enable HW flowtable offload for IPv4
over IPv6 tunnels.
Signed-off-by: Lorenzo Bianconi <lorenzo.bianconi@oss.qualcomm.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Generic XDP exposes non-linear skb fragments through an xdp_buff. If an
XDP program adjusts the fragment area, bpf_prog_run_generic_xdp() copies
xdp_frags_size back to skb->data_len but leaves skb->len containing the
old fragment contribution.
After a fragment shrink, this makes skb_headlen() larger than the actual
linear area. In the reproduced UDP receive path, __skb_datagram_iter()
copied 1024 bytes past the actual linear tail to userspace, starting at
struct skb_shared_info. The copied bytes included the affected skb's
nr_frags, xdp_frags_size and a kernel pointer from
skb_shinfo(skb)->frags[0]. Real packet data was displaced by the same
amount and truncated at the end.
Subtract the old data_len before replacing it and add the new data_len
afterwards, keeping skb->len and skb->data_len synchronized.
A 60000-byte UDP datagram on a veth pair with MTU 64000 was shortened by
1024 bytes from its fragment area. Before the fix, all 10 runs produced
corrupted payloads. After the fix, all 10 runs matched the expected
payload exactly.
Fixes: e6d5dbdd20 ("xdp: add multi-buff support for xdp running in generic mode")
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/al9T9Eto%2FhRIzP5W@boxer/
Reviewed-by: Mohsin Bashir <hmohsin@meta.com>
Acked-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>
Link: https://patch.msgid.link/20260804054040.613675-2-sun.jian.kdev@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Technically, it's illegal to take a ref on a netdev just because
we have a pointer on which we already hold a ref, with no other
protection. This is because our simple per-cpu refcount
implementation cannot atomically read the count.
Let's make sure we cancel outstanding work and never queue more
work for a device we know is dead. This way taking a ref on
a dev we know is on the netdev_work_list is always going to be safe.
Jiangshan Yi reports that the issues is caught by ref tracker infra
leading to a warning:
WARNING: lib/ref_tracker.c:322 at ref_tracker_free
WARNING: lib/ref_tracker.c:246 at ref_tracker_dir_exit
Reported-by: Jiangshan Yi <yijiangshan@kylinos.cn>
Link: https://lore.kernel.org/20260731035135.3917308-2-yijiangshan@kylinos.cn
Fixes: 12c765be84 ("net: turn the rx_mode work into a generic netdev_work facility")
Link: https://patch.msgid.link/20260806022821.2079945-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
netdev_register_kobject() calls device_add(), which emits KOBJ_ADD and
wakes udev, but register_netdevice() only makes the device findable by
name later, in list_netdevice(). A udev worker that reacts to the uevent
can therefore run against a device that no lookup can find yet.
This used to be harmless because the ethtool ioctl took the rtnl_lock
when looking the device up, and register_netdevice() runs under rtnl, so
the worker simply blocked until registration finished. The commit in the
fixes tag moved the lookup out from under rtnl for ops-locked drivers.
Now there is a short window in register_netdevice() between
netdev_register_kobject() until list_netdevice() when the device is not
findable by name.
This was reproduced with the mlx5 driver on a kernel with KASAN enabled
during devlink reload: systemd-udevd's net_driver builtin gets -ENODEV
from ETHTOOL_GDRVINFO, which was preventing interface renaming.
Suppress the uevent in netdev_register_kobject() and emit it from
register_netdevice() next to rtmsg_ifinfo(). This is the last point in
register_netdevice() where no error can happen, so only fully registered
devices are announced: the registration error paths never reach it, and
the device_del() that unwinds them stays silent as well, leaving
userspace with neither an add nor a remove.
Fixes: f994752b11 ("net: ethtool: optionally skip rtnl_lock on IOCTL path")
Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
Reviewed-by: Shahar Shitrit <shshitrit@nvidia.com>
Link: https://patch.msgid.link/20260806080758.2039586-2-dtatulea@nvidia.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add dst_entry to tunnel device path, this will allow us to remove
a duplicated route lookup.
This is a preparation patch to retrieve the tunnel route directly
from the .fill_forward_path. This new dst_entry in the tunnel will be
used by a follow up patch.
Since dst_release() works fine on NULL interface, this is still
noop until the flowtable starts using this.
Add a new dev_fill_forward_path_release() function to drop the refcount
on the tunnel device route and use it in case of error out. Export it so
to drop the refcount on the tunnel route at a later stage.
Adjust existing drivers that recycle dev_fill_forward_path() to call
dev_fill_forward_path_release() for safety reasons.
Acked-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Update stack index from dev_fill_forward_path() instead, once the
forward path slot has been populated.
Acked-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
For per-netns device unregistration, the list passed to
unregister_netdevice_many() must contain devices from a single
netns only (once all callers are converted).
Let's move collected devices in the following functions to
net->dev_unreg_head and let __rtnl_net_unlock() pass them to
unregister_netdevice_many().
* default_device_exit_batch()
* ops_exit_rtnl_list()
* __rtnl_kill_links()
This allows incremental conversion of each driver to support
per-netns device unregistration without affecting the normal
kernel where CONFIG_DEBUG_NET_SMALL_RTNL is disabled.
Note that this change unbatches synchronize_rcu() etc in
unregister_netdevice_many(), but we can later split it into
multiple stages to batch them again.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-8-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
When we need to unregister a netdev in a different netns, we will
delegate its unregistration to per-netns work.
There are three types of such cross-netns devices:
1. Paired devices (e.g., netkit, veth, vxcan)
-> Unregistering one device also deletes its peer, which
may reside in another netns.
2. Tunnel devices (e.g., bareudp, geneve, etc)
-> Destroying a netns removes devices in another netns if
their backend sockets reside in the dying netns
3. Stacked devices (e.g., ipvlan, macvlan, etc)
-> Removing the lower device also removes multiple upper
devices, each of which may reside in different namespaces.
In these cases, we will use unregister_netdevice_queue_net() to
queue such potential cross-netns devices for destruction.
Each driver must not call both unregister_netdevice_queue_net()
and unregister_netdevice_queue() for the same device. See the
subsequent veth/bareudp/ipvlan patches for how they avoid double
queueing.
unregister_netdevice_queue_net() takes net and dev. If dev resides
in the net, it simply calls unregister_netdevice_queue().
If dev_net(dev) is different from the net, it enqueues the device
to dev_net(dev)->dev_unreg_head and schedules the per-netns work.
When __rtnl_net_unlock() is called from the per-netns work (or another
thread already holding the lock), unregister_netdevice_many_net()
collects the queued devices and calls unregister_netdevice_many()
to perform the actual unregistration.
During netns dismantle, rtnl_net_flush_workqueue() is called at the
end of default_device_exit_batch() to ensure that cross-netns
devices in the other alive netns are unregistered.
Once RTNL is removed, a device could be moved to another netns while
being queued to net->dev_unreg_head.
__dev_change_net_namespace() handles this race by acquiring
net->dev_unreg_lock of both the old and new netns after dev_set_net()
and moving the device between their dev_unreg_head lists.
Since dev_set_net() and unregister_netdevice_queue_net() are
synchronised by netdev_lock(), the device is either queued to the
old netns's dev_unreg_head and then moved, or queued directly to
the new netns.
Note that unregister_netdevice_move_net() does not need to call
rtnl_net_queue_work() because __dev_change_net_namespace() is
(supposed to be) called with rtnl_net_lock(). (Not all callers
hold it yet, but the race does not happen until all callers
are converted and RTNL is removed.)
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-7-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Currently, netdev_run_todo() processes pending devices from multiple
namespaces in a batch.
To expand the per-netns RTNL coverage for NETDEV_UNREGISTER, let's
acquire __rtnl_net_lock() in netdev_wait_allrefs_any().
Note that netdev_run_todo() itself will need to be namespacified
before RTNL is removed.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-6-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
validate_xmit_xfrm() returns NULL both when a packet is dropped and
when it is stolen by async crypto (-EINPROGRESS from ->xmit()).
Callers cannot distinguish the two cases.
f53c723902 ("net: Add asynchronous callbacks for xfrm on layer 2.")
changed the semantics of a NULL return from "dropped" to "stolen or
dropped", but __dev_queue_xmit() was not updated. On virtual/bridge
interfaces (noqueue qdisc) __dev_queue_xmit() initialises rc=-ENOMEM
and jumps to out: when skb is NULL, returning -ENOMEM to the caller
even though the packet will be delivered correctly via xfrm_dev_resume().
Return ERR_PTR(-EINPROGRESS) from validate_xmit_xfrm() for the async
case so callers can tell it apart from a real drop. Update
__dev_queue_xmit() to handle ERR_PTR(-EINPROGRESS) from
validate_xmit_skb() correctly. Update validate_xmit_skb_list() to
use IS_ERR_OR_NULL() so that ERR_PTR(-EINPROGRESS) is not mistakenly
added to the transmitted list.
Fixes: f53c723902 ("net: Add asynchronous callbacks for xfrm on layer 2.")
Suggested-by: Sabrina Dubroca <sd@queasysnail.net>
Signed-off-by: Petr Wozniak <petr.wozniak@gmail.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
vlan_device_event() generates nested UP/DOWN, MTU and feature
change events. It executes an event for the VLAN device directly
from the notifier - while the locks of the lower device are held.
This causes deadlocks, for example:
bond (3) bond_update_speed_duplex(vlan)
| ^ v
vlan (2) UP(vlan) (4) vlan_ethtool_get_link_ksettings()
| ^ v
dummy (1) UP(dummy) (5) __ethtool_get_link_ksettings()
The dummy device is ops locked, vlan creates a nested event (2),
then bond wants to ask vlan for link state (3). bond uses the
"I'm already holding the instance lock" flavor of API. But in
this case the lock held refers to vlan itself. We hit vlan's
link settings trampoline (4) and call __ethtool_get_link_ksettings()
which tries to lock dummy. Deadlock. There's no clean way for us
to tell the vlan_ethtool_get_link_ksettings() that the caller
is already in lower device's critical section.
Defer the propagation to the per-netdev work facility instead:
the notifier only schedules netdev_work_sched(vlandev, VLAN_WORK_*),
and ndo_work (vlan_dev_work) applies the change later. Hopefully
nobody expects the VLAN state changes to be instantaneous.
If someone does expect the changes to be instantaneous we will
have to do the same thing Stan did for rx_mode and "strategically"
place sync calls, to make sure such delayed works are executed
after we drop the ops lock but before we drop rtnl_lock.
Stan suggests that if we need that down the line we may
consider reshaping the mechanism into "async notifications".
AFAICT only vlan does this sort of netdev open chaining,
so as a first try I think that sticking the complexity into
the vlan code makes sense.
One corner case is that we need to cancel the event if user
explicitly changes the state before work could run. Consider
the following operations with vlan0 on top of dummy0:
ip link set dev dummy0 up # queues work to up vlan0
ip link set dev vlan0 down # user explicitly downs the vlan
ndo_work # acts on the stale event
Reported-by: syzbot+09da62a8b78959ceb8bb@syzkaller.appspotmail.com
Reported-by: syzbot+cb67c392b0b8f0fd0fc1@syzkaller.appspotmail.com
Reported-by: syzbot+9bb8bd77f3966641f298@syzkaller.appspotmail.com
Fixes: 9f275c2e90 ("net: ethtool: make sure __ethtool_get_link_ksettings() is ops-locked")
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Nicolai Buchwitz <nb@tipi-net.de>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20260624182018.2445732-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The rx_mode update runs from a workqueue: drivers have their
ndo_set_rx_mode_async() callback executed by a single global
work item under RTNL and ops lock. This is a useful pattern.
Support multiple "events" that need to be serviced and make RX_MODE
sync the first one. Call the events "core" because later on
we will let drivers define and schedule their own.
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20260624182018.2445732-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Syzbot reported a KASAN slab-use-after-free in fib_rules_lookup().
The root cause is a race condition where packets can escape the backlog
flushing during device unregistration (e.g., during netns exit).
Commit e9e4dd3267 ("net: do not process device backlog during unregistration")
introduced a lockless netif_running() check in enqueue_to_backlog() to
prevent queuing packets to an unregistering device.
However, this creates a TOCTOU race window.
A lockless transmitter (like veth_xmit) can pass
the check before dev_close() clears IFF_UP. If the transmitter is then
delayed, flush_all_backlogs() can run and finish before the transmitter
grabs the backlog lock and queues the packet. The packet then escapes
the flush and triggers UAF later when processed.
Fix this by moving the netif_running() check inside the backlog lock.
This serializes the check with the flush work (which also grabs the lock).
We then either queue the packet before the flush runs (so it gets flushed),
or check netif_running() after the flush/close completes (so it gets dropped).
Fixes: e9e4dd3267 ("net: do not process device backlog during unregistration")
Reported-by: syzbot+965506b59a2de0b6905c@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/netdev/6a315824.b0403584.28d0ff.0000.GAE@google.com/T/#u
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Julian Anastasov <ja@ssi.bg>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260616141317.407791-1-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Merge in late fixes in preparation for the net-next PR.
Conflicts:
net/tls/tls_sw.c
406e8a651a ("net: skmsg: preserve sg.copy across SG transforms")
79511603a6 ("tls: remove dead sockmap (psock) handling from the SW path")
drivers/net/ethernet/microsoft/mana/mana_en.c
f8fd56977e ("net: mana: guard TX wq object destroy with INVALID_MANA_HANDLE check")
d07efe5a6e ("net: mana: Use per-queue allocation for tx_qp to reduce allocation size")
https://lore.kernel.org/ajAPXu-C_PuTgV-a@sirena.org.uk
No adjacent changes.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Maxime Chevallier