mv88e6xxx: Standalone cross-chip LAGs are broken #4
Description
Background
From the standalone port isolation approach described here:
The ideal isolation between standalone ports satisfies two properties:
1. Packets from one standalone port must not be forwarded to any other
port.
2. Packets from a standalone port must be sent to the CPU port.
mv88e6xxx solves (1) by isolating standalone ports using the PVT. Up
to this point though, (2) has not guaranteed; as the ATU is still
consulted, there is a chance that incoming packets never reach the CPU
if its DA has previously been used as the SA of an earlier packet (see
1/5 for more details). This is typically not a problem, except for one
very useful setup in which switch ports are looped in order to run the
bridge kselftests in tools/testing/selftests/net/forwarding. This
series attempts to solve (2).
Ideally, we could simply use the "ForceMap" bit of more modern chips
(Agate and newer) to classify all incoming packets as MGMT. This is
not available on older silicon that is still widely used (Opal Plus
chips like the 6097 for example).
Instead, this series takes a two pronged approach:
1/5: Always clear MapDA on standalone ports to make sure that no ATU
entry can lead packets astray. This solves (2) for single-chip
systems.
2/5: Trivial prep work for 4/5.
3/5: Trivial prep work for 4/5.
4/5: On multi-chip systems though, this is not enough. On the incoming
chip, the packet will be forced out towards the CPU thanks to
1/5, but on any intermediate chips the ATU is still consulted. We
override this behavior by marking the reserved standalone VID (0)
as a policy VID, the DSA ports' VID policy is set to TRAP. This
will cause the packet to be reclassified as MGMT on the first
intermediate chip, after which it's a straight shot towards the
CPU.
Problem
As it turns out, this is not compatible with LAGs. Consider the following setup:
.-----.
| CPU |
'--+--'
|
.--0--. .-----.
| sw1 9---0 sw2 |
'-----' '-4-5-'
A LAG is created from sw2p{4,5}, using ID 0, but it is not attached to any bridge - so port isolation is active. Let's walk through a packet's journey to the CPU:
- Packet ingresses
sw2p4, theMapDAbit is not set, so it is flooded according to the PVT, which only containssw2p0 - Packet egresses
sw2p0withFORWARD vid:0 dev:2 port:0(lag) - Packet ingresses
sw1p9, the VTU policy bit is set for VLAN 0, thus the packet is classified asMGMTand trapped to the CPU - Packet egresses
sw1p0withTO_CPU vid:0 dev:2 port:0THE LAG BIT IS LOST IN TRANSLATION HERE - Packet ingresses CPU, since no user port is mapped to
sw2p0the packet is dropped
Solution
There are (at least) two ways forward:
- Use the
ForceMapbit on the chips that support it and say "GLHF" to the older ones - Delay setting the LAG bit in port register 5 until the port joins a bridge. This should work on all devices, and since there is no offloading going on for standalone LAGs anyway, there is no real penalty either