lib, zebra: mark singleton nexthops inactive/active on link state changes for wecmp (backport #18947) #19257
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
July 24, 2025 03:33
add nexthop_same_no_weight api which compares if 2 nexthops are same ignoring the weight. This is used in W-ECMP case. Ticket: # Signed-off-by: Karthikeya Venkat Muppalla <kmuppalla@nvidia.com> (cherry picked from commit 6d0224d)
…for wecmp
Issue:
------
When interface goes down, singleton nexthops are marked INACTIVE, but
their individual entries within dependent NHG groups were not being
marked INACTIVE due to weight comparison checks in nexthop_same().
Similarly, when interface comes up, singleton nexthops are marked ACTIVE, but their
individual entries within dependent NHG groups were not being marked
ACTIVE due to weight comparision checks when comparision checks in
nexthop_same().
There is a weight mismatch issue:
- The standalone singleton nexthop has weight = 1
- The same singleton nexthop when part of an NHG has weight = 255
This weight mismatch causes nexthop_same() to return FALSE, preventing proper matching.
Fix:
----
Skip weight comparison when marking dependent nexthops to ensure marking the individual
nexthops in the nexthop group correctly by using
nexthop_same_no_weight() instead of nexthop_same()
Testing:
--------
Before fix:
-----------
root@leaf11:~# ip route | grep nhid
11.0.0.0/8 nhid 13 dev lo proto 196 metric 20
12.0.0.0/8 nhid 20 proto bgp metric 20
21.0.0.0/8 nhid 20 proto bgp metric 20
22.0.0.0/8 nhid 20 proto bgp metric 20
root@leaf11:~#
root@leaf11:~# ip nexthop | grep group
id 20 group 21,254/22,254/23,254/24,254 proto zebra
root@leaf11:~#
root@leaf11:~# vtysh -c "show nexthop-group rib 20"
ID: 20 (zebra)
RefCnt: 6
Uptime: 00:00:46
VRF: default(No AFI)
Nexthop Count: 4
Valid, Installed
Depends: (21) (22) (23) (24)
via fe80::146f:98ff:fe54:d087, leaf11-eth0 (vrf default), weight 254
via fe80::181a:f0ff:feab:f2f1, leaf11-eth1 (vrf default), weight 254
via fe80::4804:4dff:fe6f:2dfc, leaf11-eth2 (vrf default), weight 254
via fe80::6444:f3ff:fe4d:7360, leaf11-eth3 (vrf default), weight 254
root@leaf11:~#
root@leaf11:~# ip link set leaf11-eth1 down --> Bring down leaf11-eth1 link
root@leaf11:~#
root@leaf11:~# ip route | grep nhid --> routes moved to nhid 39
11.0.0.0/8 nhid 13 dev lo proto 196 metric 20
12.0.0.0/8 nhid 39 proto bgp metric 20
21.0.0.0/8 nhid 39 proto bgp metric 20
22.0.0.0/8 nhid 39 proto bgp metric 20
root@leaf11:~#
root@leaf11:~# ip nexthop | grep group
id 20 group 21,254/23,254/24,254 proto zebra
id 39 group 21,254/23,254/24,254 proto zebra --> new nhid 39 created
root@leaf11:~#
root@leaf11:~# vtysh -c "show nexthop-group rib 20"
ID: 20 (zebra)
RefCnt: 1 Time to Deletion: 00:02:46
Uptime: 00:01:06
VRF: default(No AFI)
Nexthop Count: 4
Valid, Installed
Depends: (21) (22) (23) (24)
via fe80::146f:98ff:fe54:d087, leaf11-eth0 (vrf default), weight 254
via fe80::181a:f0ff:feab:f2f1, leaf11-eth1 (vrf default), weight 254 --> even though leaf11-eth1 is down, it is not marked as inactive
via fe80::4804:4dff:fe6f:2dfc, leaf11-eth2 (vrf default), weight 254
via fe80::6444:f3ff:fe4d:7360, leaf11-eth3 (vrf default), weight 254
root@leaf11:~#
root@leaf11:~# vtysh -c "show nexthop-group rib 39" --> new nhid 39 is created
ID: 39 (zebra)
RefCnt: 6
Uptime: 00:00:19
VRF: default(No AFI)
Nexthop Count: 3
Valid, Installed
Depends: (21) (23) (24)
via fe80::146f:98ff:fe54:d087, leaf11-eth0 (vrf default), weight 254
via fe80::4804:4dff:fe6f:2dfc, leaf11-eth2 (vrf default), weight 254
via fe80::6444:f3ff:fe4d:7360, leaf11-eth3 (vrf default), weight 254
root@leaf11:~#
root@leaf11:~# ip link set leaf11-eth1 up ---> bring up leaf11-eth1
root@leaf11:~#
root@leaf11:~# ip route | grep nhid ---> all routes moved from nhid 39 to nhid 20 again
11.0.0.0/8 nhid 13 dev lo proto 196 metric 20
12.0.0.0/8 nhid 20 proto bgp metric 20
21.0.0.0/8 nhid 20 proto bgp metric 20
22.0.0.0/8 nhid 20 proto bgp metric 20
root@leaf11:~#
root@leaf11:~# ip nexthop | grep group
id 20 group 21,254/22,254/23,254/24,254 proto zebra
id 39 group 21,254/23,254/24,254 proto zebra
root@leaf11:~#
root@leaf11:~# vtysh -c "show nexthop-group rib 20"
ID: 20 (zebra)
RefCnt: 6
Uptime: 00:00:39
VRF: default(No AFI)
Nexthop Count: 4
Valid, Installed
Depends: (21) (22) (23) (24)
via fe80::146f:98ff:fe54:d087, leaf11-eth0 (vrf default), weight 254
via fe80::181a:f0ff:feab:f2f1, leaf11-eth1 (vrf default), weight 254
via fe80::4804:4dff:fe6f:2dfc, leaf11-eth2 (vrf default), weight 254
via fe80::6444:f3ff:fe4d:7360, leaf11-eth3 (vrf default), weight 254
root@leaf11:~#
root@leaf11:~# vtysh -c "show nexthop-group rib 39"
ID: 39 (zebra)
RefCnt: 1 Time to Deletion: 00:02:16 ---> nhid marked for deletion
Uptime: 00:01:21
VRF: default(No AFI)
Nexthop Count: 3
Valid, Installed
Depends: (21) (23) (24)
via fe80::146f:98ff:fe54:d087, leaf11-eth0 (vrf default), weight 254
via fe80::4804:4dff:fe6f:2dfc, leaf11-eth2 (vrf default), weight 254
via fe80::6444:f3ff:fe4d:7360, leaf11-eth3 (vrf default), weight 254
root@leaf11:~#
After fix:
----------
root@leaf11:~# ip route show | grep nhid
11.0.0.0/8 nhid 13 dev lo proto 196 metric 20
12.0.0.0/8 nhid 20 proto bgp metric 20
21.0.0.0/8 nhid 20 proto bgp metric 20
22.0.0.0/8 nhid 20 proto bgp metric 20
root@leaf11:~# ip nexthop show | grep group
id 20 group 21,254/22,254/23,254/24,254 proto zebra
root@leaf11:~# vtysh -c "show nexthop-group rib 20"
ID: 20 (zebra)
RefCnt: 6
Uptime: 00:01:52
VRF: default(No AFI)
Nexthop Count: 4
Valid, Installed
Depends: (21) (22) (23) (24)
via fe80::1c71:39ff:febc:5328, leaf11-eth2 (vrf default), weight 254
via fe80::5c54:11ff:fef3:3fe3, leaf11-eth1 (vrf default), weight 254
via fe80::8c06:3bff:fe29:752f, leaf11-eth0 (vrf default), weight 254
via fe80::f89e:7fff:fea3:8c7, leaf11-eth3 (vrf default), weight 254
root@leaf11:~#
root@leaf11:~# ip link set leaf11-eth1 down ---> leaf11-eth1 link is down
root@leaf11:~# vtysh -c "show nexthop-group rib 20" ---> NHG retained
ID: 20 (zebra)
RefCnt: 6
Uptime: 00:02:23
VRF: default(No AFI)
Nexthop Count: 4
Valid, Installed
Depends: (21) (22) (23) (24)
via fe80::1c71:39ff:febc:5328, leaf11-eth2 (vrf default), weight 254
via fe80::5c54:11ff:fef3:3fe3, leaf11-eth1 (vrf default) inactive, weight 254 <<< leaf11-eth1 NH is marked inactive
via fe80::8c06:3bff:fe29:752f, leaf11-eth0 (vrf default), weight 254
via fe80::f89e:7fff:fea3:8c7, leaf11-eth3 (vrf default), weight 254
root@leaf11:~# ip route show | grep nhid ---> routes still pointing to the same NHG 20
11.0.0.0/8 nhid 13 dev lo proto 196 metric 20
12.0.0.0/8 nhid 20 proto bgp metric 20
21.0.0.0/8 nhid 20 proto bgp metric 20
22.0.0.0/8 nhid 20 proto bgp metric 20
root@leaf11:~#
root@leaf11:~# ip nexthop show | grep group ---> no additional NHG's are created
id 20 group 21,254/23,254/24,254 proto zebra
root@leaf11:~#
root@leaf11:~# ip link set leaf11-eth1 up ---> leaf11-eth1 link is up
root@leaf11:~#
root@leaf11:~# vtysh -c "show nexthop-group rib 20" ---> NHG retained
ID: 20 (zebra)
RefCnt: 6
Uptime: 00:02:54
VRF: default(No AFI)
Nexthop Count: 4
Valid, Installed
Depends: (21) (22) (23) (24)
via fe80::1c71:39ff:febc:5328, leaf11-eth2 (vrf default), weight 254
via fe80::5c54:11ff:fef3:3fe3, leaf11-eth1 (vrf default), weight 254 <<< leaf11-eth1 NH is marked active
via fe80::8c06:3bff:fe29:752f, leaf11-eth0 (vrf default), weight 254
via fe80::f89e:7fff:fea3:8c7, leaf11-eth3 (vrf default), weight 254
root@leaf11:~#
root@leaf11:~# ip route show | grep nhid ---> routes still pointing to the same NHG 20
11.0.0.0/8 nhid 13 dev lo proto 196 metric 20
12.0.0.0/8 nhid 20 proto bgp metric 20
21.0.0.0/8 nhid 20 proto bgp metric 20
22.0.0.0/8 nhid 20 proto bgp metric 20
root@leaf11:~#
root@leaf11:~# ip nexthop show | grep group ---> no additional NHG's are created
id 20 group 21,254/22,254/23,254/24,254 proto zebra
root@leaf11:~#
Ticket: #
Signed-off-by: Karthikeya Venkat Muppalla <kmuppalla@nvidia.com>
(cherry picked from commit ed3aab6)
# Conflicts:
# zebra/zebra_nhg.c
Add tests for nexthop group (NHG) stability with Weighted ECMP in high-density CLOS topologies. This test addresses weight mismatch issues in dependent NHGs where link state changes previously caused unnecessary NHG deletion/recreation. Test topology: - 2-layer CLOS with 2 spines and 2 leafs - 64 parallel links per leaf (32 to each spine) - 1000 test routes injected via sharpd Test scenarios: - Single link down/up - Partial links down/up (16 out of 32 links to spine1) Key verification points: - NHG ID remains stable during all link state changes - No new NHG creation during local link events - All 1000 routes continue using same NHG ID - No unnecessary route reprogramming occurs Ticket: # Signed-off-by: Karthikeya Venkat Muppalla <kmuppalla@nvidia.com> (cherry picked from commit 5a792e9)
|
Cherry-pick of ed3aab6 has failed: To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With W-ECMP, when an interface goes down, we mark the singleton nexthop as INACTIVE. We then process the dependents (NHG groups containing this singleton nexthop) and attempt to mark them as INACTIVE as well.
During this process, we compare all singleton nexthops in the nexthop group with the singleton nexthop that went down using nexthop_same() in zebra_nhg_set_valid().
However, there's a weight mismatch issue:
Testing:
Ticket: #
This is an automatic backport of pull request #18947 done by Mergify.