examples/gnrc_border_router not routing between 6lo clients and Internet #14676
Description
Description
I have seen a similar issue on esp* as border router with a direct WiFi up-link, but sniffing WiFi packets is a bit of a pain, so I tried to reproduce this using Ethernet (CDC-ECM).
The issue I see is that the border router can reach hosts outside the WPAN while it's clients cant.
There is some non-determinism involved as sometimes this works, but often it does not.
Steps to reproduce the issue
This requires an upstream Router with DHCPv6 Prefix Delegation enabled. On the common Fritz!Box, this can be enabled in Network -> Network Settings -> IPv6 Addresses:
In the following, we will request a prefix for delegation from the upstream router, then delegate that prefix to the border router so it can be distributed to the 6LoWPAN clients.
But first, let's flash the border router:
make BOARD=samr21-xpro UPLINK=cdc-ecm USE_DHCPV6=1
(This requres a second micro USB cabled plugged to the TARGET USB port of the samr21-xpro)
We should now have a usb network interface, udev gives it a unique name based on the USB topology.
ifconfig output
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.42.69 netmask 255.255.255.0 broadcast 192.168.42.255
inet6 fe80::13ae:a2d4:3402:38fe prefixlen 64 scopeid 0x20<link>
inet6 2001:16b8:45ca:b600:c732:1a76:eedb:8035 prefixlen 64 scopeid 0x0<global>
ether 14:da:e9:4c:6b:2e txqueuelen 1000 (Ethernet)
RX packets 1355481 bytes 1502693074 (1.5 GB)
RX errors 0 dropped 61432 overruns 0 frame 0
TX packets 552454 bytes 72575305 (72.5 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 20 memory 0xf7f00000-f7f20000
enx0e504ad43426: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.1.2 netmask 255.255.0.0 broadcast 169.254.255.255
inet6 fe80::8520:5adc:6f1c:36f4 prefixlen 64 scopeid 0x20<link>
ether 0e:50:4a:d4:34:26 txqueuelen 1000 (Ethernet)
RX packets 13 bytes 1140 (1.1 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 198 bytes 38194 (38.1 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
I used wide-dhcpv6-client as this will automatically configure the interface.
dhcp6c.conf
interface eno1 {
send ia-pd 1;
};
id-assoc pd 1 {
prefix ::/60 infinity;
prefix-interface enx0e504ad43426 {
sla-id 2;
ifid 1;
sla-len 4;
};
};
sudo dhcp6c -c dhcp6c.conf -D -f eno1
output
Jul/31/2020 22:37:40: dhcp6_reset_timer: reset a timer on eno1, state=INIT, timeo=0, retrans=424
Jul/31/2020 22:37:40: client6_send: a new XID (27a739) is generated
Jul/31/2020 22:37:40: copy_option: set client ID (len 14)
Jul/31/2020 22:37:40: copy_option: set elapsed time (len 2)
Jul/31/2020 22:37:40: copyout_option: set IA_PD prefix
Jul/31/2020 22:37:40: copyout_option: set IA_PD
Jul/31/2020 22:37:40: client6_send: send solicit to ff02::1:2%eno1
Jul/31/2020 22:37:40: dhcp6_reset_timer: reset a timer on eno1, state=SOLICIT, timeo=0, retrans=1061
Jul/31/2020 22:37:40: client6_recv: receive advertise from fe80::de39:6fff:fe6a:6980%eno1 on eno1
Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option client ID, len 14
Jul/31/2020 22:37:40: DUID: 00:01:00:01:26:b7:24:ff:14:da:e9:4c:6b:2e
Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option server ID, len 10
Jul/31/2020 22:37:40: DUID: 00:03:00:01:dc:39:6f:6a:69:80
Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option preference, len 1
Jul/31/2020 22:37:40: preference: 0
Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option DNS, len 16
Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option opt_86, len 16
Jul/31/2020 22:37:40: dhcp6_get_options: unknown or unexpected DHCP6 option opt_86, len 16
Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option IA_PD, len 41
Jul/31/2020 22:37:40: IA_PD: ID=1, T1=1800, T2=2880
Jul/31/2020 22:37:40: copyin_option: get DHCP option IA_PD prefix, len 25
Jul/31/2020 22:37:40: copyin_option: IA_PD prefix: 2001:16b8:45ca:b6c0::/60 pltime=3600 vltime=7200
Jul/31/2020 22:37:40: client6_recvadvert: server ID: 00:03:00:01:dc:39:6f:6a:69:80, pref=0
Jul/31/2020 22:37:40: client6_recvadvert: reset timer for eno1 to 0.996041
Jul/31/2020 22:37:41: select_server: picked a server (ID: 00:03:00:01:dc:39:6f:6a:69:80)
Jul/31/2020 22:37:41: client6_send: a new XID (e7c212) is generated
Jul/31/2020 22:37:41: copy_option: set client ID (len 14)
Jul/31/2020 22:37:41: copy_option: set server ID (len 10)
Jul/31/2020 22:37:41: copy_option: set elapsed time (len 2)
Jul/31/2020 22:37:41: copyout_option: set IA_PD prefix
Jul/31/2020 22:37:41: copyout_option: set IA_PD
Jul/31/2020 22:37:41: client6_send: send request to ff02::1:2%eno1
Jul/31/2020 22:37:41: dhcp6_reset_timer: reset a timer on eno1, state=REQUEST, timeo=0, retrans=1074
Jul/31/2020 22:37:41: client6_recv: receive reply from fe80::de39:6fff:fe6a:6980%eno1 on eno1
Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option client ID, len 14
Jul/31/2020 22:37:41: DUID: 00:01:00:01:26:b7:24:ff:14:da:e9:4c:6b:2e
Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option server ID, len 10
Jul/31/2020 22:37:41: DUID: 00:03:00:01:dc:39:6f:6a:69:80
Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option preference, len 1
Jul/31/2020 22:37:41: preference: 0
Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option DNS, len 16
Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option opt_86, len 16
Jul/31/2020 22:37:41: dhcp6_get_options: unknown or unexpected DHCP6 option opt_86, len 16
Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option IA_PD, len 41
Jul/31/2020 22:37:41: IA_PD: ID=1, T1=1800, T2=2880
Jul/31/2020 22:37:41: copyin_option: get DHCP option IA_PD prefix, len 25
Jul/31/2020 22:37:41: copyin_option: IA_PD prefix: 2001:16b8:45ca:b6c0::/60 pltime=3600 vltime=7200
Jul/31/2020 22:37:41: info_printf: nameserver[0] fd00::de39:6fff:fe6a:6980
Jul/31/2020 22:37:41: get_ia: make an IA: PD-1
Jul/31/2020 22:37:41: update_prefix: create a prefix 2001:16b8:45ca:b6c0::/60 pltime=3600, vltime=7200
Jul/31/2020 22:37:41: ifaddrconf: add an address 2001:16b8:45ca:b6c2::1/64 on enx0e504ad43426
Jul/31/2020 22:37:41: dhcp6_remove_event: removing an event on eno1, state=REQUEST
Jul/31/2020 22:37:41: dhcp6_remove_event: removing server (ID: 00:03:00:01:dc:39:6f:6a:69:80)
Jul/31/2020 22:37:41: client6_recvreply: got an expected reply, sleeping.
So we got the prefix 2001:16b8:45ca:b6c0::/60.
We should also start radvd on the interface. We can use the ::/64 prefix to automatically select a prefix from the /60 we got:
sudo dist/tools/radvd/radvd.sh -c enx0e504ad43426 ::/64
ifconfig afterwards
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.42.69 netmask 255.255.255.0 broadcast 192.168.42.255
inet6 fe80::13ae:a2d4:3402:38fe prefixlen 64 scopeid 0x20<link>
inet6 2001:16b8:45ca:b600:c732:1a76:eedb:8035 prefixlen 64 scopeid 0x0<global>
ether 14:da:e9:4c:6b:2e txqueuelen 1000 (Ethernet)
RX packets 1365523 bytes 1512790982 (1.5 GB)
RX errors 0 dropped 62554 overruns 0 frame 0
TX packets 556281 bytes 73155386 (73.1 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 20 memory 0xf7f00000-f7f20000
enx0e504ad43426: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.1.2 netmask 255.255.0.0 broadcast 169.254.255.255
inet6 fe80::8520:5adc:6f1c:36f4 prefixlen 64 scopeid 0x20<link>
inet6 2001:16b8:45ca:b6c2::1 prefixlen 64 scopeid 0x0<global>
inet6 2001:16b8:45ca:b6c2:e8c6:3805:9c87:6c26 prefixlen 64 scopeid 0x0<global>
ether 0e:50:4a:d4:34:26 txqueuelen 1000 (Ethernet)
RX packets 9 bytes 816 (816.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 117 bytes 21513 (21.5 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions
Our border router should now have configured an address on it's upstream interface via router advertisements.
ifconfig RIOT
2020-07-31 22:41:38,679 # Iface 8 HWaddr: 0E:50:4A:D4:34:25
2020-07-31 22:41:38,684 # L2-PDU:1500 MTU:1500 HL:64 RTR
2020-07-31 22:41:38,687 # Source address length: 6
2020-07-31 22:41:38,690 # Link type: wired
2020-07-31 22:41:38,695 # inet6 addr: fe80::c50:4aff:fed4:3425 scope: link VAL
2020-07-31 22:41:38,703 # inet6 addr: 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425 scope: global VAL
2020-07-31 22:41:38,705 # inet6 group: ff02::2
2020-07-31 22:41:38,708 # inet6 group: ff02::1
2020-07-31 22:41:38,712 # inet6 group: ff02::1:ffd4:3425
2020-07-31 22:41:38,713 #
2020-07-31 22:41:38,720 # Iface 7 HWaddr: 71:A7 Channel: 26 Page: 0 NID: 0x23 PHY: O-QPSK
2020-07-31 22:41:38,721 #
2020-07-31 22:41:38,725 # Long HWaddr: 5A:06:59:C9:2C:36:7D:7A
2020-07-31 22:41:38,732 # TX-Power: 0dBm State: IDLE max. Retrans.: 3 CSMA Retries: 4
2020-07-31 22:41:38,739 # AUTOACK ACK_REQ CSMA L2-PDU:102 MTU:1280 HL:64 RTR
2020-07-31 22:41:38,742 # RTR_ADV 6LO IPHC
2020-07-31 22:41:38,745 # Source address length: 8
2020-07-31 22:41:38,748 # Link type: wireless
2020-07-31 22:41:38,754 # inet6 addr: fe80::5806:59c9:2c36:7d7a scope: link VAL
2020-07-31 22:41:38,757 # inet6 group: ff02::2
2020-07-31 22:41:38,759 # inet6 group: ff02::1
2020-07-31 22:41:38,763 # inet6 group: ff02::1:ff36:7d7a
To get a prefix on the downstream interface, we have to enable the DHCPv6 server
make BOARD=samr21-xpro UPLINK=cdc-ecm USE_DHCPV6=1 IPV6_PREFIX=2001:16b8:45ca:b6c0::/61 PORT= term
new ifconfig
2020-07-31 22:46:17,746 # Iface 8 HWaddr: 0E:50:4A:D4:34:25
2020-07-31 22:46:17,751 # L2-PDU:1500 MTU:1500 HL:64 RTR
2020-07-31 22:46:17,754 # Source address length: 6
2020-07-31 22:46:17,757 # Link type: wired
2020-07-31 22:46:17,765 # inet6 addr: fe80::c50:4aff:fed4:3425 scope: link VAL
2020-07-31 22:46:17,770 # inet6 addr: 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425 scope: global VAL
2020-07-31 22:46:17,773 # inet6 group: ff02::2
2020-07-31 22:46:17,775 # inet6 group: ff02::1
2020-07-31 22:46:17,779 # inet6 group: ff02::1:ffd4:3425
2020-07-31 22:46:17,780 #
2020-07-31 22:46:17,787 # Iface 7 HWaddr: 71:A7 Channel: 26 Page: 0 NID: 0x23 PHY: O-QPSK
2020-07-31 22:46:17,788 #
2020-07-31 22:46:17,792 # Long HWaddr: 5A:06:59:C9:2C:36:7D:7A
2020-07-31 22:46:17,799 # TX-Power: 0dBm State: IDLE max. Retrans.: 3 CSMA Retries: 4
2020-07-31 22:46:17,806 # AUTOACK ACK_REQ CSMA L2-PDU:102 MTU:1280 HL:64 RTR
2020-07-31 22:46:17,809 # RTR_ADV 6LO IPHC
2020-07-31 22:46:17,812 # Source address length: 8
2020-07-31 22:46:17,815 # Link type: wireless
2020-07-31 22:46:17,821 # inet6 addr: fe80::5806:59c9:2c36:7d7a scope: link VAL
2020-07-31 22:46:17,828 # inet6 addr: 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a scope: global VAL
2020-07-31 22:46:17,831 # inet6 group: ff02::2
2020-07-31 22:46:17,834 # inet6 group: ff02::1
2020-07-31 22:46:17,840 # inet6 group: ff02::1:ff36:7d7a
2020-07-31 22:46:17,841 #
> ping6 2600::
2020-07-31 22:46:26,583 # ping6 2600::
2020-07-31 22:46:26,724 # 12 bytes from 2600::: icmp_seq=0 ttl=49 time=135.199 ms
2020-07-31 22:46:27,724 # 12 bytes from 2600::: icmp_seq=1 ttl=49 time=133.682 ms
2020-07-31 22:46:28,726 # 12 bytes from 2600::: icmp_seq=2 ttl=49 time=134.162 ms
2020-07-31 22:46:28,726 #
2020-07-31 22:46:28,729 # --- 2600:: PING statistics ---
2020-07-31 22:46:28,734 # 3 packets transmitted, 3 packets received, 0% packet loss
2020-07-31 22:46:28,739 # round-trip min/avg/max = 133.682/134.347/135.199 ms
2020-07-31 22:51:22,226 # nib neigh
2020-07-31 22:51:22,233 # fe80::8520:5adc:6f1c:36f4 dev #8 lladdr 0E:50:4A:D4:34:26 router REACHABLE GC
2020-07-31 22:51:22,241 # 2001:16b8:45ca:b6c2:fec2:3d00:0:bb1 dev #7 lladdr FC:C2:3D:00:00:00:0B:B1 STALE REGISTERED
Expected results
6LoWPAN clients got a global address and can reach other global addresses.
Actual results
Clients get a global address, but packets are not routed
2020-07-31 22:48:16,109 # Iface 7 HWaddr: 0B:B1 Channel: 26 Page: 0 NID: 0x23 PHY: O-QPSK
2020-07-31 22:48:16,110 #
2020-07-31 22:48:16,115 # Long HWaddr: FC:C2:3D:00:00:00:0B:B1
2020-07-31 22:48:16,121 # TX-Power: 0dBm State: IDLE max. Retrans.: 3 CSMA Retries: 4
2020-07-31 22:48:16,128 # AUTOACK ACK_REQ CSMA L2-PDU:102 MTU:1280 HL:64 RTR
2020-07-31 22:48:16,131 # RTR_ADV 6LO IPHC
2020-07-31 22:48:16,134 # Source address length: 8
2020-07-31 22:48:16,137 # Link type: wireless
2020-07-31 22:48:16,143 # inet6 addr: fe80::fec2:3d00:0:bb1 scope: link VAL
2020-07-31 22:48:16,151 # inet6 addr: 2001:16b8:45ca:b6c2:fec2:3d00:0:bb1 scope: global VAL
2020-07-31 22:48:16,154 # inet6 group: ff02::2
2020-07-31 22:48:16,157 # inet6 group: ff02::1
2020-07-31 22:48:16,160 # inet6 group: ff02::1:ff00:bb1
2020-07-31 22:48:16,160 #
2020-07-31 22:48:16,163 # Statistics for Layer 2
2020-07-31 22:48:16,166 # RX packets 4 bytes 286
2020-07-31 22:48:16,172 # TX packets 54 (Multicast: 52) bytes 2364
2020-07-31 22:48:16,175 # TX succeeded 54 errors 0
2020-07-31 22:48:16,178 # Statistics for IPv6
2020-07-31 22:48:16,181 # RX packets 3 bytes 304
2020-07-31 22:48:16,186 # TX packets 54 (Multicast: 52) bytes 3488
2020-07-31 22:48:16,190 # TX succeeded 54 errors 0
2020-07-31 22:48:16,191 #
> ping6 2600::
2020-07-31 22:48:20,111 # ping6 2600::
2020-07-31 22:48:23,113 #
2020-07-31 22:48:23,114 # --- 2600:: PING statistics ---
2020-07-31 22:48:23,121 # 3 packets transmitted, 0 packets received, 100% packet loss
> ping6 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a
2020-07-31 22:49:39,113 # ping6 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a
2020-07-31 22:49:39,134 # 12 bytes from 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a: icmp_seq=0 ttl=64 rssi=-18 dBm time=10.944 ms
2020-07-31 22:49:40,135 # 12 bytes from 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a: icmp_seq=1 ttl=64 rssi=-18 dBm time=12.000 ms
2020-07-31 22:49:41,134 # 12 bytes from 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a: icmp_seq=2 ttl=64 rssi=-18 dBm time=10.720 ms
2020-07-31 22:49:41,135 #
2020-07-31 22:49:41,140 # --- 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a PING statistics ---
2020-07-31 22:49:41,144 # 3 packets transmitted, 3 packets received, 0% packet loss
2020-07-31 22:49:41,150 # round-trip min/avg/max = 10.720/11.221/12.000 ms
> ping6 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425
2020-07-31 22:49:45,321 # ping6 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425
2020-07-31 22:49:48,323 #
2020-07-31 22:49:48,326 # --- 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425 PING statistics ---
2020-07-31 22:49:48,333 # 3 packets transmitted, 0 packets received, 100% packet loss
> nib neigh
2020-07-31 22:50:09,942 # nib neigh
2020-07-31 22:50:09,949 # fe80::5806:59c9:2c36:7d7a dev #7 lladdr 5A:06:59:C9:2C:36:7D:7A router STALE GC
Instead a neighbor solicitation for the global address is performed.