Skip to content

Conversation

miri64
Copy link
Member

@miri64 miri64 commented Mar 24, 2020

Contribution description

This provides an implementation for lwIP of the zero-copy receive functions (see #13616). As the implementation of sock_*_recv() now uses those functions, I only adapted the tests minimally to test the success case to acknowledge the function is there.

I am not 100% sure it is okay, not to loop over the netbuf in sock_udp (for sock_ip I did not), but testing will tell.

Testing procedure

Should still pass (only LWIP_IPV6=1 is tested by the CI):

BOARD="<pick one>"
for test in tests/lwip_sock_{ip,udp}; do
    LWIP_IPV6=1 make -C ${test} clean flash -j16 test;
    LWIP_IPV4=1 make -C ${test} clean flash -j16 test;
    LWIP_IPV4=1 LWIP_IPV6=1 make -C ${test} clean flash -j16 test;
done

Also please, please, please test with (IP) fragmented packets (so larger than MTU) for sock_udp with tests/lwip. I did not have any time to test that right now, but we really should, as I removed a copying loop and replaced it just with a pointer assignment (see #13701 (comment) for procedure).

Issues/PRs references

Follow-up to #13616.
Requires #13779 (merged).

@miri64 miri64 added Area: network Area: Networking Type: new feature The issue requests / The PR implemements a new feature for RIOT Area: pkg Area: External package ports CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR labels Mar 24, 2020
@miri64 miri64 force-pushed the lwip_sock/enh/recv_buf branch from 54aefea to 1870f3e Compare April 1, 2020 11:32
@miri64 miri64 added State: waiting for other PR State: The PR requires another PR to be merged first CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR State: waiting for other PR State: The PR requires another PR to be merged first labels Apr 1, 2020
@benpicco
Copy link
Contributor

benpicco commented Apr 3, 2020

If this can be reviewed now, just squash - there hasn't been a review yet.

@pokgak
Copy link
Contributor

pokgak commented Apr 25, 2020

I'm not really familiar with lwip but I compared this PR with how sock_udp_recv_buf is implemented and used in gnrc and overall it looks good to me. For testing, I ran the command described in the testing procedure above and all tests completed successfully. I'm not sure how to tests using fragmented packets. Do you have an extra application written for that @miri64?

Sorry, I cannot add much to the review but I can help tests it locally to confirm that it is working.

@miri64 miri64 force-pushed the lwip_sock/enh/recv_buf branch from 78f11e0 to 53b5f37 Compare April 28, 2020 07:46
@miri64
Copy link
Member Author

miri64 commented Apr 28, 2020

If this can be reviewed now, just squash - there hasn't been a review yet.

Squashed

@miri64
Copy link
Member Author

miri64 commented Apr 28, 2020

I'm not sure how to tests using fragmented packets. Do you have an extra application written for that @miri64?

No, not really. However, tests/gnrc_udp has IPv6 fragmentation activated per default. You can just use that for sending UDP packets or pinging the lwIP node with payloads greater than 1432 (to exceed the Ethernet SDU).

@benpicco
Copy link
Contributor

benpicco commented May 4, 2020

Can this be tested entirely on native?

@miri64
Copy link
Member Author

miri64 commented May 15, 2020

Can this be tested entirely on native?

Yepp, the automated version of tests/lwip even only runs on native ^^"

@benpicco
Copy link
Contributor

Also please, please, please test with (IP) fragmented packets (so larger than MTU) for sock_udp with tests/lwip. I did not have any time to test that right now, but we really should, as I removed a copying loop and replaced it just with a pointer assignment.

I reckon make test will not cover that, right?
'larger than MTU' here means > 1280 bytes and the console commands take payload as hex string. I'm afraid there is no way to produce such a packet with that test.

@miri64
Copy link
Member Author

miri64 commented May 19, 2020

I reckon make test will not cover that, right?

No.

'larger than MTU' here means > 1280 bytes and the console commands take payload as hex string. I'm afraid there is no way to produce such a packet with that test.

As I wrote: use tests/gnrc_udp for that. Give me minute, then I will provide a testing procedure for that...

@miri64
Copy link
Member Author

miri64 commented May 19, 2020

I adapted tests/lwip to be able to receive such large packets:

diff --git a/tests/lwip/common.h b/tests/lwip/common.h
index 4cea2573df..6d0778a6f6 100644
--- a/tests/lwip/common.h
+++ b/tests/lwip/common.h
@@ -29,7 +29,7 @@ extern "C" {
  * @brief   Application configuration
  * @{
  */
-#define SOCK_INBUF_SIZE         (256)
+#define SOCK_INBUF_SIZE         (1500)
 #define SERVER_MSG_QUEUE_SIZE   (8)
 #define SERVER_BUFFER_SIZE      (64)
 /**

Then I started 3 terminals:

Terminal 1 (tshark for sniffing)

might also show other packets; the important part is the IPv6 fragment part

$ tshark -i tap0
Capturing on 'tap0'
    1 0.000000000 fe80::d827:1dff:fea8:6424 → fe80::e0bc:7dff:fecb:f550 IPv6 1510 IPv6 fragment (off=0 more=y ident=0xbf684be8 nxt=17)
    2 0.000158170 fe80::d827:1dff:fea8:6424 → fe80::e0bc:7dff:fecb:f550 UDP 122 1337 → 1337 Len=1500
Terminal 2 (tests/lwip as UDP server)
$ make -C tests/lwip all term
[…]
> ifconfig
ifconfig
ET_00:  inet6 fe80::e0bc:7dff:fecb:f550

> udp server start 1337
udp server start 1337
Success: started UDP server on port 1337
> Received UDP data from [fe80::d827:1dff:fea8:6424]:1337:
00000000  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000010  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000020  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000030  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000040  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000050  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000060  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000070  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000080  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000090  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000000A0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000000B0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000000C0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000000D0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000000E0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000000F0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000100  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000110  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000120  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000130  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000140  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000150  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000160  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000170  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000180  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000190  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000001A0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000001B0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000001C0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000001D0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000001E0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000001F0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000200  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000210  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000220  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000230  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000240  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000250  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000260  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000270  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000280  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000290  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000002A0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000002B0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000002C0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000002D0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000002E0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000002F0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000300  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000310  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000320  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000330  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000340  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000350  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000360  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000370  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000380  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000390  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000003A0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000003B0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000003C0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000003D0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000003E0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000003F0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000400  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000410  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000420  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000430  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000440  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000450  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000460  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000470  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000480  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000490  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000004A0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000004B0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000004C0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000004D0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000004E0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000004F0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000500  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000510  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000520  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000530  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000540  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000550  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000560  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000570  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000580  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
00000590  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000005A0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000005B0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000005C0  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01  01
000005D0  01  01  01  01  01  01  01  01  01  01  01  01

(1500 in hex is 0x5dc so we received the correct number of bytes)

Terminal 3 (tests/gnrc_udp as UDP client)
$ make -C tests/gnrc_udp all term
> udp send fe80::e0bc:7dff:fecb:f550 1337 1500                              
udp send fe80::e0bc:7dff:fecb:f550 1337 1500
Success: send 1500 byte to [fe80::e0bc:7dff:fecb:f550]:1337

Copy link
Contributor

@MichelRottleuthner MichelRottleuthner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the automated tests from the PR description, the manual one from the previous comment, and some manual UDP traffic between Linux and RIOT via netcat. I wasn't able to break it. Code also looks good to me. Lets retrigger Murdock.

@MichelRottleuthner MichelRottleuthner removed the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label May 26, 2020
@MichelRottleuthner MichelRottleuthner added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Reviewed: 1-fundamentals The fundamentals of the PR were reviewed according to the maintainer guidelines Reviewed: 2-code-design The code design of the PR was reviewed according to the maintainer guidelines Reviewed: 3-testing The PR was tested according to the maintainer guidelines Reviewed: 4-code-style The adherence to coding conventions by the PR were reviewed according to the maintainer guidelines labels May 26, 2020
@miri64 miri64 merged commit a38428b into RIOT-OS:master May 26, 2020
@miri64 miri64 deleted the lwip_sock/enh/recv_buf branch May 26, 2020 19:33
@miri64 miri64 added this to the Release 2020.07 milestone Jun 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: network Area: Networking Area: pkg Area: External package ports CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR Reviewed: 1-fundamentals The fundamentals of the PR were reviewed according to the maintainer guidelines Reviewed: 2-code-design The code design of the PR was reviewed according to the maintainer guidelines Reviewed: 3-testing The PR was tested according to the maintainer guidelines Reviewed: 4-code-style The adherence to coding conventions by the PR were reviewed according to the maintainer guidelines Type: new feature The issue requests / The PR implemements a new feature for RIOT
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants