Skip to content

pkg/wolfmqtt: add wolfmqtt #15969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open

pkg/wolfmqtt: add wolfmqtt #15969

wants to merge 20 commits into from
+3,847 −105

Conversation

HendrikVE
Copy link
Contributor

@HendrikVE HendrikVE commented Feb 9, 2021 •
edited
Loading

This PR adds the new package wolfmqtt. WolfMQTT is somewhat of an all-inclusive MQTT Client.

Features (of the library)

  • MQTT (version 3 and 5)
  • MQTT-SN
  • Secure MQTT connections by using wolfSSL
  • Several options when compiling
  • Several example applications demonstrating the capabilities
    • other examples would allow us to connect to web services like AWS and Azure!

I had to do some fixes/modifications on posix_sockets. The example sn-client requires support for MSG_PEEK, which was not supported before. Also posix_socket did not even compile for TCP at all.

Some examples were added by using the package's sourcecode. Necessary modifications to make them run on RIOT were added via patch. E.g. I had to modify the part that was using getaddrinfo which is missing in RIOT (see #7801).

Because I don't know much about developing network stacks I just hacked some solutions for my problems right now. @miri64 can give me some hints I guess? :)

@danielinux I've got confused exactly the same way as @rugo about the state of support of wolfssl in RIOT. See #14915. Could you shed some light on this? Standard TLS support is essential for this library, because this would give us the option to connect to services which require a secured MQTT connection. And that is really missing in RIOT.

Also it would be great if someone could come up with a better way to determine whether the device is connected to a network or not. Currently the esp32 sleeps for a few seconds until it is connected to my Wifi, which does not feel like the optimal solution.

My setup

  • ESP32 with Wifi connection
  • LwIP (same reason like for paho-mqtt, because of TCP)
  • bunch of posix modules to easily port the library
    • posix_inet, posix_sockets, posix_select, posix_semaphore, pthread
  • IPv4 and IPv6
  • MQTT-SN Gateway (eclipse/paho.mqtt-sn.embedded-c)
  • Compiling and running all given examples with their standard configuration given in user_settings.h

Open issues (help needed) are marked with a checkbox.

Options

  • WOLFMQTT_V5
    • compilation: OK
    • execution: OK
    • WOLFMQTT_PROPERTY_CB (requires WOLFMQTT_V5)
      • compilation: OK
      • execution: OK
  • WOLFMQTT_DISCONNECT_CB
    • compilation: OK
    • execution: OK
  • WOLFMQTT_ENABLE_STDIN_CAP
    • defining WOLFMQTT_NO_STDIO, which disables WOLFMQTT_ENABLE_STDIN_CAP, leads to a bunch of compilation errors
  • WOLFMQTT_NO_TIMEOUT
    • compilation: OK
    • execution: OK, but ...
      • seems to have no effect
  • ENABLE_MQTT_TLS
    • not part of this PR

Examples

  • mqttclient
    • compilation: OK
    • execution: OK
  • sn-client
    • compilation: OK
    • execution: OK
  • multithread
    • compilation: OK, but...
      • patch providing an empty implementation of 'pipe' needed
    • execution: OK
  • nbclient
    • compilation: OK, but ...
      • 'EWOULDBLOCK' and 'EAGAIN' were undefined, even though 'errno.h' is included. I had to define them in the Makefile manually
    • execution: OK, but ...
      • "fcntl get failed!"
      • "fcntl set failed!"

TODOs:

  • all above mentioned issues
  • see TODO at dd209a9
  • for the examples to work we need to ignore an error in 'mqttnet.c'
    • see TODO at 8ccc2e3 in mqttnet.c (FD_ISSET(sock->fd, &errfds))
  • replace xtimer_sleep call in the examples by something more convenient when waiting for the network to connect

Dependencies

@HendrikVE HendrikVE requested a review from miri64 as a code owner February 9, 2021 17:36
@HendrikVE HendrikVE added Area: examples Area: Example Applications Area: POSIX Area: POSIX API wrapper Type: new feature The issue requests / The PR implemements a new feature for RIOT Area: pkg Area: External package ports labels Feb 9, 2021
@HendrikVE HendrikVE changed the title Add wolfmqtt package WIP: Add wolfmqtt package Feb 9, 2021
@HendrikVE HendrikVE mentioned this pull request Feb 9, 2021
Open
2 tasks
@aabadie
Copy link
Contributor

aabadie commented Feb 9, 2021

Awesome PR. Many thanks for working on that. It could be of interest to @javierfileiv.

@miri64
Copy link
Member

miri64 commented Feb 9, 2021

I can only join @aabadie in saying "Awesome PR!" :-)

@danielinux I've got confused exactly the same way as @rugo about the state of support of wolfssl in RIOT. See #14915. Could you shed some light on this? Standard TLS support is essential for this library, because this would give us the option to connect to services which require a secured MQTT connection. And that is really missing in RIOT.

I think a common sock_tls, like we have for sock_dtls would be great, especially considering that such a thing could be useful in the future for implementing the QUIC protocol (which could also be used as a transport for MQTT).

@javierfileiv
Copy link
Contributor

This is great! Awesome job! I'll leave my PR aside for the moment 😉😉

@danielinux
Copy link
Contributor

@HendrikVE wolfSSL supports posix sockets when used in combination with lwIP. The sock_dtls is for use in combination with gnrc/UDP.

Check pkg/wolfssl/Makefile.dep : there is a wolfssl_socket pseudo-module that provides access to TCP via lwip posix_sockets.

@embhorn
Copy link

embhorn commented Feb 25, 2021

@HendrikVE - This is looking great! Just wanted to add that you could create a RIOT section inside the network porting layer
https://github.com/wolfSSL/wolfMQTT/blob/master/examples/mqttnet.c
That could account for the network init delay instead of modifying all the examples. That file is not inside the library proper and is intended to be modified for any needed network interface.

@HendrikVE HendrikVE force-pushed the wolfmqtt_lwip branch 2 times, most recently from d62e64f to 1ebf321 Compare March 3, 2021 21:28
@HendrikVE
Copy link
Contributor Author

HendrikVE commented Mar 3, 2021 •
edited
Loading

This is looking great! Just wanted to add that you could create a RIOT section inside the network porting layer
https://github.com/wolfSSL/wolfMQTT/blob/master/examples/mqttnet.c

Good point thx, I moved the call out of the examples into mqttnet.c. But still, does someone from RIOT know how to wait properly for the network to connect? Any value I could read in a loop until it's connected or something?

Check pkg/wolfssl/Makefile.dep : there is a wolfssl_socket pseudo-module that provides access to TCP via lwip posix_sockets.

Thx, I updated the wolfmqtt_aws example. I had to do some modifications on the existing wolfssl configuration.

  1. Move the user_settings.h from include/ to include/wolfssl/ so we can include it with #include "wolfssl/user_settings.h". Otherwise it clashes with the user_settings.h from the wolfmqtt examples (see 4bd6219)
  2. Don't define NO_MAIN_DRIVER when using MODULE_WOLFMQTT. The examples won't do their job with this variable defined. (see bc7ffdb)
  3. Define NO_ASN_TIME, because the date verification fails (see 1ebf321)

wolfmqtt_aws now runs to a certain point, but the TLS handshake fails. Here my output:

2021-03-03 22:16:14,497 # MQTT Net Init: Success (0)
2021-03-03 22:16:14,499 # MQTT Init: Success (0)
2021-03-03 22:16:14,505 # NetConnect: Host 34.210.92.83, Port 8883, Timeout 5000 ms, Use TLS 1
2021-03-03 22:16:14,765 # MQTT TLS Setup (1)
2021-03-03 22:16:15,403 # MQTT TLS Verify Callback: PreVerify 1, Error 0 (none)
2021-03-03 22:16:15,409 #   Subject's domain name is Symantec Class 3 Secure Server CA - G4
2021-03-03 22:16:15,527 # MQTT TLS Verify Callback: PreVerify 1, Error 0 (none)
2021-03-03 22:16:15,532 #   Subject's domain name is *.iot.us-west-2.amazonaws.com
2021-03-03 22:16:15,539 # MqttSocket_TlsConnect Error -1: Num -404, Handshake message too large Error
2021-03-03 22:16:15,541 # MqttSocket_Disconnect: Rc=0
2021-03-03 22:16:15,541 # 
2021-03-03 22:16:15,543 # MqttSocket_Connect: Rc=-6
2021-03-03 22:16:15,547 # MQTT Socket Connect: Error (Network) (-8)

I checked the sizes for the condition size > MAX_HANDSHAKE_SZ. While MAX_HANDSHAKE_SZ=18462 is ok, size=7354927 is insanely large. I have no idea why?

@danielinux
Copy link
Contributor

Hello @HendrikVE,
it looks like there is still an issue with the I/O towards the socket, as the MQTT messages appear mangled.

Could you try to rebase the example on the LWIP BSD socket layer (via USEMODULE+=posix_sockets)?

An example off-tree app with TLS sockets is now available here:
https://github.com/danielinux/RIOT-wolfssl-tls-posix

Thanks,

--
Daniele

@HendrikVE
Copy link
Contributor Author

@danielinux Thank you very much for providing an example tor this! Unfortunatley I am not able to make a successful connection between server and client :( I followed the steps in the README and got the following output on the RIOT node:

tlsc 192.168.7.1
This is a TLS Client!
Client is connecting to server at address 192.168.7.1 port 11111...
ERROR: failed to connect

I also added the -x flag to wolfssl/examples/server/server -b -d, but it does not output any error after it was started.

ifconfig provides:

tap0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.7.1  netmask 255.255.255.0  broadcast 192.168.7.255
        inet6 fe80::2c55:4ff:feda:4785  prefixlen 64  scopeid 0x20<link>
        ether 2e:55:04:da:47:85  txqueuelen 1000  (Ethernet)
        RX packets 11  bytes 462 (462.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 62  bytes 6750 (6.7 KB)
        TX errors 0  dropped 16 overruns 0  carrier 0  collisions 0

Executing the server and client examples from wolfssl works so far. What did I miss?

Could you try to rebase the example on the LWIP BSD socket layer (via USEMODULE+=posix_sockets)?

The wolfmqtt_aws example is already using LWIP and posix_sockets because of USEPKG += wolfmqtt. In Makefile.dep of pkg/wolfmqtt we set the posix dependencies for wolfmqtt. Or do you mean something different?

Is it possible that my changes with 0002-adapt-network-code-to-RIOT.patch on the network code (mqttnet.c) cause the problem with the TLS? I had to workaround the fact that RIOT does not provide an implementation of getaddrinfo.

I added CFLAGS += -DDEBUG_WOLFSSL to my Makefile to get a debug output for wolfssl, which looks good until Handshake message too large appears out of nowhere.

expand for the output 

2021-03-16 20:03:30,691 # MQTT Net Init: Success (0)
2021-03-16 20:03:30,693 # MQTT Init: Success (0)
2021-03-16 20:03:30,699 # NetConnect: Host 44.241.47.76, Port 8883, Timeout 5000 ms, Use TLS 1
2021-03-16 20:03:30,965 # wolfSSL Entering wolfSSL_Init
2021-03-16 20:03:30,968 # wolfSSL Entering wolfCrypt_Init
2021-03-16 20:03:30,972 # wolfSSL Entering TLSv1_2_client_method_ex
2021-03-16 20:03:30,975 # wolfSSL Entering wolfSSL_CTX_new_ex
2021-03-16 20:03:30,978 # wolfSSL Entering wolfSSL_CertManagerNew
2021-03-16 20:03:30,982 # wolfSSL Leaving WOLFSSL_CTX_new, return 0
2021-03-16 20:03:30,986 # wolfSSL Entering wolfSSL_CTX_set_verify
2021-03-16 20:03:30,990 # wolfSSL Entering wolfSSL_CTX_load_verify_buffer_ex
2021-03-16 20:03:30,992 # Processing CA PEM file
2021-03-16 20:03:30,994 # wolfSSL Entering PemToDer
2021-03-16 20:03:30,995 # Adding a CA
2021-03-16 20:03:30,998 # wolfSSL Entering GetExplicitVersion
2021-03-16 20:03:31,001 # wolfSSL Entering GetSerialNumber
2021-03-16 20:03:31,003 # Got Cert Header
2021-03-16 20:03:31,005 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:31,008 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,009 # Got Algo ID
2021-03-16 20:03:31,010 # Getting Cert Name
2021-03-16 20:03:31,012 # Getting Cert Name
2021-03-16 20:03:31,013 # Got Subject Name
2021-03-16 20:03:31,016 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:31,018 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,019 # Got Key
2021-03-16 20:03:31,020 # Parsed Past Key
2021-03-16 20:03:31,024 # wolfSSL Entering DecodeCertExtensions
2021-03-16 20:03:31,026 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,030 # wolfSSL Entering DecodeBasicCaConstraint
2021-03-16 20:03:31,033 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,035 # wolfSSL Entering DecodeKeyUsage
2021-03-16 20:03:31,038 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,041 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,044 # wolfSSL Entering DecodeSubjKeyId
2021-03-16 20:03:31,046 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:31,049 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,050 # 	Parsed new CA
2021-03-16 20:03:31,052 # 	Freeing Parsed CA
2021-03-16 20:03:31,053 # 	Freeing der CA
2021-03-16 20:03:31,055 # 		OK Freeing der CA
2021-03-16 20:03:31,058 # wolfSSL Leaving AddCA, return 0
2021-03-16 20:03:31,059 #    Processed a CA
2021-03-16 20:03:31,063 # Processed at least one valid CA. Other stuff OK
2021-03-16 20:03:31,069 # wolfSSL Leaving wolfSSL_CTX_load_verify_buffer_ex, return 1
2021-03-16 20:03:31,073 # wolfSSL Entering wolfSSL_CTX_use_certificate_buffer
2021-03-16 20:03:31,075 # wolfSSL Entering PemToDer
2021-03-16 20:03:31,078 # Checking cert signature type
2021-03-16 20:03:31,081 # wolfSSL Entering GetExplicitVersion
2021-03-16 20:03:31,084 # wolfSSL Entering GetSerialNumber
2021-03-16 20:03:31,085 # Got Cert Header
2021-03-16 20:03:31,088 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:31,090 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,091 # Got Algo ID
2021-03-16 20:03:31,093 # Getting Cert Name
2021-03-16 20:03:31,094 # Getting Cert Name
2021-03-16 20:03:31,096 # Got Subject Name
2021-03-16 20:03:31,098 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:31,101 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,102 # Got Key
2021-03-16 20:03:31,104 # Not ECDSA cert signature
2021-03-16 20:03:31,109 # wolfSSL Leaving wolfSSL_CTX_use_certificate_buffer, return 1
2021-03-16 20:03:31,113 # wolfSSL Entering wolfSSL_CTX_use_PrivateKey_buffer
2021-03-16 20:03:31,116 # wolfSSL Entering PemToDer
2021-03-16 20:03:31,121 # wolfSSL Leaving wolfSSL_CTX_use_PrivateKey_buffer, return 1
2021-03-16 20:03:31,123 # MQTT TLS Setup (1)
2021-03-16 20:03:31,125 # wolfSSL Entering SSL_new
2021-03-16 20:03:31,128 # wolfSSL Leaving SSL_new, return 0
2021-03-16 20:03:31,131 # wolfSSL Entering wolfSSL_SetCertCbCtx
2021-03-16 20:03:31,134 # wolfSSL Entering SSL_connect()
2021-03-16 20:03:31,137 # wolfSSL Entering SendClientHello
2021-03-16 20:03:31,140 # Adding signature algorithms extension
2021-03-16 20:03:31,142 # growing output buffer
2021-03-16 20:03:31,142 # 
2021-03-16 20:03:31,146 # Signature Algorithms extension to write
2021-03-16 20:03:31,148 # Shrinking output buffer
2021-03-16 20:03:31,148 # 
2021-03-16 20:03:31,152 # wolfSSL Leaving SendClientHello, return 0
2021-03-16 20:03:31,155 # connect state: CLIENT_HELLO_SENT
2021-03-16 20:03:31,376 # growing input buffer
2021-03-16 20:03:31,376 # 
2021-03-16 20:03:31,378 # received record layer msg
2021-03-16 20:03:31,379 # got HANDSHAKE
2021-03-16 20:03:31,382 # wolfSSL Entering DoHandShakeMsg()
2021-03-16 20:03:31,385 # wolfSSL Entering DoHandShakeMsgType
2021-03-16 20:03:31,387 # processing server hello
2021-03-16 20:03:31,390 # wolfSSL Entering DoServerHello
2021-03-16 20:03:31,393 # wolfSSL Entering VerifyClientSuite
2021-03-16 20:03:31,397 # wolfSSL Leaving DoServerHello, return 0
2021-03-16 20:03:31,399 # Shrinking input buffer
2021-03-16 20:03:31,399 # 
2021-03-16 20:03:31,403 # wolfSSL Leaving DoHandShakeMsgType(), return 0
2021-03-16 20:03:31,407 # wolfSSL Leaving DoHandShakeMsg(), return 0
2021-03-16 20:03:31,408 # growing input buffer
2021-03-16 20:03:31,408 # 
2021-03-16 20:03:31,682 # received record layer msg
2021-03-16 20:03:31,683 # got HANDSHAKE
2021-03-16 20:03:31,686 # wolfSSL Entering DoHandShakeMsg()
2021-03-16 20:03:31,689 # wolfSSL Entering DoHandShakeMsgType
2021-03-16 20:03:31,691 # processing certificate
2021-03-16 20:03:31,694 # wolfSSL Entering DoCertificate
2021-03-16 20:03:31,697 # wolfSSL Entering ProcessPeerCerts
2021-03-16 20:03:31,699 # Loading peer's cert chain
2021-03-16 20:03:31,702 # 	Put another cert into chain
2021-03-16 20:03:31,704 # 	Put another cert into chain
2021-03-16 20:03:31,708 # wolfSSL Entering GetExplicitVersion
2021-03-16 20:03:31,710 # wolfSSL Entering GetSerialNumber
2021-03-16 20:03:31,712 # Got Cert Header
2021-03-16 20:03:31,714 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:31,717 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,718 # Got Algo ID
2021-03-16 20:03:31,719 # Getting Cert Name
2021-03-16 20:03:31,721 # Getting Cert Name
2021-03-16 20:03:31,722 # Got Subject Name
2021-03-16 20:03:31,725 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:31,727 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,728 # Got Key
2021-03-16 20:03:31,730 # Parsed Past Key
2021-03-16 20:03:31,733 # wolfSSL Entering DecodeCertExtensions
2021-03-16 20:03:31,736 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,739 # wolfSSL Entering DecodeBasicCaConstraint
2021-03-16 20:03:31,742 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,744 # wolfSSL Entering DecodeCrlDist
2021-03-16 20:03:31,747 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,750 # wolfSSL Entering DecodeKeyUsage
2021-03-16 20:03:31,753 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,755 # wolfSSL Entering DecodeAuthInfo
2021-03-16 20:03:31,758 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,761 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,765 # Certificate Policy extension not supported yet.
2021-03-16 20:03:31,768 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,770 # wolfSSL Entering DecodeAltNames
2021-03-16 20:03:31,773 # 	Unsupported name type, skipping
2021-03-16 20:03:31,776 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,779 # wolfSSL Entering DecodeSubjKeyId
2021-03-16 20:03:31,782 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,784 # wolfSSL Entering DecodeAuthKeyId
2021-03-16 20:03:31,787 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:31,789 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,790 # CA found
2021-03-16 20:03:31,792 # 	CA boolean set
2021-03-16 20:03:31,794 # 	Extension Key Usage Set
2021-03-16 20:03:31,797 # 	maxPathLen status: set to pathLength
2021-03-16 20:03:31,800 # wolfSSL Entering ConfirmSignature
2021-03-16 20:03:31,913 # wolfSSL Leaving ConfirmSignature, return 0
2021-03-16 20:03:31,918 # MQTT TLS Verify Callback: PreVerify 1, Error 0 (none)
2021-03-16 20:03:31,924 #   Subject's domain name is Symantec Class 3 Secure Server CA - G4
2021-03-16 20:03:31,926 # Adding CA from chain
2021-03-16 20:03:31,927 # Adding a CA
2021-03-16 20:03:31,930 # wolfSSL Entering GetExplicitVersion
2021-03-16 20:03:31,933 # wolfSSL Entering GetSerialNumber
2021-03-16 20:03:31,934 # Got Cert Header
2021-03-16 20:03:31,936 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:31,939 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,940 # Got Algo ID
2021-03-16 20:03:31,942 # Getting Cert Name
2021-03-16 20:03:31,943 # Getting Cert Name
2021-03-16 20:03:31,945 # Got Subject Name
2021-03-16 20:03:31,947 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:31,950 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,950 # Got Key
2021-03-16 20:03:31,952 # Parsed Past Key
2021-03-16 20:03:31,955 # wolfSSL Entering DecodeCertExtensions
2021-03-16 20:03:31,958 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,961 # wolfSSL Entering DecodeBasicCaConstraint
2021-03-16 20:03:31,964 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,967 # wolfSSL Entering DecodeCrlDist
2021-03-16 20:03:31,969 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,972 # wolfSSL Entering DecodeKeyUsage
2021-03-16 20:03:31,975 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,978 # wolfSSL Entering DecodeAuthInfo
2021-03-16 20:03:31,980 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,983 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,987 # Certificate Policy extension not supported yet.
2021-03-16 20:03:31,990 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:31,993 # wolfSSL Entering DecodeAltNames
2021-03-16 20:03:31,995 # 	Unsupported name type, skipping
2021-03-16 20:03:31,998 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,001 # wolfSSL Entering DecodeSubjKeyId
2021-03-16 20:03:32,004 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,007 # wolfSSL Entering DecodeAuthKeyId
2021-03-16 20:03:32,009 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:32,012 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,012 # CA found
2021-03-16 20:03:32,014 # 	CA boolean set
2021-03-16 20:03:32,016 # 	Extension Key Usage Set
2021-03-16 20:03:32,019 # 	maxPathLen status: set to pathLength
2021-03-16 20:03:32,021 # 	Parsed new CA
2021-03-16 20:03:32,022 # 	Freeing Parsed CA
2021-03-16 20:03:32,024 # 	Freeing der CA
2021-03-16 20:03:32,025 # 		OK Freeing der CA
2021-03-16 20:03:32,028 # wolfSSL Leaving AddCA, return 0
2021-03-16 20:03:32,030 # Verifying Peer's cert
2021-03-16 20:03:32,033 # wolfSSL Entering GetExplicitVersion
2021-03-16 20:03:32,036 # wolfSSL Entering GetSerialNumber
2021-03-16 20:03:32,037 # Got Cert Header
2021-03-16 20:03:32,040 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:32,042 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,043 # Got Algo ID
2021-03-16 20:03:32,045 # Getting Cert Name
2021-03-16 20:03:32,047 # Getting Cert Name
2021-03-16 20:03:32,048 # Got Subject Name
2021-03-16 20:03:32,050 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:32,053 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,054 # Got Key
2021-03-16 20:03:32,055 # Parsed Past Key
2021-03-16 20:03:32,058 # wolfSSL Entering DecodeCertExtensions
2021-03-16 20:03:32,061 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,064 # wolfSSL Entering DecodeAltNames
2021-03-16 20:03:32,067 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,070 # wolfSSL Entering DecodeBasicCaConstraint
2021-03-16 20:03:32,073 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,076 # wolfSSL Entering DecodeKeyUsage
2021-03-16 20:03:32,078 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,080 # DecodeExtKeyUsage
2021-03-16 20:03:32,083 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,085 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,088 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,092 # Certificate Policy extension not supported yet.
2021-03-16 20:03:32,095 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,098 # wolfSSL Entering DecodeAuthKeyId
2021-03-16 20:03:32,100 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,103 # wolfSSL Entering DecodeCrlDist
2021-03-16 20:03:32,106 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,109 # wolfSSL Entering DecodeAuthInfo
2021-03-16 20:03:32,111 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,114 # wolfSSL Entering GetAlgoId
2021-03-16 20:03:32,116 # wolfSSL Entering GetObjectId()
2021-03-16 20:03:32,117 # CA found
2021-03-16 20:03:32,120 # wolfSSL Entering ConfirmSignature
2021-03-16 20:03:32,231 # wolfSSL Leaving ConfirmSignature, return 0
2021-03-16 20:03:32,233 # Verified Peer's cert
2021-03-16 20:03:32,238 # MQTT TLS Verify Callback: PreVerify 1, Error 0 (none)
2021-03-16 20:03:32,243 #   Subject's domain name is *.iot.us-west-2.amazonaws.com
2021-03-16 20:03:32,246 # wolfSSL Leaving ProcessPeerCerts, return 0
2021-03-16 20:03:32,250 # wolfSSL Leaving DoCertificate, return 0
2021-03-16 20:03:32,252 # Shrinking input buffer
2021-03-16 20:03:32,252 # 
2021-03-16 20:03:32,256 # wolfSSL Leaving DoHandShakeMsgType(), return 0
2021-03-16 20:03:32,260 # wolfSSL Leaving DoHandShakeMsg(), return 0
2021-03-16 20:03:32,261 # growing input buffer
2021-03-16 20:03:32,262 # 
2021-03-16 20:03:32,264 # received record layer msg
2021-03-16 20:03:32,265 # got HANDSHAKE
2021-03-16 20:03:32,268 # wolfSSL Entering DoHandShakeMsg()
2021-03-16 20:03:32,270 # Handshake message too large
2021-03-16 20:03:32,274 # wolfSSL error occurred, error = -404
2021-03-16 20:03:32,277 # wolfSSL error occurred, error = -404
2021-03-16 20:03:32,280 # wolfSSL Entering SSL_get_error
2021-03-16 20:03:32,283 # wolfSSL Leaving SSL_get_error, return -404
2021-03-16 20:03:32,290 # MqttSocket_TlsConnect Error -1: Num -404, Handshake message too large Error
2021-03-16 20:03:32,292 # wolfSSL Entering SSL_free
2021-03-16 20:03:32,295 # CTX ref count not 0 yet, no free
2021-03-16 20:03:32,297 # Shrinking input buffer
2021-03-16 20:03:32,297 # 
2021-03-16 20:03:32,300 # wolfSSL Leaving SSL_free, return 0
2021-03-16 20:03:32,303 # wolfSSL Entering SSL_CTX_free
2021-03-16 20:03:32,306 # CTX ref count down to 0, doing full free
2021-03-16 20:03:32,310 # wolfSSL Entering wolfSSL_CertManagerFree
2021-03-16 20:03:32,313 # wolfSSL Leaving SSL_CTX_free, return 0
2021-03-16 20:03:32,316 # wolfSSL Entering wolfSSL_Cleanup
2021-03-16 20:03:32,319 # wolfSSL Entering wolfCrypt_Cleanup
2021-03-16 20:03:32,322 # MqttSocket_Disconnect: Rc=0
2021-03-16 20:03:32,322 # 
2021-03-16 20:03:32,324 # MqttSocket_Connect: Rc=-6
2021-03-16 20:03:32,328 # MQTT Socket Connect: Error (Network) (-8)

it looks like there is still an issue with the I/O towards the socket, as the MQTT messages appear mangled.

How were you able to check these messages? From my understanding there should be no MQTT message until the TLS connection is successful.

Sorry for asking so many questions🙈

@MrKevinWeiss MrKevinWeiss added this to the Release 2021.07 milestone Jun 21, 2021
@github-actions github-actions bot added Area: doc Area: Documentation Area: network Area: Networking Area: sys Area: System labels Jun 30, 2021
@HendrikVE
Copy link
Contributor Author

I decided to exclude the SSL part for now as I was not yet able to run it. Furthermore the PR is already big enough, so I will open another one for this.
I added some other nice stuff though. I realized that Azure is changing its IPs multiple times per day so an address resolution is inevitable. I added a netdb module to the posix section to provide the functions getaddrinfo, freeaddrinfo, gethostbyname_r and gethostbyname. It's not doing much by itself as it is only a meta module. The work is done in the lwip package. I couldn't get the dns part of lwip running so I patched the line of code with a call to the already in RIOT existing module sock_dns.
In addition the mqtt-sn client is now working with IPv6 too!
I remove the WIP status, so we can start discussing the changes.

@HendrikVE HendrikVE changed the title WIP: Add wolfmqtt package Add wolfmqtt package Jun 30, 2021
@HendrikVE HendrikVE mentioned this pull request Jun 30, 2021
Draft
@MrKevinWeiss MrKevinWeiss removed this from the Release 2021.07 milestone Jul 15, 2021
@HendrikVE
Copy link
Contributor Author

HendrikVE commented Jan 9, 2022 •
edited
Loading

I did a major rework of this PR. Now with fewer hacks! #17493 is now a new dependency.

@HendrikVE HendrikVE mentioned this pull request Mar 10, 2022
Draft
@github-actions github-actions bot added the Area: build system Area: Build system label Mar 14, 2022
@HendrikVE HendrikVE changed the title Add wolfmqtt package pkg/wolfmqtt: add wolfmqtt Mar 30, 2022
@stale
Copy link

stale bot commented Nov 2, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

@stale stale bot added the State: stale State: The issue / PR has no activity for >185 days label Nov 2, 2022
@HendrikVE HendrikVE added State: don't stale State: Tell state-bot to ignore this issue and removed State: stale State: The issue / PR has no activity for >185 days labels Nov 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@miri64 miri64 Awaiting requested review from miri64 miri64 is a code owner

@cgundogan cgundogan Awaiting requested review from cgundogan

@haukepetersen haukepetersen Awaiting requested review from haukepetersen

@maribu maribu Awaiting requested review from maribu maribu is a code owner

@PeterKietzmann PeterKietzmann Awaiting requested review from PeterKietzmann

@aabadie aabadie Awaiting requested review from aabadie aabadie is a code owner

@fjmolinas fjmolinas Awaiting requested review from fjmolinas

@leandrolanzieri leandrolanzieri Awaiting requested review from leandrolanzieri leandrolanzieri is a code owner

@MichelRottleuthner MichelRottleuthner Awaiting requested review from MichelRottleuthner MichelRottleuthner is a code owner

@smlng smlng Awaiting requested review from smlng

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
Area: build system Area: Build system Area: doc Area: Documentation Area: examples Area: Example Applications Area: network Area: Networking Area: pkg Area: External package ports Area: POSIX Area: POSIX API wrapper Area: sys Area: System Area: tests Area: tests and testing framework State: don't stale State: Tell state-bot to ignore this issue Type: new feature The issue requests / The PR implemements a new feature for RIOT
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@HendrikVE @aabadie @miri64 @javierfileiv @danielinux @embhorn @fjmolinas @MrKevinWeiss