reverse_tunnel: Implement backoff retry upon remote server connection errors #430
Conversation
… errors Currently in reverse-tunnel mode, upon errors when client tries to establish the connection to the remote server, 'run_reverse_tunnel' simply retries every 1s. This might be too agressive in certain scenarios, for example when the tunnel server (or a load-blancer in front of it) is not yet ready and returns 503. Another example is when the tunnel server does not agree on the tunnel parameters. Implement an exponential backoff for the delay between reconnect attempts. The maximum reconnect delay is controlled by setting "--reverse-reconnect-max-delay". The default is "1s" in order to keep existing behavior.
|
Hello, |
| @@ -133,6 +137,9 @@ impl WsClient { | |||
| remote_addr: RemoteAddr, | |||
| connector: impl TunnelConnector, | |||
There was a problem hiding this comment.
The delay should be better configurable per tunnel/connection instead of putting the config globally for the whole client.
You can take as example the timeout parameter, https://github.com/erebe/wstunnel/blob/main/src/tunnel/mod.rs#L41
It should be parsed from the tunnel command line (or use a default value) and passed as parameter in run_reverse_tunnel
There was a problem hiding this comment.
ack, will try that
|
Overall looks ok, the PR only needs to make the delay configurable per tunnel :) |
actually @erebe , after consideration, i actually think this knob should be global (and not per specific reverse tunnel configuation). the exponential backoff property is regarding the attempt to communicate with the wstunnel server. we would like to avoid being too aggressive when connecting the server; hence that property makes little sense to configure per RTUNNEL-SPEC - user will most likely choose same value for all. this property is similar in spirit to the global |
|
Ok fair enough. I am going to merge the PR and make a new release tomorrow. By the way and out of curiosity what are you using wstunnel for ? |
Thanks! |
Hi @erebe No plans for an additional PR eventually. Would appreciate if you can create a new release. Many thanks! |
|
It is released in https://github.com/erebe/wstunnel/releases/tag/v10.3.0 In case you haven't seen it, I have renamed the parameter into |
Currently in reverse-tunnel mode, upon errors when client tries to establish the connection to the remote server, 'run_reverse_tunnel' simply retries every 1s.
This might be too agressive in certain scenarios, for example when the tunnel server (or a load-blancer in front of it) is not yet ready and returns 503. Another example is when the tunnel server does not agree on the tunnel parameters.
Implement an exponential backoff for the delay between reconnect attempts. The maximum reconnect delay is controlled by setting "--reverse-reconnect-max-delay". The default is "1s" in order to keep existing behavior.