resolver defaults to ipv6 even if local ipv6 not available in network stack #508
Description
Output of haproxy -vv and uname -a
$ haproxy -vv
HA-Proxy version 1.8.12-8a200c7 2018/06/27
Copyright 2000-2018 Willy Tarreau <willy@haproxy.org>
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -fno-strict-overflow -Wno-format-truncation -Wno-null-dereference -Wno-unused-label
OPTIONS = USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with OpenSSL version : OpenSSL 1.1.1c FIPS 28 May 2019
Running on OpenSSL version : OpenSSL 1.1.1c FIPS 28 May 2019
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.42 2018-03-20
Running on PCRE version : 8.42 2018-03-20
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built without compression support (neither USE_ZLIB nor USE_SLZ are set).
Compression algorithms supported : identity("identity")
Built with network namespace support.
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
$ uname -a
Linux 9f9febb8f3d7 4.15.0-1058-aws #60-Ubuntu SMP Wed Jan 15 22:35:20 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
What's the configuration?
resolvers dns
nameserver public-0 8.8.8.8:53
nameserver public-1 8.8.4.4:53
backend public_uploads_cloudfront
server uploads d724jat9k2oaz.cloudfront.net:80 resolvers dns check inter 60s
Steps to reproduce the behavior
- Configure a DNS resolver (as shown above) for any backend that supports IPv6, e.g. AWS CloudFront, from an AWS EC2 instance.
Actual behavior
Observe the following log line indicating that on server boot, an IPv4 record is used for the backend, but it changes to an IPv6 record on first DNS query:
public_uploads_cloudfront/uploads changed its IP from 13.224.180.141 to 2600:9000:2083:7a00:1d:8223:e2c0:21 by DNS cache.
From this point, all subsequent queries to this backend fail with the following error:
Connect() failed for backend public_uploads_cloudfront: no free ports.
Remember that AWS data centers don't provide an IPv6 stack, and slap your forehead in disgust.
Expected behavior
-
Haproxy should not default to using IPv6 if such a record is found, i.e. change the default on the 'resolve-prefer' flag. As web services migrate from IPv4 to IPv6, they will begin to offer both addresses. This causes a dangerous and unpredictable change of behaviour as haproxy will pick up the new IPv6 record and immediately start using it.
-
At the least, haproxy should check once at start-up whether it is capable of making an IPv6 connection before choosing to use an IPv6 address.
Do you have any idea what may have caused this?
Yes, haproxy defaults to using IPv6 addresses if it can find them - this can be changed using the 'resolve-prefer' option. This happens even if the host it's running on isn't capable of making IPv6 connections.
Do you have an idea how to solve the issue?
Either:
-
Check whether the host has an IPv6 address, and if so use IPv6 for backend connections.
-
Change the default for 'resolve-prefer' to be 'ipv4'.