Usage of IPv6 to connect to a proxy target even though the environment only has IPv4 connectivity #792
Description
Environment
- Official Nginx container image: nginx:1.29
- A complete environment to reproduce the problem can be found here
Description
We are running nginx as a proxy in AWS ECS Fargate, AWS assigns an IPv6 link local to every tasks which doesn't allow the task internet access, only the IPv4 allows internet access here.
When using a proxy config like this:
location /test {
proxy_ssl_server_name on;
proxy_pass https://google.com/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $host;
}
We can see that most of the requests to /test are using the IPv4 address of the upstream to connect to the proxy target, but in the local environment to reproduce the problem every 9th or 10th request uses an IPv6 address (which is also interestingly paired with an IPv4 address in the log).
nginx-proxy-should-work | {"timestamp":"2025-07-18T13:38:18+00:00","request":"GET /test HTTP/1.1","status": "302","request_processing_time":"1.547","upstream_addr":"[2a00:1450:4001:800::200e]:443, 142.250.186.110:443",}
nginx-proxy-should-work | {"timestamp":"2025-07-18T13:38:20+00:00","request":"GET /test HTTP/1.1","status": "302","request_processing_time":"0.354","upstream_addr":"142.250.186.110:443",}
nginx-proxy-should-work | {"timestamp":"2025-07-18T13:38:21+00:00","request":"GET /test HTTP/1.1","status": "302","request_processing_time":"0.386","upstream_addr":"142.250.186.110:443",}
nginx-proxy-should-work | {"timestamp":"2025-07-18T13:38:22+00:00","request":"GET /test HTTP/1.1","status": "302","request_processing_time":"0.320","upstream_addr":"142.250.186.110:443",}
nginx-proxy-should-work | {"timestamp":"2025-07-18T13:38:24+00:00","request":"GET /test HTTP/1.1","status": "302","request_processing_time":"0.433","upstream_addr":"142.250.186.110:443",}
nginx-proxy-should-work | {"timestamp":"2025-07-18T13:38:25+00:00","request":"GET /test HTTP/1.1","status": "302","request_processing_time":"0.293","upstream_addr":"142.250.186.110:443",}
nginx-proxy-should-work | {"timestamp":"2025-07-18T13:38:27+00:00","request":"GET /test HTTP/1.1","status": "302","request_processing_time":"0.357","upstream_addr":"142.250.186.110:443",}
nginx-proxy-should-work | {"timestamp":"2025-07-18T13:38:28+00:00","request":"GET /test HTTP/1.1","status": "302","request_processing_time":"0.484","upstream_addr":"142.250.186.110:443",}
nginx-proxy-should-work | {"timestamp":"2025-07-18T13:38:29+00:00","request":"GET /test HTTP/1.1","status": "302","request_processing_time":"0.305","upstream_addr":"142.250.186.110:443",}
nginx-proxy-should-work | {"timestamp":"2025-07-18T13:38:31+00:00","request":"GET /test HTTP/1.1","status": "302","request_processing_time":"0.463","upstream_addr":"142.250.186.110:443",}
nginx-proxy-should-work | 2025/07/18 13:38:33 [error] 13#13: *87 connect() failed (113: No route to host) while connecting to upstream, client: 127.0.0.1, server: , request: "GET /test HTTP/1.1", upstream: "https://[2a00:1450:4001:800::200e]:443/", host: "example.com"
Adding a resolver config like resolver 1.1.1.1 ipv6=off; in combination with the following proxy config works:
location /test {
proxy_ssl_server_name on;
# A variable must be used to apply the resolver config
set $upstream_dest "https://google.com/";
proxy_pass $upstream_dest;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $host;
}
But when now combining a non working and working proxy config in the same config file, we can see the same problem of intermittent IPv6 connections shown in the logs above:
location /unrelated {
proxy_ssl_server_name on;
proxy_pass https://google.com/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $host;
}
location /test {
proxy_ssl_server_name on;
# A variable must be used to apply the resolver config
set $upstream_dest "https://google.com/";
proxy_pass $upstream_dest;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $host;
}
Even when still connecting to /test that should use the non IPv6 resolver config
Problems
- When nginx is only listening on IPv4, it still tries to use IPv6 to connect to proxy targets (this is especially in environments problematic where only a link local IPv6 address is available without any public routing). I didn't found a global config to disable IPv6 in these cases completely
- When using a local variable in a proxy configuration to make use of an IPv6 disabling resolver config another proxy config using the destination directly deactivates the usage of the resolver config also for the first proxy config
Wishes
- Additions to the official documentation explaining that the resolver config has no effect when using a DNS entry directly in proxy_pass
nginx configuration
https://github.com/janritter/nginx-ipv6-proxy-problem/tree/main/confs