This repository contains a Docker setup to demonstrate an issue with Nginx reverse proxying to a dual stack (IPv4 and IPv6) destination.

After noticing intermittent proxy problems in our production environment with the proxy trying to connect to IPv6 target addresses, even though it only has outgoing IPv4 connectivity I started investigating this further.

With many completely wrong blog posts and information on the internet when it comes to nginx proxy configuration and IPv6, I was able to to nail it down to a "working" and "not working" configuration.

Our test setup:

• Incoming nginx proxy requests are IPv4 only
• We want to return the proxied page when our proxy receives requests on /test

Starting with the non working config, which might also be the most intuitive one:

location /test {
    proxy_ssl_server_name on;
    proxy_pass https://google.com/;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Host $host;
}

This one will not work since every 9th or 10th request the proxy will try to connect to the destination IPv6 address. Online you will find many blog posts prompting you to just add resolver 1.1.1.1 ipv6=off; deactivating IPv6 on the destination request, looks good, sounds good but doesn't work since with the above proxy configuration nginx resolves the DNS on startup and the resolver config is never used.

To make the resolver config work we have to introduce a temporary variable, this will force nginx to resolve the DNS during the request and not on startup:

location /test {
    proxy_ssl_server_name on;
    # A variable must be used to apply the resolver config
    set $upstream_dest "https://google.com/";
    proxy_pass $upstream_dest;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Host $host;
}

Now when we combine both proxy configurations we get the should work configuration:

location /unrelated {
    proxy_ssl_server_name on;
    proxy_pass https://google.com/;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Host $host;
}

location /test {
    proxy_ssl_server_name on;
    # A variable must be used to apply the resolver config
    set $upstream_dest "https://google.com/";
    proxy_pass $upstream_dest;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Host $host;
}

My expectation would be that the /test config still works as before, but it now shows the same behavior as the non working config, trying to connect to the IPv6 address every 9th or 10th request.

Finally you might ask yourself in which scenario a link local Ipv6 address without public routing might be available as configured in the docker-compose.yml and the answer is AWS ECS Fargate.

make start

make check-working-logs

make check-not-working-logs

make check-should-work-logs

make stop