[BUG] .env required value interpolation error when using --env-file with different file during build #13070
Description
Description
I have run into an issue with compose and buildx on compose v2.38.2. This was not an issue with the compose version I used with docker v8.1.1
The problem happens when compose uses buildx to build a docker image using env variables from an env file specified via --env-file that is not the .env file, but an .env file exists and has a variable interpolation with a required value.
It seems that even though a non .env file is specified with --env-file, compose still evaluates the interpolations in the .env file.
The issue occurs even if the variable in question is actually set, which means that even if we want to use the .env file it fails to recognize that the variable is already set, so the env var interpolation with required value does not seem to be working at all.
A workaround for this issue is to use COMPOSE_BAKE=false, or rename the .env file to something else and use --env-file to use it.
Steps To Reproduce
Minimal setup to reproduce the problem
- Create the following files
Dockerfile
FROM busybox:latest
docker-compose.yml
services:
example:
build:
context: .
image: dummy/image:v1
.env
SOME_VAR=${SOME_VAR:?Missing SOME_VAR}
.env.dev
SOME_VAR="DUMMY"
- Run the following scenarios
Scenario 1:
$ unset SOME_VAR
$ docker compose --env-file .env.dev build
[+] Building 0.0s (1/1) FINISHED
=> [internal] load local bake definitions 0.0s
=> => reading from stdin 403B 0.0s
required variable SOME_VAR is missing a value: Missing SOME_VAR
Scenario 2:
$ export SOME_VAR="some val"
$ docker compose --env-file .env.dev build
[+] Building 0.0s (1/1) FINISHED
=> [internal] load local bake definitions 0.0s
=> => reading from stdin 403B 0.0s
required variable SOME_VAR is missing a value: Missing SOME_VAR
Scenario 3:
$ export SOME_VAR="some val"
$ docker compose build
[+] Building 0.0s (1/1) FINISHED
=> [internal] load local bake definitions 0.0s
=> => reading from stdin 406B 0.0s
required variable SOME_VAR is missing a value: Missing SOME_VAR
- What was expected
A missing required value interpolation in the .env file should not prevent the build process when another env file is being specified via --env-file.
Also, when using the .env file, the required value interpolation should detect that the variable value is in fact set.
Compose Version
Docker Compose version v2.38.2
Docker Environment
Client: Docker Engine - Community
Version: 28.3.2
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.25.0
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.38.2
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 2
Running: 0
Paused: 0
Stopped: 2
Images: 70
Server Version: 28.3.2
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
CDI spec directories:
/etc/cdi
/var/run/cdi
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
runc version: v1.2.5-0-g59923ef
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.1.0-37-amd64
Operating System: Debian GNU/Linux 12 (bookworm)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.638GiB
Name: debian
ID: 88667259-f5cb-4b6c-b20d-d450887d633a
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
::1/128
127.0.0.0/8
Live Restore Enabled: false
Anything else?
No response