bake: ignore unrelated fields when parsing compose files #3292

crazy-max · 2025-07-03T13:54:43Z

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>

ndeloof · 2025-07-03T14:16:44Z

bake/compose.go

+						filteredServices[svcName] = map[string]any{}
+					} else if svcMap, ok := svc.(map[string]any); ok {
+						filteredService := make(map[string]any)
+						for _, svcField := range []string{"image", "build", "environment", "env_file"} {


AFAICT you don't need environment or env_file as those won't have any impact on the build context (they only apply on runtime environment)

We need them to evaluate build arguments:

buildx/bake/compose_test.go

Lines 396 to 424 in a007368

func TestEnv(t *testing.T) {

envf, err := os.CreateTemp("", "env")

require.NoError(t, err)

defer os.Remove(envf.Name())

_, err = envf.WriteString("FOO=bsdf -csdf\n")

require.NoError(t, err)

dt := []byte(`

services:

scratch:

build:

context: .

args:

CT_ECR: foo

FOO:

NODE_ENV:

environment:

- NODE_ENV=test

- AWS_ACCESS_KEY_ID=dummy

- AWS_SECRET_ACCESS_KEY=dummy

env_file:

- ` + envf.Name() + `

`)

c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)

require.NoError(t, err)

require.Equal(t, map[string]*string{"CT_ECR": ptrstr("foo"), "FOO": ptrstr("bsdf -csdf"), "NODE_ENV": ptrstr("test")}, c.Targets[0].Args)

}

If not set then this test fails:

Error: Not equal: expected: map[string]*string{"CT_ECR":(*string)(0xc00060a510), "FOO":(*string)(0xc00060a520), "NODE_ENV":(*string)(0xc00060a530)} actual : map[string]*string{"CT_ECR":(*string)(0xc00060a4a0)} Diff: --- Expected +++ Actual @@ -1,5 +1,3 @@ -(map[string]*string) (len=3) { - (string) (len=6) "CT_ECR": (*string)((len=3) "foo"), - (string) (len=3) "FOO": (*string)((len=10) "bsdf -csdf"), - (string) (len=8) "NODE_ENV": (*string)((len=4) "test") +(map[string]*string) (len=1) { + (string) (len=6) "CT_ECR": (*string)((len=3) "foo") } Test: TestEnv --- FAIL: TestEnv (0.00s)

There's a bug then. Runtime environment MUST not impact build environment; Sounds there's a confusion here with client environment (which is defined by .env file and --env-file flag in Compose) ... which is unfortunately a pretty common source of confusion 😭

Argh yes seems it was a mistake, this was introduced in #905. What about .env then?

For parity with Compose, .env should be loaded (from compose file's parent folder), and interpolated using os.Env

ndeloof · 2025-07-03T14:17:38Z

bake/compose.go

+
+	return loader.ModelToProject(filtered, loader.ToOptions(&cfgDetails, append([]func(*loader.Options){func(options *loader.Options) {
+		options.SkipNormalization = true
+		options.Profiles = []string{"*"}


building with all profiles enabled is not consistent with Compose. Is this expected ?

We allow all profiles because Bake doesn't support them: #1903. If we add something similar in our spec I think we could reconsider that. cc @colinhemmings

In follow-up we could consider mapping compose profiles to bake groups and maybe defining a default profile that would be used if one is defined.

bake/compose_test.go

tonistiigi · 2025-07-03T23:23:34Z

bake/compose.go

+		Environment: envs,
+	}
+
+	raw, err := loader.LoadModelWithContext(context.Background(), cfgDetails, append([]func(*loader.Options){func(opts *loader.Options) {


We can do this as follow-up as not directly related but seems fine to pass in the right context from the command.

tonistiigi · 2025-07-03T23:26:46Z

bake/compose.go

+
+	return loader.ModelToProject(filtered, loader.ToOptions(&cfgDetails, append([]func(*loader.Options){func(options *loader.Options) {
+		options.SkipNormalization = true
+		options.Profiles = []string{"*"}


In follow-up we could consider mapping compose profiles to bake groups and maybe defining a default profile that would be used if one is defined.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>

vendor: github.com/compose/compose-go/v2 891fce532a51

07e3226

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>

github-actions bot assigned crazy-max Jul 3, 2025

github-actions bot added area/bake area/bake/compose area/dependencies area/tests labels Jul 3, 2025

crazy-max added this to the v0.26.0 milestone Jul 3, 2025

crazy-max requested review from ndeloof, tonistiigi and glours July 3, 2025 14:15

crazy-max marked this pull request as ready for review July 3, 2025 14:15

ndeloof reviewed Jul 3, 2025

View reviewed changes

ndeloof approved these changes Jul 3, 2025

View reviewed changes

crazy-max force-pushed the compose-filtered-spec branch from 05451de to 0ce459f Compare July 3, 2025 16:08

tonistiigi reviewed Jul 3, 2025

View reviewed changes

bake: ignore unrelated fields when parsing compose files

c124b14

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>

crazy-max force-pushed the compose-filtered-spec branch from 0ce459f to c124b14 Compare July 4, 2025 06:09

crazy-max requested a review from tonistiigi July 4, 2025 06:28

crazy-max added the status/needs-follow-up label Jul 6, 2025

tonistiigi approved these changes Jul 14, 2025

View reviewed changes

tonistiigi merged commit f6d2c5e into docker:master Jul 14, 2025
138 checks passed

crazy-max deleted the compose-filtered-spec branch July 14, 2025 09:55

crazy-max mentioned this pull request Jul 22, 2025

bake: fix compose files validation #3329

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

bake: ignore unrelated fields when parsing compose files #3292

bake: ignore unrelated fields when parsing compose files #3292

Uh oh!

crazy-max commented Jul 3, 2025

Uh oh!

ndeloof Jul 3, 2025

Uh oh!

crazy-max Jul 3, 2025

Uh oh!

ndeloof Jul 3, 2025

Uh oh!

crazy-max Jul 3, 2025

Uh oh!

ndeloof Jul 3, 2025

Uh oh!

ndeloof Jul 3, 2025

Uh oh!

crazy-max Jul 3, 2025

Uh oh!

tonistiigi Jul 3, 2025

Uh oh!

Uh oh!

tonistiigi Jul 3, 2025

Uh oh!

tonistiigi Jul 3, 2025

Uh oh!

Uh oh!

Uh oh!

	func TestEnv(t *testing.T) {
	envf, err := os.CreateTemp("", "env")
	require.NoError(t, err)
	defer os.Remove(envf.Name())

	_, err = envf.WriteString("FOO=bsdf -csdf\n")
	require.NoError(t, err)

	dt := []byte(`
	services:
	scratch:
	build:
	context: .
	args:
	CT_ECR: foo
	FOO:
	NODE_ENV:
	environment:
	- NODE_ENV=test
	- AWS_ACCESS_KEY_ID=dummy
	- AWS_SECRET_ACCESS_KEY=dummy
	env_file:
	- ` + envf.Name() + `
	`)

	c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
	require.NoError(t, err)
	require.Equal(t, map[string]*string{"CT_ECR": ptrstr("foo"), "FOO": ptrstr("bsdf -csdf"), "NODE_ENV": ptrstr("test")}, c.Targets[0].Args)
	}

bake: ignore unrelated fields when parsing compose files #3292

bake: ignore unrelated fields when parsing compose files #3292

Uh oh!

Conversation

crazy-max commented Jul 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!