Skip to content

Conversation

cpuguy83
Copy link
Contributor

@cpuguy83 cpuguy83 commented Feb 15, 2023

While we consider coming up with a nice DX for policies, allow passing through the raw buildkit format.

@cpuguy83
Copy link
Contributor Author

cpuguy83 commented Apr 2, 2023

Notes from @tonistiigi from slack: https://dockercommunity.slack.com/archives/C7S7A40MP/p1679946536583609?thread_ts=1679946076.766389&cid=C7S7A40MP

Recording them here in case the Slack history disappears:

  • probably fine as an env. maybe we should add UNSTABLE or EXPERIMENTAL to the env name
  • would be nice if bake --print showed that policy was applied to target
  • there is a wip work atm going on with introducing controller interface. we need to be careful about reading env in these cases as if the build goes through controller it has different env. basically we need to make sure that all params configured by env are declared in controller types and we don’t expect controller side to call GetEnv()

last point is similar to #1675

@jedevc jedevc added this to the v0.11.0 milestone Apr 4, 2023
@AkihiroSuda AkihiroSuda requested a review from tonistiigi April 6, 2023 04:44
@tonistiigi
Copy link
Member

@AkihiroSuda Afaics, the points from the last comment have not been updated yet.

@jedevc
Copy link
Collaborator

jedevc commented Apr 26, 2023

@cpuguy83 are you still interested in looking at this? We're coming up to a buildx release soon, I imagine you'd probably want this in?

If you're busy, I can try and carry this with @tonistiigi's suggested points 🎉

@cpuguy83
Copy link
Contributor Author

I think I've mostly got it. Sent you a message in slack but I'll post here as well:

Trying to decide how to deal with bake.
Specifically for --print.
If we want to show the policy in there then I need to expose it to the Target type, which opens it up to being set directly from the bake file.
Is this desirable?
Alternatively I could wrap the Target type for printing and include the source policy there.
(note I really mean the path to the source policy, not the policy itself)

@cpuguy83 cpuguy83 force-pushed the policy_file branch 2 times, most recently from 3469444 to ebbb602 Compare April 27, 2023 21:05
@cpuguy83
Copy link
Contributor Author

Updated this with the requested changes.

Note that while policies will apply to bakes it does not currently print that information with docker buildx bake --print.

@tonistiigi
Copy link
Member

If we decide to remove the experimental or expose this more visibly, then it should show up in bake --print as well. Atm. I don't think it matters much.

@jedevc
Copy link
Collaborator

jedevc commented May 2, 2023

I think we still need a couple changes - we need to avoid calling ReadSourcePolicy completely inside the build package. I wrote a fixup that I've tested and works correctly with build (and both local and remote controllers) and bake. 972f40a

I'm also not convinced on the env name - I'd personally prefer to use a combination of our already existing BUILDX_EXPERIMENTAL=1 and BUILDKIT_SOURCE_POLICY? Though, then we have the issue that it's not possible to just enable source policy, without also enabling the controller API, so I'm happy to keep it separate for now.

This adds an env var which can be used to pass in a path to a file to
read a buildkit source poliy from.

This is applied to any build is executed with the env set.
It is also applied to bakes (which are calling build behind the scenes).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
@cpuguy83
Copy link
Contributor Author

cpuguy83 commented May 3, 2023

Updated with the changes from your fixup commit.

Copy link
Collaborator

@jedevc jedevc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @dvdksn we might want some docs changes to note this somewhere?

@jedevc jedevc merged commit ae3299d into docker:master May 9, 2023
Comment on lines +1672 to +1677
p := os.Getenv("EXPERIMENTAL_BUILDKIT_SOURCE_POLICY")
if p == "" {
return nil, nil
}

data, err := os.ReadFile(p)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I wanted to update this to also allow passing the policy itself via environment variable,

  1. would such a change be considered?

  2. should that be implemented with this same variable (some kind of fallback either after reading the file or after trying to parse the JSON string?) or via a different variable?

😇

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would such a change be considered?

I think this could be a bit messy to pass a big json/proto value with env. Any other option that would avoid creating temp file if that is the issue?

@cpuguy83

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess something like EXPERIMENTAL_BUILDKIT_SOURCE_POLICY=<(... generate policy here ...) docker buildx build ... would probably work, given we read the entire file all at once and don't make any assumptions about it being an actual file (like trying to seek or something).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants