Skip to content

Conversation

jedevc
Copy link
Collaborator

@jedevc jedevc commented Mar 14, 2023

This PR pushes all SOURCE_DATE_EPOCH parsing into the command layer. This fixes two issues (in separate commits):

  • For the build command, we pull out SOURCE_DATE_EPOCH early, and set it in the build-args - then in the case of the remote controller (which connects to a buildx server), we can still set SOURCE_DATE_EPOCH correctly.
  • For the bake command, we pull out SOURCE_DATE_EPOCH early, and set it as an override - this fixes docker-buildx: add developer-guy to maintainers list, upgrade docker-buildx to 0.10.4 NixOS/nixpkgs#221023 (comment), where if SOURCE_DATE_EPOCH is set in the test environment (I think this is standard on Nix, though I'm not familiar) we generate additional build args. By splitting into a separate override in the bake command, tests don't end up accidentally reading the contents of the environment.

Comment on lines -607 to -612
// Propagate SOURCE_DATE_EPOCH from the client env
if v := os.Getenv("SOURCE_DATE_EPOCH"); v != "" {
if _, ok := so.FrontendAttrs["build-arg:SOURCE_DATE_EPOCH"]; !ok {
so.FrontendAttrs["build-arg:SOURCE_DATE_EPOCH"] = v
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the case where buildx is used as lib (compose), it is not going to work anymore I guess?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm you're right. I'm not quite sure if that's a bug or a feature though 🤔

I'm leaning towards it's probably best that when used as a library buildx should avoid reading from the environment as much as possible. We also have some minor precedent here - DOCKER_DEFAULT_PLATFROM.

Though I wonder about cases like BUILDX_NO_DEFAULT_ATTESTATIONS. On the one hand, it makes sense to propagate them, on the other, it could be confusing to the library consumer if the environment overrides the options that they have passed to Buildx.

IMO, in the future we should push all environment variables that affect the build solve request (so not including buildx driver env vars, etc) into the command layer, and then we should provide a utility method to parse environment variables and modify the build options from them:

message BuildOptions {
string ContextPath = 1;
string DockerfileName = 2;
string PrintFunc = 3;
map<string, string> NamedContexts = 4;
repeated string Allow = 5;
repeated Attest Attests = 6;
map<string, string> BuildArgs = 7;
repeated CacheOptionsEntry CacheFrom = 8;
repeated CacheOptionsEntry CacheTo = 9;
string CgroupParent = 10;
repeated ExportEntry Exports = 11;
repeated string ExtraHosts = 12;
map<string, string> Labels = 13;
string NetworkMode = 14;
repeated string NoCacheFilter = 15;
repeated string Platforms = 16;
repeated Secret Secrets = 17;
int64 ShmSize = 18;
repeated SSH SSH = 19;
repeated string Tags = 20;
string Target = 21;
UlimitOpt Ulimits = 22;
CommonOptions Opts = 24;
}

Does that seem somewhat reasonable? If so, I'll add a TODO in the code - it's tricky to handle right now, since I think with the controller code, I'd expect we'd try and get library consumers of Buildx to use that API instead of using build.Build?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

./cc FYI @glours @ndeloof (for compose)

This allows the build package code to become more generic, and also
ensures that when the environment variables are not propogated (in the
case of the remote controller), that we can still correctly set
SOURCE_DATE_EPOCH.

Signed-off-by: Justin Chadwell <me@jedevc.com>
@jedevc jedevc force-pushed the source-date-epoch-fixes branch from fcb01a0 to 38b96da Compare March 14, 2023 10:21
crazy-max
crazy-max previously approved these changes Mar 14, 2023
@crazy-max crazy-max dismissed their stale review March 15, 2023 09:36

fix bake override behavior

Previously, when directly modifying the args map when reading targets,
we could end up in a scenario where bake tests that compare arg maps
would fail if SOURCE_DATE_EPOCH was set in the environment.

This patch prevents this failure by setting the SOURCE_DATE_EPOCH at the
command level (which isn't injected into tests as well), ensuring that
we test correctly even when SOURCE_DATE_EPOCH is set in the environment.

Signed-off-by: Justin Chadwell <me@jedevc.com>
@jedevc jedevc force-pushed the source-date-epoch-fixes branch from 38b96da to 7805314 Compare March 15, 2023 10:07
Copy link
Member

@crazy-max crazy-max left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jedevc jedevc requested a review from tonistiigi March 20, 2023 09:40
@tonistiigi tonistiigi merged commit fd8eaab into docker:master Mar 22, 2023
@jedevc jedevc deleted the source-date-epoch-fixes branch May 2, 2023 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants