Skip to content

Improve the Git VCS collection logic #1623

@tonistiigi

Description

@tonistiigi

Since v0.10 #1462 buildx sends basic Git info with the build request from a local directory so that builder can use it to link build result with the source. This info is Git origin address, current commit, and an indicator if the local directory is dirty.

Issues that have appeared with this:

  • When using bake, all targets run their Git commands individually, even if they check the same directory. When using lots of targets, this results in a considerable slowdown. Warnings are also duplicated. These queries should be synchronized. Probably a simple memoized function with flightcontrol would do.
  • The git info warnings and the TTY progressbar to not synchronize properly. I think there is a race condition. Often the behavior is that the program appears to be stuck while Git commands are running, then build starts, and only after it has completed we get a list of warnings from the very beginning of the command. This process always runs before actual buildkit requests, so it should not conflict with build progress.
  • Even though we have already tried to optimize this, in bigger projects getting the "dirty checkout value" still takes considerable time. It is easy to make it take 5-10sec that could be a very considerable slowdown of the build(especially if a lot of it is cached). The program looking like it is stuck doesn't help either. I think once users figure out why this is broken for them, they will opt-out of the whole VCS feature as a whole. Considering that we don't have good ideas how to improve this speed significantly, I think we should consider making the dirty-bit opt-in. Repo URL and revision would be sent automatically as these Git requests are quick, but if the user wants a dirty bit they need to set an environment variable.

@cdupuis @crazy-max @jedevc

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions