Skip to content

Conversation

whatwewant
Copy link
Contributor

@whatwewant whatwewant commented Sep 14, 2023

fixes #2049

limit builder container memory

use: docker buildx create --driver docker-container --driver-opt memory=32G --name zmicro

@whatwewant
Copy link
Contributor Author

whatwewant commented Sep 14, 2023

closes #2049

Copy link
Collaborator

@jsternberg jsternberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea and I think these options could be helpful for those using the docker-container driver. I'd like to use a different method for parsing the options though to avoid pulling in new dependencies and to make it more consistent with the other docker products that have the same functionality.

Comment on lines 44 to 50
memory string
memorySwap string
cpuQuota int64
cpuPeriod int64
cpuShares int64
cpusetCpus string
cpusetMems string
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be better if we utilize the equivalent types from the docker create command: https://github.com/docker/cli/blob/0e70f1b7b831565336006298b9443b015c3c87a5/cli/command/container/opts.go#L34-L132

This plugin also has docker/cli as a dependency so we should be able to utilize the github.com/docker/cli/opts package which I think we should use instead so the options here are consistent with the equivalent options.

An alternative is to utilize the method that compose does to parse the options, but it's a bit more adhoc. The underlying implementation seems to be the same, but we may not want to reuse that section of code mostly because it is not easily reusable outside of a YAML context.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, let's avoid adding unnecessary dep. opts.MemBytes can be used similar to

shmSize dockeropts.MemBytes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsternberg @crazy-max done, the types have been aligned to github.com/docker/cli/opts, and the unnecessary deps have been removed

Comment on lines 137 to 139
if d.memory != "" {
var memory uint64
memory, err := humanize.ParseBytes(d.memory)
if err != nil {
return err
}

hc.Resources.Memory = int64(memory)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matching with my recommendation above, I think we can do this:

if d.memory != "" {
    var memory opts.MemBytes
    if err := memory.Set(d.memory); err != nil {
        return err
    }
    hc.Resources.Memory = int64(memory)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

go.mod Outdated
@@ -16,6 +16,7 @@ require (
github.com/docker/cli-docs-tool v0.6.0
github.com/docker/docker v24.0.5+incompatible
github.com/docker/go-units v0.5.0
github.com/dustin/go-humanize v1.0.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm hesitant to add the dependencies that are added here since the functionality included in this PR is stuff that's also been done in other docker code. My preference is going to be that we utilize those methods rather than add a new way to parse the units.

The same applies to the other two dependencies here.

If there's a compelling reason why this is brand new functionality for docker and these dependencies make implementing that functionality easier, these dependencies seem solid. My preference is just going to be on not adding dependencies if there's an existing alternative.

case k == "memory-swap":
d.memorySwap = v
case k == "cpu-period":
d.cpuPeriod = int64(cast.ToInt(v))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think strconv.ParseInt makes more sense here and then we don't need a dependency. The container create command uses the cli flags and that uses strconv.ParseInt internally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@whatwewant whatwewant force-pushed the feat/support-memory-opt-for-docker-container branch from f2874f7 to 8143bf4 Compare September 28, 2023 03:17
@jsternberg
Copy link
Collaborator

@tonistiigi tagging you since this touches user-facing configuration options so.

Copy link
Member

@crazy-max crazy-max left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@whatwewant
Copy link
Contributor Author

@whatwewant whatwewant force-pushed the feat/support-memory-opt-for-docker-container branch from 313a1ca to 7c79b78 Compare September 29, 2023 08:10
@whatwewant whatwewant changed the title feat: support memory driver-opt for docker-container feat: support memory/cpu driver options for docker-container Sep 29, 2023
Copy link
Contributor

@dvdksn dvdksn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(docs) LGTM

Copy link
Member

@crazy-max crazy-max left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you squash your commits please? Otherwise LGTM

@whatwewant whatwewant force-pushed the feat/support-memory-opt-for-docker-container branch from 7c79b78 to 9941e3d Compare September 29, 2023 15:55
Signed-off-by: Zero <tobewhatwewant@outlook.com>
@whatwewant whatwewant force-pushed the feat/support-memory-opt-for-docker-container branch from 9941e3d to cfcd1d9 Compare September 29, 2023 15:57
@whatwewant
Copy link
Contributor Author

@crazy-max something wrong ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

support memory driver opt for docker-container driver
5 participants