docker: fix cgroupdriver #1312

win-t · 2025-05-03T13:48:19Z

This fix #1311

The change is that we check if the system is running systemd, and then select the appropriate cgroup driver
This PR also skips setting the cgroup driver if the user sets it manually in the configuration.

abiosoft

Thanks for this :)

k0d3r1s · 2025-05-06T07:46:03Z

@abiosoft @win-t i'm on m2 mac, i dont have systemd or cgroups and i get an error:

Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as "xxx.slice"

what am i supposed to do now?
docker, not k3s or anything

This reverts commit 6c352e4.

abiosoft · 2025-05-07T03:20:55Z

@k0d3r1s this has been reverted.

@win-t can you kindly have another look, thanks.

win-t · 2025-05-07T06:31:55Z

@k0d3r1s how did you produce the error? when docker build or just normal docker run
I think the issue is the version of the docker inside the vm
can you help run systemctl --version and docker info and cat /etc/os-release in your colima vm

win-t · 2025-05-07T06:34:11Z

quick search give this issue on buildx kubernetes/kubernetes#107403

k0d3r1s · 2025-05-07T06:46:32Z

@win-t
i have this: docker compose --profile ${PROFILE} -f docker-compose.yml -f "docker-compose.${PROFILE}.yml" -p $NAME up -d --force-recreate --build --remove-orphans so i guess its build.

systemctl --version
systemd 255 (255.4-1ubuntu8.4)
+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified

docker info
Client: Docker Engine - Community
 Version:    28.1.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.35.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 23
  Running: 23
  Paused: 0
  Stopped: 0
 Images: 22
 Server Version: 28.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-50-generic
 Operating System: Ubuntu 24.04.1 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 11.65GiB
 Name: colima
 ID: a6102c05-3f91-4242-97ef-c847079f9b7f
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: true
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.1 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo

i was back from vacation. i ran brew update and as i am using master of colima, it updated to this broken version. after update i ran start script of my work project that has the command from above for build in it. and got the error from the message above.

win-t · 2025-05-07T08:36:59Z

quick search give this issue on buildx kubernetes/kubernetes#107403

sorry, i mean docker/buildx#935

win-t · 2025-05-07T08:37:43Z

can you recreate your buildx builder, it look like that will solve the issue docker/buildx#935 (comment)

win-t · 2025-05-07T11:22:07Z

I don't suspect any issue from your setup,

btw @k0d3r1s do you have any cgroup_parent in your compose file?

In systemd mode, this value must strictly follow the systemd requirement, which must have .slice suffix

k0d3r1s · 2025-05-07T17:28:58Z

@win-t colima delete also worked :)

nope, i dont do any cgroup things in compose, dont have any advanced config parameters, basics. i cant paste it as is (work related), will try to obfuscate tomorrow if you need to see it

win-t · 2025-05-07T22:55:34Z

nope, i dont do any cgroup things in compose, dont have any advanced config parameters, basics. i cant paste it as is (work related), will try to obfuscate tomorrow if you need to see it

I don't see any other explanation here, basically that error is saying wrong --cgroup-parent when using systemd as driver

for example

So, this error is due to the client side. By client side, I mean Docker tooling installed on the mac (not the lima vm), you could try upgrading docker-buildx or docker-compose (or other tools) on your mac, other possible issue is that the tooling in the mac cache the response of docker info in the mac side, and believe that the docker daemon is still using cgroupfs and not aware that it was changed to systemd

btw, I think we in Hyrum's Law territory now, I leave it to you @abiosoft if you want to implement the proper cgroup driver or not

but, it is the requirement for systemd to manage cgroups exclusively, there is no other choice here,
native.cgroupdriver=cgroupfs only for systems without systemd, like when using alpine os as the lima vm, (or Docker-in-Docker where there is no systemd in the container)

I still believe we need to use a proper cgroup driver here. Sooner or later, this setup will break. systemd version 255 might still tolerate this behaviour, but there is no such guarantee in future upgrades

k0d3r1s · 2025-05-08T13:12:13Z

@win-t

# This is the base compose file that defines base images for the project (should be used as basis for environment
# specific images. This file is also used to build base images on build server (to use for building testing/deployment
# images)
services:
  example_base_gotenberg:
    container_name: example_base_gotenberg
    image: example_base_gotenberg
    build:
      context: gotenberg
      dockerfile: Dockerfile
      args:
        GOTENBERG_VERSION: "8"
  example_base_postgres:
    container_name: example_base_postgres
    image: example_base_postgres
    build:
      context: postgres
      dockerfile: Dockerfile
      args:
        DEBIAN_VERSION: bookworm
        POSTGRES_VERSION: "16"
  example_base_php:
    container_name: example_base_php
    image: example_base_php
    build:
      context: php
      dockerfile: Dockerfile
      args:
        DEBIAN_VERSION: bookworm
        HOST_UID: ${HOST_UID}
        HOST_GID: ${HOST_GID}
        PHP_VERSION: "8.3"
        PHP_EXTENSIONS: "curl fpm intl mbstring pgsql xml"
  example_base_nginx:
    container_name: example_base_nginx
    image: example_base_nginx
    build:
      args:
        DEBIAN_VERSION: bookworm
      context: nginx
      dockerfile: Dockerfile
  example_base_node:
    container_name: example_base_node
    image: example_base_node
    build:
      context: node
      dockerfile: Dockerfile
      args:
        DEBIAN_VERSION: bookworm
        HOST_UID: ${HOST_UID}
        HOST_GID: ${HOST_GID}
        NODE_VERSION: "20"

# This compose file defines docker compose stack for specific environment (development in this case)
# This file should not be used directly with docker compose, because it depends on other locally built base images
# Check helper scripts up.sh and rebuild.sh to see how the environment specific stack is build


services:
#  example_dev_mailpit:
#    container_name: example_dev_mailpit
#    image: axllent/mailpit
#    ports:
#      - "1025:1025"
#      - "8025:8025"
#    restart: unless-stopped
#    networks:
#      example_dev_network:
  example_dev_gotenberg:
    image: example_base_gotenberg:latest
    pull_policy: never
    restart: unless-stopped
    networks:
      example_dev_network:
  example_dev_postgres:
    container_name: example_dev_postgres
    image: example_base_postgres:latest
    pull_policy: never
    environment:
      POSTGRES_PASSWORD: postgres
    volumes:
      - example_dev_postgres_volume:/var/lib/postgresql/data
    ports:
      - "5442:5432"
    restart: unless-stopped
    networks:
      example_dev_network:
      backend:
  example_dev_php:
    container_name: example_dev_php
    pull_policy: never
    image: example_dev_php
    build:
      args:
        PHP_VERSION: "8.3"
      context: php
      dockerfile: Dockerfile
    volumes:
      - ../../src:/var/www/html
    depends_on:
#      - example_dev_mailpit
#      - example_dev_gotenberg
      - example_dev_postgres
    restart: unless-stopped
    extra_hosts:
      - "host.docker.internal:host-gateway"
    networks:
      - example_dev_network
      - frontend
      - backend
    working_dir: /var/www/html/api
  example_dev_nginx:
    container_name: example_dev_nginx
    pull_policy: never
    image: example_dev_nginx
    build:
      context: nginx
      dockerfile: Dockerfile
    volumes:
      - ../../src:/var/www/html
      - ./nginx/certs:/etc/ssl/private
#    ports:
#      - "80:80"
#      - "443:443"
    depends_on:
      - example_dev_php
    restart: unless-stopped
    extra_hosts:
      - "host.docker.internal:host-gateway"
    networks:
      example_dev_network:
        aliases:
          - admin-example.local.io # Add alias so we can also use domain to connect to this container
      frontend:
      backend:
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=frontend"
      - "traefik.http.routers.example_dev_nginx.rule=Host(`admin-example.local.io`)"
      - "traefik.http.routers.example_dev_nginx.entrypoints=https"
      - "traefik.udp.routers.example_dev_nginx.entrypoints=http3"
      - "traefik.http.routers.example_dev_nginx.tls=true"
      - "traefik.http.services.example_dev_nginx.loadbalancer.server.scheme=https"
      - "traefik.http.services.example_dev_nginx.loadbalancer.server.port=443"
      - "traefik.http.routers.example_dev_nginx.service=example_dev_nginx"
  example_dev_node:
    container_name: example_dev_node
    pull_policy: never
    image: example_base_node:latest
    volumes:
      - ../../src:/var/www/html
      - ./nginx/certs:/etc/ssl/private
    restart: unless-stopped
    working_dir: /var/www/html/portal
    networks:
      example_dev_network:
      frontend:
    ports:
      - "8009:8009"
    tty: true # If not specified as true, container will exit immediately
networks:
  example_dev_network:
    name: example_dev_network
  backend:
    name: backend
    external: true

  frontend:
    name: frontend
    external: true
volumes:
  example_dev_postgres_volume:
    name: example_dev_postgres_volume

running like this (first file, second file):

#!/bin/bash
# Build base images from base stack in ../base directory with cache
cd ../base || exit
HOST_UID="$(id -u)" HOST_GID="$(id -g)" docker compose build --pull
# Build environment specific images from environment stack in ../dev directory with cache
cd ../dev || exit
HOST_UID="$(id -u)" HOST_GID="$(id -g)" docker compose build --detach
# Start the environment specific stack
HOST_UID="$(id -u)" HOST_GID="$(id -g)" docker compose up --detach --force-recreate --remove-orphans
#docker image prune --force

first file builds correct, second file fails with error i pasted in original message

win-t · 2025-07-31T09:57:31Z

Hi @k0d3r1s, sorry for the very late response

Can you show me the /etc/docker/daemon.json in your setup?

colima ssh cat /etc/docker/daemon.json

and also the colima yaml config that you are using especially the docker part

cat ~/.colima/default/colima.yaml | yq .docker

thanks

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [abiosoft/colima](https://github.com/abiosoft/colima) | patch | `v0.8.1` -> `v0.8.2` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>abiosoft/colima (abiosoft/colima)</summary> ### [`v0.8.2`](https://github.com/abiosoft/colima/releases/tag/v0.8.2) [Compare Source](abiosoft/colima@v0.8.1...v0.8.2) #### Highlights This is an incremental release with bug fixes. #### Fixes - Report errors that occur while reading configuration file. Fixes [#1239](abiosoft/colima#1239). - Use native shasum binary for download assets verification. Fixes [#1163](abiosoft/colima#1163). - Fix download errors when K3s version when empty in config. - Reset formatting after questions prompts in the terminal. Fixes [#1319](abiosoft/colima#1319). #### Other Updates - UDP port forwarding is now supported. Resolves [#1292](abiosoft/colima#1292), [#1300](abiosoft/colima#1300). - Binfmt emulation is configurable via the `--binfmt` flag and `binfmt` config. Defaults to `true` when Rosetta is disabled. - Rename `--cpu` flag to `--cpus` for `colima start`, to align with Lima. `--cpu` still works but deprecated. - Current profile can now be set with `COLIMA_PROFILE` environment variable. Resolves [#1308](abiosoft/colima#1308). ##### Runtime version bumps **NOTE:** container runtime versions can be updated manually by running the `colima update` command. - Docker version updated to `v28.3.3` - Nerdctl version updated to `v2.1.3` - Incus version updated to `v6.14` - K3s version defaults to `v1.33.3+k3s1 ` #### Commits - ci: enable gocritic in golangci by [@PascalBourdier](https://github.com/PascalBourdier) in abiosoft/colima#1223 - faq: update to address potential need for Docker client configuration by [@carljmosca](https://github.com/carljmosca) in abiosoft/colima#1231 - cli: add quotes to k3s-arg example for colima start by [@olamilekan000](https://github.com/olamilekan000) in abiosoft/colima#1230 - chore: update LICENSE, fix copyright license year by [@JasonnnW3000](https://github.com/JasonnnW3000) in abiosoft/colima#1235 - chore: fix copyright license year (Revert [#1235](abiosoft/colima#1235)) by [@windyakin](https://github.com/windyakin) in abiosoft/colima#1237 - chore: log possible error while reading config file by [@olamilekan000](https://github.com/olamilekan000) in abiosoft/colima#1240 - build(deps): bump golang.org/x/term from 0.27.0 to 0.28.0 by [@dependabot](https://github.com/dependabot)\[bot] in abiosoft/colima#1242 - core: use native shasum binary on macOS for download verification by [@abiosoft](https://github.com/abiosoft) in abiosoft/colima#1245 - core: config initial value for kubernetes version if empty in yaml by [@jason19970210](https://github.com/jason19970210) in abiosoft/colima#1255 - k8s: update latest version for default kubernetes release by [@jason19970210](https://github.com/jason19970210) in abiosoft/colima#1259 - core: respect LIMA\_SSH\_PORT\_FORWARDER env var by [@abiosoft](https://github.com/abiosoft) in abiosoft/colima#1263 - build(deps): bump golang.org/x/term from 0.28.0 to 0.29.0 by [@dependabot](https://github.com/dependabot)\[bot] in abiosoft/colima#1268 - incus: remove core.https\_address from default config by [@abiosoft](https://github.com/abiosoft) in abiosoft/colima#1273 - build(deps): bump github.com/spf13/cobra from 1.8.1 to 1.9.1 by [@dependabot](https://github.com/dependabot)\[bot] in abiosoft/colima#1274 - docker: fix cgroupdriver by [@win-t](https://github.com/win-t) in abiosoft/colima#1312 - revert: "docker: fix cgroupdriver" by [@abiosoft](https://github.com/abiosoft) in abiosoft/colima#1314 - Properly reset formatting at the end of question prompts by [@SukritBhatt](https://github.com/SukritBhatt) in abiosoft/colima#1324 - chore: reorganize dependencies in go.mod by [@alexandear](https://github.com/alexandear) in abiosoft/colima#1325 - github workflows: pin dependencies by [@thypon](https://github.com/thypon) in abiosoft/colima#1326 - vm: support UDP port forwarding by [@stek29](https://github.com/stek29) in abiosoft/colima#1343 - core: update disk images by [@abiosoft](https://github.com/abiosoft) in abiosoft/colima#1351 - k3s: fix config defaults for Kubernetes values by [@kaisq](https://github.com/kaisq) in abiosoft/colima#1322 - vm: make binfmt emulation configurable by [@winterqt](https://github.com/winterqt) in abiosoft/colima#1315 - docker: replace also localhost with host gateway ip in docker daemon.json by [@fralken](https://github.com/fralken) in abiosoft/colima#1298 - k3s: write kubeconfig to KUBECONFIG defined file by [@Sumoa](https://github.com/Sumoa) in abiosoft/colima#978 - Multiple minor updates by [@abiosoft](https://github.com/abiosoft) in abiosoft/colima#1352 - build(deps): bump golang.org/x/term from 0.29.0 to 0.30.0 by [@dependabot](https://github.com/dependabot)\[bot] in abiosoft/colima#1281 - chore: expend the path information about current profile being used by [@jason19970210](https://github.com/jason19970210) in abiosoft/colima#1270 - core: add support for COLIMA\_PROFILE environment variable. by [@abiosoft](https://github.com/abiosoft) in abiosoft/colima#1353 - ci: bump Go version by [@abiosoft](https://github.com/abiosoft) in abiosoft/colima#1354 #### New Contributors - [@PascalBourdier](https://github.com/PascalBourdier) made their first contribution in abiosoft/colima#1223 - [@carljmosca](https://github.com/carljmosca) made their first contribution in abiosoft/colima#1231 - [@olamilekan000](https://github.com/olamilekan000) made their first contribution in abiosoft/colima#1230 - [@JasonnnW3000](https://github.com/JasonnnW3000) made their first contribution in abiosoft/colima#1235 - [@windyakin](https://github.com/windyakin) made their first contribution in abiosoft/colima#1237 - [@jason19970210](https://github.com/jason19970210) made their first contribution in abiosoft/colima#1255 - [@win-t](https://github.com/win-t) made their first contribution in abiosoft/colima#1312 - [@SukritBhatt](https://github.com/SukritBhatt) made their first contribution in abiosoft/colima#1324 - [@alexandear](https://github.com/alexandear) made their first contribution in abiosoft/colima#1325 - [@thypon](https://github.com/thypon) made their first contribution in abiosoft/colima#1326 - [@stek29](https://github.com/stek29) made their first contribution in abiosoft/colima#1343 - [@kaisq](https://github.com/kaisq) made their first contribution in abiosoft/colima#1322 - [@winterqt](https://github.com/winterqt) made their first contribution in abiosoft/colima#1315 - [@fralken](https://github.com/fralken) made their first contribution in abiosoft/colima#1298 - [@Sumoa](https://github.com/Sumoa) made their first contribution in abiosoft/colima#978 **Full Changelog**: abiosoft/colima@v0.8.1...v0.8.2 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever MR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

k0d3r1s · 2025-08-13T12:11:55Z

@win-t sorry. this was so long ago, i have no idea what was the fix etc. it is working now so - good now? idk

abiosoft · 2025-08-13T12:20:54Z

@win-t I think we can get this PR back in.

Would you like to recreate it?

win-t force-pushed the main branch 2 times, most recently from 7e1cd54 to d56856a Compare May 3, 2025 13:52

fix: abiosoft#1311

fcbe61e

win-t force-pushed the main branch from d56856a to fcbe61e Compare May 3, 2025 13:52

win-t mentioned this pull request May 3, 2025

cgroupdriver is hardcoded to cgroupfs #1311

Closed

abiosoft approved these changes May 4, 2025

View reviewed changes

abiosoft changed the title ~~fix: cgroupdriver is hardcoded to cgroupfs~~ docker: fix cgroupdriver May 4, 2025

abiosoft merged commit 6c352e4 into abiosoft:main May 4, 2025
11 checks passed

abiosoft added a commit that referenced this pull request May 6, 2025

Revert "docker: fix cgroupdriver (#1312)"

b72d64d

This reverts commit 6c352e4.

abiosoft mentioned this pull request May 6, 2025

revert: "docker: fix cgroupdriver" #1314

Merged

abiosoft added a commit that referenced this pull request May 7, 2025

revert: "docker: fix cgroupdriver (#1312)" (#1314)

5d2e91c

This reverts commit 6c352e4.

Uh oh!

docker: fix cgroupdriver #1312

docker: fix cgroupdriver #1312

Uh oh!

Conversation

win-t commented May 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

abiosoft left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

k0d3r1s commented May 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

abiosoft commented May 7, 2025

Uh oh!

win-t commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

win-t commented May 7, 2025

Uh oh!

k0d3r1s commented May 7, 2025

Uh oh!

win-t commented May 7, 2025

Uh oh!

win-t commented May 7, 2025

Uh oh!

win-t commented May 7, 2025

Uh oh!

k0d3r1s commented May 7, 2025

Uh oh!

win-t commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k0d3r1s commented May 8, 2025

Uh oh!

win-t commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k0d3r1s commented Aug 13, 2025

Uh oh!

abiosoft commented Aug 13, 2025

Uh oh!

Uh oh!

win-t commented May 3, 2025 •

edited

Loading

k0d3r1s commented May 6, 2025 •

edited

Loading

win-t commented May 7, 2025 •

edited

Loading

win-t commented May 7, 2025 •

edited

Loading

win-t commented Jul 31, 2025 •

edited

Loading