Skip to content

Containers created with docker compose cannot be updated #1019

@UncleSamSwiss

Description

@UncleSamSwiss

Describe the bug
Using docker compose under Windows 10 with Docker in WSL2 leads to wrong image name being used, thus watchtower can't check for updates properly.

To Reproduce

  1. Create a "regular" docker-compose.yml with some containers and add watchtower
  2. Execute docker compose up (no hyphen!)
  3. ❌ Watchtower can't find the images (looking for the wrong name, see log for details)
  4. Exit the above command and execute docker compose down
  5. Execute docker-compose up (with hyphen!)
  6. ✔️ Everything works as expected
  7. Exit the above command and execute docker-compose down

Expected behavior
Both docker-compose up and docker compose up should work the same way.

Logs
Below are extracts of the log file of both runs.

Using docker compose:

watchtower-ghcr_1       | time="2021-07-08T12:24:49Z" level=debug msg="Making sure everything is sane before starting"
watchtower-ghcr_1       | time="2021-07-08T12:24:49Z" level=debug msg="Retrieving running containers"
watchtower-ghcr_1       | time="2021-07-08T12:24:49Z" level=debug msg="There are no additional watchtower containers"
watchtower-ghcr_1       | time="2021-07-08T12:24:49Z" level=debug msg="Watchtower HTTP API skipped."
watchtower-ghcr_1       | time="2021-07-08T12:24:49Z" level=info msg="Watchtower 1.3.0\nUsing no notifications\nOnly checking containers in scope \"ghcr\"\nScheduling first run: 2021-07-08 12:25:19 +0000 UTC\nNote that the first check will be performed in 29 seconds"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Checking containers for updated images"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Retrieving running containers"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Trying to load authentication credentials." container=/weblate-docker-compose_weblate_1 image="sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Loaded auth credentials for user UncleSamSwiss on registry sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Got image name: sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Credentials loaded"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Checking if pull is needed" container=/weblate-docker-compose_weblate_1 image="sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Setting scope for auth token" image=sha256 scope="repository:library/sha256:pull"
watchtower-ghcr_1       | time="2021-07-08T12:25:19Z" level=debug msg="Credentials found."
watchtower-ghcr_1       | time="2021-07-08T12:25:22Z" level=debug msg="Parsing image ref" host=index.docker.io image=sha256 normalized="docker.io/library/sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8" tag=e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8
watchtower-ghcr_1       | time="2021-07-08T12:25:22Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/library/sha256/manifests/e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
watchtower-ghcr_1       | time="2021-07-08T12:25:22Z" level=warning msg="Could not do a head request for \"sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8\", falling back to regular pull." container=/weblate-docker-compose_weblate_1 image="sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
watchtower-ghcr_1       | time="2021-07-08T12:25:22Z" level=warning msg="Reason: registry responded to head request with \"401 Unauthorized\", auth: \"Bearer realm=\\\"https://auth.docker.io/token\\\",service=\\\"registry.docker.io\\\",scope=\\\"repository:library/sha256:pull\\\"\"" container=/weblate-docker-compose_weblate_1 image="sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
watchtower-ghcr_1       | time="2021-07-08T12:25:22Z" level=debug msg="Pulling image" container=/weblate-docker-compose_weblate_1 image="sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
watchtower-ghcr_1       | time="2021-07-08T12:25:27Z" level=debug msg="Error pulling image sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8, Error response from daemon: Head https://registry-1.docker.io/v2/library/sha256/manifests/e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8: unauthorized: incorrect username or password"
watchtower-ghcr_1       | time="2021-07-08T12:25:27Z" level=info msg="Unable to update container \"/weblate-docker-compose_weblate_1\": Error response from daemon: Head https://registry-1.docker.io/v2/library/sha256/manifests/e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8: unauthorized: incorrect username or password. Proceeding to next."
watchtower-ghcr_1       | time="2021-07-08T12:25:27Z" level=debug msg="Session done: 1 scanned, 0 updated, 1 failed"
watchtower-ghcr_1       | time="2021-07-08T12:25:27Z" level=debug msg="Scheduled next run: 2021-07-08 12:25:49 +0000 UTC"

Using docker-compose:

watchtower-ghcr_1       | time="2021-07-08T12:44:55Z" level=debug msg=ghcr
watchtower-ghcr_1       | time="2021-07-08T12:44:55Z" level=debug msg="Sleeping for a second to ensure the docker api client has been properly initialized."
watchtower-ghcr_1       | time="2021-07-08T12:44:56Z" level=debug msg="Making sure everything is sane before starting"
watchtower-ghcr_1       | time="2021-07-08T12:44:56Z" level=debug msg="Retrieving running containers"
watchtower-ghcr_1       | time="2021-07-08T12:44:56Z" level=debug msg="There are no additional watchtower containers"
watchtower-ghcr_1       | time="2021-07-08T12:44:56Z" level=debug msg="Watchtower HTTP API skipped."
watchtower-ghcr_1       | time="2021-07-08T12:44:56Z" level=info msg="Watchtower 1.3.0\nUsing no notifications\nOnly checking containers in scope \"ghcr\"\nScheduling first run: 2021-07-08 12:45:26 +0000 UTC\nNote that the first check will be performed in 29 seconds"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Checking containers for updated images"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Retrieving running containers"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Trying to load authentication credentials." container=/weblate-docker-compose_weblate_1 image="ghcr.io/iobrokertranslator/weblate:master"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Loaded auth credentials for user UncleSamSwiss on registry ghcr.io/iobrokertranslator/weblate:master"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Got image name: ghcr.io/iobrokertranslator/weblate:master"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Credentials loaded"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Checking if pull is needed" container=/weblate-docker-compose_weblate_1 image="ghcr.io/iobrokertranslator/weblate:master"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Building challenge URL" URL="https://ghcr.io/v2/"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://ghcr.io/token\",service=\"ghcr.io\",scope=\"repository:user/image:pull\"" status="401 Unauthorized"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Checking challenge header content" realm="https://ghcr.io/token" service=ghcr.io
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Setting scope for auth token" image=ghcr.io/iobrokertranslator/weblate scope="repository:ghcr.io/iobrokertranslator/weblate:pull"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Credentials found."
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Parsing image ref" host=ghcr.io image=iobrokertranslator/weblate normalized="ghcr.io/iobrokertranslator/weblate:master" 
tag=master
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://ghcr.io/v2/iobrokertranslator/weblate/manifests/master"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Found a remote digest to compare with" remote="sha256:a3082489de4a3bd2690cdb2efd27f83ec76e716db9832eb0fc202664dcb3b2a5"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg=Comparing local="sha256:a3082489de4a3bd2690cdb2efd27f83ec76e716db9832eb0fc202664dcb3b2a5" remote="sha256:a3082489de4a3bd2690cdb2efd27f83ec76e716db9832eb0fc202664dcb3b2a5"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Found a match"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="No pull needed. Skipping image."
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="No new images found for /weblate-docker-compose_weblate_1"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Session done: 1 scanned, 0 updated, 0 failed"
watchtower-ghcr_1       | time="2021-07-08T12:45:26Z" level=debug msg="Scheduled next run: 2021-07-08 12:45:56 +0000 UTC"

Main differences:

  level=debug msg="Checking containers for updated images"
  level=debug msg="Retrieving running containers"
- level=debug msg="Trying to load authentication credentials." container=/weblate-docker-compose_weblate_1 image="sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
+ level=debug msg="Trying to load authentication credentials." container=/weblate-docker-compose_weblate_1 image="ghcr.io/iobrokertranslator/weblate:master"
- level=debug msg="Loaded auth credentials for user UncleSamSwiss on registry sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
+ level=debug msg="Loaded auth credentials for user UncleSamSwiss on registry ghcr.io/iobrokertranslator/weblate:master"
- level=debug msg="Got image name: sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
+ level=debug msg="Got image name: ghcr.io/iobrokertranslator/weblate:master"
  level=debug msg="Credentials loaded"
- level=debug msg="Checking if pull is needed" container=/weblate-docker-compose_weblate_1 image="sha256:e2dff46bca7b31539a043a3a42c714a21873d99ab92b9a865370479fd83122d8"
+ level=debug msg="Checking if pull is needed" container=/weblate-docker-compose_weblate_1 image="ghcr.io/iobrokertranslator/weblate:master"
- level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/"
+ level=debug msg="Building challenge URL" URL="https://ghcr.io/v2/"

Environment

  • Windows 10 Pro
  • x64
  • Docker version 20.10.7, build f0df350 (WSL2 backend)
  • docker-compose version 1.29.2, build 5becea4c

Additional context
My docker-compose.yml

version: "3"
services:
  weblate:
    image: ghcr.io/iobrokertranslator/weblate:master
    volumes:
      - weblate-data:/app/data
    env_file:
      - ./environment
    restart: unless-stopped
    depends_on:
      - database
      - cache
    labels:
      com.centurylinklabs.watchtower.scope: "ghcr"
  database:
    image: postgres:12-alpine
    env_file:
      - ./environment
    volumes:
      - postgres-data:/var/lib/postgresql/data
    restart: unless-stopped
  cache:
    image: redis:6-alpine
    restart: unless-stopped
    command: ["redis-server", "--appendonly", "yes"]
    volumes:
      - redis-data:/data
  watchtower-ghcr:
    image: containrrr/watchtower
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: --debug --cleanup --interval 30 --stop-timeout 30s --scope ghcr
volumes:
  weblate-data: {}
  postgres-data: {}
  redis-data: {}

Of course you need a docker-compose.override.yml, but I can't provide it here for obvious reasons (credentials).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions