-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Description
Description
Hello,
My docker-compose file consists of one image, but it is used by multiple services. So I defined three services:
- The first service contains the build definitions and image name. This service builds image.
- The second and third service uses this image and depends on first service.
I found this trick on Stackoverflow:
https://stackoverflow.com/questions/40899236/how-to-prevent-docker-compose-building-the-same-image-multiple-times
Next, I wanted to add integration with docker compose watch
, so I added develop
sections like below:
- The first service rebuilds the image if dependencies are changed.
- Second and third service sync application code
Unfortunately, if my services with the application do not have a build section, the files are not synchronized without any error message, if I add the build section, everything starts working, but this causes docker compose build to build the same image multiple times after executing docker compose build
.
From what I understand from the source code this is not the expected behavior. The sync
action should work without a build
section. The rebuild
action requires a build
section.
Lines 337 to 339 in 11c7a25
if trigger.Action == types.WatchActionRebuild && service.Build == nil { | |
return nil, fmt.Errorf("service %s doesn't have a build section, can't apply 'rebuild' on watch", service.Name) | |
} |
Steps To Reproduce
I have a simple application as below:
#./src/main.py
import textwrap
from flask import Flask
import os;
from pathlib import Path
INSTANCE_NAME = os.environ.get('INSTANCE_NAME', 'Hello World!')
app = Flask(__name__)
@app.route("/")
def hello():
response = f"INSTANCE_NAME: {INSTANCE_NAME}\n"
response += f"REQUIREMENTS: \n{Path('requirements.txt').read_text()}\n"
response += f"MAIN_APP: \n{Path(__file__).read_text()}\n"
return textwrap.dedent(response)
if __name__ == "__main__":
app.run()
# requirements.txt
flask
# Dockerfile
# syntax = docker/dockerfile:1.9
# check=error=true
FROM python:3.10-slim-bullseye
WORKDIR /app
RUN apt-get update; apt-get install -y dumb-init curl
COPY requirements.txt /app/
RUN pip install -r requirements.txt
ENV PYTHONPATH=/app/src
ENV FLASK_APP=/app/src/main.py
COPY ./src /app/src
RUN curl "http://host.docker.internal:9500/$(hostname)" || true
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
# docker-compose.yaml
services:
my-app-image:
build:
context: .
dockerfile: Dockerfile
image: ${COMPOSE_PROJECT_NAME}-my-app
command: ['echo', 'The docker image (${COMPOSE_PROJECT_NAME}-docparser) is ready to be used.']
develop:
watch:
- path: ./requirements.txt
action: rebuild
my-app-instance-1:
depends_on:
my-app-image:
condition: service_completed_successfully
required: false
environment:
INSTANCE_NAME: instance1
build:
context: .
dockerfile: Dockerfile
image: ${COMPOSE_PROJECT_NAME}-my-app
ports:
- "3001:3001"
command: [ 'flask', 'run', '--host=0.0.0.0', '--port=3001' ]
develop:
watch:
- action: sync
path: ./src
target: /app/src
my-app-instance-2:
depends_on:
my-app-image:
condition: service_completed_successfully
required: false
image: ${COMPOSE_PROJECT_NAME}-my-app
environment:
INSTANCE_NAME: instance2
ports:
- "3002:3002"
command: [ 'flask', 'run', '--host=0.0.0.0', '--port=3002' ]
develop:
watch:
- action: sync
path: ./src
target: /app/src
Now I run my environment:
docker compose build --no-cache
docker compose up --wait
And then I run Compose Watch
docker --debug --log-level=debug compose watch
I have the following output:
DEBU[0000] Enabled Docker Desktop integration (experimental) @ unix:///Users/my-user/Library/Containers/com.docker.docker/Data/docker-cli.sock
DEBU[0000] Watch configuration for service "my-app-instance-1":
- Action sync for path "/Users/my-user/compose_develop_seperate_image/src"
DEBU[0000] Watch configuration for service "my-app-image":
- Action rebuild for path "/Users/kbregula//compose_develop_seperate_image/requirements.txt"
Watch enabled
DEBU[0005] otel error error="<nil>"
As you can see, one service is not tracked.
In a separate terminal, I check if synchronization is working.
$ curl localhost:3001
INSTANCE_NAME: instance1
REQUIREMENTS:
flask
MAIN_APP:
import textwrap
from flask import Flask
import os;
from pathlib import Path
INSTANCE_NAME = os.environ.get('INSTANCE_NAME', 'Hello World!')
app = Flask(__name__)
@app.route("/")
def hello():
response = f"INSTANCE_NAME: {INSTANCE_NAME}\n"
response += f"REQUIREMENTS: \n{Path('requirements.txt').read_text()}\n"
response += f"MAIN_APP: \n{Path(__file__).read_text()}\n"
return textwrap.dedent(response)
if __name__ == "__main__":
app.run()
$ curl localhost:3002
INSTANCE_NAME: instance2
REQUIREMENTS:
flask
MAIN_APP:
import textwrap
from flask import Flask
import os;
from pathlib import Path
INSTANCE_NAME = os.environ.get('INSTANCE_NAME', 'Hello World!')
app = Flask(__name__)
@app.route("/")
def hello():
response = f"INSTANCE_NAME: {INSTANCE_NAME}\n"
response += f"REQUIREMENTS: \n{Path('requirements.txt').read_text()}\n"
response += f"MAIN_APP: \n{Path(__file__).read_text()}\n"
return textwrap.dedent(response)
if __name__ == "__main__":
app.run()
Then I make a change in the source code:
echo "# TEST $(date)" >> src/main.py
Next I checked again:
$ curl -s localhost:3001 | grep TEST
# TEST Sat Aug 17 17:42:27 CEST 2024
$ curl -s localhost:3002 | grep TEST
Here's the full log.
$ docker --debug --log-level=debug compose watch --no-up
DEBU[0000] Enabled Docker Desktop integration (experimental) @ unix:///Users/kbregula/Library/Containers/com.docker.docker/Data/docker-cli.sock
DEBU[0000] Watch configuration for service "my-app-image":
- Action rebuild for path "/Users/my-user/compose_develop_seperate_image/requirements.txt"
DEBU[0000] Watch configuration for service "my-app-instance-1":
- Action sync for path "/Users/my-user/compose_develop_seperate_image/src"
Watch enabled
DEBU[0005] otel error error="<nil>"
DEBU[0090] change for /Users/my-user/compose_develop_seperate_image/src/main.py - comparing with /Users/my-user/compose_develop_seperate_image/src
DEBU[0090] batch start: service[my-app-instance-1] count[1]
Syncing "my-app-instance-1" after changes were detected
DEBU[0090] batch complete: service[my-app-instance-1] duration[29.016667ms] count[1]
DEBU[0095] otel error error="<nil>"
I will add that if a change is made in dependencies (requirements.txt
), the image is correctly updated.
$ echo "### TEST2 $(date)" >> requirements.txt
curl -s localhost:3001 | grep TEST2
### TEST2 Sat Aug 17 17:45:45 CEST 2024
$ curl -s localhost:3002 | grep TEST2
### TEST2 Sat Aug 17 17:45:45 CEST 2024
Compose Version
Docker Compose version v2.29.1-desktop.1
Docker Environment
Client:
Version: 27.1.1
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.16.1-desktop.1
Path: /Users/my-user/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.29.1-desktop.1
Path: /Users/my-user/.docker/cli-plugins/docker-compose
debug: Get a shell into any image or container (Docker Inc.)
Version: 0.0.34
Path: /Users/my-user/.docker/cli-plugins/docker-debug
desktop: Docker Desktop commands (Alpha) (Docker Inc.)
Version: v0.0.14
Path: /Users/my-user/.docker/cli-plugins/docker-desktop
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.2
Path: /Users/my-user/.docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.25
Path: /Users/my-user/.docker/cli-plugins/docker-extension
feedback: Provide feedback, right in your terminal! (Docker Inc.)
Version: v1.0.5
Path: /Users/my-user/.docker/cli-plugins/docker-feedback
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v1.3.0
Path: /Users/my-user/.docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /Users/my-user/.docker/cli-plugins/docker-sbom
scout: Docker Scout (Docker Inc.)
Version: v1.11.0
Path: /Users/my-user/.docker/cli-plugins/docker-scout
Server:
Containers: 10
Running: 2
Paused: 0
Stopped: 8
Images: 18
Server Version: 27.1.1
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
runc version: v1.1.13-0-g58aa920
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 6.10.0-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: aarch64
CPUs: 10
Total Memory: 16.07GiB
Name: docker-desktop
ID:
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=unix:///Users/my-user/Library/Containers/com.docker.docker/Data/docker-cli.sock
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false
WARNING: daemon is not using the default seccomp profile
Anything else?
No response