llama: fix race in parallel make #6845
Conversation
llama/Makefile
Outdated
| @@ -278,7 +278,7 @@ DIST_RUNNERS = $(addprefix $(RUNNERS_DIST_DIR)/,$(addsuffix /ollama_llama_server | |||
| PAYLOAD_RUNNERS = $(addprefix $(RUNNERS_PAYLOAD_DIR)/,$(addsuffix /ollama_llama_server$(EXE_EXT).gz,$(RUNNERS))) | |||
| BUILD_RUNNERS = $(addprefix $(RUNNERS_BUILD_DIR)/,$(addsuffix /ollama_llama_server$(EXE_EXT),$(RUNNERS))) | |||
|
|
|||
| all: clean-payload dist payload | |||
| all: clean-payload .WAIT dist payload | |||
There was a problem hiding this comment.
This didn't fix the problem on my system but I guess there might be multiple similar races.
For me, it looks like there is a race within clean-payload itself - between rm and mkdir. Adding this solves the problem:
.NOTPARALLEL clean-payload
We probably still the need the .WAIT in all to solve the race between those dependencies.
There was a problem hiding this comment.
I would also suggest not depending on clean in the default target
There was a problem hiding this comment.
clean-payload just cleans the final build products, not everything. We need to do it so we can switch between old and new runners.
There was a problem hiding this comment.
worth noting this might not work as expected. .NOTPARALLEL is used to signal its prerequisites, clean-payload in this case, should be run sequentially. all will still execute its prerequisites in parallel. what you probably want is to make all .NOTPARALLEL
f1da9b2 to
ce1415e
Compare
Ensure the cleanup step completes before starting to build targets
ce1415e to
345e276
Compare
|
It looks like |
|
Closing in favor of a more extensive refactoring in #6924 |
Ensure the cleanup step completes before starting to build targets