-
Notifications
You must be signed in to change notification settings - Fork 441
Closed
Description
instructlab is using threads, which makes it incompatible with multiprocessing's default fork
start method on Linux. Fork and threads don't mix well. They can lead to all sorts of trouble from crashes to deadlocks. The unit test warn about the problem:
tests/test_lab_generate.py: 40 warnings
/usr/lib64/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=382568) is multi-threaded, use of fork() may lead to deadlocks in the child.
self.pid = os.fork()
Some bindings for hardware AI accelerators do not like fork, either. For example HabanaLabs documentation for Intel Gaudi HPUs states that:
Default start method is fork which may result in undefined behavior.
Metadata
Metadata
Assignees
Labels
No labels