Multiprocessing support #32864
Description
Running model forwards within a process seems to get stuck. I tried to set TOKENIZERS_PARALLELISM to true and false but unfortunately both couldn't help 🥲
System Info
transformers-cli env:
- `transformers` version: 4.44.0
- Platform: Linux-6.10.0-linuxkit-aarch64-with-glibc2.35
- Python version: 3.10.14
- Huggingface_hub version: 0.24.5
- Safetensors version: 0.4.4
- Accelerate version: 0.31.0
- Accelerate config: not found
- PyTorch version (GPU?): 2.4.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: yes
Who can help?
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
Minimal example:
from transformers import AutoModelForCausalLM, AutoTokenizer
from multiprocess import Process, Queue
import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"
model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")
tok_ids = tokenizer.encode("Multiprocessing with Hugging Face could be an ", return_tensors="pt")
def fwd(model, tok_ids, queue):
print("Starting process")
print(f"{os.environ['TOKENIZERS_PARALLELISM']=}")
print(f"{type(model)=}")
print(f"{tok_ids=}")
try:
outs = model(tok_ids)
except Exception as e:
print(f"Error: {e}")
print(f"{outs=}")
queue.put(outs)
queue = Queue()
pr = Process(target=fwd, args=(model, tok_ids, queue))
pr.start()
pr.join()
outs = queue.get()
print(outs)prints
Starting process
os.environ['TOKENIZERS_PARALLELISM']='false'
type(model)=<class 'transformers.models.gpt2.modeling_gpt2.GPT2LMHeadModel'>
tok_ids=tensor([[15205, 541, 305, 919, 278, 351, 12905, 2667, 15399, 714,
307, 281, 220]])
Expected behavior
Shouldn't get stuck.