FLUX error when loading with low_cpu_mem_usage=False and ignore_mismatched_sizes=True #9343
Description
Describe the bug
I'd like to change the input layers of FLUX for training some img2img stuff, but got:
TypeError: expected str, bytes or os.PathLike object, not NoneType
when loading FluxTransformer2DModel with low_cpu_mem_usage=False, ignore_mismatched_sizes=True.
Reproduction
from diffusers.models import FluxTransformer2DModel
transformer = FluxTransformer2DModel.from_pretrained(
"black-forest-labs/FLUX.1-dev",
subfolder="transformer",
torch_dtype=weight_dtype,
low_cpu_mem_usage=False, ignore_mismatched_sizes=True,
revision=None, variant=None)
Logs
expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/model_loading_utils.py", line 104, in load_state_dict
file_extension = os.path.basename(checkpoint_file).split(".")[-1]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen posixpath>", line 142, in basename
TypeError: expected str, bytes or os.PathLike object, not NoneType
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/xxxx/repos/xxxx/train.py", line xxxx, in <module>
main()
File "/home/xxxx/repos/xxxx/train.py", line xxx, in main
transformer = load_flux(args, weight_dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/xxxx.py", line xx, in load_flux
transformer = FluxTransformer2DModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/modeling_utils.py", line 828, in from_pretrained
state_dict = load_state_dict(model_file, variant=variant)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/model_loading_utils.py", line 116, in load_state_dict
with open(checkpoint_file) as f:
^^^^^^^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneTypeSystem Info
- 🤗 Diffusers version: 0.31.0.dev0
- Platform: Linux-5.15.0-119-generic-x86_64-with-glibc2.31
- Running on Google Colab?: No
- Python version: 3.11.9
- PyTorch version (GPU?): 2.4.0+cu124 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 0.24.5
- Transformers version: 4.44.0
- Accelerate version: 0.33.0
- PEFT version: 0.12.0
- Bitsandbytes version: 0.43.3
- Safetensors version: 0.4.4
- xFormers version: 0.0.27.post2
- Accelerator: NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB - Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes