llm_guided_diffusion example code gets AttributeError: 'bool' object has no attribute '__module__' #5992
Description
Describe the bug
I simply followed the instructions here to try out the llm_grounded_diffusion pipeline and it doesn't work out of the box.
Reproduction
Started with making sure I'm using main
poetry add git+https://github.com/huggingface/diffusers.git
Then copied the example code and ran:
import torch
from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained(
"longlian/lmd_plus",
custom_pipeline="llm_grounded_diffusion",
variant="fp16", torch_dtype=torch.float16
)
pipe.enable_model_cpu_offload()
# Generate directly from a text prompt and an LLM response
prompt = "a waterfall and a modern high speed train in a beautiful forest with fall foliage"
phrases, boxes, bg_prompt, neg_prompt = pipe.parse_llm_response("""
[('a waterfall', [71, 105, 148, 258]), ('a modern high speed train', [255, 223, 181, 149])]
Background prompt: A beautiful forest with fall foliage
Negative prompt:
""")
images = pipe(
prompt=prompt,
negative_prompt=neg_prompt,
phrases=phrases,
boxes=boxes,
gligen_scheduled_sampling_beta=0.4,
output_type="pil",
num_inference_steps=50,
lmd_guidance_kwargs={}
).images
images[0].save("./lmd_plus_generation.jpg")Logs
C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\testproject-cM1rhShM-py3.10\lib\site-packages\huggingface_hub\file_download.py:672: FutureWarning: 'cached_download' is the legacy way to download files from the HF hub, please consider upgrading to 'hf_hub_download'
warnings.warn(
Loading pipeline components...: 29%|██▊ | 2/7 [00:00<00:00, 6.07it/s]C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\testproject-cM1rhShM-py3.10\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
warnings.warn(
Loading pipeline components...: 86%|████████▌ | 6/7 [00:00<00:00, 9.93it/s]`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["bos_token_id"]` will be overriden.
`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["eos_token_id"]` will be overriden.
Loading pipeline components...: 100%|██████████| 7/7 [00:00<00:00, 7.78it/s]
Traceback (most recent call last):
File "D:\GitHub\testproject\image_gen.py", line 19, in <module>
pipe = DiffusionPipeline.from_pretrained(
File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\testproject-cM1rhShM-py3.10\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 1357, in from_pretrained
model = pipeline_class(**init_kwargs)
File "C:\Users\krist\.cache\huggingface\modules\diffusers_modules\git\llm_grounded_diffusion.py", line 277, in __init__
super().__init__(
File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\testproject-cM1rhShM-py3.10\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion.py", line 236, in __init__
self.register_modules(
File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\testproject-cM1rhShM-py3.10\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 566, in register_modules
library = not_compiled_module.__module__.split(".")[0]
AttributeError: 'bool' object has no attribute '__module__'. Did you mean: '__mod__'?System Info
diffusersversion: 0.24.0.dev0- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.10.2
- PyTorch version (GPU?): 1.13.0+cu117 (True)
- Huggingface_hub version: 0.19.4
- Transformers version: 4.35.2
- Accelerate version: 0.23.0
- xFormers version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No
Who can help?
No response