Skip to content

transformers version causing errors? #2503

@isaac-chung

Description

@isaac-chung

https://github.com/embeddings-benchmark/mteb/actions/runs/14292996959/job/40056622040?pr=2501

=========================== short test summary info ============================
FAILED tests/test_cli.py::test_run_task[intfloat/multilingual-e5-small-BornholmBitextMining-fd1525a9fd15316a2d503bf26ab031a61d056e98] - NameError: name 'init_empty_weights' is not defined
FAILED tests/test_reproducible_workflow.py::test_reproducibility_workflow[8b3219a92973c328a8e22fadcfa821b5dc75636a-sentence-transformers/all-MiniLM-L6-v2-BornholmBitextMining] - NameError: name 'init_empty_weights' is not defined
FAILED tests/test_tasks/test_mteb_rerank.py::test_mteb_rerank - NameError: name 'init_empty_weights' is not defined
FAILED tests/test_tasks/test_mteb_rerank.py::test_reranker_same_ndcg1 - NameError: name 'init_empty_weights' is not defined
== 4 failed, 3241 passed, 408 skipped, 1851313 warnings in 878.32s (0:14:38) ===

Looks like the latest transformers version is causing errors:

Version:

Collecting sentence_transformers>=3.0.0 (from mteb==1.36.37)
  Using cached sentence_transformers-4.0.2-py3-none-any.whl.metadata (13 kB)
Collecting transformers<5.0.0,>=4.41.0 (from sentence_transformers>=3.0.0->mteb==1.36.37)
  Using cached transformers-4.51.0-py3-none-any.whl.metadata (38 kB)

Error:

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/mteb/cli.py:124: in run
    model = mteb.get_model(args.model, args.model_revision, device=device)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/mteb/models/overview.py:237: in get_model
    model = meta.load_model(**kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/mteb/model_meta.py:151: in load_model
    model: Encoder = loader(**kwargs)  # type: ignore
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/mteb/model_meta.py:48: in sentence_transformers_loader
    return SentenceTransformerWrapper(model=model_name, revision=revision, **kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/mteb/models/sentence_transformer_wrapper.py:37: in __init__
    self.model = SentenceTransformer(model, revision=revision, **kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py:309: in __init__
    modules, self.module_kwargs = self._load_sbert_model(
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py:1802: in _load_sbert_model
    module = module_class(model_name_or_path, cache_dir=cache_folder, backend=self.backend, **kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py:81: in __init__
    self._load_model(model_name_or_path, config, cache_dir, backend, is_peft_model, **model_args)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py:181: in _load_model
    self.auto_model = AutoModel.from_pretrained(
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:571: in from_pretrained
    return model_class.from_pretrained(
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/transformers/modeling_utils.py:279: in _wrapper
    return func(*args, **kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/transformers/modeling_utils.py:4333: in from_pretrained
    model_init_context = cls.get_init_context(is_quantized, _is_ds_init_called)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'transformers.models.bert.modeling_bert.BertModel'>
is_quantized = False, _is_ds_init_called = False

    @classmethod
    def get_init_context(cls, is_quantized: bool, _is_ds_init_called: bool):
        # With deepspeed, we cannot initialize the model on meta device
        if is_deepspeed_zero3_enabled():
            init_contexts = [no_init_weights()]
            if not is_quantized and not _is_ds_init_called:
                logger.info("Detected DeepSpeed ZeRO-3: activating zero.init() for this model")
                init_contexts.extend(
                    [
                        deepspeed.zero.Init(config_dict_or_path=deepspeed_config()),
                        set_zero3_state(),
                    ]
                )
            elif is_quantized:
                init_contexts.append(set_quantized_state())
        else:
>           init_contexts = [no_init_weights(), init_empty_weights()]
E           NameError: name 'init_empty_weights' is not defined

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions