-
Notifications
You must be signed in to change notification settings - Fork 463
Description
https://github.com/embeddings-benchmark/mteb/actions/runs/14292996959/job/40056622040?pr=2501
=========================== short test summary info ============================
FAILED tests/test_cli.py::test_run_task[intfloat/multilingual-e5-small-BornholmBitextMining-fd1525a9fd15316a2d503bf26ab031a61d056e98] - NameError: name 'init_empty_weights' is not defined
FAILED tests/test_reproducible_workflow.py::test_reproducibility_workflow[8b3219a92973c328a8e22fadcfa821b5dc75636a-sentence-transformers/all-MiniLM-L6-v2-BornholmBitextMining] - NameError: name 'init_empty_weights' is not defined
FAILED tests/test_tasks/test_mteb_rerank.py::test_mteb_rerank - NameError: name 'init_empty_weights' is not defined
FAILED tests/test_tasks/test_mteb_rerank.py::test_reranker_same_ndcg1 - NameError: name 'init_empty_weights' is not defined
== 4 failed, 3241 passed, 408 skipped, 1851313 warnings in 878.32s (0:14:38) ===
Looks like the latest transformers version is causing errors:
Version:
Collecting sentence_transformers>=3.0.0 (from mteb==1.36.37)
Using cached sentence_transformers-4.0.2-py3-none-any.whl.metadata (13 kB)
Collecting transformers<5.0.0,>=4.41.0 (from sentence_transformers>=3.0.0->mteb==1.36.37)
Using cached transformers-4.51.0-py3-none-any.whl.metadata (38 kB)
Error:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/mteb/cli.py:124: in run
model = mteb.get_model(args.model, args.model_revision, device=device)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/mteb/models/overview.py:237: in get_model
model = meta.load_model(**kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/mteb/model_meta.py:151: in load_model
model: Encoder = loader(**kwargs) # type: ignore
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/mteb/model_meta.py:48: in sentence_transformers_loader
return SentenceTransformerWrapper(model=model_name, revision=revision, **kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/mteb/models/sentence_transformer_wrapper.py:37: in __init__
self.model = SentenceTransformer(model, revision=revision, **kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py:309: in __init__
modules, self.module_kwargs = self._load_sbert_model(
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py:1802: in _load_sbert_model
module = module_class(model_name_or_path, cache_dir=cache_folder, backend=self.backend, **kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py:81: in __init__
self._load_model(model_name_or_path, config, cache_dir, backend, is_peft_model, **model_args)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py:181: in _load_model
self.auto_model = AutoModel.from_pretrained(
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:571: in from_pretrained
return model_class.from_pretrained(
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/transformers/modeling_utils.py:279: in _wrapper
return func(*args, **kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/transformers/modeling_utils.py:4333: in from_pretrained
model_init_context = cls.get_init_context(is_quantized, _is_ds_init_called)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cls = <class 'transformers.models.bert.modeling_bert.BertModel'>
is_quantized = False, _is_ds_init_called = False
@classmethod
def get_init_context(cls, is_quantized: bool, _is_ds_init_called: bool):
# With deepspeed, we cannot initialize the model on meta device
if is_deepspeed_zero3_enabled():
init_contexts = [no_init_weights()]
if not is_quantized and not _is_ds_init_called:
logger.info("Detected DeepSpeed ZeRO-3: activating zero.init() for this model")
init_contexts.extend(
[
deepspeed.zero.Init(config_dict_or_path=deepspeed_config()),
set_zero3_state(),
]
)
elif is_quantized:
init_contexts.append(set_quantized_state())
else:
> init_contexts = [no_init_weights(), init_empty_weights()]
E NameError: name 'init_empty_weights' is not defined
Metadata
Metadata
Assignees
Labels
No labels