-
Notifications
You must be signed in to change notification settings - Fork 465
Closed
Labels
miebThe image extension of MTEBThe image extension of MTEB
Description
ERROR:mteb.evaluation.MTEB:Error while evaluating Winoground: DatasetInfo.__init__() got an unexpected keyword argument 'task_templates'
Traceback (most recent call last):
File "/data/niklas/mieb/mteb/scripts/run_mieb.py", line 81, in <module>
results = evaluation.run(model, output_folder="/data/niklas/mieb/results-mieb-final", batch_size=1)
File "/data/niklas/mieb/mteb/mteb/evaluation/MTEB.py", line 464, in run
raise e
File "/data/niklas/mieb/mteb/mteb/evaluation/MTEB.py", line 394, in run
task.load_data(eval_splits=task_eval_splits, **kwargs)
File "/data/niklas/mieb/mteb/mteb/abstasks/AbsTask.py", line 192, in load_data
self.dataset = datasets.load_dataset(**self.metadata_dict["dataset"]) # type: ignore
File "/env/lib/conda/gritkto4/lib/python3.10/site-packages/datasets/load.py", line 2132, in load_dataset
builder_instance = load_dataset_builder(
File "/env/lib/conda/gritkto4/lib/python3.10/site-packages/datasets/load.py", line 1890, in load_dataset_builder
builder_instance: DatasetBuilder = builder_cls(
File "/env/lib/conda/gritkto4/lib/python3.10/site-packages/datasets/builder.py", line 353, in __init__
info.update(self._info())
File "/data/huggingface/modules/datasets_modules/datasets/facebook--winoground/72585f4d9cd5a28790bb9bc2adbdd45633f36dfbf85df529e0756e114e134285/winoground.py", line 61, in _info
return datasets.DatasetInfo(
TypeError: DatasetInfo.__init__() got an unexpected keyword argument 'task_templates'
Metadata
Metadata
Assignees
Labels
miebThe image extension of MTEBThe image extension of MTEB