-
Notifications
You must be signed in to change notification settings - Fork 465
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Here is the traceback
ERROR:mteb.evaluation.MTEB:Error while evaluating IndicQARetrieval: BuilderConfig 'corpus' not found. Available: ['as-corpus', 'as-qrels', 'as-queries', 'bn-corpus', 'bn-qrels', 'bn-queries', 'gu-corpus', 'gu-qrels', 'gu-queries', 'hi-corpus', 'hi-qrels', 'hi-queries', 'kn-corpus', 'kn-qrels', 'kn-queries', 'ml-corpus', 'ml-qrels', 'ml-queries', 'mr-corpus', 'mr-qrels', 'mr-queries', 'or-corpus', 'or-qrels', 'or-queries', 'pa-corpus', 'pa-qrels', 'pa-queries', 'ta-corpus', 'ta-qrels', 'ta-queries', 'te-corpus', 'te-qrels', 'te-queries']
Traceback (most recent call last):
File "/home/vatolin/miniconda3/envs/mteb/bin/mteb", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/vatolin/experiments/mteb/mteb/cli.py", line 388, in main
args.func(args)
File "/home/vatolin/experiments/mteb/mteb/cli.py", line 146, in run
eval.run(
File "/home/vatolin/experiments/mteb/mteb/evaluation/MTEB.py", line 672, in run
raise e
File "/home/vatolin/experiments/mteb/mteb/evaluation/MTEB.py", line 576, in run
task.load_data(**kwargs)
File "/home/vatolin/experiments/mteb/mteb/abstasks/AbsTaskRetrieval.py", line 294, in load_data
).load(split=split)
^^^^^^^^^^^^^^^^^
File "/home/vatolin/experiments/mteb/mteb/abstasks/AbsTaskRetrieval.py", line 93, in load
self._load_corpus()
File "/home/vatolin/experiments/mteb/mteb/abstasks/AbsTaskRetrieval.py", line 130, in _load_corpus
corpus_ds = load_dataset(
^^^^^^^^^^^^^
File "/home/vatolin/miniconda3/envs/mteb/lib/python3.12/site-packages/datasets/load.py", line 2606, in load_dataset
builder_instance = load_dataset_builder(
^^^^^^^^^^^^^^^^^^^^^
File "/home/vatolin/miniconda3/envs/mteb/lib/python3.12/site-packages/datasets/load.py", line 2314, in load_dataset_builder
builder_instance: DatasetBuilder = builder_cls(
^^^^^^^^^^^^
File "/home/vatolin/miniconda3/envs/mteb/lib/python3.12/site-packages/datasets/builder.py", line 374, in __init__
self.config, self.config_id = self._create_builder_config(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vatolin/miniconda3/envs/mteb/lib/python3.12/site-packages/datasets/builder.py", line 601, in _create_builder_config
raise ValueError(
ValueError: BuilderConfig 'corpus' not found. Available: ['as-corpus', 'as-qrels', 'as-queries', 'bn-corpus', 'bn-qrels', 'bn-queries', 'gu-corpus', 'gu-qrels', 'gu-queries', 'hi-corpus', 'hi-qrels', 'hi-queries', 'kn-corpus', 'kn-qrels', 'kn-queries', 'ml-corpus', 'ml-qrels', 'ml-queries', 'mr-corpus', 'mr-qrels', 'mr-queries', 'or-corpus', 'or-qrels', 'or-queries', 'pa-corpus', 'pa-qrels', 'pa-queries', 'ta-corpus', 'ta-qrels', 'ta-queries', 'te-corpus', 'te-qrels', 'te-queries']
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working