Skip to content

calulate_metadata_metrics fails with BrightRetrieval task #1483

@imenelydiaker

Description

@imenelydiaker

Btw @Samoed, BrightRetrieval raises an error when computing descriptive stats:

Traceback (most recent call last):
  File "~/mteb/scripts/tasks_descriptive_tasks.py", line 29, in <module>
    stat = task.calculate_metadata_metrics()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/mteb/mteb/abstasks/AbsTask.py", line 212, in calculate_metadata_metrics
    descriptive_stats[split] = self._calculate_metrics_from_split(
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/mteb/mteb/abstasks/AbsTaskRetrieval.py", line 341, in _calculate_metrics_from_split
    corpus.update(process_docs(self.corpus, hf_subset, split))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/mteb/mteb/abstasks/AbsTaskRetrieval.py", line 468, in process_docs
    f"{split}_{hf_subset}_{k}": v for k, v in collection[hf_subset][split].items()
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'items'

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions