-
Notifications
You must be signed in to change notification settings - Fork 463
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
After #2759, Caltech101ZeroShot will fail with the following error:
results, tick, tock = self._run_eval(
File "/usr/local/lib/python3.10/dist-packages/mteb/evaluation/MTEB.py", line 307, in _run_eval
results = task.evaluate(
File "/usr/local/lib/python3.10/dist-packages/mteb/abstasks/AbsTask.py", line 137, in evaluate
scores[hf_subset] = self._evaluate_subset(
File "/usr/local/lib/python3.10/dist-packages/mteb/abstasks/Image/AbsTaskZeroShotClassification.py", line 128, in _evaluate_subset
candidate_labels = self.get_candidate_labels()
File "/usr/local/lib/python3.10/dist-packages/mteb/tasks/Image/ZeroShotClassification/eng/Caltech101.py", line 56, in get_candidate_labels
for name in self.dataset["test"].features[self.label_column_name].names
AttributeError: 'Value' object has no attribute 'names'
@KennethEnevoldsen could you please help take a look?
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working