Skip to content

Caltech101ZeroShot fails with missing names #2770

@wwu137

Description

@wwu137

After #2759, Caltech101ZeroShot will fail with the following error:

 results, tick, tock = self._run_eval(
  File "/usr/local/lib/python3.10/dist-packages/mteb/evaluation/MTEB.py", line 307, in _run_eval
    results = task.evaluate(
  File "/usr/local/lib/python3.10/dist-packages/mteb/abstasks/AbsTask.py", line 137, in evaluate
    scores[hf_subset] = self._evaluate_subset(
  File "/usr/local/lib/python3.10/dist-packages/mteb/abstasks/Image/AbsTaskZeroShotClassification.py", line 128, in _evaluate_subset
    candidate_labels = self.get_candidate_labels()
  File "/usr/local/lib/python3.10/dist-packages/mteb/tasks/Image/ZeroShotClassification/eng/Caltech101.py", line 56, in get_candidate_labels
    for name in self.dataset["test"].features[self.label_column_name].names
AttributeError: 'Value' object has no attribute 'names'

@KennethEnevoldsen could you please help take a look?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions