results = evaluation.run(model, output_folder=f"results/{model}",
                                        batch_size=8, save_corpus_embeddings=True,
                                        save_predictions=True, export_errors=True, verbosity= 3
                                        )

*****************VRD Dataset : ImageCoDeT2IMultiChoice
─────────────────────────────────────────────────────────────────────────────────────────────── Selected tasks  ────────────────────────────────────────────────────────────────────────────────────────────────
Compositionality
    - ImageCoDeT2IMultiChoice, it2i


Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/corpus/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
INFO:datasets.info:Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/corpus/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
Found cached dataset imagecode-multi (/home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/corpus/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c)
INFO:datasets.builder:Found cached dataset imagecode-multi (/home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/corpus/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c)
Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/corpus/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
INFO:datasets.info:Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/corpus/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
Constructing Dataset for split corpus, from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/corpus/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
DEBUG:datasets.builder:Constructing Dataset for split corpus, from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/corpus/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/query/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
INFO:datasets.info:Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/query/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
Found cached dataset imagecode-multi (/home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/query/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c)
INFO:datasets.builder:Found cached dataset imagecode-multi (/home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/query/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c)
Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/query/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
INFO:datasets.info:Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/query/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
Constructing Dataset for split test, from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/query/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
DEBUG:datasets.builder:Constructing Dataset for split test, from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/query/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/qrels/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
INFO:datasets.info:Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/qrels/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
Found cached dataset imagecode-multi (/home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/qrels/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c)
INFO:datasets.builder:Found cached dataset imagecode-multi (/home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/qrels/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c)
Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/qrels/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
INFO:datasets.info:Loading Dataset info from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/qrels/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
Constructing Dataset for split test, from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/qrels/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
DEBUG:datasets.builder:Constructing Dataset for split test, from /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/qrels/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c
Set __getitem__(key) output type to arrow for no columns  (when key is int or slice) and don't output other (un-formatted) columns.
DEBUG:datasets.arrow_dataset:Set __getitem__(key) output type to arrow for no columns  (when key is int or slice) and don't output other (un-formatted) columns.
Loading cached processed dataset at /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/qrels/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c/cache-2065a22825282439.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/qrels/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c/cache-2065a22825282439.arrow
Set __getitem__(key) output type to arrow for no columns  (when key is int or slice) and don't output other (un-formatted) columns.
DEBUG:datasets.arrow_dataset:Set __getitem__(key) output type to arrow for no columns  (when key is int or slice) and don't output other (un-formatted) columns.
Set __getitem__(key) output type to python objects for ['query-id', 'corpus-id', 'score'] columns  (when key is int or slice) and don't output other (un-formatted) columns.
DEBUG:datasets.arrow_dataset:Set __getitem__(key) output type to python objects for ['query-id', 'corpus-id', 'score'] columns  (when key is int or slice) and don't output other (un-formatted) columns.

Map:   0%|          | 0/23020 [00:00<?, ? examples/s]Set __getitem__(key) output type to arrow for no columns  (when key is int or slice) and don't output other (un-formatted) columns.
DEBUG:datasets.arrow_dataset:Set __getitem__(key) output type to arrow for no columns  (when key is int or slice) and don't output other (un-formatted) columns.

Map:  14%|█▎        | 3128/23020 [00:00<00:00, 29280.20 examples/s]
Map:  27%|██▋       | 6245/23020 [00:00<00:00, 30354.08 examples/s]
Map:  46%|████▌     | 10630/23020 [00:00<00:00, 29732.78 examples/s]
Map:  60%|█████▉    | 13762/23020 [00:00<00:00, 30274.27 examples/s]
Map:  80%|███████▉  | 18381/23020 [00:00<00:00, 30489.65 examples/s]
Map:  99%|█████████▉| 22896/23020 [00:00<00:00, 30339.33 examples/s]Finished processing shard number None of 1.
DEBUG:datasets.arrow_dataset:Finished processing shard number None of 1.

Map: 100%|██████████| 23020/23020 [00:00<00:00, 29562.08 examples/s]
Loading cached processed dataset at /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/query/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c/cache-c1dcea3d461d7d75.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /home/aamita/.cache/huggingface/datasets/JamieSJS___imagecode-multi/query/0.0.0/d28adfd8b34fefa546fdf94bdc352622b2575f6c/cache-c1dcea3d461d7d75.arrow
ERROR:mteb.evaluation.MTEB:Error while evaluating ImageCoDeT2IMultiChoice: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/mnt/shared/aamita/project/image_retrieval/notebooks/load_models.py", line 141, in <module>
    results = evaluation.run(model, output_folder=f"results/{model}",
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/evaluation/MTEB.py", line 673, in run
    raise e
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/evaluation/MTEB.py", line 626, in run
    results, tick, tock = self._run_eval(
                          ^^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/evaluation/MTEB.py", line 308, in _run_eval
    results = task.evaluate(
              ^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/abstasks/Image/AbsTaskAny2AnyMultiChoice.py", line 363, in evaluate
    scores[hf_subset] = self._evaluate_subset(
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/abstasks/Image/AbsTaskAny2AnyMultiChoice.py", line 372, in _evaluate_subset
    results = retriever(corpus, queries, relevant_docs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/evaluation/evaluators/Image/Any2AnyMultiChoiceEvaluator.py", line 298, in __call__
    return self.retriever.search(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/evaluation/evaluators/Image/Any2AnyMultiChoiceEvaluator.py", line 124, in search
    query_embeddings = self.model.get_text_embeddings(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/models/gme_v_models.py", line 196, in get_text_embeddings
    return self.get_fused_embeddings(texts=texts, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/models/gme_v_models.py", line 220, in get_fused_embeddings
    instruction = self.get_instruction(task_name, prompt_type)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/models/wrapper.py", line 93, in get_instruction
    task = mteb.get_task(task_name=task_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/site-packages/mteb/overview.py", line 330, in get_task
    close_matches = difflib.get_close_matches(task_name, TASKS_REGISTRY.keys())
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/difflib.py", line 701, in get_close_matches
    s.set_seq2(word)
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/difflib.py", line 248, in set_seq2
    self.__chain_b()
  File "/home/aamita/miniconda3/envs/retrieval/lib/python3.12/difflib.py", line 280, in __chain_b
    for i, elt in enumerate(b):
                  ^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable