Speed up test_save_predictions #1630
Description
GH tests reveal the following slow tests:
============================= slowest 5 durations ==============================
210.39s call tests/test_cli.py::test_save_predictions
19.10s call tests/test_cli.py::test_run_task[intfloat/multilingual-e5-small-BornholmBitextMining-fd1525a9fd15316a2d503bf26ab031a61d056e98]
17.32s call tests/test_benchmark/test_benchmark_integration_with_datasets.py::test_benchmark_sentence_transformer[model0-task1]
16.07s call tests/test_tasks/test_mteb_rerank.py::test_mteb_rerank
12.80s call tests/test_tasks/test_mteb_rerank.py::test_reranker_same_ndcg1
========= 1577 passed, 258 skipped, 200 warnings in 393.06s (0:06:33) ==========
Right now, test_save_predictions run
command = f"{sys.executable} -m mteb run -m all-MiniLM-L6-v2 -t NFCorpus --output_folder tests/results --save_predictions"Propose to use a smaller dataset, smaller model, both, or use a mock or limit to reduce the runtime significantly.