Hi,
I’m submitting this pull request to push BarExamQA to MTEB.

BarExamQA is a dataset created by RegLab for the purposes of evaluating models on the retrieval of relevant legal provisions.

BarExamQA contains over 100 questions taken from the bar exams of states around the US, with law students having manually identified the most relevant legal provisions to each question.

We would like to improve the coverage of legal domain tasks on MTEB and we believe this dataset will contribute to increasing the diversity and difficulty of MTEB.

This pull request is being submitted courtesy of Isaacus, a legal AI research company.

You may find the original dataset here:
https://huggingface.co/datasets/reglab/barexam_qa

Our version of the dataset after having been converted to the MTEB information retrieval dataset format is available here:
https://huggingface.co/datasets/isaacus/mteb-barexam-qa

Checklist

• I have outlined why this dataset is filling an existing gap in mteb
• I have tested that the dataset runs with the mteb package.
• I have run the following models on the task (adding the results to the pr). These can be run using the mteb run -m {model_name} -t {task_name} command.
• I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
• I have considered the size of the dataset and reduced it if it is too big (2048 examples is typically large enough for most tasks)