Skip to content

Conversation

isaac-chung
Copy link
Collaborator

If you add a model or a dataset, please add the corresponding checklist:

Samoed and others added 30 commits April 25, 2025 00:26
* add tasks

* add benchmark

* fix imports

* update stsb split
Automatically generated by python-semantic-release
* add vision only bench

* add description

* correct zs task modalities

* specify tasks param
* Update benchmarks.py

* make lint

* add to left side bar
* update seed-embedding

* update seed models

* fix linting and tiktoken problem

* fix tiktoken bug

* fix lint

* update name

* Update mteb/models/seed_models.py

adopt suggestion

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

* update logging

* update lint

---------

Co-authored-by: zhangpeitian <zhangpeitian@bytedance.com>
Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>
* add 2 web SSL dino models

* add models from collection and revisions

* update memory_usage_mb and embed dim

* use automodel instead
Automatically generated by python-semantic-release
* update seed-embedding

* update seed models

* fix linting and tiktoken problem

* fix tiktoken bug

* fix lint

* update name

* Update mteb/models/seed_models.py

adopt suggestion

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

* update logging

* update lint

* update link

---------

Co-authored-by: zhangpeitian <zhangpeitian@bytedance.com>
Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>
* update benchmark table

* fix table
* update seed-embedding

* update seed models

* fix linting and tiktoken problem

* fix tiktoken bug

* fix lint

* update name

* Update mteb/models/seed_models.py

adopt suggestion

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

* update logging

* update lint

* update link

* update revision

---------

Co-authored-by: zhangpeitian <zhangpeitian@bytedance.com>
Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>
KennethEnevoldsen and others added 18 commits June 9, 2025 12:35
* Add files via upload

* Add files via upload

* Update benchmarks.py

* Update __init__.py

* Add files via upload

* Update R2MEDRetrieval.py

* Update run_mteb_r2med.py

* Delete scripts/run_mteb_r2med.py

* Update mteb/tasks/Retrieval/eng/R2MEDRetrieval.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

* Update mteb/tasks/Retrieval/eng/R2MEDRetrieval.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

* Update mteb/tasks/Retrieval/eng/R2MEDRetrieval.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

* Update mteb/tasks/Retrieval/eng/R2MEDRetrieval.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

* Add files via upload

* Delete mteb/descriptive_stats/Retrieval/R2MEDRetrieval.json

* Add files via upload

* Add files via upload

* Add files via upload

* Update R2MEDRetrieval.py

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* format citations

* Update R2MEDRetrieval.py

* Add files via upload

* Add files via upload

---------

Co-authored-by: Li Lei <34205771+ll0ruc@users.noreply.github.com>
Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>
update training datasets

Co-authored-by: zhangzeqing <zhangzeqing@zhejianglab.com>
* fix: Add adapted_from to Cmedqaretrieval

Also snuck in a fix with form=None, which is no longer valid, but was still used in a few places.

* format
Automatically generated by python-semantic-release
* Adding OpenAI client arg to init method (e.g., for already initialized AzureOpenAI client)

To use OpenAI embedding models via Azure, the model wrapper needs to be initialized with a different client.

* Update mteb/models/openai_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

* Update mteb/models/openai_models.py

* remove comment and format

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>
Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>
Add LGAI-Embedding

- Add mteb/models/lgai_embedding_models.py

- defined model metadata
Automatically generated by python-semantic-release
* add description to template

* fix typo
* Added HIT-TMG_KaLM-embedding-multilingual-mini-instruct-v1 with instruct wrapper

* Added KaLM_embedding_multilingual_mini_instruct_v1_5

* Added model to overview.py

* Fix Task Count Per Language Table in tasks.md

* resolve conflicts

* remove tasks.md

* Modified get_instruction funcion

* Added support for prompt dict in get_instruction

* fix lang code

* Address comments

* Delete mteb/models/check_models.py

* added prompts_dict support in InstructSentenceTransformerWrapper

* corrected instruction format

* corrected prompts format

* added correct instruction format

* fix implementation

* remove `if name main`

* add comment

---------

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com>
* fix: Reuploaded previously unavailable SNL datasets

closes #2477

* removed exceptions from tests

* temp fixes

* added temporary fix

* clean up commented out code

* format
Automatically generated by python-semantic-release
* Update usage.md

* Update usage.md

* Update docs/usage/usage.md

---------

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>
* add custom instructions

* fixed

* lint

* fix last instruction

---------

Co-authored-by: Kolodin Egor <eikolodin@sberbank.ru>
Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com>
@isaac-chung isaac-chung changed the base branch from main to maeb June 22, 2025 09:27
@isaac-chung isaac-chung changed the title [MAEB] Merge with main 20250622 [MAEB] Merge from main 20250622 Jun 22, 2025
@isaac-chung isaac-chung changed the title [MAEB] Merge from main 20250622 [MAEB] Merge from main Jun 22, 2025
@isaac-chung isaac-chung changed the title [MAEB] Merge from main [MAEB] Merge from main up to 1.38.30 Jun 22, 2025
@isaac-chung isaac-chung requested a review from Samoed June 22, 2025 09:28
@isaac-chung isaac-chung merged commit bdbe51f into maeb Jun 22, 2025
9 checks passed
@isaac-chung isaac-chung deleted the merge-with-main-20250622 branch June 22, 2025 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.