Adding Turkish Movie Sentiment Dataset #389

asparius · 2024-04-16T20:23:00Z

Checklist for adding MMTEB dataset

Reason for dataset addition:

It is a collected Turkish movie sentiment dataset and it is the first classification dataset for Turkish. Publication of the dataset.

I have tested that the dataset runs with the mteb package.
I have run the following models on the task (adding the results to the pr). These can be run using the mteb run -m {model_name} -t {task_name} command.
- sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
- intfloat/multilingual-e5-small
I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
I have considered the size of the dataset and reduced it if it is too big (2048 examples is typically large enough for most tasks)
Run tests locally to make sure nothing is broken using make test.
Run the formatter to format the code using make lint.
I have added points for my submission to the POINTS.md file.

Sakshamrzt

Please add points too for the dataset. 2+4(6) points for yourself and 1 point for me for the review.

Thanks a lot for the changes!

mteb/tasks/Classification/tur/TurkishMovieSentimentClassification.py

isaac-chung

A few more small things around metadata. I have found mteb/abstasks/TaskMetadata.py to be a helpful reference.

mteb/tasks/Classification/tur/TurkishMovieSentimentClassification.py

…ion.py Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

isaac-chung

Thanks for iterating. We are very close. Sorry that I missed a few things in the first round. Please add a tur/__init__.py file as well.

Looks good otherwise!

Oh, and once you have merged updates from the main branch, please add 2+4=6 points for yourself, and 1 point each for myself and @Sakshamrzt for the review.

mteb/tasks/Classification/tur/TurkishMovieSentimentClassification.py

…ion.py Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

asparius · 2024-04-17T13:00:42Z

Updated everything according to reviews @isaac-chung

isaac-chung

So close!
Could you please also resolve the merge conflicts by syncing your fork and merging in the updates from main?

mteb/tasks/Classification/tur/TurkishMovieSentimentClassification.py

asparius · 2024-04-17T13:57:25Z

So close! Could you please also resolve the merge conflicts by syncing your fork and merging in the updates from main?

Should be ok now

isaac-chung

@asparius Thanks for your contribution!

KennethEnevoldsen · 2024-04-18T08:22:38Z

@isaac-chung, when merging datasets into main, please prefix it with "fix:" to ensure that the version is bumped. You can find more on this in the contributing guidelines

isaac-chung · 2024-04-18T08:34:09Z

Yep will do. Apologies!

KennethEnevoldsen · 2024-04-18T09:49:44Z

No worries, with the number of PR atm. it is unlikely to cause any issues

Adding Turkish Movie Sentiment Dataset

8d59671

Sakshamrzt reviewed Apr 17, 2024

View reviewed changes

mteb/tasks/Classification/tur/TurkishMovieSentimentClassification.py Outdated Show resolved Hide resolved

Sakshamrzt reviewed Apr 17, 2024

View reviewed changes

mteb/tasks/Classification/tur/TurkishMovieSentimentClassification.py Outdated Show resolved Hide resolved

isaac-chung reviewed Apr 17, 2024

View reviewed changes

mteb/tasks/Classification/tur/TurkishMovieSentimentClassification.py Outdated Show resolved Hide resolved

asparius and others added 8 commits April 17, 2024 14:57

Update mteb/tasks/Classification/tur/TurkishMovieSentimentClassificat…

d410080

…ion.py Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

Update mteb/tasks/Classification/tur/TurkishMovieSentimentClassificat…

19f763c

…ion.py Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

Update mteb/tasks/Classification/tur/TurkishMovieSentimentClassificat…

f062255

…ion.py Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

Update mteb/tasks/Classification/tur/TurkishMovieSentimentClassificat…

7fc1d20

…ion.py Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

Update mteb/tasks/Classification/tur/TurkishMovieSentimentClassificat…

2f7319b

…ion.py Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

Update avg_character length and socioecon status

17972b8

Updating licence

8b08832

Update TurkishMovieSentimentClassification.py

06ff969

isaac-chung reviewed Apr 17, 2024

View reviewed changes

asparius and others added 3 commits April 17, 2024 15:37

Update mteb/tasks/Classification/tur/TurkishMovieSentimentClassificat…

17aca11

…ion.py Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

Update mteb/tasks/Classification/tur/TurkishMovieSentimentClassificat…

894314e

…ion.py Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

Update TurkishMovieSentimentClassification.py

30d1c01

isaac-chung mentioned this pull request Apr 17, 2024

Update points.md #397

Closed

9 tasks

asparius added 3 commits April 17, 2024 15:48

Update TurkishMovieSentimentClassification.py

a41ca04

Create __init__.py

4439e42

Update points.md

527dd91

isaac-chung reviewed Apr 17, 2024

View reviewed changes

mteb/tasks/Classification/tur/TurkishMovieSentimentClassification.py Outdated Show resolved Hide resolved

asparius added 4 commits April 17, 2024 16:25

Merge branch 'main' into trmoviedemirtas

2db4147

Update TurkishMovieSentimentClassification.py

c169644

Update points.md

c3d66df

Update TurkishMovieSentimentClassification.py

0033ce9

asparius and others added 2 commits April 17, 2024 17:24

formatting

c8494fb

Update points.md

0efff16

asparius added 3 commits April 17, 2024 17:34

Update points.md

cd27c30

Update points.md

1067ad0

Merge branch 'main' into trmoviedemirtas

f0df183

isaac-chung approved these changes Apr 17, 2024

View reviewed changes

isaac-chung merged commit b01382b into embeddings-benchmark:main Apr 17, 2024

isaac-chung mentioned this pull request Apr 17, 2024

Add points missing from PR#389 #403

Merged

asparius deleted the trmoviedemirtas branch April 17, 2024 15:03

asparius mentioned this pull request Apr 17, 2024

Turkish multidomain product review data #406

Merged

9 tasks

Adding Turkish Movie Sentiment Dataset #389

Adding Turkish Movie Sentiment Dataset #389

Uh oh!

Conversation

asparius commented Apr 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist for adding MMTEB dataset

Uh oh!

Sakshamrzt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

isaac-chung left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

isaac-chung left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

asparius commented Apr 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

isaac-chung left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asparius commented Apr 17, 2024

Uh oh!

isaac-chung left a comment

Choose a reason for hiding this comment

Uh oh!

KennethEnevoldsen commented Apr 18, 2024

Uh oh!

isaac-chung commented Apr 18, 2024

Uh oh!

KennethEnevoldsen commented Apr 18, 2024

Uh oh!

Uh oh!

asparius commented Apr 16, 2024 •

edited

Loading

asparius commented Apr 17, 2024 •

edited

Loading