Skip to content

Conversation

ZiyiXia
Copy link
Contributor

@ZiyiXia ZiyiXia commented May 23, 2025

(eng, v2) only contains the 41 filtered tasks. (eng, v1) contain the whole 56 tasks

Checklist

  • I did not add a dataset, or if I did, I added the dataset checklist to the PR and completed it.
  • I did not add a model, or if I did, I added the model checklist to the PR and completed it.

@Samoed
Copy link
Member

Samoed commented May 23, 2025

Why do we need this change? V2 version is better and faster to use

@ZiyiXia
Copy link
Contributor Author

ZiyiXia commented May 23, 2025

It's a typo. The text above it states:

For instance to select the 56 English datasets that form the English leaderboard

while the code is using v2.

@Samoed
Copy link
Member

Samoed commented May 23, 2025

Then I think it would be better to change number of tasks than benchmark version

@Samoed Samoed requested a review from KennethEnevoldsen May 23, 2025 08:54
@KennethEnevoldsen
Copy link
Contributor

@ZiyiXia, thanks for letting us know about the mistake. We will fix the number instead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants