Skip to content

add "batcher stopped because" field to batch objects #5488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Apr 17, 2025

Conversation

dureuill
Copy link
Contributor

@dureuill dureuill commented Apr 2, 2025

Fixes #5511

Public Usage


This PR makes forward-compatible changes

Forward-compatible changes are changes to the database such that databases created in an older version of Meilisearch are still valid in the new version of Meilisearch. They usually represent additive changes, like adding a new optional attribute or setting.

  • Detail the change to the DB format and why they are forward compatible
    • Adds a new stopReason field in the batch objects in the DB
    • When deserializing a batch object, if that field is missing, as is the case for batch objects created before this PR, then the field will be set to unspecified
  • Forward-compatibility: A database created before this PR and using the features touched by this PR was able to be opened by a Meilisearch produced by the code of this PR.
    • I checked that the batch reason was unspecified

@dureuill dureuill added the db change A database was modified label Apr 3, 2025
Copy link

github-actions bot commented Apr 3, 2025

Hello, I'm a bot 🤖

You are receiving this message because you declared that this PR make changes to the Meilisearch database.
Depending on the nature of the change, additional actions might be required on your part. The following sections detail the additional actions depending on the nature of the change, please copy the relevant section in the description of your PR, and make sure to perform the required actions.

Thank you for contributing to Meilisearch ❤️

This PR makes forward-compatible changes

Forward-compatible changes are changes to the database such that databases created in an older version of Meilisearch are still valid in the new version of Meilisearch. They usually represent additive changes, like adding a new optional attribute or setting.

  • Detail the change to the DB format and why they are forward compatible
  • Forward-compatibility: A database created before this PR and using the features touched by this PR was able to be opened by a Meilisearch produced by the code of this PR.

This PR makes breaking changes

Breaking changes are changes to the database such that databases created in an older version of Meilisearch need changes to remain valid in the new version of Meilisearch. This typically happens when the way to store the data changed (change of database, new required key, etc). This can also happen due to breaking changes in the API of an experimental feature. ⚠️ This kind of changes are more difficult to achieve safely, so proceed with caution and test dumpless upgrade right before merging the PR.

  • Detail the changes to the DB format,
    • which are compatible, and why
    • which are not compatible, why, and how they will be fixed up in the upgrade
  • /!\ Ensure all the read operations still work!
    • If the change happened in milli, you may need to check the version of the database before doing any read operation
    • If the change happened in the index-scheduler, make sure the new code can immediately read the old database
    • If the change happened in the meilisearch-auth database, reach out to the team; we don't know yet how to handle these changes
  • Write the code to go from the old database to the new one
    • If the change happened in milli, the upgrade function should be written and called here
    • If the change happened in the index-scheduler, we've never done it yet, but the right place to do it should be here
  • Write an integration test here ensuring you can read the old database, upgrade to the new database, and read the new database as expected

@dureuill dureuill added this to the v1.15.0 milestone Apr 14, 2025
Copy link
Member

@irevoire irevoire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you 👓

@irevoire irevoire added this pull request to the merge queue Apr 17, 2025
Merged via the queue into main with commit e9b4794 Apr 17, 2025
12 checks passed
@irevoire irevoire deleted the try-batch-end-reason branch April 17, 2025 09:12
@meili-bot meili-bot added the v1.15.0 PRs/issues solved in v1.15.0 released on 2025-06-09 label Jun 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
db change A database was modified v1.15.0 PRs/issues solved in v1.15.0 released on 2025-06-09
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Display the reason why the batcher stopped batching tasks
4 participants