Skip to content

Regenerate all fragments when coming from a user provided vector #5763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 22, 2025

Conversation

dureuill
Copy link
Contributor

@dureuill dureuill commented Jul 17, 2025

This PR makes the following changes:

  1. This changes the indexers in order to regenerate all fragments if the embeddings for that document were previously marked as user-provided.
  2. Modifies the dump creation (+ meilitool) to inject vectors with regenerate set to false when fragments are involved. This is to avoid regenerating too many embeddings at once when fragments are involved.

This is important especially in the context of dumps, as otherwise documents would have their embeddings removed unexpectedly without any new embedding being generated.

@dureuill dureuill added this to the v1.16.0 milestone Jul 17, 2025
@dureuill dureuill requested a review from Kerollmops July 17, 2025 15:18
@dureuill dureuill added the no db change The database didn't change label Jul 17, 2025
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 👍

@dureuill dureuill added this pull request to the merge queue Jul 22, 2025
Merged via the queue into release-v1.16.0 with commit 080d5f9 Jul 22, 2025
20 of 21 checks passed
@dureuill dureuill deleted the embedding-fixes branch July 22, 2025 09:37
@meili-bot meili-bot added the v1.16.0 PRs/issues solved in v1.16.0 released on 2025-08-04 label Aug 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no db change The database didn't change v1.16.0 PRs/issues solved in v1.16.0 released on 2025-08-04
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants