Skip to content

Conversation

timvisee
Copy link
Member

@timvisee timvisee commented Jul 16, 2025

Fixes #6880

When upserting new points, the is_empty filter did not work properly. It did not always include the newly upserted points during filtering.

More specifically, this problem appears when upserting points in which the payload field (of the index) is absent. The null index relies on knowing the exact (segment) point count to reliably filter points. When upserting a point in which the payload field is absent, we don't add the point to the index and therefore don't bump the point count.

When the payload field is absent, instead of adding (set, overwrite) the point to the index, we actually remove (set, overwrite) it from the index. To fix the filtering problem, I now also bump the point count when removing a point. This will ensure that new upsertions without the field still set the correct point count.

This PR fixes two problems:

  1. not setting the total point count (described above)
  2. loading the null index from the wrong path (in some scenarios)

I also add a test to assert correct behavior.

Existing deployments do not have permanent damage because of this bug. All (existing) null indices are fixed on restart.

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  1. Does your submission pass tests?
  2. Have you formatted your code locally using cargo +nightly fmt --all command prior to submission?
  3. Have you checked your code using cargo clippy --all --all-features command?

Copilot

This comment was marked as resolved.

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@timvisee timvisee merged commit cd1a314 into dev Jul 16, 2025
16 checks passed
@timvisee timvisee deleted the fix-is-empty-new-points branch July 16, 2025 12:25
generall pushed a commit that referenced this pull request Jul 17, 2025
* Add test to catch bug

Bug: <#6880>

* Fix loading null index from wrong path

* Simplify test

* Also bump total point count when removing a point

* Add debug assertion, ensure we don't create null index in selector

* Update bug link

* Remove unused import

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants