Skip to content

Bulk deletion of expired snapshots / deleting many snapshots serially is impractically slow #2036

@chrisjsimpson

Description

@chrisjsimpson

Is there an existing issue for this?

  • There is no existing issue for this feature

What are you currently unable to do

Related to #1837

Problem statement: When instance(s) have many thounsands of snapshots, it's infeasibly slow for them to be deleted since they are removed one-by one (a full round trip) for each deletion.

Proposed solution: Refactor the deletion of snapshots logic so that multiple expired snaoshots may be deleted in bulk, rather than serially. Perhaps Lean on the Automatic snapshot names

When a snapshot policy generates many (thousands) of snapshots, for example with a regular cadence schedule , it can take many

There are a few options to delete snaphots within Incus:

  • Change the attached snapshot expiry to something less than before- incus will then gradually remove snapshots
  • Remove (delete) the instance, after which incus will start deleting associated snapshots one-by-one

What the user must not do, if my understanding is correct, is manually zfs delete any Incus deleted snapshots- whilst this would be very fast, Inus would loose state in it's sqlite style database, causing a confused state.

What do you think would need to be added

Proposed solution: Refactor the deletion of snapshots logic so that multiple expired snaoshots may be deleted in bulk, rather than serially. Perhaps Lean on the Automatic snapshot names

There's two scenarios I can think of for this:

  • Primary use case: When Incus has generated a large volume of snapshots (> 10,000) (e.g.) and wants to keep the instance in situ, but delete all the snapshots.
  • Secondary use case: When an incus instance has been deleted, to expediate the deletion of the snapshots

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions