Skip to content

Pinning is slow when there are many pins #5221

@Stebalien

Description

@Stebalien

We store all pins in a single massive object so adding and removing pins is really slow when we have many pins.

This affects:

  • ipfs dag add --pin=true
  • ipfs add
  • ipfs pin add
  • ipfs pin rm

Listing pins also appears to be slow but for a different reason:

  1. ipfs pin ls buffers pins in memory before sending them back to the user (see pin ls should stream the result #6304).
  2. ipfs pin ls lists all pinned blocks, directly or indirectly, by default. Calling ipfs pin ls --type=recursive is much faster.

Proposed solutions:


I'm filing this issue so we can have a single issue that succinctly describes the entire issue and all variants.

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugA bug in existing code (including security flaws)status/deferredConscious decision to pause or backlogtopic/perfPerformance

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions