Skip to content

Standardising matrix arrays (adopt Array API spec) #941

@KennethEnevoldsen

Description

@KennethEnevoldsen

Currently MTEB does not consistently use a ground truth for vector manipulation. A solution would be to adopt the Array API spec. A potentially simpler solution is to use either torch or numpy.

  • The benefit of using numpy is that it is very stable and rarely leads to OOM and related GPU issues.
  • The benefit of using PyTorch is that it potentially allows evaluation using GPUs (currently, sklearn does not support this, but might in the future, when they adopt they API spec)

partly related to the discussion in: #883

This is not a pressing concern atm.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestwontfixThis will not be worked on

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions