Skip to content

Feature to encode MLXArrays to .safetensors binary data in-memory #214

@grananqvist

Description

@grananqvist

for data transfer purposes it would be very useful to be able to encode [String:MLXArray] into .safetensors format in the form of in-memory binary data type Data.

e.g. in addition to a FileWriter in mlx core https://github.com/ml-explore/mlx/blob/fd0d63ba5b83da41d6a0e75a0bcb5d70ec93eb40/mlx/io/load.h#L109, we can have a reader and writer to a binary data buffer, and expose this to swift apis, or something similar.

The purpose look like this:

weights: [String: MLXArray]
let blob: Data = save(weights) 
// ... blob can be transfered over the network
let sameWeights = load(blob)
// weights and sameWeights should be equal

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions