Skip to content

Reader oriented encoder #210

@oakad

Description

@oakad

Right now, Writer is used for compression and Reader is used for decompression. However, some storage APIs, namely AWS S3 and kind, insist on having a Reader as a body of file to upload.

S3 style APIs are incredibly popular and will stay with us for the indefinite future.

We, of course, can work around this limitation by means of a helper goroutine and an io.Pipe, but this adds quite a bit of overhead when many objects are to be uploaded in compressed form.

From the look of it, it should not be too difficult to make an EncodingReader object, which takes a plain data Reader as a parameter and exposes a Reader for compressed data (that is, performs a logical opposite of the presently available lz4.Reader object).

I think, such a feature will be most useful for people using cloud storage.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions