Skip to content

Suggestion: Implement a way to clear Sep's internal buffer #170

@Camderman106

Description

@Camderman106

Hi @nietras

This is a really cool library. Full disclosure, I only have a few years of programming experience, so there might be a way to do this I just haven't identified.
As far as I can tell, Sep's internal buffer can't be cleared. This makes it impossible to seek through the file stream to a specific point in the file (whilst maintaining the header information etc).

I wish to be able to create maps of large files. E.g. a Dictionary<TKey, long> where the long is the raw byte offset into the file for the start of a row/record. I'm able to do this with csv helper as it uses a plain streamreader which we can seek through, and discard the buffered values. (this is helpful for indexing large files and doing random/out of order reads without having it all in memory). However the same trick doesn't work on Sep.

As far as I can tell this is due to the array pool mechanism or some other internal buffer but I dont fully understand it. All i can see is that manually making the stream reader and then seeking on it and clearing the buffer data doesn't seem to work.

Would it be difficult to implement this feature? Just the raw ability to seek the underlying stream. I can manage the byte offsets externally.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions