@suyashkumar could you please take a look on this fork? We're seeing 5-10x speed improvements due to reduced hot path allocations.

hi @kaxap, thanks for reaching out, looks like we are both pulling at the same thread here!

Note that there will be a larger apparent percent latency reduction for files with more frames or larger frames (don't have such examples in testdata/ yet, so it's not as apparent in the benchmarks above).

Similar to your implementation, this change removes an unnecessary slice allocation in the loop. In this implementation, two slices are allocated per frame in advance of parsing the frame. In your implementation, I see you read the data into a single dimension slice until the user wants to unpack it (where the rest of the work is done). This is something I considered myself, however I was able to achieve similar results without changing the API surface (in the struct) in this change so I prefer that until the next minor release with api breaking changes. The reason for this is because slicing a golang slice doesn't lead to copying (though there is still some minor overhead).

One of the things I see in your implementation is replacing the calls to binary.Read, which seems very interesting and useful--I didn't know it allocated a slice under the hood!

Would you be able to run the benchmarks on your fork so we can see how they look? And/or, would you be willing to open a PR here with the uint read changes to move away from binary.Read (but we can keep the slice structure the same for now)? If you open a PR, it will automatically run the benchmarks as a GitHub action and compare to baseline. Alternatively, I can make a change to that effect and add you as a co-author.

I will probably go ahead and merge this change to separate it from the others (there are some others I'd like to make too).

@suyashkumar Thanks for the feedback. I was thinking about creating PR but I was not sure about the struct change I made, glad we cleared that up.
I'll create PR with binary<order>.Uintxx asap.