The Rerun Viewer is currently limited to only being able to display ~two million points at a time, and will have pretty terrible frame rate at 1 million points already.

The reason for this is that we upload the point cloud every frame. This is very simple, but obviously very wasteful.

A solution is to detect if the same points are being rendered this frame as the previous, and if so not re-upload them. This is similar to how we already handle tensors.