fix: panic in arrow table buffer #5549
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
An empty arrow.TableBuffer can panic when calling Len() if it is in the process of being appended to by a table.BufferBuilder. Normally the TableBuffer maintains the invariant that the size of Columns is the same as the size if Values so to check one is equivalent to checkint the other. However when a BufferBuilder is merging buffers with different schema it will add new null-valued columns to the TableBuffer. This checks the TableBuffer.Len() between adding the column and adding the null-valued column.
In the case that the TableBuffer is empty, but the Columns array is a zero-length slice, rather than nil, a panic occurs.
There are two fixes here. Either of which fix the symptom, but both are worthwhile changes. TableBuffer.Len() now checks the length of the slice it is attempting to access before reading it.
BufferBuilder.normalizeSchema() now checks that the Columns slice length is 0 (rather than the value being nil) in order to use the shortcut normalize code path.
Checklist
Dear Author 👋, the following checks should be completed (or explicitly dismissed) before merging.
experimental/
docs/Spec.md
has been updatedDear Reviewer(s) 👋, you are responsible (among others) for ensuring the completeness and quality of the above before approval.