Make CSV Parser strict_mode=True fail on a mix of new line delimiters. #15959
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Mytherin
added a commit
that referenced
this pull request
Feb 11, 2025
At some point one of the CI refactors, disabled the Parallel CSV Tests. This PR makes the CI run it again, all tests are slow, and they take about ~20 seconds, so the extra flag is not really necessary. The file `data/csv/mixed_new_line.csv` is breaking, but should be fixed with: #15959 Hence I'm skipping it for now.
|
This still seems to be bundling too many files |
|
Thanks! Looks good now - can you just resolve the merge conflict? |
Done! :-) |
|
Thanks! |
krlmlr
added a commit
to duckdb/duckdb-r
that referenced
this pull request
May 15, 2025
Make CSV Parser strict_mode=True fail on a mix of new line delimiters. (duckdb/duckdb#15959) [Julia] support binding for vectors (duckdb/duckdb#16701) JSON Reader: make read_position atomic so this can be read by the progress bar while processing the JSON file (duckdb/duckdb#16692)
krlmlr
added a commit
to duckdb/duckdb-r
that referenced
this pull request
May 15, 2025
Make CSV Parser strict_mode=True fail on a mix of new line delimiters. (duckdb/duckdb#15959) [Julia] support binding for vectors (duckdb/duckdb#16701) JSON Reader: make read_position atomic so this can be read by the progress bar while processing the JSON file (duckdb/duckdb#16692)
krlmlr
added a commit
to duckdb/duckdb-r
that referenced
this pull request
May 16, 2025
Make CSV Parser strict_mode=True fail on a mix of new line delimiters. (duckdb/duckdb#15959) [Julia] support binding for vectors (duckdb/duckdb#16701) JSON Reader: make read_position atomic so this can be read by the progress bar while processing the JSON file (duckdb/duckdb#16692)
krlmlr
added a commit
to duckdb/duckdb-r
that referenced
this pull request
May 16, 2025
Make CSV Parser strict_mode=True fail on a mix of new line delimiters. (duckdb/duckdb#15959) [Julia] support binding for vectors (duckdb/duckdb#16701) JSON Reader: make read_position atomic so this can be read by the progress bar while processing the JSON file (duckdb/duckdb#16692)
krlmlr
added a commit
to duckdb/duckdb-r
that referenced
this pull request
May 17, 2025
Make CSV Parser strict_mode=True fail on a mix of new line delimiters. (duckdb/duckdb#15959) [Julia] support binding for vectors (duckdb/duckdb#16701) JSON Reader: make read_position atomic so this can be read by the progress bar while processing the JSON file (duckdb/duckdb#16692)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With
strict_modeset to True, the CSV parser will only accept files that consistently use either\r\n,\r, or\nas line endings. Any mixture of these line endings will result in a failure, accompanied by a suggestion to setstrict_mode=False.Fix: #15783