Skip to content

Conversation

pdet
Copy link
Contributor

@pdet pdet commented Sep 9, 2024

This PR checks for potential unquoted errors at the end of buffer scans.

Fix: #13787

If detected, it will throw an error with the following error message:

Invalid Input Error: CSV Error on Line: 1
Value with unterminated quote found.

Possible fixes:
* Enable ignore errors (ignore_errors=true) to skip this row
* Set quote do empty or to a different value (e.g., quote='')

  file=data/csv/click_mini.tsv.gz
  delimiter = 	 (Auto-Detected)
  quote = " (Auto-Detected)
  escape = " (Auto-Detected)
  new_line = \n (Auto-Detected)
  header = false (Auto-Detected)
  skip_rows = 0 (Auto-Detected)
  comment = \0 (Auto-Detected)
  date_format =  (Auto-Detected)
  timestamp_format =  (Auto-Detected)
  null_padding=0
  sample_size=20480
  ignore_errors=false
  all_varchar=0

@duckdb-draftbot duckdb-draftbot marked this pull request as draft September 9, 2024 18:51
@pdet pdet marked this pull request as ready for review September 9, 2024 18:53
@Mytherin Mytherin merged commit 329bb53 into duckdb:main Sep 10, 2024
41 checks passed
@Mytherin
Copy link
Collaborator

Thanks!

github-actions bot pushed a commit to duckdb/duckdb-r that referenced this pull request Sep 25, 2024
github-actions bot added a commit to duckdb/duckdb-r that referenced this pull request Sep 25, 2024
Merge pull request duckdb/duckdb#13826 from pdet/unquoted_error

Co-authored-by: krlmlr <krlmlr@users.noreply.github.com>
@pdet pdet deleted the unquoted_error branch November 27, 2024 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CSV import swallows some lines for large CSVs.
2 participants