Skip to content

CSV import: IntParser removes decimal when value ends with ".0" during parsing #732

@szabolcskarolyi

Description

@szabolcskarolyi

We have a case when our input csv has a column that has many values of "0.0" and some real double values.
Problem is that during the sample phase the columnType is set to Integer because the sample values pass in the parse test due to the check at:
https://github.com/jtablesaw/tablesaw/blob/master/core/src/main/java/tech/tablesaw/columns/numbers/IntParser.java#L28

Wondering if that check could be removed or be optional.

We came up with a solution, where we created a custom IntColumnType and IntParser, trying to set columnTypesToDetect in ReadOptions, however it looks like the setter is missing in CsvReadOptions.Builder so the default columnTypesToDetect list cannot be overriden.
We used reflection, but would like to avoid this solution.
Ideal would be to handle "x.0" values as double instead.

We would appreciate your help :)
Szabolcs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions