Skip to content

WARNING: Error: JSON data must be UTF-8 encoded #464

@He1my

Description

@He1my

rmlint is unable to read a 50M json file it has produced after scanning a large number of files
$file large.json
large.json: UTF-8 Unicode text

see attached file,
large.tar.gz

Edit:
doing other tests with a 10M json file found that the json file was ISO-8859-1 encoded
converted it using
iconv -f ISO-8859-1 -t UTF-8//TRANSLIT rmlint.json -o rmlintutf8.json
and it worked

it seems that for some reason the resulting json file from rmlint is ISO-8859-1

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions