sqlite-utils insert --flatten option to flatten nested JSON #310
Description
I had to do this with a jq recipe today: https://til.simonwillison.net/cloudrun/tailing-cloud-run-request-logs
cat log.json | jq -c '[leaf_paths as $path | {
"key": $path | join("_"), "value": getpath($path)
}] | from_entries' \
| sqlite-utils insert /tmp/logs.db logs - --nl --alter --batch-size 1
That was to turn something like this:
{
"httpRequest": {
"latency": "0.112114537s",
"requestMethod": "GET",
"requestSize": "534",
"status": 200,
},
"insertId": "6111722f000b5b4c4d4071e2",
"labels": {
"service": "datasette-io"
}
}Into this instead:
{
"httpRequest_latency": "0.112114537s",
"httpRequest_requestMethod": "GET",
"httpRequest_requestSize": "534",
"httpRequest_status": 200,
"insertId": "6111722f000b5b4c4d4071e2",
"labels_service": "datasette-io"
}I have to do this often enough that I think it should be an option, --flatten - so I can do this instead:
cat log.json | sqlite-utils insert /tmp/logs.db logs - --flatten