Skip to content

Use jsonb_strip_nulls in read_schema function #925

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Jun 20, 2025
Merged

Use jsonb_strip_nulls in read_schema function #925

merged 5 commits into from
Jun 20, 2025

Conversation

andrew-farries
Copy link
Collaborator

@andrew-farries andrew-farries commented Jun 20, 2025 •
edited
Loading

Stop creating empty objects and lists in the JSON returned by the read_schema function for each table.

Empty fields are now now omitted from the returned JSON object, rather than being present with empty/lists maps.

Example:

  • Create a simple table:
CREATE TABLE items(id int)
  • Run:
SELECT jsonb_pretty(pgroll.read_schema('public'))

Before this change:

+--------------------------------------------+
| jsonb_pretty                               |
|--------------------------------------------|
| {                                          |
|     "name": "public",                      |
|     "tables": {                            |
|         "items": {                         |
|             "oid": "16437",                |
|             "name": "items",               |
|             "columns": {                   |
|                 "id": {                    |
|                     "name": "id",          |
|                     "type": "integer",     |
|                     "unique": false,       |
|                     "comment": null,       |
|                     "default": null,       |
|                     "nullable": true,      |
|                     "enumvalues": null,    |
|                     "postgrestype": "base" |
|                 }                          |
|             },                             |
|             "comment": null,               |
|             "indexes": {                   |
|             },                             |
|             "primaryKey": [                |
|             ],                             |
|             "foreignKeys": {               |
|             },                             |
|             "checkConstraints": {          |
|             },                             |
|             "uniqueConstraints": {         |
|             },                             |
|             "excludeConstraints": {        |
|             }                              |
|         }                                  |
|     }                                      |
| }                                          |
+--------------------------------------------+

After this change

+--------------------------------------------+
| jsonb_pretty                               |
|--------------------------------------------|
| {                                          |
|     "name": "public",                      |
|     "tables": {                            |
|         "items": {                         |
|             "oid": "16437",                |
|             "name": "items",               |
|             "columns": {                   |
|                 "id": {                    |
|                     "name": "id",          |
|                     "type": "integer",     |
|                     "unique": false,       |
|                     "nullable": true,      |
|                     "postgrestype": "base" |
|                 }                          |
|             }                              |
|         }                                  |
|     }                                      |
| }                                          |
+--------------------------------------------+

null and empty fields are removed from the resulting JSON, reducing bloat in the pgroll.migrations table for large schema.

@andrew-farries
Stop using COALESCE in read_schema
099f7c2
@andrew-farries
jsonb_strip_nulls to remove null table fields
5673667
@andrew-farries
Run make format
5cd0f3f
@andrew-farries
Update TestReadSchema
7f4b276 
Remove all the empty fields from the test expectations, as these fields
will be `nil` in the JSON output from the `read_schem` function.
@andrew-farries
Update TestBaseline
20b22d3 
Remove all the empty fields from the test expectations, as these fields
will be `nil` in the JSON output from the `read_schem` function.
@github-actions github-actions bot temporarily deployed to Docs Preview June 20, 2025 08:50 Inactive
@github-actions GitHub Actions
Copy link

Merging this branch will not change overall coverage

Impacted Packages Coverage Δ 🤖
github.com/xataio/pgroll/pkg/roll 80.00% (ø)
github.com/xataio/pgroll/pkg/state 70.78% (ø)
Coverage by file

Changed unit test files

  • github.com/xataio/pgroll/pkg/roll/baseline_test.go
  • github.com/xataio/pgroll/pkg/state/state_test.go

@andrew-farries andrew-farries marked this pull request as ready for review June 20, 2025 08:54
@andrew-farries andrew-farries requested a review from kvch June 20, 2025 08:54
kvch
kvch approved these changes Jun 20, 2025
View reviewed changes
Copy link
Contributor

@kvch kvch left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay, I've always found the empty objects annoying. Thank you for this cleanup.

@andrew-farries andrew-farries merged commit 803868a into main Jun 20, 2025
30 checks passed
@andrew-farries andrew-farries deleted the stop-using-coalesce-in-read-schema branch June 20, 2025 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kvch kvch kvch approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@andrew-farries @kvch