Skip to content

Conversation

andrew-farries
Copy link
Collaborator

Implement the change column type operation. A change type migration looks like this:

{
  "name": "18_change_column_type",
  "operations": [
    {
      "change_type": {
        "table": "reviews",
        "column": "rating",
        "type": "integer",
        "up": "CAST(rating AS integer)",
        "down": "CAST(rating AS text)"
      }
    }
  ]
}

This migration changes the type of the rating column from TEXT to INTEGER.

The implementation is very similar to the set NOT NULL operation (#63):

  • On Start:
    • Create a new column having the new type
    • Backfill the new column with values from the existing column, converting the types using the up SQL.
    • Create a trigger to populate the new column when values are written to the old column, converting types with up.
    • Create a trigger to populate the old column when values are written to the new column, converting types with down.
  • On Complete
    • Remove triggers
    • Drop the old column
    • Rename the new column to the old column name.
  • On Rollback
    • Remove the new column and both triggers.

The migration can fail in at least 2 ways:

  • The initial backfill of existing rows on Start fails due to the type conversion not being possible on one or more rows. In the above example, any existing rows with rating values not representable as an INTEGER will cause a failure on Start.
  • During the rollout period, unconvertible values are written to the old version schema. The up trigger will fail to convert the values and the INSERT/UPDATE will fail.
    • Some form of data quarantine needs to be implemented here, copying the invalid rows elsewhere and blocking completion of the migration until those rows are handled in some way).

The PR also adds example migrations to the /examples directory.

@andrew-farries andrew-farries requested a review from exekias August 31, 2023 07:43
@andrew-farries andrew-farries mentioned this pull request Sep 1, 2023
43 tasks
Copy link
Member

@exekias exekias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's great to see that we have all the building blocks in place to implement hard operations!

@andrew-farries andrew-farries force-pushed the support-aliased-cols-in-trigger branch from 5705ad9 to d408a38 Compare September 1, 2023 12:14
Base automatically changed from support-aliased-cols-in-trigger to main September 1, 2023 12:16
Add a new column to the `reviews` table with one migration, then change
its type with another migration.
Allow assertions on the type of a column.
@andrew-farries andrew-farries merged commit 6b91f43 into main Sep 1, 2023
@andrew-farries andrew-farries deleted the change-column-type branch September 1, 2023 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants