Skip to content

Allow migration files to specify a versionSchema field #884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jun 13, 2025

Conversation

andrew-farries
Copy link
Collaborator

@andrew-farries andrew-farries commented Jun 12, 2025

#852 removed support for specifying a migration name via the name field in a migration file. While having an unambiguous source of truth for the migration name (the filename) is good, it is still often desirable to decouple the filename from the name of the version schema that the migration will create when applied because filenames:

  • can be longer than the limit Postgres imposes on schema names
  • can contain characters that are not legal in Postgres schema names
  • must be named so that migration files are ordered lexicographically on disk

This PR adds support for a new version_schema field in migration files that allows the migration author to specify the name of the version schema that the migration will create.

Example

# migration files can now specify the version schema 
# name to be created by the migration
version_schema: my_version_schema
operations:
  - create_table:
      name: items
      columns:
        - name: id
          type: serial
          pk: true
        - name: name
          type: varchar(255)

Running this migration:

$ pgroll start migrations/01_create_table.yaml --complete

Creates this version schema:

+-----------------------+-------------------+
| Name                  | Owner             |
|-----------------------+-------------------|
...
| public_my_version_schema | postgres       |
+-----------------------+-------------------+

Had the version_schema field not been specified in the migration file the version schema would have been:

+-----------------------+-------------------+
| Name                  | Owner             |
|-----------------------+-------------------|
...
| public_01_create_table | postgres         |
+-----------------------+-------------------+

This PR is part of a stack:

Part of #882

@github-actions github-actions bot temporarily deployed to Docs Preview June 12, 2025 06:38 Inactive
@github-actions github-actions bot temporarily deployed to Docs Preview June 12, 2025 06:55 Inactive
Use the migration's `VersionSchema` if it is set, otherwise use the
migration's `Name`.
Use the version schema name instead of the migration name for the view
name in the ouput message.
If the migration has a `versionSchema` field, use that as the version
returned by the `latest_version` function. Otherwise, use the name of
the migration.
Ensure that the previous version schema is dropped after the next
migration completes, even when the previous version schema sets a
non-default version schema name.
This function behaves like `latest_version` but returns the *name* of
the latest migration, rather than the name of its version schema.

Use the new function:
* in `previous_version` because migration parent-child relationships are
  based on the migration name, not the version schema name.
* in `raw_migration` when populating the parent migration for the
  captured inferred migration, for the same reason as above.
Now that a migration name is not necessarily the same as the version
schema name, ensure that this function returns the version schema name
of the previous migration, rather than the migration name.
Use `latest_migration` instead of `latest_version` in the SQL statements
to ensure that the correct migration name is used.

Parent - child relationships use the migration name, not the version
schema name.
Ensure that the version schema is dropped after a migration rollback,
in both cases where the migration does not set an explicit version
schema name and where it does.
Return the name of the latest migration, as oppoosed to the name of the
version schema of the latest migration.
Display the latest migration name if there is an active migration rather
than the latest version schema name.
This is the partner to the `previous_version` function; this one returns
tha **name** of the previous migration, while the other returns the
version-schema name of the previous migration.
Return the name of the previous migration for a given schema.
Ensure that we get the name of the previous migration rather than the
previous version schema name during the rollback process.
@andrew-farries andrew-farries force-pushed the add-version-schema-field branch from fd35f55 to 3dddf26 Compare June 12, 2025 07:01
@github-actions github-actions bot temporarily deployed to Docs Preview June 12, 2025 07:01 Inactive
@andrew-farries andrew-farries marked this pull request as ready for review June 12, 2025 07:23
@andrew-farries andrew-farries requested a review from kvch June 12, 2025 07:23
Copy link
Contributor

@kvch kvch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change the attribute name.

For consistency with other fields in migration files.
@github-actions github-actions bot temporarily deployed to Docs Preview June 13, 2025 07:27 Inactive
Copy link

Merging this branch will decrease overall coverage

Impacted Packages Coverage Δ 🤖
github.com/xataio/pgroll/cmd 0.00% (ø)
github.com/xataio/pgroll/pkg/migrations 74.03% (-0.01%) 👎
github.com/xataio/pgroll/pkg/roll 79.07% (ø)
github.com/xataio/pgroll/pkg/state 53.58% (-2.10%) 👎

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/xataio/pgroll/cmd/migrate.go 0.00% (ø) 58 0 58
github.com/xataio/pgroll/cmd/start.go 0.00% (ø) 47 (-1) 0 47 (-1)
github.com/xataio/pgroll/pkg/migrations/migrations.go 90.48% (-3.97%) 21 (+3) 19 (+2) 2 (+1) 👎
github.com/xataio/pgroll/pkg/migrations/op_common.go 85.71% (ø) 112 96 16
github.com/xataio/pgroll/pkg/roll/execute.go 78.79% (ø) 165 130 35
github.com/xataio/pgroll/pkg/state/state.go 42.02% (-2.36%) 188 (+10) 79 109 (+10) 👎

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/xataio/pgroll/pkg/roll/execute_test.go

@andrew-farries andrew-farries requested a review from kvch June 13, 2025 07:42
@andrew-farries andrew-farries merged commit 1dd00ff into main Jun 13, 2025
30 checks passed
@andrew-farries andrew-farries deleted the add-version-schema-field branch June 13, 2025 08:16
andrew-farries added a commit that referenced this pull request Jun 13, 2025
Ensure that at least one test for every operation type sets a
`VersionSchema` for the the migration.

This PR adds a `VersionSchema` field to one test for every operation
type.

--- 

This PR is part of a stack:
* #884
* #886 (this PR)
* #887
andrew-farries added a commit that referenced this pull request Jun 13, 2025
Update documentation for the new `versionSchema` field and add an
example migration.

--- 

This PR is part of a stack:
* #884
* #886 
* #887 (this PR)
andrew-farries added a commit that referenced this pull request Jun 13, 2025
…896)

#884 added support for the `version_schema` field in migration files,
allowing a migration to specify the name of the version schema that will
be created when the migration is applied:

```yaml
# migration files can now specify the version schema 
# name to be created by the migration
version_schema: my_version_schema
operations:
  - create_table:
      name: items
      columns:
        - name: id
          type: serial
          pk: true
        - name: name
          type: varchar(255)
```

When applied as `pgroll start 01_create_table.yaml`, this migration will
take the name of its version schema from the migration file:

```
+-----------------------+-------------------+
| Name                  | Owner             |
|-----------------------+-------------------|
...
| public_my_version_schema | postgres       |
+-----------------------+-------------------+
```

If `version_schema` is not specified, the name of the version schema
defaults to the filename, as before.

---

This PR updates the `pgroll latest` command to take account of this new
behaviour, where the version schema name can be decoupled from its
filename.

Previous behaviour:

* `pgroll latest schema`: would return the most recent migration name
prefixed with the schema name, eg `public_01_create_table`.
* `pgroll latest migration`: would return the most receent migration
name without the schema prefix, eg `01_create_table`.

New behaviour:

* `pgroll latest schema`: returns the version schema name of the most
recent migration prefixed with the schema name, eg
`public_01_set_by_version_schema_field`
* `pgroll latest migration`: returns the name of the most recent
migration, without the schema prefix, eg, `01_create_table`.

## Examples

Run:

```
pgroll migrate examples/ --complete
```

to apply all example migrations. Now:

```bash
# get the **name** of the latest migration applied to the target database
$ pgroll latest migration
56_with_version_schema
```

```bash
# get the **version schema name** of the latest migration applied to the target database
$ pgroll latest schema
public_with_version_schema
```

```bash
# get the **name** of the latest migration in the migrations/ folder
$ pgroll latest migration --local examples/
56_with_version_schema
```


```bash
# get the **version schema name** of the latest migration in the migrations/ folder
$ pgroll latest schema --local examples/
public_with_version_schema
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants