-
Notifications
You must be signed in to change notification settings - Fork 389
box: disable DDL with old schema #8705
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
locker
merged 1 commit into
tarantool:master
from
locker:gh-7149-forbid-ddl-until-box-schema-upgrade
May 26, 2023
Merged
box: disable DDL with old schema #8705
locker
merged 1 commit into
tarantool:master
from
locker:gh-7149-forbid-ddl-until-box-schema-upgrade
May 26, 2023
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andreyaksenov
approved these changes
May 25, 2023
alyapunov
approved these changes
May 26, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM,
Nit: upgrde
-> upgrade
in the commit message.
** Implementation details ** We disable DDL by patching the existing on_replace_dd_system_space trigger callback installed for each system space so that now it raises an error in case the current schema version is less than the most recent one known to this build. Since to perform a schema upgrade we need to execute DDL, we suppress the error for the fiber that is currently running a schema upgrade. To achieve that, the upgrade script calls box_schema_upgrade_begin and box_schema_upgrade_end before starting and after completing a schema upgrade. The functions keep track of the fiber that is currently running a schema upgrade so that we can allow all DDL operations for it. We also allow DDL during recovery so that we can replay DDL statements written to the WAL. Since there may be a bug in the `box.schema.upgrade` implementation, we export `box.internal.run_schema_upgrade`, which runs the given function as a schema upgrade script (allowing DDL). The user may use this function to recover after a schema upgrade failure. ** Note about the tests ** A test server instance started by luatest grants permissions to the guest user so that luatest can execute commands on it. It means that if a test uses a generated snap file committed to the repository for a test server instance, it will fail because granting permissions is a DDL operation. To prevent this, we have to regenerate snap files so that they contain all required permissions. This works because a test server instance grants permissions with the `if_not_exists` flag. The problem is that it isn't easy to regenerate the snap files for the following tests because there's no generator script: - `test/box-luatest/gh_6794_recover_nonmatching_xlogs_test.lua` - `test/box-luatest/gh_7974_force_recovery_bugs_test.lua` So we temporarily disable these tests and file tickets to fix them. Other notes: - We drop `test/box-luatest/upgrade/2.9.1` and make the test using it use `test/box-luatest/upgrade/2.10.0` instead. We do this because 2.9.1 was never released and the earliest Tarantool version using the 2.9.1 schema version is 2.10.0. This shouldn't affect the test anyhow. - We drop the part of the `user_auth_history_last_modified_upgrade` test that checks that creating users/roles with an old schema works fine because this is forbidden now. - We wrap the code that creates a space with an old schema in the downgrade test in `box.internal.run_schema_upgrade`. Even though it's unsupported now, we still need to check that space creation works after a downgrade. Closes tarantool#7149 @TarantoolBot document Title: Document that DDL is disabled with an old system schema Executing DDL operations with an old (not upgraded) system schema is dangerous and might result in unexpected breakages. So we decided to explicitly forbid all DDL operations with an old system schema until `box.schema.upgrade()` is called. Note, one can still call `box.schema` functions with an old schema provided they do nothing, for example, if an object is created with the `if_not_exists` flag and the object with same id already exists: ```lua box.schema.create_space('test', {if_not_exists = true}) ``` Otherwise an attempt to create a space with an old schema will raise an error like shown below: ```yaml tarantool> box.schema.space.create('test') --- - error: Your schema version is 1.6.8 while Tarantool 3.0.0-entrypoint-262-g3eaba1cef686 requires a more recent schema version. Please, consider using box.schema.upgrade(). ... ```
5f7fa26
to
8163305
Compare
Totktonada
reviewed
May 26, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Executing DDL operations with an old (not upgraded) system schema is dangerous and might result in unexpected breakages. So we decided to explicitly forbid all DDL operations with an old system schema until
box.schema.upgrade()
is called. Note, one can still callbox.schema
functions with an old schema provided they do nothing, for example, if an object is created with theif_not_exists
flag and the object with same id already exists:Otherwise an attempt to create a space with an old schema will raise an error like shown below:
Implementation details
We disable DDL by patching the existing on_replace_dd_system_space trigger callback installed for each system space so that now it raises an error in case the current schema version is less than the most recent one known to this build. Since to perform a schema upgrade we need to execute DDL, we suppress the error for the fiber that is currently running a schema upgrade. To achieve that, the upgrade script calls box_schema_upgrade_begin and box_schema_upgrade_end before starting and after completing a schema upgrade. The functions keep track of the fiber that is currently running a schema upgrade so that we can allow all DDL operations for it. We also allow DDL during recovery so that we can replay DDL statements written to the WAL.
Since there may be a bug in the
box.schema.upgrade
implementation, we exportbox.internal.run_schema_upgrade
, which runs the given function as a schema upgrade script (allowing DDL). The user may use this function to recover after a schema upgrade failure.Note about the tests
A test server instance started by luatest grants permissions to the guest user so that luatest can execute commands on it. It means that if a test uses a generated snap file committed to the repository for a test server instance, it will fail because granting permissions is a DDL operation. To prevent this, we have to regenerate snap files so that they contain all required permissions. This works because a test server instance grants permissions with the
if_not_exists
flag.The problem is that it isn't easy to regenerate the snap files for the following tests because there's no generator script:
test/box-luatest/gh_6794_recover_nonmatching_xlogs_test.lua
test/box-luatest/gh_7974_force_recovery_bugs_test.lua
So we temporarily disable these tests and file tickets to fix them, see #8701 #8702.
Other notes:
test/box-luatest/upgrade/2.9.1
and make the test using it usetest/box-luatest/upgrade/2.10.0
instead. We do this because 2.9.1 was never released and the earliest Tarantool version using the 2.9.1 schema version is 2.10.0. This shouldn't affect the test anyhow.user_auth_history_last_modified_upgrade
test that checks that creating users/roles with an old schema works fine because this is forbidden now.box.internal.run_schema_upgrade
. Even though it's unsupported now, we still need to check that space creation works after a downgrade.Closes #7149