Don't enforce DBRecoveryMode::AbsoluteConsistency for the metadata server #3161
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AhmedSoliman
approved these changes
Apr 15, 2025
1bb8feb to
f332939
Compare
With this commit we are now properly setting the default values when using a deprecated option. This is particularly relevant if the deprecated option is a struct that contains mutliple values and the user only specifies a few of them. In this case, we want to a) not fail because not all fields have been specified and b) fill in the default values that are specified in the configuration. Before, the problem was that default values weren't respected for flattened structs (see serde-rs/serde#1879). Moreover, if a field of a sub- struct was specified, then the default value of the sub-struct would be used instead of the default value specified in the parent. This was problematic with RocksDbOptions, for example. Since all fields are optional, it would deserialize as all not specified fields being None. If the parent enabled the WAL, then it would not have been respected. This fixes restatedev#3139.
Disabling WAL by default has the risk that the WAL gets disabled if we are using RocksDbOptions::default(). This has happened when using the deprecated metadata-store option because of a serde bug where defaults are not respected for flattened structs.
…rver Since we cannot prevent tail writes from being incomplete in the presence of hard crashes, using the DBRecoveryMode::AbsoluteConsistency might be too restrictive because it prevents the node from restarting. If we haven't completed the tail write, then we can be sure that we haven't acted on it yet. Therefore, it is ok to resume and ignore it. The only problem is that the tail entry might also have been corrupted after having been acted upon. This commit weakens the recovery mode in favor of DBRecoveryMode::TolerateCorruptedTailRecords. This fixes restatedev#3154.
f332939 to
85cbaf0
Compare
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since we cannot prevent tail writes from being incomplete in the presence of hard crashes, using the DBRecoveryMode::AbsoluteConsistency might be too restrictive because it prevents the node from restarting. If we haven't completed the tail write, then we can be sure that we haven't acted on it yet. Therefore, it is ok to resume and ignore it. The only problem is that the tail entry might also have been corrupted after having been acted upon. This commit weakens the recovery mode in favor of DBRecoveryMode::TolerateCorruptedTailRecords.
This fixes #3154.