[RayService] Add a safeguard to prevent overriding the pending cluster during a upgrade #2887
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3fcc4a7 to
9ca4b4a
Compare
kevin85421
reviewed
Feb 4, 2025
| @@ -243,6 +243,7 @@ func (r *RayServiceReconciler) calculateStatus(ctx context.Context, rayServiceIn | |||
| rayServiceInstance.Status.ActiveServiceStatus.Applications = activeClusterServeApplications | |||
| rayServiceInstance.Status.PendingServiceStatus.Applications = pendingClusterServeApplications | |||
|
|
|||
| var servingClusterName string | |||
There was a problem hiding this comment.
Maybe use a boolean to indicate whether the services point to the pending cluster instead?
There was a problem hiding this comment.
Done, but I think using a Boolean feels like leaving the actual check outside of the shouldXXX function.
8e02563 to
2311404
Compare
kevin85421
reviewed
Feb 5, 2025
| assert.True(t, shouldPrepareNewCluster) | ||
| } | ||
|
|
||
| func TestShouldPrepareNewCluster_RecreatePendingCluster(t *testing.T) { |
There was a problem hiding this comment.
These two tests look pretty similar how about merging it into one test with t.Run(...).
| @@ -243,6 +243,7 @@ func (r *RayServiceReconciler) calculateStatus(ctx context.Context, rayServiceIn | |||
| rayServiceInstance.Status.ActiveServiceStatus.Applications = activeClusterServeApplications | |||
| rayServiceInstance.Status.PendingServiceStatus.Applications = pendingClusterServeApplications | |||
|
|
|||
| var isPendingClusterServing bool | |||
There was a problem hiding this comment.
explicitly initialing it to false
5e23364 to
5b226cb
Compare
…r during a upgrade Signed-off-by: Rueian <rueiancsie@gmail.com>
5b226cb to
1981f08
Compare
kevin85421
approved these changes
Feb 5, 2025
4 tasks
andrewsykim
added a commit
that referenced
this pull request
Feb 10, 2025
* [RayService] More envtests that follow the most common scenario in the RayService code path (#2880) Signed-off-by: Rueian <rueiancsie@gmail.com> * [RayService] Remove outdated env tests (#2886) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [RayService] Refactor envtests (#2888) * [docs][ray-operator] fix typo in Golang version (#2893) Project uses Golang 1.22 not 1.20. Signed-off-by: David Xia <david@davidxia.com> * [Fix][kubectl-plugin] Fix no context nil error SIGSEGV in tests (#2892) Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [CI] Enable testifylint rule (#2896) Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [CI] Enable `testifylint` `error-nil` rule (#2907) * [kubectl-plugin][feat] support specifying number of head GPUs (#2895) when creating a RayCluster with `kubectl ray create cluster NAME --head-gpu N`. Similar to the `--worker-gpu` switch. Signed-off-by: David Xia <david@davidxia.com> * Use webhook.CustomValidator instead of deprecated webhook.Validator. (#2803) * Use webhook.CustomValidator instead of deprecated webhook.Validator. * Move RayClusterWebhook to pkg/webhook/v1. * [kubectl-plugin] update context error messages (#2891) to tell user they can use `--context` to set the K8s context. Add tests Signed-off-by: David Xia <david@davidxia.com> Co-authored-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [Fix][kubectl-plugin] make tests use a temporary kube config (#2894) file that's not at the default path. Right now tests fail if there's a K8s current context set with a command like `kubectl config use-context my-context`. This change allows tests to pass regardless of current context. Signed-off-by: David Xia <david@davidxia.com> * [CI] Enable testifylint formatter rule (#2915) Signed-off-by: 400Ping <43886578+400Ping@users.noreply.github.com> * [CI] Enable `testifylint` `empty` rule (#2908) * [CI] Enable testifylint bool-compare rule (#2911) Signed-off-by: 400Ping <43886578+400Ping@users.noreply.github.com> * [CI] Auto download golang tools in pre-commit (#2917) Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [RayService] a safeguard for preventing overriding the pending cluster during a upgrade (#2887) Signed-off-by: Rueian <rueiancsie@gmail.com> * [RayService] Refactor unit tests for ShouldPrepareNewCluster (#2928) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [chore][kubectl-plugin] use consistent capitalization (#2922) in comments and console messages. * capitalize "Ray" when used as a proper noun * capitalize Ray K8s CRDs like "RayJob" and "RayCluster" * capitalize acronyms like "YAML" * fix some minor typos No functional changes. Signed-off-by: David Xia <david@davidxia.com> * [CI] Enable `testifylint` `require-error` rule (#2909) * [kubectl-plugin] support general `kubectl` switches like `--context` (#2883) * [CI] Fix lint error (require-error) (#2931) Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [CI] Enable `testifylint` `float-compare` rule (#2910) * [docs] move pre-commit instructions to main dev docs (#2921) * [CI] Enable `testifylint` `expected-actual` rule (#2914) * [CI] Generate CRD json schema separately in pre-commit (#2930) * [release][1/N] Update YAMLs from Ray 2.9 to Ray 2.41 (#2934) * Delete `[raycluster|rayjob|rayservice]_types_test.go` unnecessary tests (#2935) * [release][2/N] Update RayCluster Helm chart from Ray 2.9 to Ray 2.41 (#2936) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [release][3/N] Update RayService e2e tests YAML files from Ray 2.9 to Ray 2.41 (#2937) * [release][4/N] Update Ray images / versions in kubectl plugin (#2938) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [release][5/N] Update some RayJob YAMLs from Ray 2.9 to Ray 2.41 (#2941) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [release][6/N] Remove unnecessary YAMLs (#2946) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [docs][kubectl-plugin] add dev docs (#2912) Signed-off-by: David Xia <david@davidxia.com> Co-authored-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [CI] Add shellcheck and fix error of it (#2933) * [chore][kubectl-plugin] use better test assertions (#2955) Fix two places. * use `assert.EqualError()` instead of `assert.Error()` to check the error string is what we expect * use testify assertion instead of string checking for better error output Signed-off-by: David Xia <david@davidxia.com> * [chore] add Markdown linting pre-commit hook (#2953) Ignore most rules we violate for now. Fix these two violations. ``` pre-commit run markdownlint --all-files --show-diff-on-failure ... clients/python-client/README.md:106 MD001/heading-increment Heading levels should only increment by one level at a time [Expected: h3; Actual: h4] CHANGELOG.md:420:1 MD030/list-marker-space Spaces after list markers [Expected: 1; Actual: 2] ``` Signed-off-by: David Xia <david@davidxia.com> * [chore][kubectl-plugin] use consistent capitalization (#2950) in comments and console messages. capitalize "Ray" when used as a proper noun. Similar to #2922. Signed-off-by: David Xia <david@davidxia.com> * [Fix][RayJob] Invalid quote for RayJob submitter (#2949) Closes: #2943 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [release][7/N] Update RayService YAMLs (#2956) * [CI] Enable testifylint len rule (#2945) Signed-off-by: LeoLiao123 <leoyeepaa@gmail.com> * [docs][kubectl-plugin] improve help messages (#2952) Signed-off-by: David Xia <david@davidxia.com> * [kubectl-plugin] Fix panic when GPU resource is not set (#2954) Signed-off-by: win5923 <ken89@kimo.com> * [release][8/N] Upgrade Stable Diffusion RayService to Ray 2.41 (#2960) * [docs][kubectl-plugin] fix incorrect example commands (#2951) Signed-off-by: David Xia <david@davidxia.com> Co-authored-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [fix][kubectl-plugin] set worker group CPU limit (#2958) when creating a new worker group with `kubectl ray create workergroup`. Write a unit test. I noticed we are setting resource limits equal to resource requests everywhere else but in this command. I have a K8s [LimitRange] that prevented the creation of these worker Pods that had CPU limit defaulting to less than their CPU requests. Describing the RayCluster showed this warning event. `Failed to create worker Pod hyperkube/, Pod "dxia-test-other-group-worker-pm2sh" is invalid: spec.containers[0].resources.requests: Invalid value: "2": must be less than or equal to cpu limit of 250m` Signed-off-by: David Xia <david@davidxia.com> [LimitRange]: https://kubernetes.io/docs/concepts/policy/limit-range/ * [RayJob] Deflaky RayJob e2e tests (#2963) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [RayService] Deflaky RayService envtest (#2962) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [release][9/N] Update text summarizer RayService to Ray 2.41 (#2961) * [RayService] adapter vllm 0.6.1.post2 (#2823) * adapter vllm 0.6.1.post2 * fix var define * Unify the cpu Settings in serve.py in service.yaml, all set to 1 * Maintain the configuration with vllm0.5x * [Release] Upgrade ray-job.batch-inference.yaml image to 2.41 (#2971) Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [chore][docs] enable Markdownlint rule MD010 (#2975) * [CI] Change Pre-commit-shellcheck-to-shellcheck-py (#2974) Signed-off-by: owenowenisme <mses010108@gmail.com> * [release] Update Yunikorn YAML file to Ray 2.41 (#2969) Signed-off-by: Cheng-Yeh Chung <kenchung285@gmail.com> * [release] Update YuniKorn YAML files to Ray 2.41 (#2976) Signed-off-by: win5923 <ken89@kimo.com> * [chore][docs] enable Markdownlint rule MD004 (#2973) [Unordered list style][1] [1]: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md004---unordered-list-style * [Test] Use GcsFaultToleranceOptions in test and backward compatibility (#2972) * Update samples to use Ray 2.41.0 images (#2964) Signed-off-by: Andrew Sy Kim <andrewsy@google.com> * Update TPU Ray CR manifests to use Ray 2.41.0 (#2965) * [Refactor] Use constants for image tag, image repo, and versions in golang to avoid hard-coded strings (#2978) --------- Signed-off-by: Rueian <rueiancsie@gmail.com> Signed-off-by: kaihsun <kaihsun@anyscale.com> Signed-off-by: David Xia <david@davidxia.com> Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> Signed-off-by: 400Ping <43886578+400Ping@users.noreply.github.com> Signed-off-by: LeoLiao123 <leoyeepaa@gmail.com> Signed-off-by: win5923 <ken89@kimo.com> Signed-off-by: owenowenisme <mses010108@gmail.com> Signed-off-by: Cheng-Yeh Chung <kenchung285@gmail.com> Signed-off-by: Andrew Sy Kim <andrewsy@google.com> Co-authored-by: Rueian <rueiancsie@gmail.com> Co-authored-by: Kai-Hsun Chen <kaihsun@anyscale.com> Co-authored-by: David Xia <david@davidxia.com> Co-authored-by: Chi-Sheng Liu <chishengliu@chishengliu.com> Co-authored-by: David Xia <dxia@spotify.com> Co-authored-by: Mykhailo Bobrovskyi <mikhail.bobrovsky@gmail.com> Co-authored-by: Ping <43886578+400Ping@users.noreply.github.com> Co-authored-by: Owen Lin <106612301+owenowenisme@users.noreply.github.com> Co-authored-by: Leo Liao <93932709+LeoLiao123@users.noreply.github.com> Co-authored-by: Blocka <ken89@kimo.com> Co-authored-by: zrant <37032227+pxp531@users.noreply.github.com> Co-authored-by: kenchung285 <kenchung285@gmail.com> Co-authored-by: fscnick <6858627+fscnick@users.noreply.github.com> Co-authored-by: ryanaoleary <113500783+ryanaoleary@users.noreply.github.com>
Ygnas
pushed a commit
to Ygnas/kuberay
that referenced
this pull request
Mar 20, 2025
* [RayService] More envtests that follow the most common scenario in the RayService code path (ray-project#2880) Signed-off-by: Rueian <rueiancsie@gmail.com> * [RayService] Remove outdated env tests (ray-project#2886) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [RayService] Refactor envtests (ray-project#2888) * [docs][ray-operator] fix typo in Golang version (ray-project#2893) Project uses Golang 1.22 not 1.20. Signed-off-by: David Xia <david@davidxia.com> * [Fix][kubectl-plugin] Fix no context nil error SIGSEGV in tests (ray-project#2892) Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [CI] Enable testifylint rule (ray-project#2896) Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [CI] Enable `testifylint` `error-nil` rule (ray-project#2907) * [kubectl-plugin][feat] support specifying number of head GPUs (ray-project#2895) when creating a RayCluster with `kubectl ray create cluster NAME --head-gpu N`. Similar to the `--worker-gpu` switch. Signed-off-by: David Xia <david@davidxia.com> * Use webhook.CustomValidator instead of deprecated webhook.Validator. (ray-project#2803) * Use webhook.CustomValidator instead of deprecated webhook.Validator. * Move RayClusterWebhook to pkg/webhook/v1. * [kubectl-plugin] update context error messages (ray-project#2891) to tell user they can use `--context` to set the K8s context. Add tests Signed-off-by: David Xia <david@davidxia.com> Co-authored-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [Fix][kubectl-plugin] make tests use a temporary kube config (ray-project#2894) file that's not at the default path. Right now tests fail if there's a K8s current context set with a command like `kubectl config use-context my-context`. This change allows tests to pass regardless of current context. Signed-off-by: David Xia <david@davidxia.com> * [CI] Enable testifylint formatter rule (ray-project#2915) Signed-off-by: 400Ping <43886578+400Ping@users.noreply.github.com> * [CI] Enable `testifylint` `empty` rule (ray-project#2908) * [CI] Enable testifylint bool-compare rule (ray-project#2911) Signed-off-by: 400Ping <43886578+400Ping@users.noreply.github.com> * [CI] Auto download golang tools in pre-commit (ray-project#2917) Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [RayService] a safeguard for preventing overriding the pending cluster during a upgrade (ray-project#2887) Signed-off-by: Rueian <rueiancsie@gmail.com> * [RayService] Refactor unit tests for ShouldPrepareNewCluster (ray-project#2928) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [chore][kubectl-plugin] use consistent capitalization (ray-project#2922) in comments and console messages. * capitalize "Ray" when used as a proper noun * capitalize Ray K8s CRDs like "RayJob" and "RayCluster" * capitalize acronyms like "YAML" * fix some minor typos No functional changes. Signed-off-by: David Xia <david@davidxia.com> * [CI] Enable `testifylint` `require-error` rule (ray-project#2909) * [kubectl-plugin] support general `kubectl` switches like `--context` (ray-project#2883) * [CI] Fix lint error (require-error) (ray-project#2931) Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [CI] Enable `testifylint` `float-compare` rule (ray-project#2910) * [docs] move pre-commit instructions to main dev docs (ray-project#2921) * [CI] Enable `testifylint` `expected-actual` rule (ray-project#2914) * [CI] Generate CRD json schema separately in pre-commit (ray-project#2930) * [release][1/N] Update YAMLs from Ray 2.9 to Ray 2.41 (ray-project#2934) * Delete `[raycluster|rayjob|rayservice]_types_test.go` unnecessary tests (ray-project#2935) * [release][2/N] Update RayCluster Helm chart from Ray 2.9 to Ray 2.41 (ray-project#2936) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [release][3/N] Update RayService e2e tests YAML files from Ray 2.9 to Ray 2.41 (ray-project#2937) * [release][4/N] Update Ray images / versions in kubectl plugin (ray-project#2938) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [release][5/N] Update some RayJob YAMLs from Ray 2.9 to Ray 2.41 (ray-project#2941) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [release][6/N] Remove unnecessary YAMLs (ray-project#2946) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [docs][kubectl-plugin] add dev docs (ray-project#2912) Signed-off-by: David Xia <david@davidxia.com> Co-authored-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [CI] Add shellcheck and fix error of it (ray-project#2933) * [chore][kubectl-plugin] use better test assertions (ray-project#2955) Fix two places. * use `assert.EqualError()` instead of `assert.Error()` to check the error string is what we expect * use testify assertion instead of string checking for better error output Signed-off-by: David Xia <david@davidxia.com> * [chore] add Markdown linting pre-commit hook (ray-project#2953) Ignore most rules we violate for now. Fix these two violations. ``` pre-commit run markdownlint --all-files --show-diff-on-failure ... clients/python-client/README.md:106 MD001/heading-increment Heading levels should only increment by one level at a time [Expected: h3; Actual: h4] CHANGELOG.md:420:1 MD030/list-marker-space Spaces after list markers [Expected: 1; Actual: 2] ``` Signed-off-by: David Xia <david@davidxia.com> * [chore][kubectl-plugin] use consistent capitalization (ray-project#2950) in comments and console messages. capitalize "Ray" when used as a proper noun. Similar to ray-project#2922. Signed-off-by: David Xia <david@davidxia.com> * [Fix][RayJob] Invalid quote for RayJob submitter (ray-project#2949) Closes: ray-project#2943 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [release][7/N] Update RayService YAMLs (ray-project#2956) * [CI] Enable testifylint len rule (ray-project#2945) Signed-off-by: LeoLiao123 <leoyeepaa@gmail.com> * [docs][kubectl-plugin] improve help messages (ray-project#2952) Signed-off-by: David Xia <david@davidxia.com> * [kubectl-plugin] Fix panic when GPU resource is not set (ray-project#2954) Signed-off-by: win5923 <ken89@kimo.com> * [release][8/N] Upgrade Stable Diffusion RayService to Ray 2.41 (ray-project#2960) * [docs][kubectl-plugin] fix incorrect example commands (ray-project#2951) Signed-off-by: David Xia <david@davidxia.com> Co-authored-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [fix][kubectl-plugin] set worker group CPU limit (ray-project#2958) when creating a new worker group with `kubectl ray create workergroup`. Write a unit test. I noticed we are setting resource limits equal to resource requests everywhere else but in this command. I have a K8s [LimitRange] that prevented the creation of these worker Pods that had CPU limit defaulting to less than their CPU requests. Describing the RayCluster showed this warning event. `Failed to create worker Pod hyperkube/, Pod "dxia-test-other-group-worker-pm2sh" is invalid: spec.containers[0].resources.requests: Invalid value: "2": must be less than or equal to cpu limit of 250m` Signed-off-by: David Xia <david@davidxia.com> [LimitRange]: https://kubernetes.io/docs/concepts/policy/limit-range/ * [RayJob] Deflaky RayJob e2e tests (ray-project#2963) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [RayService] Deflaky RayService envtest (ray-project#2962) Signed-off-by: kaihsun <kaihsun@anyscale.com> * [release][9/N] Update text summarizer RayService to Ray 2.41 (ray-project#2961) * [RayService] adapter vllm 0.6.1.post2 (ray-project#2823) * adapter vllm 0.6.1.post2 * fix var define * Unify the cpu Settings in serve.py in service.yaml, all set to 1 * Maintain the configuration with vllm0.5x * [Release] Upgrade ray-job.batch-inference.yaml image to 2.41 (ray-project#2971) Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> * [chore][docs] enable Markdownlint rule MD010 (ray-project#2975) * [CI] Change Pre-commit-shellcheck-to-shellcheck-py (ray-project#2974) Signed-off-by: owenowenisme <mses010108@gmail.com> * [release] Update Yunikorn YAML file to Ray 2.41 (ray-project#2969) Signed-off-by: Cheng-Yeh Chung <kenchung285@gmail.com> * [release] Update YuniKorn YAML files to Ray 2.41 (ray-project#2976) Signed-off-by: win5923 <ken89@kimo.com> * [chore][docs] enable Markdownlint rule MD004 (ray-project#2973) [Unordered list style][1] [1]: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md004---unordered-list-style * [Test] Use GcsFaultToleranceOptions in test and backward compatibility (ray-project#2972) * Update samples to use Ray 2.41.0 images (ray-project#2964) Signed-off-by: Andrew Sy Kim <andrewsy@google.com> * Update TPU Ray CR manifests to use Ray 2.41.0 (ray-project#2965) * [Refactor] Use constants for image tag, image repo, and versions in golang to avoid hard-coded strings (ray-project#2978) --------- Signed-off-by: Rueian <rueiancsie@gmail.com> Signed-off-by: kaihsun <kaihsun@anyscale.com> Signed-off-by: David Xia <david@davidxia.com> Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> Signed-off-by: 400Ping <43886578+400Ping@users.noreply.github.com> Signed-off-by: LeoLiao123 <leoyeepaa@gmail.com> Signed-off-by: win5923 <ken89@kimo.com> Signed-off-by: owenowenisme <mses010108@gmail.com> Signed-off-by: Cheng-Yeh Chung <kenchung285@gmail.com> Signed-off-by: Andrew Sy Kim <andrewsy@google.com> Co-authored-by: Rueian <rueiancsie@gmail.com> Co-authored-by: Kai-Hsun Chen <kaihsun@anyscale.com> Co-authored-by: David Xia <david@davidxia.com> Co-authored-by: Chi-Sheng Liu <chishengliu@chishengliu.com> Co-authored-by: David Xia <dxia@spotify.com> Co-authored-by: Mykhailo Bobrovskyi <mikhail.bobrovsky@gmail.com> Co-authored-by: Ping <43886578+400Ping@users.noreply.github.com> Co-authored-by: Owen Lin <106612301+owenowenisme@users.noreply.github.com> Co-authored-by: Leo Liao <93932709+LeoLiao123@users.noreply.github.com> Co-authored-by: Blocka <ken89@kimo.com> Co-authored-by: zrant <37032227+pxp531@users.noreply.github.com> Co-authored-by: kenchung285 <kenchung285@gmail.com> Co-authored-by: fscnick <6858627+fscnick@users.noreply.github.com> Co-authored-by: ryanaoleary <113500783+ryanaoleary@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Fixes #2877.
Currently, a pending cluster will only be overridden if it is unhealthy, which is safe, but this behavior implicitly relies on the current implementation. This PR introduces an explicit check to prevent the overriding of a pending cluster that is actively serving.
Related issue number
Checks