Skip to content

[Test][Autoscaler] deflaky autoscaler idle timeout e2e tests by a longer timeout #3727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 1, 2025
Merged

[Test][Autoscaler] deflaky autoscaler idle timeout e2e tests by a longer timeout #3727

merged 1 commit into from
Jun 1, 2025

Conversation

rueian
Copy link
Contributor

@rueian rueian commented Jun 1, 2025 •
edited
Loading

Give the test case more time to scale down.

The original flaky rate of this test is about 10% on my mac. Now it can pass 100 times without flaky.

▶ while go test -count=1 ./... -run=TestRayClusterAutoscalerV2IdleTimeout; do :; done

ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	96.222s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.784s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.683s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.702s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.944s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.898s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.874s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	95.137s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.967s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.632s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.951s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.076s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.656s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.324s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.028s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.483s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.926s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.547s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.739s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.546s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.204s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.733s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.741s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.087s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.840s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	95.212s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.311s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.097s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.837s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.502s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.673s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.628s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.149s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.693s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.718s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.852s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	100.106s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.967s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.824s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.861s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.826s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.891s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	100.224s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.747s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.722s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.875s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.979s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.820s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.729s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.135s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.008s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.684s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.573s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.000s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.948s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.640s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.807s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.770s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.443s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.611s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.000s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.444s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.981s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.819s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.534s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.819s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.978s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.461s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.839s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.908s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.694s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.981s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.709s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.628s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.826s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.665s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.121s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.974s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.413s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.389s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.958s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.449s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.588s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.958s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.440s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.825s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.965s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.219s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.679s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.074s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.842s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.743s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.833s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.164s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.851s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.330s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.670s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.857s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	100.949s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.538s

Why are these changes needed?

Related issue number

Closes #3701

Checks

  • I've made sure the tests are passing.
  • Testing Strategy
    • Unit tests
    • Manual tests
    • This PR is not tested :(

@rueian
[Test][Autoscaler] deflaky autoscaler idle timeout e2e tests by a lon…
c03fdd7 
…ger timeout

Signed-off-by: Rueian <rueiancsie@gmail.com>
@rueian rueian marked this pull request as ready for review June 1, 2025 19:15
@rueian rueian mentioned this pull request Jun 1, 2025
Closed
2 tasks
@kevin85421 kevin85421 merged commit 764971f into ray-project:master Jun 1, 2025
25 checks passed
@davidxia davidxia mentioned this pull request Jun 5, 2025
Closed
4 tasks
pawelpaszki pushed a commit to opendatahub-io/kuberay that referenced this pull request Jun 10, 2025
@rueian @pawelpaszki
[Test][Autoscaler] deflaky autoscaler idle timeout e2e tests by a lon…
2d0823c 
…ger timeout (ray-project#3727)

Signed-off-by: Rueian <rueiancsie@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kevin85421 kevin85421 kevin85421 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CI] Deflaky Autoscaler V2 e2e tests
2 participants
@rueian @kevin85421