fix(test): address flaky TestAPIV2_ThroughRPC #13148
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull Request Overview
Fix flaky TestAPIV2_ThroughRPC by introducing a stable execution guard and refactoring test helpers to avoid chain reorgs during RPC calls.
- Add
mkStableExecuteto repeat operations until the chain tipset remains unchanged through execution. - Refactor tipset and certificate helper closures to return
(value, error)instead of using testing helpers directly. - Wrap RPC requests, JSON unmarshalling, and assertions inside the stable execution flow for all test sections.
718937d to
1fca973
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR aims to address flaky behavior in the TestAPIV2_ThroughRPC test by adding context cancellation checks and enforcing chain stability during test execution. Key changes include:
- Adding a context cancellation check in TestEthAPIWithF3 to prevent potential infinite loops
- Introducing a stable execution wrapper (mkStableExecute) in TestAPIV2_ThroughRPC to ensure consistent chain tipset state during test runs
- Refactoring several helper functions to return errors instead of immediately failing the test
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| itests/eth_api_f3_test.go | Added context cancellation check inside a retry loop |
| itests/api_v2_test.go | Introduced mkStableExecute and updated helper functions for better error handling and stability during RPC test execution |
There was a problem hiding this comment.
Pull Request Overview
This pull request addresses flakiness in API tests by introducing a stable execution wrapper that ensures the blockchain state (tipset) remains unchanged during test execution. Key changes include:
- Adding a context check inside a loop in TestEthAPIWithF3 to fail early if the context is cancelled.
- Introducing mkStableExecute in TestAPIV2_ThroughRPC to repeatedly invoke operations until a stable tipset is achieved.
- Refactoring test cases in TestAPIV2_ThroughRPC to use the new stable execution helper and replacing direct actor expectation with tipset-based resolutions.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| itests/eth_api_f3_test.go | Inserts a context cancellation check in the execution loop. |
| itests/api_v2_test.go | Implements a stable execution wrapper and updates test expectations to reduce flakiness. |
Comments suppressed due to low confidence (2)
itests/api_v2_test.go:43
- [nitpick] Consider renaming mkStableExecute to a more descriptive name (e.g., withStableChain) to better communicate its purpose.
mkStableExecute := func(getTipSet func(t *testing.T) *types.TipSet) func(fn func()) *types.TipSet {
itests/api_v2_test.go:527
- [nitpick] For consistency with the StateGetActor tests, consider capturing the return value from stableExecute in the StateGetID tests to enable additional validations and improve test clarity.
_ = stableExecute(func() { ... })
5a16d6c to
5125066
Compare
|
Rebased on top of #13149 so I'd like for that one to land before this one so it'll be removed from here. |
5125066 to
df96ce9
Compare
rjan90
approved these changes
May 30, 2025
8 tasks
8 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes: #13116
This was mostly AI, needs some clearer review than I can afford right at this moment, but seems good on the surface. It also should have that job run in CI a bunch of times to confirm that it's actually fixed.