Upgrade Playwright to v1.54.2 #70503
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Size Change: 0 B Total Size: 1.91 MB ℹ️ View Unchanged
|
49233d2 to
1bc72b4
Compare
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
1bc72b4 to
3a8a669
Compare
3a8a669 to
001afbe
Compare
cee1736 to
d9dae52
Compare
t-hamano
reviewed
Aug 4, 2025
| const { createBlock, registerBlockType } = wp.blocks; | ||
|
|
||
| registerBlockType( 'test/alternative-group-block', { | ||
| apiVersion: 3, |
There was a problem hiding this comment.
I got here from this comment: #69932 (comment)
In the future, all editors will likely run in iframes, so I think this approach is fine. If the other tests failing in this PR are also due to non-iframe editors, we might be able to change them to test in the iframe editor.
There was a problem hiding this comment.
I'll double-check current failing tests. Only annoying problem is that I fix one test and then others start failing on CI 😅
d9dae52 to
38e8b7c
Compare
|
Flaky tests detected in 3f60660. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/16802720484
|
Mamaduka
commented
Aug 7, 2025
Comment on lines
+1092
to
+1096
| // Delay ensures selection can catch up. | ||
| await pageUtils.pressKeys( 'Shift+ArrowLeft', { | ||
| times: 5, | ||
| delay: 50, | ||
| } ); |
There was a problem hiding this comment.
It looks like Playwright just got faster, so I had to add delays to ensure text selection can catch up before performing actions 😅
I don't think a similar bug can be reproduced manually by humans.
Mamaduka
commented
Aug 7, 2025
| @@ -582,7 +582,7 @@ test.describe( 'Copy/cut/paste', () => { | |||
| } ); | |||
|
|
|||
| await pageUtils.pressKeys( 'primaryAlt+v' ); | |||
| expect( await editor.getBlocks() ).toMatchObject( [ | |||
| await expect.poll( editor.getBlocks ).toMatchObject( [ | |||
There was a problem hiding this comment.
The expect.poll should be the preferred method for similar assertions. Not sure if the whole overhaul is worth the effort, so let's fix this case by case.
|
The upgrade discovers an increasing number of flaky end-to-end tests on each CI run. I tried to fix as much as possible, but it's hard to do that all at once. How do we like to proceed here? Should we merge this as long as CI checks are green and handle flaky tests in follow-ups? cc @t-hamano, @swissspidy |
|
It may affect the release of Gutenberg 21.4 RC1, so it may be better to merge this PR at least after the 21.4RC1 release process is complete. I think the RC1 release process was scheduled for today. If we have some idea of what's causing the flaky tests, and it doesn't seem difficult to fix them, then we might be able to merge this PR. |
Makes sense.
There's no one solution, and I left inline comments regarding some cases. It seems that test interactions run much faster, and some of our current tests rely on implicit behaviors, which can occasionally fail. Example: The tests use this standard pattern for adding blocks - type That pattern was inherited from old tests, and it doesn't make explicit assertions. |
|
OK, so let's merge this PR once it passes all the tests and RC1 is released 👍 |
t-hamano
approved these changes
Aug 7, 2025
|
Okay, I'm going to merge this. I think I've fixed most of the flaky tests. The one that happened on the latest run also fails on trunk from time to time. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What?
Upgrade Playwright to v1.54.2. Supersedes #69932.
Debugging
What's new?
Testing Instructions