Skip to content

[Mobile] - E2E tests - Only upload artifacts when the tests fail to pass #47870

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

geriux
Copy link
Member

@geriux geriux commented Feb 8, 2023

What?

I've been seeing a few issues where the E2E tests would pass but the job would fail during the upload-artifacts step, I think it might be an issue with GitHub Actions itself but for now, I think we should only upload the artifacts if it fails.

Why?

To avoid disruptions while developing in this repo and constant CI failures.

How?

By changing from always to failure.

As it says in the docs leveraging on conditional artifact uploads.

Testing Instructions

CI checks should pass.

Testing Instructions for Keyboard

N/A

Screenshots or screencast

N/A

@geriux geriux added the Mobile App - i.e. Android or iOS Native mobile impl of the block editor. (Note: used in scripts, ping mobile folks to change) label Feb 8, 2023
@geriux geriux requested a review from fluiddot February 8, 2023 11:53
@geriux geriux marked this pull request as ready for review February 8, 2023 11:53
@fluiddot
Copy link
Contributor

fluiddot commented Feb 8, 2023

I've been seeing a few issues where the E2E tests would pass but the job would fail during the upload-artifacts step, I think it might be an issue with GitHub Actions itself but for now, I think we should only upload the artifacts if it fails.

@geriux It might be an edge case, but I think the video captured while running the tests can be useful to double-check that the test succeeded, in case of potential false positives like visual regressions. Probably this won't be an issue when we incorporate visual regression tests, but until then, I'm a bit hesitant to only upload the artifacts when the test fails.

Regarding the upload failure, if it's an infrastructure issue, I understand that it will be temporary, isn't it?

@geriux
Copy link
Member Author

geriux commented Feb 8, 2023

@geriux It might be an edge case, but I think the video captured while running the tests can be useful to double-check that the test succeeded, in case of potential false positives like visual regressions.

I'm not sure if I understand it correctly but the video will be recorded but it won't be uploaded to GitHub unless the tests fail.

To be honest I never check these videos unless something fails 😅

Regarding the upload failure, if it's an infrastructure issue, I understand that it will be temporary, isn't it?
I'm not really sure, we could wait and see, but just noting will be having E2E failures for now.

I'll have this PR open just in case it gets worse and it blocks development in the repo.

@fluiddot
Copy link
Contributor

fluiddot commented Feb 8, 2023

I'm not sure if I understand it correctly but the video will be recorded but it won't be uploaded to GitHub unless the tests fail.

So, if the video is not uploaded then we won't be able to check it in the PR, is this accurate?

To be honest I never check these videos unless something fails 😅

To be honest, I don't check it regularly and when I do is mainly on failed tests. Nevertheless, leaving a trace of what happened via a video might be interesting. I don't have a strong opinion on uploading it on successful runs, so in case this change helps to stabilize the E2E tests we can go with it.

I'll have this PR open just in case it gets worse and it blocks development in the repo.

👍

@github-actions github-actions bot added the Stale label Aug 8, 2023
@gziolo gziolo added the [Status] Stale Gives the original author opportunity to update before closing. Can be reopened as needed. label Sep 9, 2024
Copy link

github-actions bot commented Sep 9, 2024

Warning: Type of PR label mismatch

To merge this PR, it requires exactly 1 label indicating the type of PR. Other labels are optional and not being checked here.

  • Type-related labels to choose from: [Type] Automated Testing, [Type] Breaking Change, [Type] Bug, [Type] Build Tooling, [Type] Code Quality, [Type] Copy, [Type] Developer Documentation, [Type] Enhancement, [Type] Experimental, [Type] Feature, [Type] New API, [Type] Task, [Type] Technical Prototype, [Type] Performance, [Type] Project Management, [Type] Regression, [Type] Security, [Type] WP Core Ticket, Backport from WordPress Core.
  • Labels found: [Status] Stale, Mobile App - i.e. Android or iOS.

Read more about Type labels in Gutenberg. Don't worry if you don't have the required permissions to add labels; the PR reviewer should be able to help with the task.

@gziolo gziolo removed the Stale label Sep 9, 2024
@Mamaduka
Copy link
Member

Mamaduka commented Aug 4, 2025

Closing this as Stale.

@Mamaduka Mamaduka closed this Aug 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Mobile App - i.e. Android or iOS Native mobile impl of the block editor. (Note: used in scripts, ping mobile folks to change) [Status] Stale Gives the original author opportunity to update before closing. Can be reopened as needed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants