-
Notifications
You must be signed in to change notification settings - Fork 29.2k
Description
Currently we run almost all tests in flutter/plugins twice: once for master
, and once for stable
. By far our most computationally expensive tests are the *-platform_tests
tests, which build the plugins, run their native unit tests, and run their integration tests. In the vast majority of cases, those tests have the same outcome on the two branches (ignoring flake); branch-specific failures are much more likely to show up in analyze
(and for native build failures, we have duplicate coverage in the form of the build_all_plugins
test).
Given that, we could consider substantially reducing our presubmit CI resource usage by making *-platform-tests
only run on master
in presubmit, and include stable only post submit. The post-submit tests would still prevent us from accidentally publishing stable
breakage; the downside would be that in those cases we would not catch the failure until after it broke the tree, and we had to revert.
I think that's probably a reasonable trade-off to make, at least on an experimental basis to see how it goes. We might want to exempt Web from this change if we do though; my impression is that plugin runtime differences in master
vs stable
are more likely there than on other platforms.
/cc @jmagman @blasten @ditman @godofredoc @bparrishMines for thoughts.