Use case

Integration tests today are run by users using the following command:

flutter drive \
  --driver=test_driver/<driver_file> \
  --target=integration_test/<test file>

There is some friction when doing things like globbing over and running all tests in the integration_test/ directory.

Also, most of the time the test driver file is just the following copy-pasted:

import 'package:integration_test/integration_test_driver.dart';

Future<void> main() => integrationDriver();

Proposal

Provide some sort of executable as a thin wrapper around flutter drive.

This tool will have two main purposes:

• Deciding where / what integration tests to run
• Reducing user boilerplate, such as the placeholder driver script above.

The result is that users can use the following command to run all their tests in integration_test/, without the need to add the aforementioned placeholder script to test_driver/ as mentioned above.

cd <application directory>
pub run integration_test

Deciding Where / What integration tests to run

Some rough ideas:

# Runs all integration tests
pub run integration_test

# Runs just `foo_test.dart`
pub run integration_test integration_test/foo_test

# Run on iOS
pub run integration_test --device-id ios

We can forward all other args (or maybe just some like --device-id ?) to the flutter drive command.

Reducing user boilerplate

Host-side boilerplate

This tool will will make it no longer necessary for users to copy paste the placeholder driver file into the test_driver/ directory. We will provide a --driver parameter for users to override the driver script, for use cases such as handling screenshots (as mentioned in flutter/plugins#2986).

Application-side boilerplate

At the same time, it faciliates the change to use the reporter from package test (#50816). With the new API (currently in progress dart-lang/test#1332) from package:test, users will likely need to add boilerplate code to their tests that run on the device.

// In package:integration_test.

// Expose a new API so that [testMain] runs with the reporter from package:test.
Future<void> runIntegrationTest(FutureOr<void> Function() testMain) async {
  // Import and use `directRunTests` from `package:test`.
}

// In the users test code, in their integration_test/ directory.

import 'package:integration_test/integration_test.dart';

// Users need to wrap their tests with [run].
void main() => runIntegrationTest(testMain);

Future<void> testMain() {
  testWidgets('A widget test', (tester) async {
    await tester.pumpWidget(CounterApp());
    expect(find.byType(Text), findsNWidgets(2));
  });

  test('a test',   () {
    expect(1 + 1, 2);
  });
}

With this proposed tool, we can eliminate the boilerplate that users need to use in their tests. They do not have to import runIntegrationTest to wrap their tests; they can write their tests just like they know how with package:test and package:flutter_test.