Skip to content

Conversation

noomorph
Copy link
Collaborator

@noomorph noomorph commented Apr 18, 2020

New projects by default will be using a new test runner integration between Detox and jest-circus.

Motivation

  1. Initiate Detox as early as possible, even before we know what tests we are going to run. That potentially enables discarding --testNameRegexPattern=(^(^()^()) workaround for Jest, as in this position we will be able to decide in runtime which tests we want to skip, and which - not. Also, the initialization errors won't be bound to some specific test - less clutter and misleading information for users.

  2. Compared to previous jest-circus integration, we remove that a bit alien global.detoxCircus environment variable. Now, registering the listeners happens right in a user-defined environment class which derives from DetoxCircusEnvironment. On the other hand, we add a global detox variable, because of the very same sandboxing issues (to avoid creating many isolated Detox instances with every next require('detox') - see detox/src/index.js for the actual place).

  3. Together with removing detoxCircus variable, and moving concerns into environment class, at the end of the day, we gain freedom from ./init.js environment setup files. We don't depend on user-defined hooks (beforeAll, beforeEach, afterEach, afterAll) - now we run above that. That's why the new DetoxCircusEnvironmnent has less complexity because it does not mediate between things happening in user-facing code (setupFilesAfterEnv: ['init.js']) and jest-circus' handleTestEvent events.

  4. We used to run afterEach lifecycle actions as a first thing in the consequent test's beforeEach (or detox.cleanup() if there were no next tests). That led sometimes to issues when failures in detox.afterEach() (hence, incomplete execution of all planned routines) of the previous test could impact the behavior of detox.beforeEach() of the current one.

  5. It resolves Take screenshot after test failure, before afterEach is executed #1661 - artifacts plugins become able to react ASAP to specific failure inside tests, before any hook (e.g. afterEach) executes. That should improve the relevance of post-error screenshots, as one of the consequences.

Description

Instead of a well-known e2e/init.js file, you are going to have e2e/environment.js file.

const {
  DetoxCircusEnvironment,
  SpecReporter,
  WorkerAssignReporter,
} = require('detox/runners/jest-circus');

class CustomDetoxEnvironment extends DetoxCircusEnvironment {
  constructor(config) {
    super(config);

    // Can be safely removed, if you are content with the default value (=300000ms)
    this.initTimeout = 300000;

    // This takes care of generating status logs on a per-spec basis. By default, Jest only reports at file-level.
    // This is strictly optional.
    this.registerListeners({
      SpecReporter,
      WorkerAssignReporter,
    });
  }
}

module.exports = CustomDetoxEnvironment;

The entire class is optional if you don't plan to use the customizations above.

Jest's e2e/config.json has the following changes:

 {
-  "setupFilesAfterEnv": ["./init.js"],
-  "testEnvironment": "node",
+  "testEnvironment": "./environment",
+  "testRunner": "jest-circus/runner",
+  "testTimeout": 120000,
   "reporters": ["detox/runners/jest/streamlineReporter"],
   "verbose": true
 }

Miscallenous

CI scripts are targeting jest-circus by default. jest-jasmine is tested only for the edge case with detox.init() timeout.

@d4vidi
Copy link
Collaborator

d4vidi commented Apr 19, 2020

You are my hero

@d4vidi
Copy link
Collaborator

d4vidi commented Apr 19, 2020

@noomorph sorry for repeating myself, but really - do we really need to keep jasmine as 'legacy'? Can we not just deprecate in a major version and move on?
Doing so would also enable you to remove all of the hackish part of init/cleanup code you recently added in the Detox wrapper, won't it?

@noomorph
Copy link
Collaborator Author

noomorph commented Apr 20, 2020

@d4vidi, I really really want to rewrite adapter in a very breaking manner, so I would go an extra mile postponing a major release to not breakchange two times in a row, with jasmine and with adapters.

P. S. Actually this time I plan to kill adapters. No adapters anymore. This will become a major version.

P. P. S. The problem is that you will be very hesitant to review extra 500 lines of code in a single pull request, most likely, if I continue adding changes to here, so that's why I considered splitting into two PRs.

@noomorph noomorph force-pushed the feat/deprecate-jest-jasmine branch from f7a54e4 to addd549 Compare April 24, 2020 07:17
@stale
Copy link

stale bot commented May 24, 2020

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Thank you for your contributions.
For more information on bots in this reporsitory, read this discussion.

@stale stale bot added the 🏚 stale label May 24, 2020
@coveralls
Copy link

coveralls commented May 24, 2020

Pull Request Test Coverage Report for Build 2282

  • 158 of 180 (87.78%) changed or added relevant lines in 12 files are covered.
  • 2931 unchanged lines in 118 files lost coverage.
  • Overall coverage decreased (-5.3%) to 73.429%

Changes Missing Coverage Covered Lines Changed/Added Lines %
detox/src/artifacts/ArtifactsManager.js 35 37 94.59%
detox/src/artifacts/templates/plugin/ArtifactPlugin.js 30 32 93.75%
detox/src/utils/isPromise.js 2 5 40.0%
detox/src/utils/logger.js 13 16 81.25%
detox/src/artifacts/templates/plugin/TwoSnapshotsPerTestPlugin.js 25 29 86.21%
detox/src/utils/timely.js 12 20 60.0%
Files with Coverage Reduction New Missed Lines %
detox/src/artifacts/utils/AndroidDevicePathBuilder.js 1 87.5%
detox/src/devices/drivers/android/emulator/AVDsResolver.js 1 88.89%
detox/src/devices/drivers/android/tools/AAPT.js 1 93.02%
detox/src/devices/drivers/android/tools/APKPath.js 1 92.11%
detox/src/errors/DetoxConfigError.js 1 92.86%
detox/src/utils/debug.js 1 87.5%
detox/src/utils/Deferred.js 1 92.86%
detox/src/utils/safeAsync.js 1 81.82%
detox/src/utils/sleep.js 1 78.57%
detox/src/artifacts/log/ios/SimulatorLogRecording.js 2 90.91%
Totals Coverage Status
Change from base Build 1221: -5.3%
Covered Lines: 9502
Relevant Lines: 13001

💛 - Coveralls

@stale stale bot removed the 🏚 stale label May 24, 2020
@noomorph noomorph changed the title feat: deprecate jest-jasmine2 feat(runner): new integration with jest-circus Jun 4, 2020
@noomorph noomorph force-pushed the feat/deprecate-jest-jasmine branch 7 times, most recently from 57a69c6 to 425bca6 Compare June 4, 2020 15:53
@rotemmiz
Copy link
Contributor

rotemmiz commented Jun 4, 2020

@noomorph I still see there is no documentation re the huge change in #2027, just making sure its going to be a part of this PR.

@noomorph
Copy link
Collaborator Author

noomorph commented Jun 5, 2020

@rotemmiz, that's right, because #2027 was being treated as a part of this one. I will be working on the docs today. I am not bananas not to supply such PRs without docs, hehe, don't worry.

I'll ask you to review user-facing places on Sunday.

@rotemmiz
Copy link
Contributor

rotemmiz commented Jun 7, 2020

This is so important, we should have done this kind of thing a long time ago, tapping in at the configuration space instead of the user space.
Can you please clarify regarding point no 2, why would this type of setup disregard the module cache?

@d4vidi
Copy link
Collaborator

d4vidi commented Jun 7, 2020

  • side note -

Initiate Detox as early as possible, even before we know what tests we are going to run.

That would also enable a fix for #2097, as the device's platform would already be known before it calls register tests

  • Documentation: Need to update the whole setup process - We should prefer jest over mocha and make it the 1st-class solution in our description.

@noomorph noomorph force-pushed the feat/deprecate-jest-jasmine branch from a1933bf to ad48933 Compare June 11, 2020 16:17
@noomorph noomorph marked this pull request as ready for review June 12, 2020 14:55
@noomorph noomorph requested a review from rotemmiz as a code owner June 12, 2020 14:55
@noomorph noomorph merged commit 53d46b9 into master Jun 12, 2020
@noomorph noomorph deleted the feat/deprecate-jest-jasmine branch June 12, 2020 16:41
@refactornator
Copy link

With this new integration with jest-circus. How should I launch the app with the permissions I need?

I got my tests working again by overriding the initDetox function in my CustomDetoxEnvironment like this:

  async initDetox() {
    const instance = await this.detox.init(undefined, {launchApp: false});
    await instance.device.launchApp({permissions: {notifications: 'YES'}});
    return instance;
  }

Is this the best way to define permissions for my device?

@noomorph
Copy link
Collaborator Author

noomorph commented Jun 25, 2020

@wlindner, at the moment, not much time has passed since this feature entered production; hence I can't say for sure what the best method is.

Alternative approach

In one of our projects, we apply another approach since the initialization phase is more complicated, e.g., starting a mock server and some CI preparations.

detox.config.js

module.exports = {
  // ...
  behavior: {
    init: { launchApp: false },
  },
  // ...
};

init.js

beforeAll(async () => {
  // launch mock server and other stuff
  await device.launchApp({permissions: { /* ... */ }});
});

jest.config.js

module.exports = {
...
+  setupFilesAfterEnv: ["./init.js"],
...
};

Pros and cons

initDetox()

Pros. If you keep all your initialization steps in the protected initDetox, then in case if the app cannot start correctly, the failure phase is going to be much shorter and cleaner - for example, at the moment, Detox simply skips all the tests and promptly exits with a failure.

Cons. Any potential timeout errors of initDetox() are handled by handwritten wrapper functions (bypassing Jest timeout mechanisms), and there's a chance you might have sometimes subpar experience in unstudied aspects (use with sourcemaps, etc). Also, although there is no intention to break initDetox(): Promise<Detox> signature, I cannot give guarantees either that its signature won't undergo some changes later.

beforeAll

Pros. The beforeAll hook approach looks a bit cleaner and totally in line with both the Jest and Detox approaches and less dependent on our recent semi-documented initDetox and cleanupDetox virtual methods.

Cons. Most likely, if you have SpecReporter enabled, you'll have a bit more messy and verbose output if there is a failure in the very beginning. If your setup is very simple, you might feel that it is redundant to create one more file and add it to setupFilesAfterEnv, but that's a purely aesthetical reason with no practical reasoning underneath.

Conclusion

Try out beforeAll approach, if this is acceptable for you.

If you nevertheless prefer overriding the initDetox protected method, your code excerpt looks valid to me, thus you are free to do that either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Take screenshot after test failure, before afterEach is executed
5 participants