• Fully automate parts of smoke test on Windows - npm install & run a script. Anybody in team can run it.
• Explore visual review by talking screenshots
• Can run smoke test against insiders/stable
• Support to run smoketest during integration builds/CI (moved to Build definitions for smoke test #27845)
• Support incremental wait timeouts on test failures
• Make tests more robust by adding 'test classes' to DOM elements, add data attributes (investigated)
• Implement the full smoke test
  • Data Migration
  • Data Loss
  • First User Experience
  • Explorer
  • Configuration and views
  • Search
  • CSS
  • Markdown (skipped, webview has nodeIntegration == false)
  • JavaScript
  • Debugging JavaScript (no debug run-time test)
  • Git
  • Integrated Terminal
  • Status bar
  • Tasks
  • Extensions
  • Accessibility (skipped due to no voice-over coverage from Spectron)
  • Localization
• Support to run smoke test against fresh user data and populated user data
• Contributing.md that documents how to contribute a new smoke test
• Run on multiple platforms
• Support Insiders (e.g. window title checks)
• Explore smarter addressing of editor span tags

Issues

• Passing the user data directory to the spectron tests (Passing --user-data-dir is interpreted as Spectron, not Chrome argument electron-userland/spectron#189)
• WebDriverIO command deprecation warnings due to ChromeDriver not implementing W3C WebDriver spec (solved by redirecting warnings to separate log)

Unresolvable Issues

• Faking native dialogs - context menu, dialog popups (e.g. in getting asked for save on dirty editor close)