Testing settings sync features

• windows @alexr00
• mac @eamodio
• linux @weinand

Complexity: 5

Set Up

• Follow instructions here
• To test, have two insiders open with different user data directories and extension directories
• Make sure to back up your settings & keybindings since some test scenarios involve deleting data from server.

Reset

Reset Completely

To reset completely Turn off sync and select Turn off sync on all devices checkbox. This will remove all your data on the server and also remove the sync state on your local machine.

Reset Machine

You can reset a machine and remove the sync state using following two commands

• Turn off (do not check the Turn off sync on all devices)
• Developer: Reset local (Sync) - This will remove the sync state on your local machine.

Exploratory Testing

Getting Started Experience

• Test setting up Settings Sync on a machine for the first time ever. (Reset completely first to test)
• Test setting up Settings Sync on another machine for the first time. You can always reset your local machine using Reset machine step.
• Test turning off sync completely also turn off sync on other machines

Synchornising Settings

• Test settings file (including settings positions and comments) is synchronised across machines as expected.
• Test showing and resolving conflicts flow and UX when there are conflicts while synchronising settings and keybindings.
• For settings, conflicts should be detected not only with settings values but also with their positions and comments.

Synchronising Extension enablement

• Test extension's enablement is synchronised across devices.

Synchronising Display Language (UI State)

• Test synchronising display language across machines

Back ups

• Test local settings and keybindings file are backed up when ever updated with remote content. Local data is backed up in .sync folder under the same directory as settings and keybindings files.

Error handling & logging

• Test errors are logged (in sync log channel) and reported. Errors are reported and sync is disabled in following cases
  • When the file to sync is too large (1MB)
  • If sync fails consecutively 5 times
• Auto sync interval increases exponentially every time there is an error. Regular interval is 5 mins. It increases by number of sync failures.
• Test Sync Output channel logs are having useful information when set to different log levels (eg: trace).