Also see #77140

Plan of attack:

• Adopt vscode-textmate refactorings in vscode #167288
• Make sure rule ids are stable by initializing injection rules eagerly
• Tokenize in the webworker
• Send over states by diffing against the state of the previous line. Send the delta as pop(n), push(state) instructions.

Sync tokenization:

Async tokenization: