Add developer docs to explain pagination tokens.

The comment docs explain general things around the pagination tokens well but when I was confronted with s2633508_17_338_6732159_1082514_541479_274711_265584_1, it wasn't obvious to me how to decipher it. I knew the stream_ordering (s2633508) part but it was really fuzzy what the other numbers were and the comment docs don't explain that part. I only really figured it out while drafting this issue and looking at the code more.

Relevant code:

• RoomStreamToken
• StreamToken

Relevant endpoints:

• /sync
  • ?since
  • next_batch
  • prev_batch
• /messages
  • ?from
  • ?to
  • start
  • end
• others

Live tokens (stream_ordering)

ex.

• s2633508_17_338_6732159_1082514_541479_274711_265584_1
  1. room_key: s2633508 -> 2633508 stream_ordering
  2. presence_key: 17
  3. typing_key: 338
  4. receipt_key: 6732159
  5. account_data_key: 1082514
  6. push_rules_key: 541479
  7. to_device_key: 274711
  8. device_list_key: 265584
  9. groups_key: 1
• s1_33_0_1_1_1_1_7_1
• s843_0_0_0_0_0_0_0_0

Each number key are concatenated together in this order:

And represent the position of the various fields in the /sync response:

{
  "next_batch": "s12_4_0_1_1_1_1_4_1",
  "presence": {
    "events": [
      {
        "type": "m.presence",
        "sender": "@the-bridge-user:hs1",
        "content": {
          "presence": "offline",
          "last_active_ago": 103
        }
      }
    ]
  },
  "device_lists": {
    "changed": [
      "@alice:hs1"
    ]
  },
  "device_one_time_keys_count": {
    "signed_curve25519": 0
  },
  "org.matrix.msc2732.device_unused_fallback_key_types": [],
  "device_unused_fallback_key_types": [],
  "rooms": {
    "join": {
      "!QrZlfIDQLNLdZHqTnt:hs1": {
        "timeline": {
          "events": [],
          "prev_batch": "s10_4_0_1_1_1_1_4_1",
          "limited": false
        },
        "state": {
          "events": []
        },
        "account_data": {
          "events": []
        },
        "ephemeral": {
          "events": []
        },
        "unread_notifications": {
          "notification_count": 1,
          "highlight_count": 0
        },
        "summary": {},
        "org.matrix.msc2654.unread_count": 1
      }
    }
  }
}

Historic tokens (topological_ordering/depth)

• t175-530_0_0_0_0_0_0_0_0
  1. topological_ordering: t175 -> 175 (depth)
  2. stream_ordering: 530
  3. presence_key: 0
  4. typing_key: 0
  5. receipt_key: 0
  6. account_data_key: 0
  7. push_rules_key: 0
  8. to_device_key: 0
  9. device_list_key: 0
  10. groups_key: 0
  • You will see this from /messages probably because the endpoint is scoped to the room and so is depth
  • topological_ordering which is the same as depth in Synapse

Min-position tokens

This one seems pretty well explained by the comment docs already:

ex. m56~2.58~3.59