Hi there, thank you for sharing, good work.

I want to run the code on windows and it says NCCL error.

So i changed the backend from NCCL to GLOO, and an invalid scalar type error pop up.

Do you have any idea why? What is your environment running the code? Mine is python3.10 Cudatoolkit11.3 with torch 1.12.1+cu113

Appreciate!