Thanks to @rollingcookies, we can see in this issue that this schedulers works great with the Hyper and probably also Lighting loras/unets.

It'd be fantastic if someone can contribute this scheduler to diffusers.

Please let me know if someone is willing to do this.