Request per #96 (comment). Current finetuning script is quite slow because it doesn't support accelerate due to #7, but a lot of people are finetuning using a single GPU, so we don't really have to worry about DP or DDP and can directly turn to accelerate which supports mixed precision that saves time and RAM. It would be great if someone could implement this feature for a single GPU. For multi-GPU we are still trying to solve #7 .