Feature request

The CLI currently supports training models with SFT/DPO/KTO:

It would be good to extend this support to all trainers so that we have a consistent API and also learn which parts of our scripts need refactoring to support this usage.

This could be tackled in separate PRs to keep things lightweight, and I'll track here the trainers in terms of priority to add (based on Hub usage):

• orpo
• reward modelling
• ppo
• rloo
• online dpo
• gkd
• cpo
• bco
• xpo
• nash-md

Motivation

It is somewhat annoying that one cannot train a model through the CLI as this is helpful for fast debugging / iterations.

Your contribution

Happy to open PRs, but this could be a good first issue for new contributors!