TypeError: '>=' not supported between instances of 'list' and 'tuple'

Thanks a lot @jamesbraza

by the way, is this change needed as well? #2871

by the way, is this change needed as well? #2871

Yes, testing it now. Clearly I didn't test this PR previously, as @yeruoforever reported it had a TypeError haha, my bad.

Hi @qgallouedec I have completed my validations, this PR is ready for merge. I had to also pull in:

• 🫔 [GRPO] Pass wrapped model to unwrap_model_for_generation for DeepSpeed Stage-3 compatibility #2871 (thanks for sharing it)
• I hit [GRPO] Gradient mode issue when num_iterations > 1 #2953, and to fix it, I changed this line to with torch.no_grad() instead of with torch.inference_mode()

Thanks, but I can't see the above mentionned changes. Did you pushed them?

Thanks, but I can't see the above mentionned changes. Did you pushed them?

The other ones I mentioned weren't related to this PR, so they're not here. They are all here: https://github.com/Future-House/trl/tree/working-grpo-2025-02-27

Thanks! Just commit the suggestions and we are good to merge.
Usually allowing maintainer to edit the PR makes things easier for us.

Usually allowing maintainer to edit the PR makes things easier for us.

Yeah I always do, but for some reason that checkbox isn't present in this PR's right panel :/

Regardless, PR is ready for review again

Thanks!

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.