Reproduction

Simply just try running grpo with colocate mode and don't pass self.args.generation_kwargs

outputs:

Traceback (most recent call last):
generatigeneration_kwargson_kwargs.update(self.args.)
TypeError: 'NoneType' object is not iterable
    ...

System Info

vllm=0.9.1
trl[vllm]=0.19.0
transformers=4.52.3
torch=2.6.0

Checklist

• I have checked that my issue isn't already filed (see open issues)
• I have included my system information
• Any code provided is minimal, complete, and reproducible (more on MREs)
• Any code provided is properly formatted in code blocks, (no screenshot, more on code blocks)
• Any traceback provided is complete