This impact this test: https://github.com/NVIDIA/NeMo-RL/blob/3309631ccd1a51bbb1c622a58f14d2aa01501e77/tests/unit/models/generation/test_vllm_generation.py#L367