Skip to content

Conversation

parthchadha
Copy link
Contributor

What does this PR do ?

  1. Allow rollouts to use async generate.
  2. Fix async generate sampling params to allow only the required number of tokens to be generated.

Issues

List issues that this PR closes (syntax):

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this 

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

…ens than allowed

Signed-off-by: Parth Chadha <pchadha@nvidia.com>
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha parthchadha added the CI:L0 Run doctests and unit tests label Jun 4, 2025
@parthchadha parthchadha changed the title fix : async rollouts; incorrect tokens generated fix: async rollouts; incorrect tokens generated Jun 4, 2025
@parthchadha parthchadha added CI:L0 Run doctests and unit tests and removed CI:L0 Run doctests and unit tests labels Jun 4, 2025
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha parthchadha added CI:L0 Run doctests and unit tests and removed CI:L0 Run doctests and unit tests labels Jun 4, 2025
@SahilJain314
Copy link
Contributor

LGTM. Hopefully we can remove the specific 'vllm_cfg' indicator in rollouts.py eventually when we move to predominantly async

@parthchadha parthchadha added this pull request to the merge queue Jun 4, 2025
Merged via the queue into main with commit 2093fba Jun 4, 2025
13 of 14 checks passed
@parthchadha parthchadha deleted the pchadha/async-perf branch June 4, 2025 21:06
yfw pushed a commit that referenced this pull request Jun 6, 2025
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
Co-authored-by: Charlie Truong <chtruong@nvidia.com>
YzjiaoNvd pushed a commit to YzjiaoNvd/NeMo-RL that referenced this pull request Jun 10, 2025
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
Co-authored-by: Charlie Truong <chtruong@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI:L0 Run doctests and unit tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants