Skip to content

feat: use async vllm engine (only used in unit tests) #418

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 15 commits into from
May 29, 2025
Merged

feat: use async vllm engine (only used in unit tests) #418

merged 15 commits into from
May 29, 2025

Conversation

parthchadha
Copy link
Contributor

What does this PR do ?

This PR adds the capability to use async vllm engine and verify its correctness in unit tests. This is first of N PR's that will enable async processing of rollout requests.

Issues

List issues that this PR closes (syntax):

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

@parthchadha parthchadha added the CI:L0 Run doctests and unit tests label May 20, 2025
@parthchadha parthchadha added CI:L0 Run doctests and unit tests and removed CI:L0 Run doctests and unit tests labels May 20, 2025
@parthchadha
fix: Fix failing unit test
fb47ba9 
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha parthchadha force-pushed the pchadha/async-vllm branch from 09c45a8 to fb47ba9 Compare May 20, 2025 15:13
@parthchadha parthchadha added CI:L0 Run doctests and unit tests and removed CI:L0 Run doctests and unit tests labels May 20, 2025
terrykong
terrykong reviewed May 20, 2025
View reviewed changes
terrykong
terrykong reviewed May 20, 2025
View reviewed changes
@parthchadha parthchadha added CI:L0 Run doctests and unit tests and removed CI:L0 Run doctests and unit tests labels May 26, 2025
@parthchadha
Create separate async vs sync apis
cca95cc 
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha
Add async flag to recipes
14e821c 
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha parthchadha added CI:L0 Run doctests and unit tests and removed CI:L0 Run doctests and unit tests labels May 26, 2025
@parthchadha
Update uv.lock
8840fef 
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha parthchadha added CI:L0 Run doctests and unit tests and removed CI:L0 Run doctests and unit tests labels May 27, 2025
@parthchadha parthchadha added the CI:L0 Run doctests and unit tests label May 28, 2025
terrykong
terrykong previously approved these changes May 28, 2025
View reviewed changes
@parthchadha parthchadha added this pull request to the merge queue May 28, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 28, 2025
@parthchadha
Fix failing test due to falling back on v0 runtime
51b145b 
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha parthchadha force-pushed the pchadha/async-vllm branch from e9709a0 to 51b145b Compare May 28, 2025 22:55
terrykong
terrykong previously approved these changes May 28, 2025
View reviewed changes
@parthchadha parthchadha enabled auto-merge May 28, 2025 22:56
@parthchadha parthchadha added this pull request to the merge queue May 28, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 28, 2025
@parthchadha
Use vllm v1 runtime for non-async case
3c15f18 
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha parthchadha enabled auto-merge May 28, 2025 23:42
@parthchadha parthchadha removed the CI:L0 Run doctests and unit tests label May 29, 2025
terrykong
terrykong previously approved these changes May 29, 2025
View reviewed changes
@parthchadha parthchadha added this pull request to the merge queue May 29, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 29, 2025
@parthchadha
Add async_engine to vllm_cfg for eval
d2185f5 
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha parthchadha enabled auto-merge May 29, 2025 04:07
@parthchadha parthchadha added this pull request to the merge queue May 29, 2025
Merged via the queue into main with commit 89f98ae May 29, 2025
13 of 14 checks passed
@parthchadha parthchadha deleted the pchadha/async-vllm branch May 29, 2025 05:23
yfw pushed a commit that referenced this pull request Jun 6, 2025
@parthchadha @yfw
feat: use async vllm engine (only used in unit tests) (#418)
914e76b 
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
YzjiaoNvd pushed a commit to YzjiaoNvd/NeMo-RL that referenced this pull request Jun 10, 2025
@parthchadha
feat: use async vllm engine (only used in unit tests) (NVIDIA-NeMo#418)
b7e4c95 
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SahilJain314 SahilJain314 SahilJain314 approved these changes

@terrykong terrykong terrykong left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@parthchadha @terrykong @SahilJain314