Skip to content

feat: multi-modal example with vLLM v1 and UX v2 #2040

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 45 commits into from
Aug 12, 2025
Merged

feat: multi-modal example with vLLM v1 and UX v2 #2040

merged 45 commits into from
Aug 12, 2025

Conversation

GuanLuo
Copy link
Contributor

@GuanLuo GuanLuo commented Jul 22, 2025 •
edited by coderabbitai bot
Loading

Overview:

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

Summary by CodeRabbit

  • New Features

    • Introduced a comprehensive multimodal example suite for distributed inference, including new processor, worker, and encoder components, as well as utilities for image loading, protocol handling, and model management.
    • Added RDMA-based data transfer framework and documentation for high-throughput GPU Direct RDMA between distributed workers.
    • Provided multiple launch scripts for aggregated and disaggregated multimodal serving pipelines, including support for Llama 4 models.
    • Included detailed documentation and usage instructions for deploying and testing multimodal pipelines on Kubernetes and local environments.

  • Documentation

    • Added extensive READMEs covering multimodal deployment scenarios, RDMA library usage, and step-by-step guides for launching and testing pipelines.

  • Chores

    • Updated Dockerfile and installation scripts to support building from a forked vLLM repository with configurable build arguments.
    • Re-exported the Endpoint class in the Python runtime bindings for streamlined access.

@GuanLuo GuanLuo changed the title Gluo/multi modal ux feat: multi-modal example with vLLM v1 and UX v2 Jul 24, 2025
@GuanLuo GuanLuo marked this pull request as ready for review July 24, 2025 02:24
@github-actions github-actions bot added the feat label Jul 24, 2025
@GuanLuo GuanLuo requested review from nnshah1 and whoisj as code owners July 24, 2025 02:24
@GuanLuo
Copy link
Contributor Author

GuanLuo commented Aug 11, 2025

@grahamking yea I am reviving it, was blocked by vLLM merge and now we can proceed.
@atchernych can you approve this PR? Blocked by your request for change.

@GuanLuo
Copy link
Contributor Author

GuanLuo commented Aug 12, 2025

/ok to test

@athreesh athreesh merged commit 9b87c89 into main Aug 12, 2025
11 of 12 checks passed
@athreesh athreesh deleted the gluo/multi-modal-ux branch August 12, 2025 23:55
hhzhang16 pushed a commit that referenced this pull request Aug 27, 2025
@GuanLuo @krishung5 @hhzhang16
feat: multi-modal example with vLLM v1 and UX v2 (#2040)
834d988 
Co-authored-by: krishung5 <krish@nvidia.com>
Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
@coderabbitai coderabbitai bot mentioned this pull request Aug 28, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments

@biswapanda biswapanda biswapanda approved these changes

@whoisj whoisj whoisj approved these changes

@indrajit96 indrajit96 indrajit96 approved these changes

@rmccorm4 rmccorm4 rmccorm4 approved these changes

@krishung5 krishung5 krishung5 approved these changes

@atchernych atchernych atchernych approved these changes

@nnshah1 nnshah1 Awaiting requested review from nnshah1 nnshah1 is a code owner

@tanmayv25 tanmayv25 Awaiting requested review from tanmayv25 tanmayv25 is a code owner

@richardhuo-nv richardhuo-nv Awaiting requested review from richardhuo-nv richardhuo-nv is a code owner

@ptarasiewiczNV ptarasiewiczNV Awaiting requested review from ptarasiewiczNV ptarasiewiczNV is a code owner

@ishandhanani ishandhanani Awaiting requested review from ishandhanani ishandhanani is a code owner

@alec-flowers alec-flowers Awaiting requested review from alec-flowers alec-flowers is a code owner

@piotrm-nvidia piotrm-nvidia Awaiting requested review from piotrm-nvidia

@ryanolson ryanolson Awaiting requested review from ryanolson

@grahamking grahamking Awaiting requested review from grahamking

@paulhendricks paulhendricks Awaiting requested review from paulhendricks

@tmonty12 tmonty12 Awaiting requested review from tmonty12

@tedzhouhk tedzhouhk Awaiting requested review from tedzhouhk

@kkranen kkranen Awaiting requested review from kkranen

@jthomson04 jthomson04 Awaiting requested review from jthomson04

@PeaBrane PeaBrane Awaiting requested review from PeaBrane

@athreesh athreesh Awaiting requested review from athreesh

@nealvaidya nealvaidya Awaiting requested review from nealvaidya nealvaidya is a code owner

Assignees
No one assigned
Labels
feat size/XXL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@GuanLuo @grahamking @biswapanda @whoisj @indrajit96 @rmccorm4 @krishung5 @athreesh @atchernych