Skip to content

fix: Revert "feat: add vLLM v1 multi-modal example. Add llama4 Maverick ex… #2017

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 19, 2025
Merged

fix: Revert "feat: add vLLM v1 multi-modal example. Add llama4 Maverick ex… #2017

merged 1 commit into from
Jul 19, 2025

Conversation

alec-flowers
Copy link
Contributor

@alec-flowers alec-flowers commented Jul 18, 2025 •
edited by coderabbitai bot
Loading

This reverts commit 353146e.

This PR used an outdated UX version that we don't want to show to users anymore. It also switches the vLLM commit to be based on a fork of vLLM which we cannot have on main.

Summary by CodeRabbit

  • Chores

    • Removed all multimodal vLLM example files, including documentation, configuration files, utility scripts, component implementations, and pipeline definitions.
    • Updated Dockerfile to use the official vLLM repository for package installation.

  • Documentation

    • Deleted user guides and technical documentation related to multimodal deployment and RDMA data transfer.

End-users will no longer see or be able to use the multimodal vLLM examples or related documentation in the application.

@github-actions github-actions bot added the fix label Jul 18, 2025
@coderabbitai coderabbitai
Copy link
Contributor

coderabbitai bot commented Jul 18, 2025

Walkthrough

This change removes the entire multimodal vLLM example from the codebase, including all Python components, utility modules, configuration files, pipeline graphs, and documentation. Additionally, it updates the vLLM Dockerfile to clone from the official vLLM repository at a new commit, removing references to a previously used fork.

Changes

File(s) Change Summary
container/Dockerfile.vllm Updated to clone vLLM from the official repository at a new commit; removed fork references.
examples/multimodal_v1/components/*, examples/multimodal_v1/utils/*, ... Deleted all Python component, worker, processor, utility, and protocol files for multimodal vLLM.
examples/multimodal_v1/configs/* Deleted all YAML configuration files for multimodal model setups.
examples/multimodal_v1/graphs/* Deleted all pipeline graph definition files for multimodal examples.
examples/multimodal_v1/connect/* Deleted all RDMA connector implementation and documentation files.
examples/multimodal_v1/README.md, examples/multimodal_v1/backup_README.md Deleted all documentation and user guides for multimodal vLLM examples.

Sequence Diagram(s)

No sequence diagram generated as the change is a complete removal of features and documentation, not a modification of control flow.

Possibly related PRs

Poem

🐇
Farewell to multimodal dreams,
Configs and graphs, now gone it seems.
Workers and docs, swept away—
In Docker’s file, a cleaner way.
The codebase lighter, hops anew,
Here’s to fresh fields, and skies of blue!

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

coderabbitai[bot]
coderabbitai bot reviewed Jul 18, 2025
View reviewed changes
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
container/Dockerfile.vllm (1)

176-214: Factor out duplicated clone logic & use shallow clone

The identical git clone https://github.com/vllm-project/vllm.git appears in both the arm64 and amd64 branches. This duplicates ~40 lines and reruns clone twice, slowing builds and harming maintainability.

Proposed refactor (outline):

-ARG VLLM_REF="059d4cd..."
+# Also expose repo to keep URL in one place.
+ARG VLLM_REPO="https://github.com/vllm-project/vllm.git"
 
 ...
-    mkdir /opt/vllm && \
-    cd /opt/vllm && \
-    git clone https://github.com/vllm-project/vllm.git && \
-    cd vllm && \
-    git checkout $VLLM_REF && \
+    git clone --depth 1 --branch $VLLM_REF $VLLM_REPO /opt/vllm/vllm && \
+    cd /opt/vllm/vllm && \

Then delete the duplicate clone block in the else branch and directly jump to cd /opt/vllm/vllm && ….

Benefits:
• Single source-of-truth for the repo URL.
• Shallower clone reduces image size & build time.
• Easier to bump the ref in one place.
This is purely a Dockerfile re-ordering; functionality stays intact.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 343a481 and cf0022e.

📒 Files selected for processing (23)
  • container/Dockerfile.vllm (3 hunks)
  • examples/multimodal_v1/README.md (0 hunks)
  • examples/multimodal_v1/backup_README.md (0 hunks)
  • examples/multimodal_v1/components/direct_processor.py (0 hunks)
  • examples/multimodal_v1/components/encode_worker.py (0 hunks)
  • examples/multimodal_v1/components/processor.py (0 hunks)
  • examples/multimodal_v1/components/web.py (0 hunks)
  • examples/multimodal_v1/components/worker.py (0 hunks)
  • examples/multimodal_v1/configs/agg-llava.yaml (0 hunks)
  • examples/multimodal_v1/configs/agg-phi3v.yaml (0 hunks)
  • examples/multimodal_v1/configs/agg-qwen.yaml (0 hunks)
  • examples/multimodal_v1/configs/disagg.yaml (0 hunks)
  • examples/multimodal_v1/configs/llama.yaml (0 hunks)
  • examples/multimodal_v1/connect/README.md (0 hunks)
  • examples/multimodal_v1/connect/__init__.py (0 hunks)
  • examples/multimodal_v1/graphs/agg.py (0 hunks)
  • examples/multimodal_v1/graphs/disagg.py (0 hunks)
  • examples/multimodal_v1/utils/args.py (0 hunks)
  • examples/multimodal_v1/utils/chat_processor.py (0 hunks)
  • examples/multimodal_v1/utils/image_loader.py (0 hunks)
  • examples/multimodal_v1/utils/logging.py (0 hunks)
  • examples/multimodal_v1/utils/model.py (0 hunks)
  • examples/multimodal_v1/utils/protocol.py (0 hunks)
💤 Files with no reviewable changes (22)
  • examples/multimodal_v1/graphs/disagg.py
  • examples/multimodal_v1/graphs/agg.py
  • examples/multimodal_v1/configs/agg-qwen.yaml
  • examples/multimodal_v1/utils/logging.py
  • examples/multimodal_v1/configs/disagg.yaml
  • examples/multimodal_v1/README.md
  • examples/multimodal_v1/utils/args.py
  • examples/multimodal_v1/configs/llama.yaml
  • examples/multimodal_v1/components/web.py
  • examples/multimodal_v1/configs/agg-llava.yaml
  • examples/multimodal_v1/utils/image_loader.py
  • examples/multimodal_v1/backup_README.md
  • examples/multimodal_v1/components/encode_worker.py
  • examples/multimodal_v1/configs/agg-phi3v.yaml
  • examples/multimodal_v1/utils/model.py
  • examples/multimodal_v1/components/direct_processor.py
  • examples/multimodal_v1/components/processor.py
  • examples/multimodal_v1/connect/README.md
  • examples/multimodal_v1/utils/chat_processor.py
  • examples/multimodal_v1/connect/init.py
  • examples/multimodal_v1/components/worker.py
  • examples/multimodal_v1/utils/protocol.py
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings 
Learnt from: biswapanda
PR: ai-dynamo/dynamo#1412
File: lib/bindings/python/src/dynamo/runtime/logging.py:100-100
Timestamp: 2025-06-06T21:48:35.214Z
Learning: In the Dynamo codebase, BentoML has been completely removed from all executable code, with only documentation and attribution references remaining. The error_loggers configuration in lib/bindings/python/src/dynamo/runtime/logging.py should not include "bentoml" since those modules no longer exist.

Learnt from: krishung5
PR: ai-dynamo/dynamo#1388
File: examples/multimodal/utils/model.py:47-53
Timestamp: 2025-06-09T17:52:06.761Z
Learning: The current get_vision_embeddings_size() function in examples/multimodal/utils/model.py uses a hardcoded fallback of 4096 for hidden_size, which assumes all VLMs follow the LLaVA architecture pattern. This is problematic because different VLMs like Qwen2-VL, MiniCPM-V, and others have different hidden dimensions, making the fallback unreliable and potentially causing tensor shape mismatches.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Build and Test - vllm

@alec-flowers alec-flowers enabled auto-merge (squash) July 19, 2025 00:12
@alec-flowers alec-flowers merged commit fe63c17 into main Jul 19, 2025
11 of 12 checks passed
@alec-flowers alec-flowers deleted the aflowers/revert-multimodal-commit branch July 19, 2025 00:45
ZichengMa added a commit that referenced this pull request Jul 21, 2025
@ZichengMa
Squashed commit of the following:
dd0fa85 
commit cb6de94
Author: ptarasiewiczNV <104908264+ptarasiewiczNV@users.noreply.github.com>
Date:   Sun Jul 20 22:34:50 2025 +0200

    chore: Install vLLM and WideEP kernels in vLLM runtime container (#2010)

    Signed-off-by: Alec <35311602+alec-flowers@users.noreply.github.com>
    Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>
    Co-authored-by: alec-flowers <aflowers@nvidia.com>

commit fe63c17
Author: Alec <35311602+alec-flowers@users.noreply.github.com>
Date:   Fri Jul 18 17:45:08 2025 -0700

    fix: Revert "feat: add vLLM v1 multi-modal example. Add llama4 Maverick ex… (#2017)

commit bf1998f
Author: jthomson04 <jwillthomson19@gmail.com>
Date:   Fri Jul 18 17:23:50 2025 -0700

    fix: Don't detokenize twice in TRT-LLM examples (#1955)

commit 343a481
Author: Ryan Olson <ryanolson@users.noreply.github.com>
Date:   Fri Jul 18 16:22:43 2025 -0600

    feat: http disconnects (#2014)

commit e330d96
Author: Yan Ru Pei <yanrpei@gmail.com>
Date:   Fri Jul 18 13:40:54 2025 -0700

    feat: enable / disable chunked prefill for mockers (#2015)

    Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
    Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

commit 353146e
Author: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Date:   Fri Jul 18 13:33:36 2025 -0700

    feat: add vLLM v1 multi-modal example. Add llama4 Maverick example (#1990)

    Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
    Co-authored-by: krishung5 <krish@nvidia.com>

commit 1f07dab
Author: Jacky <18255193+kthui@users.noreply.github.com>
Date:   Fri Jul 18 13:04:20 2025 -0700

    feat: Add migration to LLM requests (#1930)

commit 5f17918
Author: Tanmay Verma <tanmayv@nvidia.com>
Date:   Fri Jul 18 12:59:34 2025 -0700

    refactor: Migrate to new UX2 for python launch (#2003)

commit fc12436
Author: Graham King <grahamk@nvidia.com>
Date:   Fri Jul 18 14:52:57 2025 -0400

    feat(frontend): router-mode settings (#2001)

commit dc75cf1
Author: ptarasiewiczNV <104908264+ptarasiewiczNV@users.noreply.github.com>
Date:   Fri Jul 18 18:47:28 2025 +0200

    chore: Move NIXL repo clone to Dockerfiles (#2009)

commit f6f392c
Author: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>
Date:   Thu Jul 17 18:44:17 2025 -0700

    Remove link to the fix for disagg + eagle3 for TRT-LLM example (#2006)

    Signed-off-by: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>

commit cc90ca6
Author: atchernych <atchernych@nvidia.com>
Date:   Thu Jul 17 18:34:40 2025 -0700

    feat: Create a convenience script to uninstall Dynamo Deploy CRDs (#1933)

commit 267b422
Author: Greg Clark <grclark@nvidia.com>
Date:   Thu Jul 17 20:44:21 2025 -0400

    chore: loosed python requirement versions (#1998)

    Signed-off-by: Greg Clark <grclark@nvidia.com>

commit b8474e5
Author: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Date:   Thu Jul 17 16:35:05 2025 -0700

    chore: update cmake and gap installation and sgl in wideep container (#1991)

commit 157a3b0
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Thu Jul 17 15:38:12 2025 -0700

    fix: incorrect helm upgrade command (#2000)

commit 0dfca2c
Author: Ryan McCormick <rmccormick@nvidia.com>
Date:   Thu Jul 17 15:33:33 2025 -0700

    ci: Update trtllm gitlab triggers for new components directory and test script (#1992)

commit f3fb09e
Author: Kris Hung <krish@nvidia.com>
Date:   Thu Jul 17 14:59:59 2025 -0700

    fix: Fix syntax for tokio-console (#1997)

commit dacffb8
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Thu Jul 17 14:57:10 2025 -0700

    fix: use non-dev golang image for operator (#1993)

commit 2b29a0a
Author: zaristei <zaristei@berkeley.edu>
Date:   Thu Jul 17 13:10:42 2025 -0700

    fix: Working Arm Build Dockerfile for Vllm_v1 (#1844)

commit 2430d89
Author: Ryan McCormick <rmccormick@nvidia.com>
Date:   Thu Jul 17 12:57:46 2025 -0700

    test: Add trtllm kv router tests (#1988)

commit 1eadc01
Author: Graham King <grahamk@nvidia.com>
Date:   Thu Jul 17 15:07:41 2025 -0400

    feat(runtime): Support tokio-console (#1986)

commit b62e633
Author: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Date:   Thu Jul 17 11:16:28 2025 -0700

    feat: support separate chat_template.jinja file (#1853)

commit 8ae3719
Author: Hongkuan Zhou <tedzhouhk@gmail.com>
Date:   Thu Jul 17 11:12:35 2025 -0700

    chore: add some details to dynamo deploy quickstart and fix deploy.sh (#1978)

    Signed-off-by: Hongkuan Zhou <tedzhouhk@gmail.com>
    Co-authored-by: julienmancuso <161955438+julienmancuso@users.noreply.github.com>

commit 08891ff
Author: Ryan McCormick <rmccormick@nvidia.com>
Date:   Thu Jul 17 10:57:42 2025 -0700

    fix: Update trtllm tests to use new scripts instead of dynamo serve (#1979)

commit 49b7a0d
Author: Ryan Olson <ryanolson@users.noreply.github.com>
Date:   Thu Jul 17 08:35:04 2025 -0600

    feat: record + analyze logprobs (#1957)

commit 6d2be14
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Thu Jul 17 00:17:58 2025 -0700

    refactor: replace vllm with vllm_v1 container (#1953)

    Co-authored-by: alec-flowers <aflowers@nvidia.com>

commit 4d2a31a
Author: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Date:   Wed Jul 16 18:04:09 2025 -0700

    chore: add port reservation to utils (#1980)

commit 1e3e4a0
Author: Alec <35311602+alec-flowers@users.noreply.github.com>
Date:   Wed Jul 16 15:54:04 2025 -0700

    fix: port race condition through deterministic ports (#1937)

commit 4ad281f
Author: Tanmay Verma <tanmayv@nvidia.com>
Date:   Wed Jul 16 14:33:51 2025 -0700

    refactor: Move TRTLLM example to the component/backends (#1976)

commit 57d24a1
Author: Misha Chornyi <99709299+mc-nv@users.noreply.github.com>
Date:   Wed Jul 16 14:10:24 2025 -0700

    build: Removing shell configuration violations. It's bad practice to hardcod… (#1973)

commit 182d3b5
Author: Graham King <grahamk@nvidia.com>
Date:   Wed Jul 16 16:12:40 2025 -0400

    chore(bindings): Remove mistralrs / llama.cpp (#1970)

commit def6eaa
Author: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>
Date:   Wed Jul 16 15:50:23 2025 -0400

    feat: attributions for debian deps of sglang, trtllm, vllm runtime containers (#1971)

commit f31732a
Author: Yan Ru Pei <yanrpei@gmail.com>
Date:   Wed Jul 16 11:22:15 2025 -0700

    feat: integrate mocker with dynamo-run and python cli (#1927)

commit aba6099
Author: Graham King <grahamk@nvidia.com>
Date:   Wed Jul 16 12:26:32 2025 -0400

    perf(router): Remove lock from router hot path (#1963)

commit b212103
Author: Hongkuan Zhou <tedzhouhk@gmail.com>
Date:   Wed Jul 16 08:55:33 2025 -0700

    docs: add notes in docs to deprecate local connector (#1959)

commit 7b325ee
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Tue Jul 15 18:52:00 2025 -0700

    fix: vllm router examples (#1942)

commit a50be1a
Author: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Date:   Tue Jul 15 17:58:01 2025 -0700

    feat: update CODEOWNERS (#1926)

commit e260fdf
Author: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>
Date:   Tue Jul 15 18:49:21 2025 -0400

    feat: add bitnami helm chart attribution (#1943)

    Signed-off-by: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>
    Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

commit 1c03404
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Tue Jul 15 14:26:24 2025 -0700

    fix: update inference gateway deployment instructions (#1940)

commit 5ca570f
Author: Graham King <grahamk@nvidia.com>
Date:   Tue Jul 15 16:54:03 2025 -0400

    chore: Rename dynamo.ingress to dynamo.frontend (#1944)

commit 7b9182f
Author: Graham King <grahamk@nvidia.com>
Date:   Tue Jul 15 16:33:07 2025 -0400

    chore: Move examples/cli to lib/bindings/examples/cli (#1952)

commit 40d40dd
Author: Graham King <grahamk@nvidia.com>
Date:   Tue Jul 15 16:02:19 2025 -0400

    chore(multi-modal): Rename frontend.py to web.py (#1951)

commit a9e0891
Author: Ryan Olson <ryanolson@users.noreply.github.com>
Date:   Tue Jul 15 12:30:30 2025 -0600

    feat: adding http clients and recorded response stream (#1919)

commit 4128d58
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Tue Jul 15 10:30:47 2025 -0700

    feat: allow helm upgrade using deploy script (#1936)

commit 4da078b
Author: Graham King <grahamk@nvidia.com>
Date:   Tue Jul 15 12:57:38 2025 -0400

    fix: Remove OpenSSL dependency, use Rust TLS (#1945)

commit fc004d4
Author: jthomson04 <jwillthomson19@gmail.com>
Date:   Tue Jul 15 08:45:42 2025 -0700

    fix: Fix TRT-LLM container build when using a custom pip wheel (#1825)

commit 3c6fc6f
Author: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Date:   Mon Jul 14 22:35:20 2025 -0700

    chore: fix typo (#1938)

commit de7fe38
Author: Alec <35311602+alec-flowers@users.noreply.github.com>
Date:   Mon Jul 14 21:47:12 2025 -0700

    feat: add vllm e2e integration tests (#1935)

commit 860f3f7
Author: Keiven C <213854356+keivenchang@users.noreply.github.com>
Date:   Mon Jul 14 21:44:19 2025 -0700

    chore: metrics endpoint variables renamed from HTTP_SERVER->SYSTEM (#1934)

    Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>

commit fc402a3
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Mon Jul 14 21:21:20 2025 -0700

    feat: configurable namespace for vllm v1 example (#1909)

commit df40d2c
Author: ZichengMa <zichengma1225@gmail.com>
Date:   Mon Jul 14 21:11:29 2025 -0700

    docs: fix typo and add mount-workspace to vllm doc (#1931)

    Signed-off-by: ZichengMa <zichengma1225@gmail.com>
    Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>

commit 901715b
Author: Tanmay Verma <tanmayv@nvidia.com>
Date:   Mon Jul 14 20:14:51 2025 -0700

    refactor:  Refactor the TRTLLM examples remove dynamo SDK (#1884)

commit 5bf23d5
Author: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Date:   Mon Jul 14 18:29:19 2025 -0700

    feat: update DynamoGraphDeployments for vllm_v1 (#1890)

    Co-authored-by: mohammedabdulwahhab <furkhan324@berkeley.edu>

commit 9e76590
Author: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Date:   Mon Jul 14 17:29:56 2025 -0700

    docs: organize sglang readme (#1910)

commit ef59ac8
Author: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
Date:   Mon Jul 14 16:16:44 2025 -0700

    docs: TRTLLM Example of Llama4+Eagle3 (Speculative Decoding) (#1828)

    Signed-off-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
    Co-authored-by: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>

commit 053041e
Author: Jorge António <matroid@outlook.com>
Date:   Tue Jul 15 00:06:38 2025 +0100

    fix: resolve incorrect finish reason propagation (#1857)

commit 3733f58
Author: Graham King <grahamk@nvidia.com>
Date:   Mon Jul 14 19:04:22 2025 -0400

    feat(backends): Python llama.cpp engine (#1925)

commit 6a1350c
Author: Tushar Sharma <tusharma@nvidia.com>
Date:   Mon Jul 14 14:56:36 2025 -0700

    build: minor improvements to sglang dockerfile (#1917)

commit e2a619b
Author: Neelay Shah <neelays@nvidia.com>
Date:   Mon Jul 14 14:52:53 2025 -0700

    fix: remove environment variable passing (#1911)

    Signed-off-by: Neelay Shah <neelays@nvidia.com>
    Co-authored-by: Neelay Shah <neelays@a4u8g-0057.ipp2u2.colossus.nvidia.com>

commit 3d17a49
Author: Schwinn Saereesitthipitak <17022745+galletas1712@users.noreply.github.com>
Date:   Mon Jul 14 14:41:56 2025 -0700

    refactor: remove dynamo build (#1778)

    Signed-off-by: Schwinn Saereesitthipitak <17022745+galletas1712@users.noreply.github.com>

commit 3e0cb07
Author: Anant Sharma <anants@nvidia.com>
Date:   Mon Jul 14 15:43:48 2025 -0400

    fix: copy attributions and license to trtllm runtime container (#1916)

commit fc36bf5
Author: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Date:   Mon Jul 14 12:31:49 2025 -0700

    feat: receive kvmetrics from sglang scheduler (#1789)

    Co-authored-by: zixuanzhang226 <zixuanzhang@bytedance.com>

commit df91fce
Author: Yan Ru Pei <yanrpei@gmail.com>
Date:   Mon Jul 14 12:24:04 2025 -0700

    feat: prefill aware routing (#1895)

commit ad8ad66
Author: Graham King <grahamk@nvidia.com>
Date:   Mon Jul 14 15:20:35 2025 -0400

    feat: Shrink the ai-dynamo wheel by 35 MiB (#1918)

    Remove http and llmctl binaries. They have been unused for a while.

commit 480b41d
Author: Graham King <grahamk@nvidia.com>
Date:   Mon Jul 14 15:06:45 2025 -0400

    feat: Python frontend / ingress node (#1912)
ZichengMa added a commit that referenced this pull request Jul 21, 2025
@ZichengMa
Squashed commit of the following:
537e633 
commit d4b5414
Author: atchernych <atchernych@nvidia.com>
Date:   Mon Jul 21 13:10:24 2025 -0700

    fix: mypy error (#2029)

commit 79337c7
Author: Ryan McCormick <rmccormick@nvidia.com>
Date:   Mon Jul 21 12:12:16 2025 -0700

    build: support custom TRTLLM build for commits not on main branch (#2021)

commit 95dd942
Author: atchernych <atchernych@nvidia.com>
Date:   Mon Jul 21 12:09:33 2025 -0700

    docs: Post-Merge cleanup of the deploy documentation (#1922)

commit cb6de94
Author: ptarasiewiczNV <104908264+ptarasiewiczNV@users.noreply.github.com>
Date:   Sun Jul 20 22:34:50 2025 +0200

    chore: Install vLLM and WideEP kernels in vLLM runtime container (#2010)

    Signed-off-by: Alec <35311602+alec-flowers@users.noreply.github.com>
    Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>
    Co-authored-by: alec-flowers <aflowers@nvidia.com>

commit fe63c17
Author: Alec <35311602+alec-flowers@users.noreply.github.com>
Date:   Fri Jul 18 17:45:08 2025 -0700

    fix: Revert "feat: add vLLM v1 multi-modal example. Add llama4 Maverick ex… (#2017)

commit bf1998f
Author: jthomson04 <jwillthomson19@gmail.com>
Date:   Fri Jul 18 17:23:50 2025 -0700

    fix: Don't detokenize twice in TRT-LLM examples (#1955)

commit 343a481
Author: Ryan Olson <ryanolson@users.noreply.github.com>
Date:   Fri Jul 18 16:22:43 2025 -0600

    feat: http disconnects (#2014)

commit e330d96
Author: Yan Ru Pei <yanrpei@gmail.com>
Date:   Fri Jul 18 13:40:54 2025 -0700

    feat: enable / disable chunked prefill for mockers (#2015)

    Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
    Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

commit 353146e
Author: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Date:   Fri Jul 18 13:33:36 2025 -0700

    feat: add vLLM v1 multi-modal example. Add llama4 Maverick example (#1990)

    Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
    Co-authored-by: krishung5 <krish@nvidia.com>

commit 1f07dab
Author: Jacky <18255193+kthui@users.noreply.github.com>
Date:   Fri Jul 18 13:04:20 2025 -0700

    feat: Add migration to LLM requests (#1930)

commit 5f17918
Author: Tanmay Verma <tanmayv@nvidia.com>
Date:   Fri Jul 18 12:59:34 2025 -0700

    refactor: Migrate to new UX2 for python launch (#2003)

commit fc12436
Author: Graham King <grahamk@nvidia.com>
Date:   Fri Jul 18 14:52:57 2025 -0400

    feat(frontend): router-mode settings (#2001)

commit dc75cf1
Author: ptarasiewiczNV <104908264+ptarasiewiczNV@users.noreply.github.com>
Date:   Fri Jul 18 18:47:28 2025 +0200

    chore: Move NIXL repo clone to Dockerfiles (#2009)

commit f6f392c
Author: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>
Date:   Thu Jul 17 18:44:17 2025 -0700

    Remove link to the fix for disagg + eagle3 for TRT-LLM example (#2006)

    Signed-off-by: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>

commit cc90ca6
Author: atchernych <atchernych@nvidia.com>
Date:   Thu Jul 17 18:34:40 2025 -0700

    feat: Create a convenience script to uninstall Dynamo Deploy CRDs (#1933)

commit 267b422
Author: Greg Clark <grclark@nvidia.com>
Date:   Thu Jul 17 20:44:21 2025 -0400

    chore: loosed python requirement versions (#1998)

    Signed-off-by: Greg Clark <grclark@nvidia.com>

commit b8474e5
Author: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Date:   Thu Jul 17 16:35:05 2025 -0700

    chore: update cmake and gap installation and sgl in wideep container (#1991)

commit 157a3b0
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Thu Jul 17 15:38:12 2025 -0700

    fix: incorrect helm upgrade command (#2000)

commit 0dfca2c
Author: Ryan McCormick <rmccormick@nvidia.com>
Date:   Thu Jul 17 15:33:33 2025 -0700

    ci: Update trtllm gitlab triggers for new components directory and test script (#1992)

commit f3fb09e
Author: Kris Hung <krish@nvidia.com>
Date:   Thu Jul 17 14:59:59 2025 -0700

    fix: Fix syntax for tokio-console (#1997)

commit dacffb8
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Thu Jul 17 14:57:10 2025 -0700

    fix: use non-dev golang image for operator (#1993)

commit 2b29a0a
Author: zaristei <zaristei@berkeley.edu>
Date:   Thu Jul 17 13:10:42 2025 -0700

    fix: Working Arm Build Dockerfile for Vllm_v1 (#1844)

commit 2430d89
Author: Ryan McCormick <rmccormick@nvidia.com>
Date:   Thu Jul 17 12:57:46 2025 -0700

    test: Add trtllm kv router tests (#1988)

commit 1eadc01
Author: Graham King <grahamk@nvidia.com>
Date:   Thu Jul 17 15:07:41 2025 -0400

    feat(runtime): Support tokio-console (#1986)

commit b62e633
Author: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Date:   Thu Jul 17 11:16:28 2025 -0700

    feat: support separate chat_template.jinja file (#1853)

commit 8ae3719
Author: Hongkuan Zhou <tedzhouhk@gmail.com>
Date:   Thu Jul 17 11:12:35 2025 -0700

    chore: add some details to dynamo deploy quickstart and fix deploy.sh (#1978)

    Signed-off-by: Hongkuan Zhou <tedzhouhk@gmail.com>
    Co-authored-by: julienmancuso <161955438+julienmancuso@users.noreply.github.com>

commit 08891ff
Author: Ryan McCormick <rmccormick@nvidia.com>
Date:   Thu Jul 17 10:57:42 2025 -0700

    fix: Update trtllm tests to use new scripts instead of dynamo serve (#1979)

commit 49b7a0d
Author: Ryan Olson <ryanolson@users.noreply.github.com>
Date:   Thu Jul 17 08:35:04 2025 -0600

    feat: record + analyze logprobs (#1957)

commit 6d2be14
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Thu Jul 17 00:17:58 2025 -0700

    refactor: replace vllm with vllm_v1 container (#1953)

    Co-authored-by: alec-flowers <aflowers@nvidia.com>

commit 4d2a31a
Author: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Date:   Wed Jul 16 18:04:09 2025 -0700

    chore: add port reservation to utils (#1980)

commit 1e3e4a0
Author: Alec <35311602+alec-flowers@users.noreply.github.com>
Date:   Wed Jul 16 15:54:04 2025 -0700

    fix: port race condition through deterministic ports (#1937)

commit 4ad281f
Author: Tanmay Verma <tanmayv@nvidia.com>
Date:   Wed Jul 16 14:33:51 2025 -0700

    refactor: Move TRTLLM example to the component/backends (#1976)

commit 57d24a1
Author: Misha Chornyi <99709299+mc-nv@users.noreply.github.com>
Date:   Wed Jul 16 14:10:24 2025 -0700

    build: Removing shell configuration violations. It's bad practice to hardcod… (#1973)

commit 182d3b5
Author: Graham King <grahamk@nvidia.com>
Date:   Wed Jul 16 16:12:40 2025 -0400

    chore(bindings): Remove mistralrs / llama.cpp (#1970)

commit def6eaa
Author: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>
Date:   Wed Jul 16 15:50:23 2025 -0400

    feat: attributions for debian deps of sglang, trtllm, vllm runtime containers (#1971)

commit f31732a
Author: Yan Ru Pei <yanrpei@gmail.com>
Date:   Wed Jul 16 11:22:15 2025 -0700

    feat: integrate mocker with dynamo-run and python cli (#1927)

commit aba6099
Author: Graham King <grahamk@nvidia.com>
Date:   Wed Jul 16 12:26:32 2025 -0400

    perf(router): Remove lock from router hot path (#1963)

commit b212103
Author: Hongkuan Zhou <tedzhouhk@gmail.com>
Date:   Wed Jul 16 08:55:33 2025 -0700

    docs: add notes in docs to deprecate local connector (#1959)

commit 7b325ee
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Tue Jul 15 18:52:00 2025 -0700

    fix: vllm router examples (#1942)

commit a50be1a
Author: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Date:   Tue Jul 15 17:58:01 2025 -0700

    feat: update CODEOWNERS (#1926)

commit e260fdf
Author: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>
Date:   Tue Jul 15 18:49:21 2025 -0400

    feat: add bitnami helm chart attribution (#1943)

    Signed-off-by: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>
    Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

commit 1c03404
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Tue Jul 15 14:26:24 2025 -0700

    fix: update inference gateway deployment instructions (#1940)

commit 5ca570f
Author: Graham King <grahamk@nvidia.com>
Date:   Tue Jul 15 16:54:03 2025 -0400

    chore: Rename dynamo.ingress to dynamo.frontend (#1944)

commit 7b9182f
Author: Graham King <grahamk@nvidia.com>
Date:   Tue Jul 15 16:33:07 2025 -0400

    chore: Move examples/cli to lib/bindings/examples/cli (#1952)

commit 40d40dd
Author: Graham King <grahamk@nvidia.com>
Date:   Tue Jul 15 16:02:19 2025 -0400

    chore(multi-modal): Rename frontend.py to web.py (#1951)

commit a9e0891
Author: Ryan Olson <ryanolson@users.noreply.github.com>
Date:   Tue Jul 15 12:30:30 2025 -0600

    feat: adding http clients and recorded response stream (#1919)

commit 4128d58
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Tue Jul 15 10:30:47 2025 -0700

    feat: allow helm upgrade using deploy script (#1936)

commit 4da078b
Author: Graham King <grahamk@nvidia.com>
Date:   Tue Jul 15 12:57:38 2025 -0400

    fix: Remove OpenSSL dependency, use Rust TLS (#1945)

commit fc004d4
Author: jthomson04 <jwillthomson19@gmail.com>
Date:   Tue Jul 15 08:45:42 2025 -0700

    fix: Fix TRT-LLM container build when using a custom pip wheel (#1825)

commit 3c6fc6f
Author: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Date:   Mon Jul 14 22:35:20 2025 -0700

    chore: fix typo (#1938)

commit de7fe38
Author: Alec <35311602+alec-flowers@users.noreply.github.com>
Date:   Mon Jul 14 21:47:12 2025 -0700

    feat: add vllm e2e integration tests (#1935)

commit 860f3f7
Author: Keiven C <213854356+keivenchang@users.noreply.github.com>
Date:   Mon Jul 14 21:44:19 2025 -0700

    chore: metrics endpoint variables renamed from HTTP_SERVER->SYSTEM (#1934)

    Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>

commit fc402a3
Author: Biswa Panda <biswa.panda@gmail.com>
Date:   Mon Jul 14 21:21:20 2025 -0700

    feat: configurable namespace for vllm v1 example (#1909)

commit df40d2c
Author: ZichengMa <zichengma1225@gmail.com>
Date:   Mon Jul 14 21:11:29 2025 -0700

    docs: fix typo and add mount-workspace to vllm doc (#1931)

    Signed-off-by: ZichengMa <zichengma1225@gmail.com>
    Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>

commit 901715b
Author: Tanmay Verma <tanmayv@nvidia.com>
Date:   Mon Jul 14 20:14:51 2025 -0700

    refactor:  Refactor the TRTLLM examples remove dynamo SDK (#1884)

commit 5bf23d5
Author: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Date:   Mon Jul 14 18:29:19 2025 -0700

    feat: update DynamoGraphDeployments for vllm_v1 (#1890)

    Co-authored-by: mohammedabdulwahhab <furkhan324@berkeley.edu>

commit 9e76590
Author: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Date:   Mon Jul 14 17:29:56 2025 -0700

    docs: organize sglang readme (#1910)

commit ef59ac8
Author: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
Date:   Mon Jul 14 16:16:44 2025 -0700

    docs: TRTLLM Example of Llama4+Eagle3 (Speculative Decoding) (#1828)

    Signed-off-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
    Co-authored-by: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>

commit 053041e
Author: Jorge António <matroid@outlook.com>
Date:   Tue Jul 15 00:06:38 2025 +0100

    fix: resolve incorrect finish reason propagation (#1857)

commit 3733f58
Author: Graham King <grahamk@nvidia.com>
Date:   Mon Jul 14 19:04:22 2025 -0400

    feat(backends): Python llama.cpp engine (#1925)

commit 6a1350c
Author: Tushar Sharma <tusharma@nvidia.com>
Date:   Mon Jul 14 14:56:36 2025 -0700

    build: minor improvements to sglang dockerfile (#1917)

commit e2a619b
Author: Neelay Shah <neelays@nvidia.com>
Date:   Mon Jul 14 14:52:53 2025 -0700

    fix: remove environment variable passing (#1911)

    Signed-off-by: Neelay Shah <neelays@nvidia.com>
    Co-authored-by: Neelay Shah <neelays@a4u8g-0057.ipp2u2.colossus.nvidia.com>

commit 3d17a49
Author: Schwinn Saereesitthipitak <17022745+galletas1712@users.noreply.github.com>
Date:   Mon Jul 14 14:41:56 2025 -0700

    refactor: remove dynamo build (#1778)

    Signed-off-by: Schwinn Saereesitthipitak <17022745+galletas1712@users.noreply.github.com>

commit 3e0cb07
Author: Anant Sharma <anants@nvidia.com>
Date:   Mon Jul 14 15:43:48 2025 -0400

    fix: copy attributions and license to trtllm runtime container (#1916)

commit fc36bf5
Author: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Date:   Mon Jul 14 12:31:49 2025 -0700

    feat: receive kvmetrics from sglang scheduler (#1789)

    Co-authored-by: zixuanzhang226 <zixuanzhang@bytedance.com>

commit df91fce
Author: Yan Ru Pei <yanrpei@gmail.com>
Date:   Mon Jul 14 12:24:04 2025 -0700

    feat: prefill aware routing (#1895)

commit ad8ad66
Author: Graham King <grahamk@nvidia.com>
Date:   Mon Jul 14 15:20:35 2025 -0400

    feat: Shrink the ai-dynamo wheel by 35 MiB (#1918)

    Remove http and llmctl binaries. They have been unused for a while.

commit 480b41d
Author: Graham King <grahamk@nvidia.com>
Date:   Mon Jul 14 15:06:45 2025 -0400

    feat: Python frontend / ingress node (#1912)
@coderabbitai coderabbitai bot mentioned this pull request Jul 24, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments

@GuanLuo GuanLuo GuanLuo approved these changes

@ishandhanani ishandhanani ishandhanani approved these changes

@nnshah1 nnshah1 Awaiting requested review from nnshah1 nnshah1 is a code owner

@whoisj whoisj Awaiting requested review from whoisj whoisj is a code owner

@rmccorm4 rmccorm4 Awaiting requested review from rmccorm4 rmccorm4 is a code owner

@tanmayv25 tanmayv25 Awaiting requested review from tanmayv25 tanmayv25 is a code owner

@richardhuo-nv richardhuo-nv Awaiting requested review from richardhuo-nv richardhuo-nv is a code owner

@ptarasiewiczNV ptarasiewiczNV Awaiting requested review from ptarasiewiczNV ptarasiewiczNV is a code owner

Assignees
No one assigned
Labels
fix size/XXL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alec-flowers @GuanLuo @ishandhanani