requirements: Remove mamba_ssm from test.txt #20047

seemethere · 2025-06-25T00:04:47Z

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

This dependency wasn't actually getting used in the original PR (#14323) since most of the code was just lifted and shifted from the mamba repository and is now causing issues once we updated torch to 2.7.0.

Related issues / PRs:

There was even a workaround PR where we have to install this from source using --no-build-isolation here:

[CI/Build] workaround for CI build failure #17070

Test Plan

CI

Test Result

(Optional) Documentation Update

This dependency wasn't actually getting used and caused issues once we updated torch to 2.7.0. Signed-off-by: Eli Uriegas <eliuriegas@meta.com>

github-actions · 2025-06-25T00:04:56Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

gemini-code-assist

Summary of Changes

Hello @seemethere, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on streamlining the test environment by removing an unnecessary and problematic dependency. The mamba_ssm package, which was not actively used and caused conflicts with updated PyTorch versions, has been entirely removed from the test requirements, leading to a cleaner and more stable dependency tree for testing purposes.

Highlights

Dependency Removal: I've removed the mamba_ssm dependency from requirements/test.in. This dependency was identified as unused in a previous PR and was causing issues with torch==2.7.0, necessitating its removal to resolve build conflicts.
Transitive Dependency Cleanup: As a direct result of removing mamba_ssm from requirements/test.in, the generated requirements/test.txt has been automatically cleaned up. This includes the removal of mamba-ssm itself, its associated comments, and its transitive dependency ninja.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request removes the mamba_ssm dependency, which is reported as unused and is causing build issues with newer versions of torch. The changes are confined to the requirements files (test.in and the auto-generated test.txt), correctly removing the package and its transitive dependencies.

The change is clean and well-justified by the PR description. The test plan relies on CI, which is appropriate for this type of change. Assuming CI passes, this is a good cleanup and ready to merge.

Signed-off-by: Eli Uriegas <eliuriegas@meta.com>

seemethere · 2025-06-25T00:09:53Z

cc @Alnusjaponica for a review since you originally added it and to sanity check my assumption that this dependency is not used.

mgoin · 2025-06-25T03:50:05Z

cc @tlrmchlsmth

aarnphm · 2025-06-25T05:39:30Z

we vendored the mamba_ssm kernel onto vllm, so probably safe to remove this.

On a side note, do you know the timeline for supporting mamba in v1 🤔?

tdoublep · 2025-06-25T06:54:17Z

This requirement is there to support the plamo2 model:

vllm/requirements/test.in

Line 29 in ba7ba35

mamba_ssm # required for plamo2 test

This model has not yet been rebased against the Mamba2 implementation that is vendored within vLLM:

vllm/vllm/model_executor/models/plamo2.py

Lines 120 to 121 in ba7ba35

    
           class Plamo2MambaMixer(nn.Module): 
        
               # TODO(Shinichi): Rebase on Mamba2 implementation.

There is a test that runs, optionally, in the CI:

vllm/tests/models/language/generation/test_hybrid.py

Line 34 in ba7ba35

"pfnet/plamo-2-1b",

which will may no longer work if we remove the mamba_ssm package. Although I think transformers has a "slow" path that runs without installing this package, so the test could still work actually.

If we remove the mamba_ssm dependency, we should also remove the installation of causal_conv1d which is related to the same model.

vllm/tests/models/language/generation/test_hybrid.py

Lines 31 to 34 in ba7ba35

    
           # NOTE: Running Plamo2 in transformers implementation requires to install 
        
           # causal-conv1d package, which is not listed as a test dependency as it's 
        
           # not compatible with pip-compile. 
        
           "pfnet/plamo-2-1b",

The causal_conv1d is installed in the CI pipeline (rather than being an explicit dependency):

vllm/.buildkite/test-pipeline.yaml

Lines 519 to 520 in ba7ba35

    
           # Install causal-conv1d for plamo2 models here, as it is not compatible with pip-compile. 
        
           - pip install 'git+https://github.com/Dao-AILab/causal-conv1d@v1.5.0.post8'

Another option could be to install the mamba_ssm package in the CI pipeline in the same way that causal_conv1d is installed. That way it doesn't mess up local installations for people, but the model can still be tested?

It would be good to run the "Language Models Test (Extended Generation)" test in Buildkite to see if this change breaks anything. It would also be good to get the team who contributed the model to weigh in here cc @Alnusjaponica

seemethere · 2025-06-25T16:09:34Z

Yeah I think in particular I don't want to land this PR until we have a full CI run.

My main takeaway from grepping through the code is that we never directly import mamba_ssm hence the PR here.

aarnphm · 2025-06-25T16:43:04Z

Let me enable full CI run to see if there are any failure, but probably echo Thomas comment here.

tdoublep · 2025-06-27T12:40:25Z

It turns out that transformers has a hard dependency on this package when running Nemotron-H models. If you don't have it installed it throws:

    try:
        #from mamba_ssm.ops.triton.layernorm_gated import RMSNorm as RMSNormGated
>       from mamba_ssm.ops.triton.layernorm_gated import rmsnorm_fn
E       ModuleNotFoundError: No module named 'mamba_ssm'

../.cache/huggingface/modules/transformers_modules/nvidia/Nemotron-H-8B-Base-8K/281935db305672111f043428fe4982969876613c/modeling_nemotron_h.py:63: ModuleNotFoundError

While we don't have any tests that run for Nemotron-H yet, perhaps we might want to add some?

I think I would advocate for removing the package from the explicit dependencies but installing it inside the CI job (similar to how we handle causal_conv1d right now).

seemethere · 2025-06-27T22:52:27Z

It turns out that transformers has a hard dependency on this package when running Nemotron-H models. If you don't have it installed it throws:
    try:
        #from mamba_ssm.ops.triton.layernorm_gated import RMSNorm as RMSNormGated
>       from mamba_ssm.ops.triton.layernorm_gated import rmsnorm_fn
E       ModuleNotFoundError: No module named 'mamba_ssm'

../.cache/huggingface/modules/transformers_modules/nvidia/Nemotron-H-8B-Base-8K/281935db305672111f043428fe4982969876613c/modeling_nemotron_h.py:63: ModuleNotFoundError
While we don't have any tests that run for Nemotron-H yet, perhaps we might want to add some?

I think I would advocate for removing the package from the explicit dependencies but installing it inside the CI job (similar to how we handle causal_conv1d right now).

Okay sounds good! I'll update this PR to install it for CI similar to what we do for causal_conv1d

Alnusjaponica · 2025-06-30T10:55:35Z

Sorry for my delayed reply and any confusion.

Based on my understanding, mamba_ssm and causal_conv1d are dependencies only for unit tests, as shown here. The original transformers implementation also relies on these libraries.
I do not fully understand why mamba-ssm is included in the Docker image by #17070, but as long as the unit tests are not run with the plain image, they are unnecessary. In that context, we would appreciate it if you could maintain the unit tests by moving the installation step to the CI pipeline, as mentioned in this comment. If that doesn’t work, one possible workaround is to hard-code the expected inference results in the unit test instead of running the transformers implementation.

DarkLight1337 · 2025-07-25T13:51:04Z

Closing as superseded by #21421

requirements: Remove mamba_ssm from test.txt

0044bdf

This dependency wasn't actually getting used and caused issues once we updated torch to 2.7.0. Signed-off-by: Eli Uriegas <eliuriegas@meta.com>

gemini-code-assist bot reviewed Jun 25, 2025

View reviewed changes

mergify bot added the ci/build label Jun 25, 2025

gemini-code-assist bot reviewed Jun 25, 2025

View reviewed changes

also remove from dockerfile

9bd1ad5

Signed-off-by: Eli Uriegas <eliuriegas@meta.com>

aarnphm added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 25, 2025

tdoublep mentioned this pull request Jun 27, 2025

[Bugfix] Correct behavior of GraniteMoeHybrid for TensorParallel execution #20137

Merged

DarkLight1337 closed this Jul 25, 2025

Alnusjaponica mentioned this pull request Aug 8, 2025

Remove mamba-ssm package #22409

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

requirements: Remove mamba_ssm from test.txt #20047

requirements: Remove mamba_ssm from test.txt #20047

Uh oh!

seemethere commented Jun 25, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Jun 25, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

seemethere commented Jun 25, 2025

Uh oh!

mgoin commented Jun 25, 2025

Uh oh!

aarnphm commented Jun 25, 2025 •

edited

Loading

Uh oh!

tdoublep commented Jun 25, 2025

Uh oh!

seemethere commented Jun 25, 2025

Uh oh!

aarnphm commented Jun 25, 2025

Uh oh!

tdoublep commented Jun 27, 2025

Uh oh!

seemethere commented Jun 27, 2025

Uh oh!

Alnusjaponica commented Jun 30, 2025

Uh oh!

DarkLight1337 commented Jul 25, 2025

Uh oh!

Uh oh!

Uh oh!

requirements: Remove mamba_ssm from test.txt #20047

requirements: Remove mamba_ssm from test.txt #20047

Uh oh!

Conversation

seemethere commented Jun 25, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Essential Elements of an Effective PR Description Checklist

Purpose

Test Plan

Test Result

(Optional) Documentation Update

Uh oh!

github-actions bot commented Jun 25, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

seemethere commented Jun 25, 2025

Uh oh!

mgoin commented Jun 25, 2025

Uh oh!

aarnphm commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tdoublep commented Jun 25, 2025

Uh oh!

seemethere commented Jun 25, 2025

Uh oh!

aarnphm commented Jun 25, 2025

Uh oh!

tdoublep commented Jun 27, 2025

Uh oh!

seemethere commented Jun 27, 2025

Uh oh!

Alnusjaponica commented Jun 30, 2025

Uh oh!

DarkLight1337 commented Jul 25, 2025

Uh oh!

Uh oh!

seemethere commented Jun 25, 2025 •

edited by github-actions bot

Loading

aarnphm commented Jun 25, 2025 •

edited

Loading