Skip to content

Conversation

haydn-jones
Copy link
Contributor

@haydn-jones haydn-jones commented Aug 4, 2025

Fixes #287

Changes proposed in this pull request:

  • Forward unknown command line arguments to vLLM.

This might be a bit of a silly approach, but theres a ton of arguments for vLLM, and patching in argument by argument as a new one is needed seems unnecessary, so I'm just forwarding any unknown argument to vLLM. Happy to hear alternatives.

I was testing OlmOCR + vLLM 0.10.0 with data parallelism on some B200s as 0.10.0 has improved support for Blackwell, but I was running into issues with the multimodal input cache so I needed to add --disable-mm-preprocessor-cache (does solve the issue), this allows me to do it from the command line.

Edit: I also needed to increase the wait time for vLLM startup, but I didn't add that here. Reasonable to add as a command line arg?

Before submitting

  • I've read and followed all steps in the Making a pull request
    section of the CONTRIBUTING docs.
  • I've updated or added any relevant docstrings following the syntax described in the
    Writing docstrings section of the CONTRIBUTING docs.
  • If this PR fixes a bug, I've added a test that will fail without my fix.
  • If this PR adds a new feature, I've added tests that sufficiently cover my new functionality.

@jakep-allenai
Copy link
Collaborator

Hey, yeah, I like this approach, I didn't know argparse could support it so cleanly.

@jakep-allenai jakep-allenai merged commit 5e991b6 into allenai:main Aug 4, 2025
1 check passed
@jakep-allenai
Copy link
Collaborator

As for the extra delay time, yes, if you want to make a new PR, I'll approve that one as well.

@jakep-allenai
Copy link
Collaborator

And it's merged and built into v0.2.3 now, ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add --enforce-eager to vllm passthrough args in pipeline.py
2 participants