Skip to content

Support limiting max loaded loras in CPU. #8650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Aug 3, 2025
Merged

Support limiting max loaded loras in CPU. #8650

merged 4 commits into from
Aug 3, 2025

Conversation

lifuhuang
Copy link
Collaborator

Motivation

Allowing customer to limit the maximal # of LoRAs loaded in CPU.

Modifications

Accuracy Test

Benchmark & Profiling

Checklist

@gemini-code-assist Gemini Code Assist
Copy link
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@lifuhuang lifuhuang mentioned this pull request Aug 2, 2025
Merged
Fridge003
Fridge003 approved these changes Aug 3, 2025
View reviewed changes
Copy link
Collaborator

@Fridge003 Fridge003 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhyncs zhyncs merged commit 8675bdf into main Aug 3, 2025
121 of 128 checks passed
@zhyncs zhyncs deleted the lifuhuang/max-lora branch August 3, 2025 07:02
lifuhuang added a commit that referenced this pull request Aug 3, 2025
@lifuhuang
Support limiting max loaded loras in CPU. (#8650)
76e3baf
htiennv pushed a commit to htiennv/sglang that referenced this pull request Aug 5, 2025
@lifuhuang @htiennv
Support limiting max loaded loras in CPU. (sgl-project#8650)
49d12d2
ShangmingCai pushed a commit that referenced this pull request Aug 5, 2025
@lifuhuang @ssssnow
Support limiting max loaded loras in CPU. (#8650)
e638d5e
ShangmingCai pushed a commit that referenced this pull request Aug 5, 2025
@lifuhuang @ssssnow
Support limiting max loaded loras in CPU. (#8650)
66b7c53
narutolhy pushed a commit to narutolhy/sglang that referenced this pull request Aug 17, 2025
@lifuhuang @narutolhy
Support limiting max loaded loras in CPU. (sgl-project#8650)
3796078
narutolhy pushed a commit to narutolhy/sglang that referenced this pull request Aug 18, 2025
@lifuhuang @narutolhy
Support limiting max loaded loras in CPU. (sgl-project#8650)
8297839
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Fridge003 Fridge003 Fridge003 approved these changes

@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy is a code owner

@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 is a code owner

@hnyls2002 hnyls2002 Awaiting requested review from hnyls2002 hnyls2002 is a code owner

@xiezhq-hermann xiezhq-hermann Awaiting requested review from xiezhq-hermann xiezhq-hermann is a code owner

Assignees

@Fridge003 Fridge003

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lifuhuang @Fridge003 @zhyncs