fix high qps crash when enable mtp #3592

zhyncs · 2025-02-15T13:24:28Z

Motivation

Format your code according to the Code Formatting with Pre-Commit.
Add unit tests as outlined in the Running Unit Tests.
Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

Co-authored-by: ispobock <ispobaoke@hotmail.com>

zhyncs requested review from merrymercy, Ying1123, hnyls2002 and ispobock as code owners February 15, 2025 13:24

fix high qps crash when enable mtp

024d20c

Co-authored-by: ispobock <ispobaoke@hotmail.com>

zhyncs force-pushed the zhyncs/fix-mtp branch from 18bb08b to 024d20c Compare February 15, 2025 14:45

Merge branch 'main' into zhyncs/fix-mtp

94c011c

zhyncs merged commit dfce926 into main Feb 15, 2025
2 of 9 checks passed

zhyncs deleted the zhyncs/fix-mtp branch February 15, 2025 15:11