Skip to content

Conversation

yinghai
Copy link
Contributor

@yinghai yinghai commented Mar 16, 2025

Motivation

Improvement of sgl-router to avoid hang.

Modifications

  • Fix sgl-router side of http payload handling to drain the payload when
    we hit early error. This covers the scenario of 404, and 413 large
    payload for json scenario.
  • Align sglang client/server keepalive expectations to avoid sgl-router client hang

Checklist

Copy link
Collaborator

@ByronHsu ByronHsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@ByronHsu
Copy link
Collaborator

I have opened a PR to fix the CI. There is some interface change. #4483

@ByronHsu ByronHsu merged commit f4d7ab7 into sgl-project:main Mar 17, 2025
3 of 4 checks passed
@yinghai yinghai deleted the yinghai/s3 branch March 17, 2025 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants