[sglang] refactor: Unify async rollout under SGLangRollout, and support sglang==0.4.6.post5 #1717

zyzshishui · 2025-05-27T08:24:28Z

Checklist Before Starting

Search for similar PR(s).

What does this PR do?

Unify the functionality of SGLangRollout and AsyncSGLangRollout, remove original SGLangRollout and rename AsyncSGLangRollout to SGLangRollout.
Make trivial changes due to modification in sglang==0.4.6.post5.

High-Level Design

Demonstrate the high-level design if this PR is complex.

Specific Changes

List the specific changes.

API

Demonstrate how the API changes if any.

Usage Example

Provide usage example(s) for easier usage.

# Add code snippet or script demonstrating how to use this

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluatuion results, etc.

Additional Info.

Issue Number: Fixes issue # or discussion # if any.
Training: [Note which backend this PR will affect: FSDP, Megatron, both, or none]
Inference: [Note which backend this PR will affect: vLLM, SGLang, both, or none]

Checklist Before Submitting

Read the Contribute Guide.
Apply pre-commit checks.
Add [BREAKING] to the PR title if it breaks any API.
Update the documentation about your changes in the docs.
Add CI test(s) if necessary.

…manager

Clean up codes and comments for SGLang rollout

refactor with block for sampling parameter update

verl/tools/sandbox_fusion_tool.py

tests/workers/rollout/test_sglang_async_rollout_w_sandbox_fusion_tool.py

examples/sglang_multiturn/config/tool_config/sandbox_fusion_tool_config.yaml

verl/workers/fsdp_workers.py

verl/workers/megatron_workers.py

verl/workers/rollout/sglang_rollout/async_sglang_rollout.py

…ge sglang dependency to post5

SwordFaith · 2025-05-28T07:39:04Z

LGTM, please resolve conflicts, after that @zhaochenyang20 can help on trigger CI

eric-haibin-lin

if i understand it correct, this PR is a breaking change, removing the sglang_async option. Is that correct? If so, i just added [breaking] to the title. Nonetheless, it looks like the codebase already depends on sglang post5 and older version of sglang wont work...

zyzshishui · 2025-05-29T23:08:43Z

if i understand it correct, this PR is a breaking change, removing the sglang_async option. Is that correct? If so, i just added [breaking] to the title. Nonetheless, it looks like the codebase already depends on sglang post5 and older version of sglang wont work...

I preserved the original behavior by supporting rollout_name in ["sglang", "sglang_async"] and issuing a deprecation warning. So the old sglang_async name is still accepted (with a warning). Older versions of sglang are no longer compatible, as we only keep align with the latest sgl.

chenhaiq

Please keep the sglang async mode branch of code in fsdp_workers

verl/workers/fsdp_workers.py

…2.0-te2.3

…rt sglang==0.4.6.post5 (volcengine#1717) ### Checklist Before Starting - [x] Search for similar PR(s). ### What does this PR do? - Unify the functionality of SGLangRollout and AsyncSGLangRollout, remove original SGLangRollout and rename AsyncSGLangRollout to SGLangRollout. - Make trivial changes due to modification in sglang==0.4.6.post5. ### High-Level Design > Demonstrate the high-level design if this PR is complex. ### Specific Changes > List the specific changes. ### API > Demonstrate how the API changes if any. ### Usage Example > Provide usage example(s) for easier usage. ```python # Add code snippet or script demonstrating how to use this ``` ### Test > For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluatuion results, etc. ### Additional Info. - **Issue Number**: Fixes issue # or discussion # if any. - **Training**: [Note which backend this PR will affect: FSDP, Megatron, both, or none] - **Inference**: [Note which backend this PR will affect: vLLM, SGLang, both, or none] ### Checklist Before Submitting - [ ] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [ ] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting). - [ ] Add `[BREAKING]` to the PR title if it breaks any API. - [ ] Update the documentation about your changes in the [docs](https://github.com/volcengine/verl/tree/main/docs). - [ ] Add CI test(s) if necessary. --------- Co-authored-by: zyzshishui <@qq.com> Co-authored-by: Xiang Long <mindsculptor@yeah.net> Co-authored-by: ocss884 <ocss.lin@gmail.com> Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com> Co-authored-by: H <linhaibin.eric@gmail.com>

zyzshishui and others added 24 commits May 20, 2025 17:27

change ENGINE to sglang_async

0e10c96

Add megatron support to sglang_async

edd3f70

Fix rebase error and share device mesh between rolllout and sharding …

c06a169

…manager

Fix formatting issue

393eb98

bump 0.4.6.post4

b269356

Fix megatron support

d79548d

Add tmp sandbox fusion tool

b7bae38

A tmp version for sandbox fusion test, need to improve

c6aa6e8

Unified processing of batch-level and request-level generate sequence

3c20523

remove async in naming

a8a9dad

fix review

5eb8913

add multi-turn fsdp2 ci

041232f

remove redundant fsdp2 test script

f6090e4

add tp size comment for megatron

8ad476f

clean up comments for sglang rollout

3306786

Merge pull request #1 from zyzshishui/refactor_refactor

6e58735

Clean up codes and comments for SGLang rollout

delete comment and remove deprecated VerlEngine

26f5415

Merge branch 'refactor' of github.com:zyzshishui/verl into refactor

9bedb2c

refactor with block for sampling parameter update

0c325e9

Fix review

dbb568d

Merge pull request #2 from zyzshishui/refactor_with

7be85f7

refactor with block for sampling parameter update

adapt new FunctionCallParser path to sgl post5

93ca6ea

merge main

c649aeb

merge main

eec7724

ocss884 self-assigned this May 27, 2025

SwordFaith reviewed May 27, 2025

View reviewed changes

zyzshishui added 2 commits May 27, 2025 17:35

remove files related to 1525

c4c58a3

solve mistakes in merge, change deprecated warning location, and chan…

467fdd4

…ge sglang dependency to post5

SwordFaith mentioned this pull request May 28, 2025

Multi-turn rollout & agentic RL Status & Roadmap zhaochenyang20/Awesome-ML-SYS-Tutorial#131

Open

27 tasks

eric-haibin-lin changed the title ~~[Refactor] Unify async rollout under SGLangRollout, and support sglang==0.4.6.post5~~ [BREAKING] [refactor] Unify async rollout under SGLangRollout, and support sglang==0.4.6.post5 May 29, 2025

eric-haibin-lin reviewed May 29, 2025

View reviewed changes

make sure required sglang version is installed

8e387ba

zyzshishui and others added 2 commits May 29, 2025 16:12

Merge branch 'volcengine:main' into remove-async

010b790

install deps in sppo test

0444f4a

eric-haibin-lin mentioned this pull request May 29, 2025

[DataProto] Supporting new operations for DataProto #1761

Merged

6 tasks

eric-haibin-lin previously approved these changes May 29, 2025

View reviewed changes

zyzshishui requested a review from zhaochenyang20 May 29, 2025 23:37

zhaochenyang20 previously approved these changes May 29, 2025

View reviewed changes

add sglang license

62aa913

zyzshishui dismissed stale reviews from zhaochenyang20 and eric-haibin-lin via c3e161b May 29, 2025 23:51

remove deprecated AsyncSGLangRollout

68496bf

zyzshishui force-pushed the remove-async branch from c3e161b to 68496bf Compare May 30, 2025 00:07

chenhaiq reviewed May 30, 2025

View reviewed changes

verl/workers/fsdp_workers.py Show resolved Hide resolved

zyzshishui and others added 6 commits May 30, 2025 04:35

compatible with old versions of sglang

94da29e

change sgl docker

a5d875a

add no-deps option

392279f

use whatcanyousee/verl:ngc-cu124-vllm0.8.5-sglang0.4.6.post5-mcore0.1…

11bf1cc

…2.0-te2.3

Merge branch 'main' into remove-async

5b87be2

remove AsyncSGLangRollout in Search-R1 test

994c167

zhaochenyang20 approved these changes May 31, 2025

View reviewed changes

eric-haibin-lin approved these changes Jun 1, 2025

View reviewed changes

eric-haibin-lin changed the title ~~[BREAKING] [refactor] Unify async rollout under SGLangRollout, and support sglang==0.4.6.post5~~ [sglang] refactor: Unify async rollout under SGLangRollout, and support sglang==0.4.6.post5 Jun 1, 2025

eric-haibin-lin merged commit 4de247f into volcengine:main Jun 1, 2025
33 of 34 checks passed

chenhaiq mentioned this pull request Jun 10, 2025

[rollout] feat: ChatScheduler requests sglang fully async #1769

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[sglang] refactor: Unify async rollout under SGLangRollout, and support sglang==0.4.6.post5 #1717

[sglang] refactor: Unify async rollout under SGLangRollout, and support sglang==0.4.6.post5 #1717

Uh oh!

zyzshishui commented May 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SwordFaith commented May 28, 2025

Uh oh!

eric-haibin-lin left a comment

Uh oh!

zyzshishui commented May 29, 2025

Uh oh!

chenhaiq left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

[sglang] refactor: Unify async rollout under SGLangRollout, and support sglang==0.4.6.post5 #1717

[sglang] refactor: Unify async rollout under SGLangRollout, and support sglang==0.4.6.post5 #1717

Uh oh!

Conversation

zyzshishui commented May 27, 2025

Checklist Before Starting

What does this PR do?

High-Level Design

Specific Changes

API

Usage Example

Test

Additional Info.

Checklist Before Submitting

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SwordFaith commented May 28, 2025

Uh oh!

eric-haibin-lin left a comment

Choose a reason for hiding this comment

Uh oh!

zyzshishui commented May 29, 2025

Uh oh!

chenhaiq left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!