Multi-turn rollout Update #2 #132
Description
- Add real world tools support (at least Search & Code Interpreter)
- Add global rate limit at tool-level (Haiquan Chen @ Bytedance volcengine is working on it)
- Code Interpreter (set ReTool like workload as example)
- ReTool like cold start SFT data https://huggingface.co/datasets/swordfaith/ReTool-SFT-multi-turn
- Qwen3 no-think SFT model
- sandbox fusion based code interpreter impl
- First version impl (Xiaocheng Wang @ Bytedance volcengine is working on it) feat: sandbox fusion for multi-turn volcengine/verl#1525
- Support global rate limit & req queue
- Support global registry and seperate resources pool
- ReTool like RL example & wandb log
- Search-R1 like RL (Ling Chang @ Baidu & CAS is working on it)
- First run with sglang 0.4.5.post3
- check if it stable and result right
- Add Server-based Rollout
- Add http engine support in async rollout
- Add http engine replace http verl engine
- Port sgl-router as a ray actor in verl
- Add register http_engine to router logic
- Refactor to common ray actor registry impl
- Discuss if it could be ray native grpc service register and dispatch
- Add Qwen3 training example @SwordFaith
- Add multi-node support (Shenggui Li & Jin Pan @ SGLang verified it), trouble shooting doc
- Add VLM support (geo3k + examples) Multi-turn Update #2 VLM Support Tracker #137
- Add Megatron support @SwordFaith [sglang] Fix megatron support in sglang and add sglang_async support & CI tasks volcengine/verl#1602
- Refactor sglang in verl
- init & generate_sequences impl align @ocss884 @SwordFaith https://github.com/SwordFaith/verl/tree/refactor/merge_sgl_rollouts_and_bump_to_0.4.6.post4
- sharding manager + rollout 合并验证，将所有 sglang 单测切换到 sglang_async 测试，环境 torch 2.6 + sglang 0.4.6.post4 (Yuzhen Zhou & Jin Pan @ SGLang help test on this)
- e2e_ppo_trainer_sglang
- e2e_ppo_trainer_sglang_async
- e2e_ppo_trainer_sglang_async_with_tool
- e2e_ppo_trainer_sglang_vlm
- e2e_ppo_trainer_megatron-qwen + megatron-core 0.12.0 @SwordFaith
- sgl.yml
- gsm8k regression training sglang & sglang_async @ocss884
- geo3k regression training sglang & sglang_async @GeLee-Q
- gsm8k with tool @SwordFaith
- sglang rollout 和 sharding manager switch to sglang_async as default
- megatron verify w generate_sequences_with_tools @SwordFaith
- Update CI & unit-tests
- Update requirements
- bump to 0.4.6.post4 Bump to sglang 0.4.6.post4 & unified generate sequences ability between sgl and sgl async volcengine/verl#1577
- fix megatron support
- merge sglang impl