[sglang] fix: Fix tool call parser not found error for SGLang==0.4.6.post5 #1852
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Adding a test to the function call parser, as Haibin suggested. Thanks! The other parts look good to me. |
vermouth1992
approved these changes
Jun 10, 2025
yellowbee686
pushed a commit
to yellowbee686/verl
that referenced
this pull request
Jun 10, 2025
…post5 (volcengine#1852) ### Checklist Before Starting - [x] Search for similar PR(s). ### What does this PR do? SGLang multiturn rollout relies on bos and eos token in the tool parser to retrieve the right tool parser. SGLang==0.4.6.post5 changed those tokens for Qwen2 parser([PR](https://github.com/sgl-project/sglang/pull/6597/files#diff-725eae87b1043c063d85c22b71f415941e2983c60eb52ef1a0d0be89f13b1110)) so it breaks async rollout. This PR updates the logic in Verl to fix the issue. Error example: ``` Traceback (most recent call last): File "/home/jobuser/resources/verl/trainer/main_ppo.py", line 28, in main run_ppo(config) File "/home/jobuser/resources/verl/trainer/main_ppo.py", line 40, in run_ppo ray.get(runner.run.remote(config)) File "/home/jobuser/.local/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper return fn(*args, **kwargs) File "/home/jobuser/.local/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper return func(*args, **kwargs) File "/home/jobuser/.local/lib/python3.10/site-packages/ray/_private/worker.py", line 2822, in get values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout) File "/home/jobuser/.local/lib/python3.10/site-packages/ray/_private/worker.py", line 930, in get_objects raise value.as_instanceof_cause() ray.exceptions.RayTaskError(ValueError): ray::TaskRunner.run() (pid=64101, ip=100.96.58.10, actor_id=5f74f8de7594144240b2dbcf01000000, repr=<main_ppo.TaskRunner object at 0x7bab01cdd9f0>) File "/home/jobuser/resources/verl/trainer/main_ppo.py", line 155, in run trainer.init_workers() File "/home/jobuser/resources/verl/trainer/ppo/ray_trainer.py", line 837, in init_workers self.actor_rollout_wg.init_model() File "/home/jobuser/resources/verl/single_controller/ray/base.py", line 51, in __call__ output = ray.get(output) ray.exceptions.RayTaskError(ValueError): ray::WorkerDict.actor_rollout_init_model() (pid=84769, ip=100.96.58.10, actor_id=d2da89f7763ecc0e0681bcdd01000000, repr=<verl.single_controller.ray.base.WorkerDict object at 0x7409bc4c7fa0>) File "/home/jobuser/resources/verl/single_controller/ray/base.py", line 645, in func return getattr(self.worker_dict[key], name)(*args, **kwargs) File "/home/jobuser/resources/verl/single_controller/base/decorator.py", line 534, in inner return func(*args, **kwargs) File "/home/jobuser/resources/verl/workers/fsdp_workers.py", line 564, in init_model self.rollout, self.rollout_sharding_manager = self._build_rollout(trust_remote_code=self.config.model.get("trust_remote_code", False)) File "/home/jobuser/resources/verl/workers/fsdp_workers.py", line 474, in _build_rollout rollout = SGLangRollout( File "/home/jobuser/resources/verl/workers/rollout/sglang_rollout/sglang_rollout.py", line 161, in __init__ ) = self._initialize_tools(config, tokenizer) File "/home/jobuser/resources/verl/workers/rollout/sglang_rollout/sglang_rollout.py", line 384, in _initialize_tools tool_call_parser_type = get_tool_call_parser_type(tokenizer) File "/home/jobuser/resources/verl/workers/rollout/sglang_rollout/sglang_rollout.py", line 113, in get_tool_call_parser_type raise ValueError(f"No tool call parser found for tokenizer {tokenizer}") ValueError: No tool call parser found for tokenizer Qwen2TokenizerFast(name_or_path='/shared/public/elr-models/Qwen/Qwen2.5-7B-Instruct/52e20a6f5f475e5c8f6a8ebda4ae5fa6b1ea22ac', vocab_size=151643, model_max_length=131072, is_fast=True, padding_side='right', truncation_side='right', special_tokens={'eos_token': '<|im_end|>', 'pad_token': '<|endoftext|>', 'additional_special_tokens': ['<|im_start|>', '<|im_end|>', '<|object_ref_start|>', '<|object_ref_end|>', '<|box_start|>', '<|box_end|>', '<|quad_start|>', '<|quad_end|>', '<|vision_start|>', '<|vision_end|>', '<|vision_pad|>', '<|image_pad|>', '<|video_pad|>']}, clean_up_tokenization_spaces=False, added_tokens_decoder={ 151643: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151644: AddedToken("<|im_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151645: AddedToken("<|im_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151646: AddedToken("<|object_ref_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151647: AddedToken("<|object_ref_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151648: AddedToken("<|box_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151649: AddedToken("<|box_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151650: AddedToken("<|quad_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151651: AddedToken("<|quad_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151652: AddedToken("<|vision_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151653: AddedToken("<|vision_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151654: AddedToken("<|vision_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151655: AddedToken("<|image_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151656: AddedToken("<|video_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151657: AddedToken("<tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151658: AddedToken("</tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151659: AddedToken("<|fim_prefix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151660: AddedToken("<|fim_middle|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151661: AddedToken("<|fim_suffix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151662: AddedToken("<|fim_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151663: AddedToken("<|repo_name|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151664: AddedToken("<|file_sep|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), } ) ``` ### Test Tested with both SGLang==0.4.6.post4 and SGLang==0.4.6.post5, successfully executed multiturn RL experiments that failed with SGLang==0.4.6.post5 before this change . ### Additional Info. - **Inference**: SGLang ### Checklist Before Submitting - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting). - [x] Add `[BREAKING]` to the PR title if it breaks any API. - [x] Update the documentation about your changes in the [docs](https://github.com/volcengine/verl/tree/main/docs). - [x] New CI unit test(s) are added to cover the code path. - [x] Rely on existing unit tests on CI that covers the code path.
whatadayG
pushed a commit
to whatadayG/verl
that referenced
this pull request
Sep 5, 2025
…post5 (volcengine#1852) ### Checklist Before Starting - [x] Search for similar PR(s). ### What does this PR do? SGLang multiturn rollout relies on bos and eos token in the tool parser to retrieve the right tool parser. SGLang==0.4.6.post5 changed those tokens for Qwen2 parser([PR](https://github.com/sgl-project/sglang/pull/6597/files#diff-725eae87b1043c063d85c22b71f415941e2983c60eb52ef1a0d0be89f13b1110)) so it breaks async rollout. This PR updates the logic in Verl to fix the issue. Error example: ``` Traceback (most recent call last): File "/home/jobuser/resources/verl/trainer/main_ppo.py", line 28, in main run_ppo(config) File "/home/jobuser/resources/verl/trainer/main_ppo.py", line 40, in run_ppo ray.get(runner.run.remote(config)) File "/home/jobuser/.local/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper return fn(*args, **kwargs) File "/home/jobuser/.local/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper return func(*args, **kwargs) File "/home/jobuser/.local/lib/python3.10/site-packages/ray/_private/worker.py", line 2822, in get values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout) File "/home/jobuser/.local/lib/python3.10/site-packages/ray/_private/worker.py", line 930, in get_objects raise value.as_instanceof_cause() ray.exceptions.RayTaskError(ValueError): ray::TaskRunner.run() (pid=64101, ip=100.96.58.10, actor_id=5f74f8de7594144240b2dbcf01000000, repr=<main_ppo.TaskRunner object at 0x7bab01cdd9f0>) File "/home/jobuser/resources/verl/trainer/main_ppo.py", line 155, in run trainer.init_workers() File "/home/jobuser/resources/verl/trainer/ppo/ray_trainer.py", line 837, in init_workers self.actor_rollout_wg.init_model() File "/home/jobuser/resources/verl/single_controller/ray/base.py", line 51, in __call__ output = ray.get(output) ray.exceptions.RayTaskError(ValueError): ray::WorkerDict.actor_rollout_init_model() (pid=84769, ip=100.96.58.10, actor_id=d2da89f7763ecc0e0681bcdd01000000, repr=<verl.single_controller.ray.base.WorkerDict object at 0x7409bc4c7fa0>) File "/home/jobuser/resources/verl/single_controller/ray/base.py", line 645, in func return getattr(self.worker_dict[key], name)(*args, **kwargs) File "/home/jobuser/resources/verl/single_controller/base/decorator.py", line 534, in inner return func(*args, **kwargs) File "/home/jobuser/resources/verl/workers/fsdp_workers.py", line 564, in init_model self.rollout, self.rollout_sharding_manager = self._build_rollout(trust_remote_code=self.config.model.get("trust_remote_code", False)) File "/home/jobuser/resources/verl/workers/fsdp_workers.py", line 474, in _build_rollout rollout = SGLangRollout( File "/home/jobuser/resources/verl/workers/rollout/sglang_rollout/sglang_rollout.py", line 161, in __init__ ) = self._initialize_tools(config, tokenizer) File "/home/jobuser/resources/verl/workers/rollout/sglang_rollout/sglang_rollout.py", line 384, in _initialize_tools tool_call_parser_type = get_tool_call_parser_type(tokenizer) File "/home/jobuser/resources/verl/workers/rollout/sglang_rollout/sglang_rollout.py", line 113, in get_tool_call_parser_type raise ValueError(f"No tool call parser found for tokenizer {tokenizer}") ValueError: No tool call parser found for tokenizer Qwen2TokenizerFast(name_or_path='/shared/public/elr-models/Qwen/Qwen2.5-7B-Instruct/52e20a6f5f475e5c8f6a8ebda4ae5fa6b1ea22ac', vocab_size=151643, model_max_length=131072, is_fast=True, padding_side='right', truncation_side='right', special_tokens={'eos_token': '<|im_end|>', 'pad_token': '<|endoftext|>', 'additional_special_tokens': ['<|im_start|>', '<|im_end|>', '<|object_ref_start|>', '<|object_ref_end|>', '<|box_start|>', '<|box_end|>', '<|quad_start|>', '<|quad_end|>', '<|vision_start|>', '<|vision_end|>', '<|vision_pad|>', '<|image_pad|>', '<|video_pad|>']}, clean_up_tokenization_spaces=False, added_tokens_decoder={ 151643: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151644: AddedToken("<|im_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151645: AddedToken("<|im_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151646: AddedToken("<|object_ref_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151647: AddedToken("<|object_ref_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151648: AddedToken("<|box_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151649: AddedToken("<|box_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151650: AddedToken("<|quad_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151651: AddedToken("<|quad_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151652: AddedToken("<|vision_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151653: AddedToken("<|vision_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151654: AddedToken("<|vision_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151655: AddedToken("<|image_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151656: AddedToken("<|video_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), 151657: AddedToken("<tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151658: AddedToken("</tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151659: AddedToken("<|fim_prefix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151660: AddedToken("<|fim_middle|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151661: AddedToken("<|fim_suffix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151662: AddedToken("<|fim_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151663: AddedToken("<|repo_name|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), 151664: AddedToken("<|file_sep|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False), } ) ``` ### Test Tested with both SGLang==0.4.6.post4 and SGLang==0.4.6.post5, successfully executed multiturn RL experiments that failed with SGLang==0.4.6.post5 before this change . ### Additional Info. - **Inference**: SGLang ### Checklist Before Submitting - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting). - [x] Add `[BREAKING]` to the PR title if it breaks any API. - [x] Update the documentation about your changes in the [docs](https://github.com/volcengine/verl/tree/main/docs). - [x] New CI unit test(s) are added to cover the code path. - [x] Rely on existing unit tests on CI that covers the code path.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist Before Starting
What does this PR do?
SGLang multiturn rollout relies on bos and eos token in the tool parser to retrieve the right tool parser. SGLang==0.4.6.post5 changed those tokens for Qwen2 parser(PR) so it breaks async rollout. This PR updates the logic in Verl to fix the issue.
Error example:
Test
Tested with both SGLang==0.4.6.post4 and SGLang==0.4.6.post5, successfully executed multiturn RL experiments that failed with SGLang==0.4.6.post5 before this change .
Additional Info.
Checklist Before Submitting
[BREAKING]to the PR title if it breaks any API.