fix: remove dependency on latest transformers impl #3635

mickqian · 2025-02-17T11:42:06Z

Motivation

Current is_valid_list_of_images import in qwen2_5_vl_config.py relies on latest transformers, leading to an import issue shown here

Modifications

qwen2_5_vl_config.py , is_valid_list_of_images copied from transformers

Checklist

Format your code according to the Code Formatting with Pre-Commit.
Add unit tests as outlined in the Running Unit Tests.
Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

vhain · 2025-02-17T20:08:57Z

Thanks for the fix. but I am seeing another error:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/dist-packages/sglang/launch_server.py", line 6, in <module>
    from sglang.srt.entrypoints.http_server import launch_server
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/http_server.py", line 41, in <module>
    from sglang.srt.entrypoints.engine import _launch_subprocesses
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/engine.py", line 36, in <module>
    from sglang.srt.managers.data_parallel_controller import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/data_parallel_controller.py", line 27, in <module>
    from sglang.srt.managers.io_struct import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/io_struct.py", line 24, in <module>
    from sglang.srt.managers.schedule_batch import BaseFinishReason
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/schedule_batch.py", line 42, in <module>
    from sglang.srt.configs.model_config import ModelConfig
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/__init__.py", line 4, in <module>
    from sglang.srt.configs.qwen2_5_vl_config import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/qwen2_5_vl_config.py", line 1005, in <module>
    AutoImageProcessor.register(Qwen2_5_VLConfig, Qwen2_5_VLImageProcessor)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py", line 628, in register
    IMAGE_PROCESSOR_MAPPING.register(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 833, in register
    raise ValueError(f"'{key}' is already used by a Transformers model.")
ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model.

when using latest transformers:

transformers==4.49.0

mickqian · 2025-02-18T00:59:34Z

Thanks for the fix. but I am seeing another error:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/dist-packages/sglang/launch_server.py", line 6, in <module>
    from sglang.srt.entrypoints.http_server import launch_server
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/http_server.py", line 41, in <module>
    from sglang.srt.entrypoints.engine import _launch_subprocesses
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/engine.py", line 36, in <module>
    from sglang.srt.managers.data_parallel_controller import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/data_parallel_controller.py", line 27, in <module>
    from sglang.srt.managers.io_struct import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/io_struct.py", line 24, in <module>
    from sglang.srt.managers.schedule_batch import BaseFinishReason
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/schedule_batch.py", line 42, in <module>
    from sglang.srt.configs.model_config import ModelConfig
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/__init__.py", line 4, in <module>
    from sglang.srt.configs.qwen2_5_vl_config import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/qwen2_5_vl_config.py", line 1005, in <module>
    AutoImageProcessor.register(Qwen2_5_VLConfig, Qwen2_5_VLImageProcessor)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py", line 628, in register
    IMAGE_PROCESSOR_MAPPING.register(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 833, in register
    raise ValueError(f"'{key}' is already used by a Transformers model.")
ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model.

when using latest transformers:

transformers==4.49.0

Please use the transformers version suggested by sglang, thanks

zhaochenyang20 · 2025-02-18T16:28:41Z

I will give credit to Chen-0210 also. @Chen-0210

panpan0000 · 2025-02-23T04:00:33Z

Thanks for the fix. but I am seeing another error:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/dist-packages/sglang/launch_server.py", line 6, in <module>
    from sglang.srt.entrypoints.http_server import launch_server
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/http_server.py", line 41, in <module>
    from sglang.srt.entrypoints.engine import _launch_subprocesses
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/engine.py", line 36, in <module>
    from sglang.srt.managers.data_parallel_controller import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/data_parallel_controller.py", line 27, in <module>
    from sglang.srt.managers.io_struct import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/io_struct.py", line 24, in <module>
    from sglang.srt.managers.schedule_batch import BaseFinishReason
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/schedule_batch.py", line 42, in <module>
    from sglang.srt.configs.model_config import ModelConfig
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/__init__.py", line 4, in <module>
    from sglang.srt.configs.qwen2_5_vl_config import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/qwen2_5_vl_config.py", line 1005, in <module>
    AutoImageProcessor.register(Qwen2_5_VLConfig, Qwen2_5_VLImageProcessor)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py", line 628, in register
    IMAGE_PROCESSOR_MAPPING.register(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 833, in register
    raise ValueError(f"'{key}' is already used by a Transformers model.")
ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model.

when using latest transformers:

transformers==4.49.0

I've tried with a workaround as this PR with transformers==4.49.0: #3792

zhaochenyang20 · 2025-02-23T06:35:00Z

@panpan0000 I don't think you can work with transformers==4.49.0. Use transformers==4.48.3 please.

yinghai · 2025-03-17T22:51:22Z

Trying to understand the nature of the issue, looks like in new transformers version if it already has the Qwen2VL image processors.

ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model. ('Qwen2VLImageProcessor', 'Qwen2VLImageProcessorFast')

so the question is why do we have our own versions here in sglang. If it's expected, we can just register and accept that exist_ok?

+AutoImageProcessor.register(Qwen2_5_VLConfig, None, Qwen2_5_VLImageProcessor, None, exist_ok=True)
+AutoProcessor.register(Qwen2_5_VLConfig, Qwen2_5_VLProcessor, exist_ok=True)

mickqian · 2025-03-18T01:04:56Z

Thanks @yinghai. The latest transformers has compatibility issue with sglang, that's why we have to keep a version of configs / processors.
However the modification you mentioned is also ok. By the time the issues above is resolved here, all the associated codes will be removed.

mickqian requested review from merrymercy, Ying1123, hnyls2002, zhyncs, ispobock and ByronHsu as code owners February 17, 2025 11:42

zhyncs mentioned this pull request Feb 17, 2025

use transformers 4.48.3 #3650

Merged

6 tasks

fix: remove dependency on latest transformers impl

95e4361

mickqian force-pushed the remove-dependency-on-transformers branch from 2db1cc6 to 95e4361 Compare February 18, 2025 01:06

simveit mentioned this pull request Feb 18, 2025

Added installation docs. #3648

Closed

zhaochenyang20 added 2 commits February 18, 2025 07:53

Merge branch 'main' into remove-dependency-on-transformers

c8d0cbe

Update qwen2_vl.py

0d6f3f8

zhaochenyang20 mentioned this pull request Feb 18, 2025

remove input_ids = None in qwen2-vl #3625

Closed

6 tasks

zhyncs merged commit 424848d into sgl-project:main Feb 18, 2025
19 of 20 checks passed

panpan0000 mentioned this pull request Feb 23, 2025

Workaround for transformers 4.49 #3792

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: remove dependency on latest transformers impl #3635

fix: remove dependency on latest transformers impl #3635

Uh oh!

mickqian commented Feb 17, 2025 •

edited

Loading

Uh oh!

vhain commented Feb 17, 2025

Uh oh!

mickqian commented Feb 18, 2025

Uh oh!

zhaochenyang20 commented Feb 18, 2025 •

edited

Loading

Uh oh!

Uh oh!

panpan0000 commented Feb 23, 2025 •

edited

Loading

Uh oh!

zhaochenyang20 commented Feb 23, 2025

Uh oh!

yinghai commented Mar 17, 2025

Uh oh!

mickqian commented Mar 18, 2025 •

edited

Loading

Uh oh!

Uh oh!

fix: remove dependency on latest transformers impl #3635

fix: remove dependency on latest transformers impl #3635

Uh oh!

Conversation

mickqian commented Feb 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Checklist

Uh oh!

vhain commented Feb 17, 2025

Uh oh!

mickqian commented Feb 18, 2025

Uh oh!

zhaochenyang20 commented Feb 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

panpan0000 commented Feb 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhaochenyang20 commented Feb 23, 2025

Uh oh!

yinghai commented Mar 17, 2025

Uh oh!

mickqian commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

mickqian commented Feb 17, 2025 •

edited

Loading

zhaochenyang20 commented Feb 18, 2025 •

edited

Loading

panpan0000 commented Feb 23, 2025 •

edited

Loading

mickqian commented Mar 18, 2025 •

edited

Loading