[Bug] [CI regression] [AMD] TestVisionChunkedPrefill #7701
Description
Checklist
- 1. I have searched related issues but cannot get the expected help.
- 2. The bug has not been fixed in the latest version.
- 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
- 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- 5. Please use English, otherwise it will be closed.
Describe the bug
CI unit-test-backend-1-gpu-amd seems to be broken since the past few days.
output with chunked prefill:
The video features a person standing on a stage with a dark background. The individual is dressed in a black outfit and appears to be speaking or presenting. The stage
output without chunked prefill:
The video features a person standing on a stage with a dark background. The individual is dressed in a black outfit and appears to be speaking or presenting. The stage
output with chunked prefill:
['The video features a close-up shot of a person holding a small, rectangular electronic device, which appears to be an iPod. The individual is wearing a black shirt', 'The video features a close-up shot of a person holding a small, white electronic device, which appears to be an iPod. The individual is wearing a black shirt', 'The video features a close-up shot of a person holding a small, white electronic device, which appears to be an iPod. The individual is wearing a black shirt', 'The video features a close-up shot of a person holding a small, white electronic device, which appears to be an iPod. The individual is wearing a black shirt']
output without chunked prefill:
['The video features a close-up shot of a person holding a small, rectangular electronic device, which appears to be an iPod. The individual is wearing a black shirt', 'The video features a close-up shot of a person holding a small, white electronic device, which appears to be an iPod. The individual is wearing a black shirt', 'The video features a close-up shot of a person holding a small, white electronic device, which appears to be an iPod. The individual is wearing a black shirt', 'The video features a close-up of a person holding a small, white electronic device, which appears to be an iPod. The individual is wearing a black shirt and']
E
======================================================================
ERROR: test_chunked_prefill (test_vision_chunked_prefill.TestVisionChunkedPrefill.test_chunked_prefill)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/sgl-workspace/llama4_sgl/python/sglang/srt/utils.py", line 2052, in retry
return fn()
^^^^
File "/sgl-workspace/llama4_sgl/python/sglang/test/test_utils.py", line 1237, in <lambda>
lambda: super(CustomTestCase, self)._callTestMethod(method),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Lists differ: ['The[511 chars]e-up shot of a person holding a small, white e[83 chars]irt'] != ['The[511 chars]e-up of a person holding a small, white electr[82 chars]and']
First differing element 3:
'The [21 chars]e-up shot of a person holding a small, white e[82 chars]hirt'
'The [21 chars]e-up of a person holding a small, white electr[81 chars] and'
Diff is 1108 characters long. Set self.maxDiff to None to see it.
I suggest to temporarily disable the test in CI to avoid blocking other PRs until we resolve the issue.
Reproduction
Sample failure: https://github.com/sgl-project/sglang/actions/runs/15965626491/job/45029188822
SGLANG_AMD_CI=1 SGLANG_IS_IN_CI=1 SGLANG_USE_AITER=1 python3 -m unittest test_vision_chunked_prefill.TestVisionChunkedPrefill.test_chunked_prefill
Environment
lmsysorg/sglang:v0.4.8.post1-rocm630
CC: @saienduri @HaiShaw