[ROCm] Facilitate shfl_xor usage in AMD platform #3664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Closed

yiakwy-xpu-ml-framework-team wants to merge 5 commits into sgl-project:use__shfl_xor_sync_style_reduce from yiakwy-xpu-ml-framework-team:enable_shfl_xor_sync_style_reduce_on_amd

Contributor

yiakwy-xpu-ml-framework-team commented Feb 18, 2025 •

edited

Loading

Motivation

Enabe compiling in AMD platform ：

SDK : ROCM 6.3
CHIP ARCH : MI series

Modifications

Checklist

Format your code according to the Code Formatting with Pre-Commit.
Add unit tests as outlined in the Running Unit Tests.
Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.


          facilitate compiling in AMD platform

595d180

yiakwy-xpu-ml-framework-team requested review from zhyncs, ispobock, HandH1998, BBuf, yizhang2077 and merrymercy as code owners

February 18, 2025 09:23

BBuf reviewed

View reviewed changes

sgl-kernel/src/sgl-kernel/csrc/per_token_group_quant_fp8.cu Outdated Show resolved Hide resolved


          remove redundant static_cast<float>

d996df5

BBuf reviewed

View reviewed changes

sgl-kernel/src/sgl-kernel/include/utils.h Show resolved Hide resolved

BBuf reviewed

View reviewed changes

sgl-kernel/src/sgl-kernel/include/hip_math_def.h Outdated Show resolved Hide resolved

BBuf reviewed

View reviewed changes

sgl-kernel/src/sgl-kernel/include/utils.h Outdated

-              #ifndef USE_ROCM
+              #ifdef USE_ROCM
+              #define CHECK_INPUT(x) \

Collaborator

BBuf Feb 18, 2025

Aten has already defined CHECK_INPUT?


          add castToFloat symbol for CUDA platform

e053898

BBuf reviewed

View reviewed changes

sgl-kernel/src/sgl-kernel/ops/__init__.py Outdated Show resolved Hide resolved

BBuf reviewed

View reviewed changes

sgl-kernel/src/sgl-kernel/include/utils.h Show resolved Hide resolved


          remove guard

1d75eaf

yiakwy-xpu-ml-framework-team mentioned this pull request

fix sgl-kernel build failure on AMD #3352

Merged

5 tasks

yiakwy-xpu-ml-framework-team requested a review from BBuf

February 18, 2025 10:14


          update setup_rocm.py

f5514fc

yiakwy-xpu-ml-framework-team changed the title ~~facilitate compiling in AMD platform~~ [ROCm] facilitate compiling in AMD platform

yiakwy-xpu-ml-framework-team changed the title ~~[ROCm] facilitate compiling in AMD platform~~ [ROCm] Facilitate shlf_xor usage in AMD platform

yiakwy-xpu-ml-framework-team changed the title ~~[ROCm] Facilitate shlf_xor usage in AMD platform~~ [ROCm] Facilitate shfl_xor usage in AMD platform

zhyncs deleted the branch sgl-project:use__shfl_xor_sync_style_reduce

February 19, 2025 12:53

zhyncs closed this

This was referenced Feb 19, 2025

[ROCm] Enable shfl xor sync style reduce on amd #3701

Closed

[ROCm] Enable per token group quant fp8 in amd #3702

Open

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

zhyncs Awaiting requested review from zhyncs

ispobock Awaiting requested review from ispobock

HandH1998 Awaiting requested review from HandH1998

yizhang2077 Awaiting requested review from yizhang2077

merrymercy Awaiting requested review from merrymercy

BBuf Awaiting requested review from BBuf

Labels

None yet