Skip to content

Update benchmark scripts #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Jan 16, 2024
Merged

Update benchmark scripts #8

merged 9 commits into from
Jan 16, 2024

Conversation

merrymercy
Copy link
Contributor

No description provided.

@merrymercy merrymercy merged commit 70359bf into main Jan 16, 2024
@merrymercy merrymercy deleted the benchmark branch January 16, 2024 00:13
@wonderisland wonderisland mentioned this pull request Sep 19, 2024
Closed
5 tasks
@CSEEduanyu CSEEduanyu mentioned this pull request Jan 26, 2025
Closed
5 tasks
@lambert0312 lambert0312 mentioned this pull request Feb 18, 2025
Merged
@ToughK ToughK mentioned this pull request Feb 18, 2025
Closed
5 tasks
@mahaocong90 mahaocong90 mentioned this pull request Feb 26, 2025
Closed
5 tasks
CatherineSue added a commit to CatherineSue/sglang that referenced this pull request Mar 4, 2025
@CatherineSue
Pull request sgl-project#8: [Log] Support request_id in OpenAI API se…
3030d92 
…rver

Merge in GENAICORE/sglang from chang/request-id to features-based-on-v0.4.2.post1

Squashed commit of the following:

commit 9b3b1488feea6f5b310570a3060cae3e2786fc38
Author: Chang Su <chang.s.su@oracle.com>
Date:   Wed Feb 19 13:35:33 2025 -0800

    [Log] Support request_id in OpenAI API server

    - Add `request_id` to ChatCompletionRequest and EmbeddingRequest
    - Pass request_ids to GenerateReqInput and EmbeedingReqInput
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
@merrymercy
Update benchmark scripts (sgl-project#8)
9ae401c
chunyuan-w added a commit to chunyuan-w/sglang that referenced this pull request Mar 12, 2025
@chunyuan-w
Use fused_experts_cpu and add weight packing (sgl-project#8)
8f5ffaa 
* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once
chunyuan-w added a commit to chunyuan-w/sglang that referenced this pull request Mar 14, 2025
@chunyuan-w
Use fused_experts_cpu and add weight packing (sgl-project#8)
dbe4060 
* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once
chunyuan-w added a commit to chunyuan-w/sglang that referenced this pull request Mar 14, 2025
@chunyuan-w
Use fused_experts_cpu and add weight packing (sgl-project#8)
81594f2 
* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once
chunyuan-w added a commit to chunyuan-w/sglang that referenced this pull request Mar 14, 2025
@chunyuan-w
Use fused_experts_cpu and add weight packing (sgl-project#8)
e69ee02 
* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once
This was referenced Apr 16, 2025
Open
Closed
Closed
@riou-chen riou-chen mentioned this pull request Apr 17, 2025
Closed
ch-wan pushed a commit to ch-wan/sglang that referenced this pull request Apr 25, 2025
@sleepcoo
Merge pull request sgl-project#8 from xutizhou/deepgemm
84a25cd 
support fp8 dispatch
NorthmanPKU pushed a commit to NorthmanPKU/sglang that referenced this pull request May 16, 2025
@xinhaoc
Merge pull request sgl-project#8 from linsj20/multi_gpu_SJ
d93fde1 
Finish all to all
@dongyibo dongyibo mentioned this pull request May 19, 2025
Open
5 tasks
@yuleiqin yuleiqin mentioned this pull request May 26, 2025
Closed
5 tasks
chunyuan-w added a commit to chunyuan-w/sglang that referenced this pull request May 28, 2025
@chunyuan-w
Use fused_experts_cpu and add weight packing (sgl-project#8)
1b3f82a 
* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once
chunyuan-w added a commit to chunyuan-w/sglang that referenced this pull request May 28, 2025
@chunyuan-w @jianan-gu
Use fused_experts_cpu and add weight packing (sgl-project#8)
88011f1 
* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once
chunyuan-w added a commit to chunyuan-w/sglang that referenced this pull request Jun 3, 2025
@chunyuan-w
Use fused_experts_cpu and add weight packing (sgl-project#8)
8bcf948 
* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once
chunyuan-w added a commit to chunyuan-w/sglang that referenced this pull request Jun 6, 2025
@chunyuan-w
Use fused_experts_cpu and add weight packing (sgl-project#8)
dd54bcf 
* Use fused_experts_cpu and add weight packing

* add check on whether AMX is supported

* move utils to cpu_utils.py

* address comment

* no need to pass in is_vnni since it's True by default; change inplace to True

* refactor prepack_weight_if_needed

* Only import sgl_kernel.cpu once
@EricKing626 EricKing626 mentioned this pull request Jun 10, 2025
Closed
5 tasks
pengxin99 pushed a commit to pengxin99/sglang that referenced this pull request Jun 19, 2025
@jianan-gu
Port dsv2 cpu opt (sgl-project#8)
9c2decb
sleepcoo pushed a commit to shuaills/sglang that referenced this pull request Jun 24, 2025
@FrankLeeeee
added configuration (sgl-project#8)
07a9db6
yichiche pushed a commit to yichiche/sglang that referenced this pull request Jul 7, 2025
@Hardcode84 @xintin
Wave Backend decode (sgl-project#8)
1db105f 
* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
pi314ever pushed a commit to pi314ever/sglang that referenced this pull request Jul 10, 2025
@rahulvijayaraghavan
[SW-224336]: Enable internal profiling of each forward batch (sgl-pro…
6322cf0 
…ject#8)

Signed-off-by: Rahul Vijayaraghavan <rahul.vijayaraghavan@intel.com>
@Xerxes-cn Xerxes-cn mentioned this pull request Jul 14, 2025
Open
5 tasks
siuhunh pushed a commit to xing-wenjin/sglang that referenced this pull request Jul 21, 2025
@ZhengdQin
Merge pull request sgl-project#8 from zhuyijie88/dpsk_dev
605bfce 
npu matmul weight nz
yichiche pushed a commit to yichiche/sglang that referenced this pull request Jul 23, 2025
@Hardcode84 @yichiche
Wave Backend decode (sgl-project#8)
78b689e 
* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
yichiche pushed a commit to yichiche/sglang that referenced this pull request Jul 25, 2025
@Hardcode84 @yichiche
Wave Backend decode (sgl-project#8)
9976cb9 
* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
@lingyaoluu lingyaoluu mentioned this pull request Jul 25, 2025
Open
5 tasks
yichiche pushed a commit to yichiche/sglang that referenced this pull request Jul 30, 2025
@Hardcode84
Wave Backend decode (sgl-project#8)
b20719f 
* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
yichiche pushed a commit to yichiche/sglang that referenced this pull request Aug 1, 2025
@Hardcode84 @yichiche
Wave Backend decode (sgl-project#8)
b80050e 
* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
yichiche pushed a commit to yichiche/sglang that referenced this pull request Aug 6, 2025
@Hardcode84 @yichiche
Wave Backend decode (sgl-project#8)
7896755 
* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
yichiche pushed a commit to yichiche/sglang that referenced this pull request Aug 7, 2025
@Hardcode84 @yichiche
Wave Backend decode (sgl-project#8)
140b6da 
* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
yichiche pushed a commit to yichiche/sglang that referenced this pull request Aug 11, 2025
@Hardcode84 @yichiche
Wave Backend decode (sgl-project#8)
7693a2a 
* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
yichiche pushed a commit to yichiche/sglang that referenced this pull request Aug 11, 2025
@Hardcode84 @yichiche
Wave Backend decode (sgl-project#8)
f514ea3 
* align shapes

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

* fix

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>

---------

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
@ericschreiber ericschreiber mentioned this pull request Aug 13, 2025
Open
5 tasks
@gaolaobao gaolaobao mentioned this pull request Aug 25, 2025
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@merrymercy