[AMD] Default to hipblaslt in gemm #127944

xw285cornell · 2024-06-04T19:05:16Z

Summary: It has been a constant pain that we have to specify env var to go with the hipblaslt path. The default path is very slow on MI300. Therefore, let's default to hipblaslt.

Differential Revision: D58150764

pytorch-bot · 2024-06-04T19:05:18Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/127944

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 5c34b21 with merge base df43d58 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / linux-focal-py3_8-clang9-xla / test (xla, 1, 1, linux.12xlarge) (gh) (similar failure)
test_all_reduce_no_op_with_one_replica

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-06-04T19:05:23Z

This pull request was exported from Phabricator. Differential Revision: D58150764

Summary: only to trigger ci test Differential Revision: D58150764

facebook-github-bot · 2024-06-05T03:44:28Z

This pull request was exported from Phabricator. Differential Revision: D58150764

Summary: only to trigger ci test Differential Revision: D58150764

facebook-github-bot · 2024-06-05T15:21:56Z

This pull request was exported from Phabricator. Differential Revision: D58150764

xw285cornell · 2024-06-05T17:36:30Z

@jeffdaily I added ciflow/rocm, wondering if you can help take a look if all the rocm tests are triggered

jeffdaily · 2024-06-05T23:02:49Z

@jeffdaily I added ciflow/rocm, wondering if you can help take a look if all the rocm tests are triggered

Added ciflow/inductor and ciflow/periodic.

Summary: only to trigger ci test Differential Revision: D58150764

facebook-github-bot · 2024-06-06T03:37:47Z

This pull request was exported from Phabricator. Differential Revision: D58150764

facebook-github-bot · 2024-06-07T08:36:58Z

This pull request was exported from Phabricator. Differential Revision: D58150764

Summary: Pull Request resolved: #127944 It has been a constant pain that we have to specify env var to go with the hipblaslt path. The default path is very slow on MI300. Therefore, let's default to hipblaslt. Test Plan: OSS CI Differential Revision: D58150764

aaronenyeshi

LGTM

aten/src/ATen/Context.h

Summary: Pull Request resolved: pytorch#127944 It has been a constant pain that we have to specify env var to go with the hipblaslt path. The default path is very slow on MI300. Therefore, let's default to hipblaslt. Test Plan: OSS CI Reviewed By: aaronenyeshi, houseroad Differential Revision: D58150764

facebook-github-bot · 2024-06-10T08:00:47Z

This pull request was exported from Phabricator. Differential Revision: D58150764

facebook-github-bot · 2024-06-10T19:53:28Z

@pytorchbot merge -f 'Landed internally'

(Initiating merge automatically since Phabricator Diff has merged, using force because this PR might not pass merge_rules.json but landed internally)

pytorchmergebot · 2024-06-10T19:55:09Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Summary: It has been a constant pain that we have to specify env var to go with the hipblaslt path. The default path is very slow on MI300. Therefore, let's default to hipblaslt. Differential Revision: D58150764 Pull Request resolved: pytorch#127944 Approved by: https://github.com/aaronenyeshi, https://github.com/houseroad

… hipblasLT (#128753) This PR is needed to resolve usability issues with PyTorch ROCm nightly wheels on non-gfx90a/gf94x architectures as a result of #127944. Addresses #119081 (comment) ### With this PR's changes, I get the following on a gfx908 (unsupported by hipblasLT) architecture: _Using setter function:_ ``` >>> torch.backends.cuda.preferred_blas_library(backend="cublaslt") [W617 19:58:58.286088851 Context.cpp:280] Warning: torch.backends.cuda.preferred_blas_library is an experimental feature. If you see any error or unexpected behavior when this flag is set please file an issue on GitHub. (function operator()) [W617 19:59:02.125161985 Context.cpp:291] Warning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas backend to hipblas (function operator()) <_BlasBackend.Cublas: 0> ``` _Using `TORCH_BLAS_PREFER_HIPBLASLT` env var:_ ``` root@9d47bf40d4d4:/tmp/pytorch# TORCH_BLAS_PREFER_CUBLASLT=1 python >>> import torch >>> torch.backends.cuda.preferred_blas_library() [W619 06:14:11.627715807 Context.cpp:274] Warning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas backend to hipblas (function operator()) <_BlasBackend.Cublas: 0> ``` ### and the following on a gfx90a (supported by hipblasLT) architecture: _Using setter function:_ ``` >>> import torch >>> torch.backends.cuda.preferred_blas_library() <_BlasBackend.Cublaslt: 1> >>> torch.backends.cuda.preferred_blas_library(backend="cublas") <_BlasBackend.Cublas: 0> >>> torch.backends.cuda.preferred_blas_library(backend="cublaslt") [W620 18:38:29.404265518 Context.cpp:293] Warning: torch.backends.cuda.preferred_blas_library is an experimental feature. If you see any error or unexpected behavior when this flag is set please file an issue on GitHub. (function operator()) <_BlasBackend.Cublaslt: 1> ``` _Using `TORCH_BLAS_PREFER_HIPBLASLT` env var:_ ``` root@9d47bf40d4d4:/tmp/pytorch# TORCH_BLAS_PREFER_HIPBLASLT=1 python >>> import torch >>> torch.backends.cuda.preferred_blas_library() <_BlasBackend.Cublaslt: 1> ``` (Same result for _Using `TORCH_BLAS_PREFER_CUBLASLT` env var:_) Pull Request resolved: #128753 Approved by: https://github.com/malfet

… hipblasLT (#128753) This PR is needed to resolve usability issues with PyTorch ROCm nightly wheels on non-gfx90a/gf94x architectures as a result of #127944. Addresses #119081 (comment) ### With this PR's changes, I get the following on a gfx908 (unsupported by hipblasLT) architecture: _Using setter function:_ ``` >>> torch.backends.cuda.preferred_blas_library(backend="cublaslt") [W617 19:58:58.286088851 Context.cpp:280] Warning: torch.backends.cuda.preferred_blas_library is an experimental feature. If you see any error or unexpected behavior when this flag is set please file an issue on GitHub. (function operator()) [W617 19:59:02.125161985 Context.cpp:291] Warning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas backend to hipblas (function operator()) <_BlasBackend.Cublas: 0> ``` _Using `TORCH_BLAS_PREFER_HIPBLASLT` env var:_ ``` root@9d47bf40d4d4:/tmp/pytorch# TORCH_BLAS_PREFER_CUBLASLT=1 python >>> import torch >>> torch.backends.cuda.preferred_blas_library() [W619 06:14:11.627715807 Context.cpp:274] Warning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas backend to hipblas (function operator()) <_BlasBackend.Cublas: 0> ``` ### and the following on a gfx90a (supported by hipblasLT) architecture: _Using setter function:_ ``` >>> import torch >>> torch.backends.cuda.preferred_blas_library() <_BlasBackend.Cublaslt: 1> >>> torch.backends.cuda.preferred_blas_library(backend="cublas") <_BlasBackend.Cublas: 0> >>> torch.backends.cuda.preferred_blas_library(backend="cublaslt") [W620 18:38:29.404265518 Context.cpp:293] Warning: torch.backends.cuda.preferred_blas_library is an experimental feature. If you see any error or unexpected behavior when this flag is set please file an issue on GitHub. (function operator()) <_BlasBackend.Cublaslt: 1> ``` _Using `TORCH_BLAS_PREFER_HIPBLASLT` env var:_ ``` root@9d47bf40d4d4:/tmp/pytorch# TORCH_BLAS_PREFER_HIPBLASLT=1 python >>> import torch >>> torch.backends.cuda.preferred_blas_library() <_BlasBackend.Cublaslt: 1> ``` (Same result for _Using `TORCH_BLAS_PREFER_CUBLASLT` env var:_) Pull Request resolved: #128753 Approved by: https://github.com/malfet (cherry picked from commit e16276b)

… hipblasLT (#133359) [ROCm] Check supported archs before setting preferred blas backend to hipblasLT (#128753) This PR is needed to resolve usability issues with PyTorch ROCm nightly wheels on non-gfx90a/gf94x architectures as a result of #127944. Addresses #119081 (comment) ### With this PR's changes, I get the following on a gfx908 (unsupported by hipblasLT) architecture: _Using setter function:_ ``` >>> torch.backends.cuda.preferred_blas_library(backend="cublaslt") [W617 19:58:58.286088851 Context.cpp:280] Warning: torch.backends.cuda.preferred_blas_library is an experimental feature. If you see any error or unexpected behavior when this flag is set please file an issue on GitHub. (function operator()) [W617 19:59:02.125161985 Context.cpp:291] Warning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas backend to hipblas (function operator()) <_BlasBackend.Cublas: 0> ``` _Using `TORCH_BLAS_PREFER_HIPBLASLT` env var:_ ``` root@9d47bf40d4d4:/tmp/pytorch# TORCH_BLAS_PREFER_CUBLASLT=1 python >>> import torch >>> torch.backends.cuda.preferred_blas_library() [W619 06:14:11.627715807 Context.cpp:274] Warning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas backend to hipblas (function operator()) <_BlasBackend.Cublas: 0> ``` ### and the following on a gfx90a (supported by hipblasLT) architecture: _Using setter function:_ ``` >>> import torch >>> torch.backends.cuda.preferred_blas_library() <_BlasBackend.Cublaslt: 1> >>> torch.backends.cuda.preferred_blas_library(backend="cublas") <_BlasBackend.Cublas: 0> >>> torch.backends.cuda.preferred_blas_library(backend="cublaslt") [W620 18:38:29.404265518 Context.cpp:293] Warning: torch.backends.cuda.preferred_blas_library is an experimental feature. If you see any error or unexpected behavior when this flag is set please file an issue on GitHub. (function operator()) <_BlasBackend.Cublaslt: 1> ``` _Using `TORCH_BLAS_PREFER_HIPBLASLT` env var:_ ``` root@9d47bf40d4d4:/tmp/pytorch# TORCH_BLAS_PREFER_HIPBLASLT=1 python >>> import torch >>> torch.backends.cuda.preferred_blas_library() <_BlasBackend.Cublaslt: 1> ``` (Same result for _Using `TORCH_BLAS_PREFER_CUBLASLT` env var:_) Pull Request resolved: #128753 Approved by: https://github.com/malfet (cherry picked from commit e16276b) Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>

facebook-github-bot added the fb-exported label Jun 4, 2024

xw285cornell force-pushed the export-D58150764 branch from 192f931 to cdf1c35 Compare June 5, 2024 03:44

xw285cornell added a commit to xw285cornell/pytorch that referenced this pull request Jun 5, 2024

[TEST ONLY] always use hipblaslt (pytorch#127944)

cdf1c35

Summary: only to trigger ci test Differential Revision: D58150764

xw285cornell added the ciflow/rocm Trigger "default" config CI on ROCm label Jun 5, 2024

xw285cornell force-pushed the export-D58150764 branch from cdf1c35 to 0d1fe70 Compare June 5, 2024 15:21

xw285cornell added a commit to xw285cornell/pytorch that referenced this pull request Jun 5, 2024

[TEST ONLY] always use hipblaslt (pytorch#127944)

0d1fe70

Summary: only to trigger ci test Differential Revision: D58150764

xw285cornell requested a review from jeffdaily June 5, 2024 17:35

jeffdaily added ciflow/periodic Trigger jobs ran periodically on master (periodic.yml) on the PR ciflow/inductor labels Jun 5, 2024

xw285cornell force-pushed the export-D58150764 branch from 0d1fe70 to d6cef6b Compare June 6, 2024 03:37

xw285cornell added a commit to xw285cornell/pytorch that referenced this pull request Jun 6, 2024

[TEST ONLY] always use hipblaslt (pytorch#127944)

d6cef6b

Summary: only to trigger ci test Differential Revision: D58150764

xw285cornell changed the title ~~[TEST ONLY] always use hipblaslt~~ [AMD] Default to hipblaslt in gemm Jun 7, 2024

xw285cornell force-pushed the export-D58150764 branch from d6cef6b to 2edb9c2 Compare June 7, 2024 08:37

aaronenyeshi approved these changes Jun 7, 2024

View reviewed changes

aten/src/ATen/Context.h Show resolved Hide resolved

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jun 7, 2024

houseroad approved these changes Jun 7, 2024

View reviewed changes

xw285cornell force-pushed the export-D58150764 branch from 2edb9c2 to 5c34b21 Compare June 10, 2024 08:00

pytorchmergebot added the merging label Jun 10, 2024

pytorchmergebot closed this in 38e0a04 Jun 10, 2024

pytorchmergebot added Merged and removed merging labels Jun 10, 2024

AngryLoki mentioned this pull request Jun 13, 2024

ROCm loses some supported GPUs by requiring hipblaslt #119081

Closed

jithunnair-amd mentioned this pull request Jun 22, 2024

[ROCm] Check supported archs before setting preferred blas backend to hipblasLT #128753

Closed

pytorchbot mentioned this pull request Aug 13, 2024

[ROCm] Check supported archs before setting preferred blas backend to hipblasLT #133359

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[AMD] Default to hipblaslt in gemm #127944

[AMD] Default to hipblaslt in gemm #127944

Uh oh!

xw285cornell commented Jun 4, 2024 •

edited

Loading

Uh oh!

pytorch-bot bot commented Jun 4, 2024 •

edited

Loading

Uh oh!

facebook-github-bot commented Jun 4, 2024

Uh oh!

facebook-github-bot commented Jun 5, 2024

Uh oh!

facebook-github-bot commented Jun 5, 2024

Uh oh!

xw285cornell commented Jun 5, 2024

Uh oh!

jeffdaily commented Jun 5, 2024

Uh oh!

facebook-github-bot commented Jun 6, 2024

Uh oh!

facebook-github-bot commented Jun 7, 2024

Uh oh!

aaronenyeshi left a comment

Uh oh!

Uh oh!

facebook-github-bot commented Jun 10, 2024

Uh oh!

facebook-github-bot commented Jun 10, 2024

Uh oh!

pytorchmergebot commented Jun 10, 2024

Uh oh!

Uh oh!

[AMD] Default to hipblaslt in gemm #127944

[AMD] Default to hipblaslt in gemm #127944

Uh oh!

Conversation

xw285cornell commented Jun 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jun 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/127944

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

facebook-github-bot commented Jun 4, 2024

Uh oh!

facebook-github-bot commented Jun 5, 2024

Uh oh!

facebook-github-bot commented Jun 5, 2024

Uh oh!

xw285cornell commented Jun 5, 2024

Uh oh!

jeffdaily commented Jun 5, 2024

Uh oh!

facebook-github-bot commented Jun 6, 2024

Uh oh!

facebook-github-bot commented Jun 7, 2024

Uh oh!

aaronenyeshi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

facebook-github-bot commented Jun 10, 2024

Uh oh!

facebook-github-bot commented Jun 10, 2024

Uh oh!

pytorchmergebot commented Jun 10, 2024

Merge started

Uh oh!

Uh oh!

xw285cornell commented Jun 4, 2024 •

edited

Loading

pytorch-bot bot commented Jun 4, 2024 •

edited

Loading