Skip to content

[MPS] adding weight_norm_interface support for mps #108008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 8 commits into from
Closed

[MPS] adding weight_norm_interface support for mps #108008

wants to merge 8 commits into from

Conversation

igm503
Copy link
Contributor

@igm503 igm503 commented Aug 26, 2023 •
edited
Loading

Fixes #104513

Adds support for aten::_weight_norm_interface to the mps backend.

Also adds a consistency test for the output and the grad.

@pytorch-bot pytorch-bot bot added ciflow/mps Run MPS tests (subset of trunk) release notes: mps Release notes category labels Aug 26, 2023
@pytorch-bot
Copy link

pytorch-bot bot commented Aug 26, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/108008

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit bd79d8d with merge base 0bf30c1 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@igm503 igm503 closed this Sep 2, 2023
@igm503 igm503 deleted the mpw_weight_norm branch September 2, 2023 20:08
@igm503 igm503 restored the mpw_weight_norm branch September 2, 2023 20:13
@igm503 igm503 reopened this Sep 2, 2023
@igm503 igm503 marked this pull request as ready for review September 4, 2023 01:47
@igm503 igm503 requested a review from kulinseth as a code owner September 4, 2023 01:47
@igm503 igm503 changed the title adding weight_norm_interface support for mps [MPS] adding weight_norm_interface support for mps Sep 4, 2023
@soulitzer soulitzer added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Sep 4, 2023
@igm503
Copy link
Contributor Author

igm503 commented Sep 19, 2023

@kulinseth This should be good to go now. I've added a test in test/test_mps.py

kulinseth
kulinseth approved these changes Sep 19, 2023
View reviewed changes
Copy link
Collaborator

@kulinseth kulinseth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great thanks !!

@igm503
Copy link
Contributor Author

igm503 commented Sep 19, 2023

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 19, 2023
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@varunneal varunneal mentioned this pull request Nov 9, 2023
Open
@fakerybakery
Copy link

Hi, for some reason this still isn't working...

@fakerybakery
Copy link

I'm on 2.1.1 and getting:

NotImplementedError: The operator 'aten::_weight_norm_interface' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

@igm503
Copy link
Contributor Author

igm503 commented Nov 16, 2023

General MPS op coverage tracking issue #77764

I'm on 2.1.1 and getting:

NotImplementedError: The operator 'aten::_weight_norm_interface' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

@fakerybakery This wasn't added in time for 2.1.1. It should be in the next release of PyTorch. In the meantime, you should be able to use it in the latest nightly versions of PyTorch.

@fakerybakery
Copy link

Ok, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kulinseth kulinseth kulinseth approved these changes

Assignees
No one assigned
Labels
ciflow/mps Run MPS tests (subset of trunk) ciflow/trunk Trigger trunk jobs on your pull request Merged open source release notes: mps Release notes category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

implement weight_norm on mps
6 participants
@igm503 @pytorchmergebot @fakerybakery @kulinseth @soulitzer @pytorchbot