Skip to content

add GroupNormalization-18 #4621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 18 commits into from
Nov 4, 2022
Merged

add GroupNormalization-18 #4621

merged 18 commits into from
Nov 4, 2022

Conversation

yuanyao-nv
Copy link
Contributor

@yuanyao-nv yuanyao-nv commented Oct 25, 2022 •
edited
Loading

Signed-off-by: Yuan Yao yuanyao@nvidia.com

Description

Add new function op GroupNormalization to opset 18.

Motivation and Context

Group norm is a commonly used normalization op, similar to some of the existing ONNX ops such as BatchNormalization, InstanceNormalization, and LayerNormalization.

Original paper that introduced the op: https://arxiv.org/abs/1803.08494

Related issue: #2020

Implementations in other frameworks:
https://www.tensorflow.org/addons/api_docs/python/tfa/layers/GroupNormalization
https://pytorch.org/docs/stable/generated/torch.nn.GroupNorm.html

@yuanyao-nv
add GroupNormalization-18
5025a24 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv yuanyao-nv requested review from a team as code owners October 25, 2022 22:12
github-advanced-security[bot]
github-advanced-security bot found potential problems Oct 25, 2022
View reviewed changes
@yuanyao-nv
clean up function body
f842aff 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@gramalingam gramalingam added the topic: operator Issues related to ONNX operators label Oct 26, 2022
@yuanyao-nv
fix lint error
5275f87 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv
fix pytest error in function body
9186ada 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv
fix lint error
fbaf918 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv
fix lint error 2
fee8e08 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv
fix lint error 3
03d0113 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv
fix lint error 4
7f13339 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv
update md files
a0a27e5 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@rajeevsrao rajeevsrao added this to the 1.13 milestone Oct 31, 2022
gramalingam
gramalingam reviewed Nov 1, 2022
View reviewed changes
gramalingam
gramalingam reviewed Nov 1, 2022
View reviewed changes
gramalingam
gramalingam reviewed Nov 1, 2022
View reviewed changes
gramalingam
gramalingam reviewed Nov 1, 2022
View reviewed changes
gramalingam
gramalingam reviewed Nov 1, 2022
View reviewed changes
@yuanyao-nv
Merge branch 'main' of https://github.com/onnx/onnx into dev-groupnorm
e9b4ad8
@yuanyao-nv
review feedback
f19c46b 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv
fix lint error
a8764bc 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
gramalingam
gramalingam reviewed Nov 2, 2022
View reviewed changes
gramalingam
gramalingam reviewed Nov 2, 2022
View reviewed changes
@gramalingam
Copy link
Contributor

LGTM. I think there may be a minor interaction between this PR and #4512 (which is rationalizing all Reduce* functions to take axes as an input instead of attribute) which will need to be resolved when both get merged.

@yuanyao-nv
update python test script per review feedback
7797d3c 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv
fix python script lint issue
d990c67 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv
fix more python lint issues
944362f 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
@yuanyao-nv
Copy link
Contributor Author

@gramalingam Thanks for the suggestions. I've made the updates. Please review again. Thanks!

@gramalingam
Copy link
Contributor

gramalingam commented Nov 3, 2022 •
edited
Loading

LGTM, thanks! I was wondering whether you had a chance to test the correctness of the function-body definition (eg., by running onnxruntime, or any other backend, on the generated expanded test-cases)? You may need to manually patch the opset of the generated test-cases to be the previous opset, and then it should be a valid test-case for the previous opset, I think, runnable by onnxruntime etc.

@yuanyao-nv
Copy link
Contributor Author

LGTM, thanks! I was wondering whether you had a chance to test the correctness of the function-body definition (eg., by running onnxruntime, or any other backend, on the generated expanded test-cases)? You may need to manually patch the opset of the generated test-cases to be the previous opset, and then it should be a valid test-case for the previous opset, I think, runnable by onnxruntime etc.

Thanks for pointing this out. I just tested and actually found a small error in the function body. Now fixed.

@gramalingam gramalingam merged commit fe78700 into onnx:main Nov 4, 2022
@yuanyao-nv yuanyao-nv deleted the dev-groupnorm branch November 4, 2022 22:24
@sunjiahao1999 sunjiahao1999 mentioned this pull request Dec 26, 2022
Merged
justinchuby pushed a commit to justinchuby/onnx that referenced this pull request Jan 27, 2023
@yuanyao-nv @gramalingam @justinchuby
add GroupNormalization-18 (onnx#4621)
13bdb33 
* add GroupNormalization-18

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* clean up function body

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix pytest error in function body

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error 2

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error 3

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error 4

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* update md files

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* review feedback

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* update python test script per review feedback

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix python script lint issue

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix more python lint issues

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix Reshape error

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
Co-authored-by: G. Ramalingam <grama@microsoft.com>
broune pushed a commit to broune/onnx that referenced this pull request May 6, 2023
@yuanyao-nv @gramalingam
add GroupNormalization-18 (onnx#4621)
a076b57 
* add GroupNormalization-18

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* clean up function body

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix pytest error in function body

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error 2

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error 3

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error 4

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* update md files

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* review feedback

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix lint error

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* update python test script per review feedback

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix python script lint issue

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix more python lint issues

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

* fix Reshape error

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>

Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
Co-authored-by: G. Ramalingam <grama@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gramalingam gramalingam gramalingam approved these changes

Assignees
No one assigned
Labels
topic: operator Issues related to ONNX operators
Projects
None yet
Milestone
1.13.0
Development

Successfully merging this pull request may close these issues.
3 participants
@yuanyao-nv @gramalingam @rajeevsrao