Skip to content

Makes to_array, from_array support custom numpy dtype, support float16 type in parser #6170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 41 commits into from
Jul 9, 2024
Merged

Makes to_array, from_array support custom numpy dtype, support float16 type in parser #6170

merged 41 commits into from
Jul 9, 2024

Conversation

xadupre
Copy link
Contributor

@xadupre xadupre commented Jun 11, 2024 •
edited
Loading

Description

from_array, to_array do not support float 8 types, int4, uint4 bfloat16. It is confusing.
Fixes #6152 as well.

@xadupre
Makes to_array, from_array support custom numpy dtype
5141ed5 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre xadupre requested review from a team as code owners June 11, 2024 11:13
@xadupre xadupre mentioned this pull request Jun 11, 2024
Open
@codecov Codecov
Copy link

codecov bot commented Jun 11, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 87.05357% with 29 lines in your changes missing coverage. Please review.

Project coverage is 57.14%. Comparing base (83194ed) to head (622e9c2).
Report is 63 commits behind head on main.

Files Patch % Lines
onnx/numpy_helper.py 89.28% 4 Missing and 5 partials ⚠️
onnx/reference/custom_element_types.py 27.27% 7 Missing and 1 partial ⚠️
onnx/test/reference_evaluator_test.py 20.00% 4 Missing ⚠️
onnx/test/test_backend_reference.py 25.00% 2 Missing and 1 partial ⚠️
onnx/helper.py 75.00% 1 Missing and 1 partial ⚠️
onnx/backend/test/case/node/cast.py 0.00% 1 Missing ⚠️
onnx/test/numpy_helper_test.py 97.22% 1 Missing ⚠️
onnx/test/parser_test.py 87.50% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #6170      +/-   ##
==========================================
+ Coverage   56.95%   57.14%   +0.18%     
==========================================
  Files         506      507       +1     
  Lines       30467    31211     +744     
  Branches     4592     4657      +65     
==========================================
+ Hits        17353    17834     +481     
- Misses      12285    12531     +246     
- Partials      829      846      +17

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@xadupre
improves
19ab1ba 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 11, 2024
View reviewed changes
gramalingam
gramalingam reviewed Jun 11, 2024
View reviewed changes
@xadupre
fix int4
b228b1a 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 11, 2024
View reviewed changes
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 11, 2024
View reviewed changes
xadupre added 6 commits June 12, 2024 12:11
@xadupre
style
6dc6193 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
Merge branch 'main' of https://github.com/onnx/onnx into map
45884a4
@xadupre
better style
0df7bda 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
add missing names
a4335bb 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
Merge branch 'main' of https://github.com/onnx/onnx into map
a8829af
@xadupre
lint
27ed758 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@justinchuby
Copy link
Member

Do we want to make custom_element_types.py public? It would be helpful to understand the pros and cons of doing so. (And mention it in the PR description when we make a decision)

xadupre added 2 commits June 12, 2024 20:34
@xadupre
lint
3f775ea 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
isort
58dc9a5 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
Copy link
Contributor Author

xadupre commented Jun 13, 2024

Do we want to make custom_element_types.py public? It would be helpful to understand the pros and cons of doing so. (And mention it in the PR description when we make a decision)

The types defined here are returned by to_array. Why not making them public?

@xadupre
lint
e050e5d 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 13, 2024
View reviewed changes
@xadupre
fix implementation
97deeae 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 13, 2024
View reviewed changes
@xadupre
lint
076b00b 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 13, 2024
View reviewed changes
xadupre added 2 commits June 13, 2024 12:02
@xadupre
check types in the parser
0cf7812 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
fix parser
7c67bc3 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre xadupre changed the title Makes to_array, from_array support custom numpy dtype Makes to_array, from_array support custom numpy dtype, support float16 type in parser Jun 13, 2024
justinchuby
justinchuby reviewed Jun 21, 2024
View reviewed changes
justinchuby
justinchuby reviewed Jun 21, 2024
View reviewed changes
justinchuby
justinchuby reviewed Jun 21, 2024
View reviewed changes
justinchuby
justinchuby reviewed Jun 21, 2024
View reviewed changes
xadupre and others added 6 commits June 23, 2024 13:31
@xadupre @justinchuby
Update onnx/reference/ops/op_batch_normalization.py
c47a231 
Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>
Signed-off-by: Xavier Dupré <xadupre@users.noreply.github.com>
@xadupre @justinchuby
Update onnx/test/helper_test.py
9fe307d 
Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>
Signed-off-by: Xavier Dupré <xadupre@users.noreply.github.com>
@xadupre @justinchuby
Update onnx/reference/ops/op_batch_normalization.py
bbd5bf8 
Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>
Signed-off-by: Xavier Dupré <xadupre@users.noreply.github.com>
@xadupre
raise an exception for odd size
1eac003 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
lint
67781f3 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@gramalingam
Merge branch 'main' into map
0ac10a3
@gramalingam gramalingam enabled auto-merge July 3, 2024 20:52
@gramalingam gramalingam added this to the 1.17 milestone Jul 3, 2024
xadupre and others added 9 commits July 5, 2024 13:05
@xadupre
Merge branch 'main' of https://github.com/onnx/onnx into map
06a29aa
@xadupre
fix numpy issue
fd2b0d1 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
update pillow requirement for numpy2.0
61d5d5d 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
tries
65cdbce 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
disable test for older version of pillow
c5d47f1 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@xadupre
disable one more test
0041c7a 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@gramalingam
Address lintrunner complaint
ff9b02e
@gramalingam
Merge branch 'main' into map
ef3021b
@xadupre
disable
622e9c2 
Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
@gramalingam gramalingam added this pull request to the merge queue Jul 9, 2024
Merged via the queue into onnx:main with commit 6179ca1 Jul 9, 2024
@xadupre xadupre mentioned this pull request Jul 12, 2024
Closed
andife pushed a commit to andife/onnx that referenced this pull request Jul 20, 2024
@xadupre @justinchuby
@gramalingam @andife
Makes to_array, from_array support custom numpy dtype, support float1…
584982a 
…6 type in parser (onnx#6170)

### Description
from_array, to_array do not support float 8 types, int4, uint4 bfloat16.
It is confusing.
Fixes onnx#6152 as well.

---------

Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
Signed-off-by: Xavier Dupré <xadupre@users.noreply.github.com>
Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>
Co-authored-by: G. Ramalingam <grama@microsoft.com>
Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
linshokaku pushed a commit to linshokaku/onnx that referenced this pull request Oct 2, 2024
@xadupre @justinchuby
@gramalingam @linshokaku
Makes to_array, from_array support custom numpy dtype, support float1…
e1f5a3c 
…6 type in parser (onnx#6170)

### Description
from_array, to_array do not support float 8 types, int4, uint4 bfloat16.
It is confusing.
Fixes onnx#6152 as well.

---------

Signed-off-by: Xavier Dupre <xadupre@microsoft.com>
Signed-off-by: Xavier Dupré <xadupre@users.noreply.github.com>
Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>
Co-authored-by: G. Ramalingam <grama@microsoft.com>
Signed-off-by: Linsho Kaku <linsho@preferred.jp>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gramalingam gramalingam gramalingam approved these changes

@justinchuby justinchuby justinchuby approved these changes

Assignees

@xadupre xadupre

Labels
run release CIs Use this label to trigger release tests in CI
Projects
PR Tracker
Status: Done
Milestone
1.17
Development

Successfully merging this pull request may close these issues.

Missing type support in parser for various types (float16, bfloat16, ...)
3 participants
@xadupre @justinchuby @gramalingam