Skip to content

Conversation

justinchuby
Copy link
Member

@justinchuby justinchuby commented Jan 10, 2025

Description

Refactor to_array to simplify and fix conversion logic.

  • The _to_array function was renamed back to to_array, consolidating with existing logic that leverages custom dtypes.
  • Created an unpack_4bit helper to unpack packed 4bit dtype data, replacing usage of the 4bit->float32 conversion functions because the original logic was not actually needed.
  • Removed test skips because bugs were fixed.
  • Type casting functions for custom dtypes will be marked private in the next PR. Public versions of them will be scheduled for deprecation. (Deprecate all type casting functions #6639) Users should use other libraries like ml_dtypes for efficient type conversion in numpy.

Motivation and Context

Fix #6604

Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Copy link

codecov bot commented Jan 10, 2025

Codecov Report

Attention: Patch coverage is 98.48485% with 1 line in your changes missing coverage. Please review.

Project coverage is 57.46%. Comparing base (5ab1e42) to head (c9269dc).
Report is 4 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
onnx/numpy_helper.py 97.95% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6638      +/-   ##
==========================================
+ Coverage   57.44%   57.46%   +0.02%     
==========================================
  Files         507      507              
  Lines       31612    31584      -28     
  Branches     3058     3046      -12     
==========================================
- Hits        18158    18151       -7     
+ Misses      12610    12607       -3     
+ Partials      844      826      -18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can commit the suggested changes from lintrunner.

Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
@justinchuby justinchuby marked this pull request as ready for review January 11, 2025 01:56
@justinchuby justinchuby requested a review from a team as a code owner January 11, 2025 01:56
@justinchuby justinchuby added this to the 1.18 milestone Jan 11, 2025
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
@justinchuby justinchuby added the run release CIs Use this label to trigger release tests in CI label Jan 13, 2025
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
@justinchuby justinchuby added run release CIs Use this label to trigger release tests in CI and removed run release CIs Use this label to trigger release tests in CI labels Jan 13, 2025
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
@justinchuby justinchuby enabled auto-merge January 13, 2025 21:53
@justinchuby justinchuby requested a review from andife January 13, 2025 22:17
@justinchuby justinchuby added this pull request to the merge queue Jan 14, 2025
Merged via the queue into main with commit 8275472 Jan 14, 2025
35 checks passed
@justinchuby justinchuby deleted the justinchu/bfloat16-to-array branch January 14, 2025 19:33
@justinchuby justinchuby added the release notes Important changes to call out in release notes label Jan 22, 2025
@justinchuby justinchuby temporarily deployed to testpypi_onnxweekly January 22, 2025 14:59 — with GitHub Actions Inactive
@justinchuby justinchuby temporarily deployed to testpypi_onnxweekly January 22, 2025 15:12 — with GitHub Actions Inactive
andife pushed a commit that referenced this pull request Jan 25, 2025
### Description

Refactor `to_array` to simplify and fix conversion logic.

- The `_to_array` function was renamed back to `to_array`, consolidating
with existing logic that leverages custom dtypes.
- Created an unpack_4bit helper to unpack packed 4bit dtype data,
replacing usage of the 4bit->float32 conversion functions because the
original logic was not actually needed.
- Removed test skips because bugs were fixed.
- Type casting functions for custom dtypes will be marked private in the
next PR. Public versions of them will be scheduled for deprecation.
(#6639) Users should use other libraries like ml_dtypes for efficient
type conversion in numpy.

### Motivation and Context

Fix #6604

---------

Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
seungwoo-ji-03 pushed a commit to seungwoo-ji-03/onnx that referenced this pull request Feb 17, 2025
### Description

Refactor `to_array` to simplify and fix conversion logic.

- The `_to_array` function was renamed back to `to_array`, consolidating
with existing logic that leverages custom dtypes.
- Created an unpack_4bit helper to unpack packed 4bit dtype data,
replacing usage of the 4bit->float32 conversion functions because the
original logic was not actually needed.
- Removed test skips because bugs were fixed.
- Type casting functions for custom dtypes will be marked private in the
next PR. Public versions of them will be scheduled for deprecation.
(onnx#6639) Users should use other libraries like ml_dtypes for efficient
type conversion in numpy.

### Motivation and Context

Fix onnx#6604

---------

Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Signed-off-by: seungwoo-ji <seungwoo.ji@nuvilab.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: utility Helper modules release notes Important changes to call out in release notes run release CIs Use this label to trigger release tests in CI topic: bug fix
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

numpy_helper.to_array returns empty array when given bfloat16 TensorProto with raw_data
2 participants