Skip to content

Add ImageDecoder operator #5294

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 17 commits into from
Aug 10, 2023
Merged

Add ImageDecoder operator #5294

merged 17 commits into from
Aug 10, 2023

Conversation

jantonguirao
Copy link
Contributor

@jantonguirao jantonguirao commented Jun 8, 2023 •
edited
Loading

Description

  • Adds a new ImageDecoder operator, to be used in preprocessing models.
  • Tests use generated images and opencv as a reference.

Motivation and Context

Currently, image decoding happens outside of ONNX, bring problems of portability and deployment.

@jantonguirao jantonguirao force-pushed the image_decoder_op branch 3 times, most recently from 9d4181b to bc35a8c Compare June 8, 2023 09:46
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 8, 2023
View reviewed changes
Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

@jantonguirao jantonguirao force-pushed the image_decoder_op branch 9 times, most recently from a5b6aeb to a17f761 Compare June 14, 2023 13:23
xadupre
xadupre reviewed Jun 14, 2023
View reviewed changes
xadupre
xadupre reviewed Jun 14, 2023
View reviewed changes
xadupre
xadupre reviewed Jun 14, 2023
View reviewed changes
@jantonguirao jantonguirao force-pushed the image_decoder_op branch 2 times, most recently from d0d618a to 9cb3740 Compare June 14, 2023 19:24
@gramalingam gramalingam added the topic: operator Issues related to ONNX operators label Jun 21, 2023
gramalingam
gramalingam reviewed Jul 11, 2023
View reviewed changes
gramalingam
gramalingam reviewed Jul 11, 2023
View reviewed changes
@jantonguirao
Add ImageDecoder operator
c7bfab5 
Signed-off-by: Joaquin Anton <janton@nvidia.com>
@jantonguirao
Make opencv an optional dependency
5b43cfb 
Signed-off-by: Joaquin Anton <janton@nvidia.com>
@jantonguirao
Add ai.onnx.io.image domain
e5a6ad5 
Signed-off-by: Joaquin Anton <janton@nvidia.com>
@jantonguirao jantonguirao force-pushed the image_decoder_op branch 3 times, most recently from 3ce0ed0 to f78c40f Compare July 12, 2023 12:34
@jantonguirao jantonguirao changed the title Add ImageDecoder operator Add ai.onnx.io.ImageDecoder operator Jul 12, 2023
@jantonguirao jantonguirao force-pushed the image_decoder_op branch 2 times, most recently from 106abca to af982b2 Compare July 12, 2023 13:11
@jantonguirao
Copy link
Contributor Author

As agreed during the Operators SIG meeting, we will not use a separate domain for this operator, at least for the time being.

@jantonguirao
Fixes
c207d3b 
Signed-off-by: Joaquin Anton <janton@nvidia.com>
@jantonguirao
Fix CodeQL errors
f521b14 
Signed-off-by: Joaquin Anton <janton@nvidia.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jul 31, 2023
View reviewed changes
onnx/reference/ops/_op_list.py
@@ -113,6 +113,7 @@
from onnx.reference.ops.op_hardmax import Hardmax
from onnx.reference.ops.op_identity import Identity
from onnx.reference.ops.op_if import If
from onnx.reference.ops.op_image_decoder import ImageDecoder

Check notice

Code scanning / CodeQL

Unused import

Import of 'ImageDecoder' is not used.
@jantonguirao
Merge remote-tracking branch 'upstream/main' into image_decoder_op
d0251d2 
Signed-off-by: Joaquin Anton <janton@nvidia.com>
xadupre
xadupre reviewed Aug 4, 2023
View reviewed changes
@jantonguirao
Merge remote-tracking branch 'upstream/main' into image_decoder_op
d03c247 
Signed-off-by: Joaquin Anton <janton@nvidia.com>
adityagoel4512
adityagoel4512 reviewed Aug 8, 2023
View reviewed changes
@jantonguirao
Remove duplicated dependency
00310de 
Signed-off-by: Joaquin Anton <janton@nvidia.com>
@xadupre xadupre added this pull request to the merge queue Aug 9, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 9, 2023
@jantonguirao
Merge remote-tracking branch 'upstream/main' into image_decoder_op
5d8e38a 
Signed-off-by: Joaquin Anton <janton@nvidia.com>
@xadupre xadupre added this pull request to the merge queue Aug 10, 2023
Merged via the queue into onnx:main with commit 4ad6d31 Aug 10, 2023
@lutzroeder lutzroeder mentioned this pull request Aug 10, 2023
Closed
justinchuby added a commit to microsoft/onnxscript that referenced this pull request Aug 25, 2023
@justinchuby
Pin onnx-weekly to dev20230807 in requirements-dev | fix(release)
43df3bf 
onnx/onnx#5294 introduced a dependency to cv2 in tests, which fails our release pipeline. Instead of requiring cv2 we pin onnx to dev20230807 for now to unblock the release pipeline.
@justinchuby justinchuby mentioned this pull request Aug 25, 2023
Merged
justinchuby added a commit to microsoft/onnxscript that referenced this pull request Aug 25, 2023
@justinchuby @titaiwangms
Pin onnx-weekly to dev20230807 in requirements-dev | fix(release) (#1026
61466de 
)

onnx/onnx#5294 introduced a dependency to cv2 in
tests, which fails our release pipeline. Instead of requiring cv2 we pin
onnx to dev20230807 for now to unblock the release pipeline.

---------

Co-authored-by: Ti-Tai Wang <titaiwang@microsoft.com>
@liqunfu
Copy link
Collaborator

liqunfu commented Sep 18, 2023

Hi @jantonguirao , thank you for adding this op. Now I am preparing ONNX 1.15.0 release and integrating the release with ORT, I wonder if you have a plan to implement the runtime kernel for this new op in ORT? If so, I will let you know once ORT has ONNX 1.15.0 integrated. Thank you in advance.

@liqunfu liqunfu mentioned this pull request Sep 18, 2023
Open
@jantonguirao
Copy link
Contributor Author

Hi @jantonguirao , thank you for adding this op. Now I am preparing ONNX 1.15.0 release and integrating the release with ORT, I wonder if you have a plan to implement the runtime kernel for this new op in ORT? If so, I will let you know once ORT has ONNX 1.15.0 integrated. Thank you in advance.

Hi @liqunfu. I was hoping that the ORT team will handle the implementation. It should be simple using a library such as OpenCV.

@liqunfu
Copy link
Collaborator

liqunfu commented Sep 19, 2023

Hi @jantonguirao , thank you for adding this op. Now I am preparing ONNX 1.15.0 release and integrating the release with ORT, I wonder if you have a plan to implement the runtime kernel for this new op in ORT? If so, I will let you know once ORT has ONNX 1.15.0 integrated. Thank you in advance.

Hi @liqunfu. I was hoping that the ORT team will handle the implementation. It should be simple using a library such as OpenCV.

Thank you @jantonguirao , I will add this to ORT release work item and have ORT team work on it.

@ScottTodd ScottTodd mentioned this pull request Jun 13, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@justinchuby justinchuby justinchuby left review comments

@gramalingam gramalingam gramalingam approved these changes

@xadupre xadupre xadupre approved these changes

+1 more reviewer

@adityagoel4512 adityagoel4512 adityagoel4512 left review comments

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
topic: operator Issues related to ONNX operators
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@jantonguirao @liqunfu @gramalingam @justinchuby @xadupre @adityagoel4512