Introduce CenterCropPad, add `axes` to Pad #4190

jantonguirao · 2022-05-05T16:02:52Z

Signed-off-by: Joaquin Anton janton@nvidia.com

Description

Adds a new function: CenterCropPad-18.
CenterCropPad is a typical operation in image processing application.
It has been implemented as a function in terms of Pad and Slice
Pad-18 to have axes attribute

Motivation and Context

Why is this change required? What problem does it solve?
Enables writing image preprocessing pipelines (such as ResNet-50)

Signed-off-by: Joaquin Anton <janton@nvidia.com>

gramalingam · 2022-05-26T15:50:50Z

onnx/defs/tensor/defs.cc

+        .SetContextDependentFunctionBodyBuilder(
+            [](const FunctionBodyBuildContext& ctx, const OpSchema& schema, FunctionProto& functionProto) {
+              const auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
+              const int64_t input_rank = input_shape.dim_size();


Unfortunately, this is going to be a problem. The shape we can get is only a static-shape, not a dynamic-shape. Statically, we may not know the rank of the input.

Instead, one alternative is to get the (full) shape, convert the axes attribute into a tensor, and use it to index into the full-shape using some form of gather op.

I went with a separate code path when axes are not provided. Also, I took your suggestion and used Gather instead of extending Shape.

Signed-off-by: Joaquin Anton <janton@nvidia.com>

onnx/defs/tensor/defs.cc

Signed-off-by: Joaquin Anton <janton@nvidia.com>

gramalingam · 2022-06-21T20:41:19Z

LGTM, thanks. Have the test-cases been validated with ORT? Unfortunately, there is no other way to validate function definitions.

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao · 2022-06-22T17:44:05Z

LGTM, thanks. Have the test-cases been validated with ORT? Unfortunately, there is no other way to validate function definitions.

Yes. I have validated the test cases that don't depend on Pad-18 having support for axes

p-wysocki

Left one comment, LGTM otherwise.

p-wysocki · 2022-06-23T15:22:35Z

onnx/defs/tensor/defs.cc

+            3,
+            "axes",
+            "1-D tensor of axes that `pads` apply to. Negative value means counting dimensions "
+            "from the back. Accepted range is [-r, r-1] where r = rank(data). Behavior is undefined if an "


Is it possible to add a warning/exception when an axis is repeated, like in case of Resize-18?

Signed-off-by: Joaquin Anton <janton@nvidia.com>

* Introduce ai.onnx.image.CenterCropPad Signed-off-by: Joaquin Anton <janton@nvidia.com> * Remove ai.onnx.image domain Signed-off-by: Joaquin Anton <janton@nvidia.com> * Add Pad axes Signed-off-by: Joaquin Anton <janton@nvidia.com> * Add axes to Shape, update CenterCropPad to work with arbitrary axes Signed-off-by: Joaquin Anton <janton@nvidia.com> * Fix shape inference file Signed-off-by: Joaquin Anton <janton@nvidia.com> * Undo Shape-17 Signed-off-by: Joaquin Anton <janton@nvidia.com> * Update doc Signed-off-by: Joaquin Anton <janton@nvidia.com> * Linter fix Signed-off-by: Joaquin Anton <janton@nvidia.com> * Create Const directly from attribute Signed-off-by: Joaquin Anton <janton@nvidia.com> * Move to opset 18 Signed-off-by: Joaquin Anton <janton@nvidia.com> * Apply clang-format Signed-off-by: Joaquin Anton <janton@nvidia.com> * Update opset in shape_inference_tests Signed-off-by: Joaquin Anton <janton@nvidia.com> * Fix shape inference tests Signed-off-by: Joaquin Anton <janton@nvidia.com> * Add automatic upgrade test Signed-off-by: Joaquin Anton <janton@nvidia.com> * Update CenterCropPad tests Signed-off-by: Joaquin Anton <janton@nvidia.com> * Refactor tests Signed-off-by: Joaquin Anton <janton@nvidia.com> * Update tests Signed-off-by: Joaquin Anton <janton@nvidia.com> * Check for repeated axes Signed-off-by: Joaquin Anton <janton@nvidia.com> * Fix flake8 issues Signed-off-by: Joaquin Anton <janton@nvidia.com> * Fix clang-format issue Signed-off-by: Joaquin Anton <janton@nvidia.com> * Fix CenterCropPad tests with axes Signed-off-by: Joaquin Anton <janton@nvidia.com> * Update doc Signed-off-by: Joaquin Anton <janton@nvidia.com>

fdwr · 2024-09-20T02:18:00Z

What is the behavior for odd input sizes? Floor? Ceil?
When padded, what is the fill value? Zero? Edge repetition?

The documentation currently lacks these salient details - https://onnx.ai/onnx/operators/onnx__CenterCropPad.html#l-onnx-doc-centercroppad. Is the following correct?

If the input dimensions are bigger than the crop shape, a centered cropping window is extracted from the input. If the input dimensions are smaller than the crop shape, the input is padded on each side equally <with zero values>, so that the input is centered in the output. <If the input dimensions are odd relative to the cropping window, the offset is floored>.

jantonguirao · 2024-10-16T11:50:50Z

What is the behavior for odd input sizes? Floor? Ceil?

When padded, what is the fill value? Zero? Edge repetition?

The documentation currently lacks these salient details - https://onnx.ai/onnx/operators/onnx__CenterCropPad.html#l-onnx-doc-centercroppad. Is the following correct?

If the input dimensions are bigger than the crop shape, a centered cropping window is extracted from the input. If the input dimensions are smaller than the crop shape, the input is padded on each side equally <with zero values>, so that the input is centered in the output. <If the input dimensions are odd relative to the cropping window, the offset is floored>.

You are right, the documentation is ambiguous in those cases.
I've created a PR to enhance the documentation: #6464

jantonguirao added 2 commits May 5, 2022 18:00

Introduce ai.onnx.image.CenterCropPad

44cf3c4

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Merge remote-tracking branch 'upstream/main' into CenterCrop

3537b17

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao marked this pull request as ready for review May 9, 2022 08:01

jantonguirao requested review from a team as code owners May 9, 2022 08:01

jantonguirao force-pushed the CenterCrop branch from 0a61800 to 5cd42b8 Compare May 13, 2022 08:01

jantonguirao changed the title ~~Introduce ai.onnx.image.CenterCropPad~~ Introduce CenterCropPad May 13, 2022

Remove ai.onnx.image domain

b869ef9

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao force-pushed the CenterCrop branch from 5cd42b8 to b869ef9 Compare May 13, 2022 08:02

jantonguirao marked this pull request as draft May 13, 2022 12:55

jantonguirao added 3 commits May 13, 2022 14:56

Add Pad axes

fa32a8b

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Merge remote-tracking branch 'upstream/main' into CenterCrop

90b44ae

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Add axes to Shape, update CenterCropPad to work with arbitrary axes

ca4995f

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao changed the title ~~Introduce CenterCropPad~~ Introduce CenterCropPad, add axes to Pad and Shape May 17, 2022

Merge remote-tracking branch 'upstream/main' into CenterCrop

bb9e6f6

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao force-pushed the CenterCrop branch from d6e4b42 to bb9e6f6 Compare May 17, 2022 13:51

Fix shape inference file

f183885

Signed-off-by: Joaquin Anton <janton@nvidia.com>

gramalingam added the topic: operator Issues related to ONNX operators label May 25, 2022

gramalingam reviewed May 26, 2022

View reviewed changes

jantonguirao added 2 commits June 13, 2022 15:49

Merge remote-tracking branch 'upstream/main' into CenterCrop

232929e

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Undo Shape-17

9bb7a28

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao force-pushed the CenterCrop branch from 32623ac to 9bb7a28 Compare June 13, 2022 14:00

Update doc

e3e08de

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao force-pushed the CenterCrop branch from 62be0b0 to e3e08de Compare June 13, 2022 14:50

jantonguirao marked this pull request as ready for review June 14, 2022 07:29

jantonguirao added 2 commits June 14, 2022 09:35

Merge remote-tracking branch 'upstream/main' into CenterCrop

84d457d

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Linter fix

49b674a

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao changed the title ~~Introduce CenterCropPad, add axes to Pad and Shape~~ Introduce CenterCropPad, add axes to Pad Jun 14, 2022

gramalingam reviewed Jun 15, 2022

View reviewed changes

onnx/defs/tensor/defs.cc Outdated Show resolved Hide resolved

Merge remote-tracking branch 'upstream/main' into CenterCrop

c00182f

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Create Const directly from attribute

48ed678

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao added 10 commits June 22, 2022 11:02

Merge remote-tracking branch 'upstream/main' into CenterCrop

80c84d2

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Move to opset 18

7029cc8

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Apply clang-format

2550271

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Update opset in shape_inference_tests

8252b3a

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Fix shape inference tests

b8359a3

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Add automatic upgrade test

360abe7

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Update CenterCropPad tests

bbbaa87

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Refactor tests

b8ff15d

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Update tests

5d8835d

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Merge remote-tracking branch 'upstream/main' into CenterCrop

e1e5ca6

Signed-off-by: Joaquin Anton <janton@nvidia.com>

p-wysocki reviewed Jun 23, 2022

View reviewed changes

jantonguirao added 6 commits June 27, 2022 10:34

Check for repeated axes

11c6923

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Merge remote-tracking branch 'upstream/main' into CenterCrop

05294ba

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Fix flake8 issues

cf18a2e

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Fix clang-format issue

f7daf02

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Fix CenterCropPad tests with axes

8e3de23

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Merge remote-tracking branch 'upstream/main' into CenterCrop

93ac996

Signed-off-by: Joaquin Anton <janton@nvidia.com>

gramalingam approved these changes Jun 29, 2022

View reviewed changes

jantonguirao added 4 commits July 6, 2022 09:19

Merge remote-tracking branch 'upstream/main' into CenterCrop

b8c858d

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Merge remote-tracking branch 'upstream/main' into CenterCrop

6611a5d

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Merge remote-tracking branch 'upstream/main' into CenterCrop

f434f3b

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Update doc

23b008a

Signed-off-by: Joaquin Anton <janton@nvidia.com>

gramalingam merged commit 1400c98 into onnx:main Jul 12, 2022

YUNQIUGUO mentioned this pull request Mar 16, 2023

Missing ONNX Shape inferencing support for handling negative axes for Pad-18 #5003

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Introduce CenterCropPad, add `axes` to Pad #4190

Introduce CenterCropPad, add `axes` to Pad #4190

Uh oh!

jantonguirao commented May 5, 2022 •

edited

Loading

Uh oh!

gramalingam May 26, 2022

Uh oh!

gramalingam May 27, 2022

Uh oh!

jantonguirao Jun 14, 2022

Uh oh!

Uh oh!

gramalingam commented Jun 21, 2022

Uh oh!

jantonguirao commented Jun 22, 2022

Uh oh!

p-wysocki left a comment

Uh oh!

p-wysocki Jun 23, 2022 •

edited

Loading

Uh oh!

jantonguirao Jun 27, 2022

Uh oh!

fdwr commented Sep 20, 2024 •

edited

Loading

Uh oh!

jantonguirao commented Oct 16, 2024 •

edited

Loading

Uh oh!

Uh oh!

Introduce CenterCropPad, add axes to Pad #4190

Introduce CenterCropPad, add axes to Pad #4190

Uh oh!

Conversation

jantonguirao commented May 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gramalingam May 26, 2022

Choose a reason for hiding this comment

Uh oh!

gramalingam May 27, 2022

Choose a reason for hiding this comment

Uh oh!

jantonguirao Jun 14, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gramalingam commented Jun 21, 2022

Uh oh!

jantonguirao commented Jun 22, 2022

Uh oh!

p-wysocki left a comment

Choose a reason for hiding this comment

Uh oh!

p-wysocki Jun 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jantonguirao Jun 27, 2022

Choose a reason for hiding this comment

Uh oh!

fdwr commented Sep 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jantonguirao commented Oct 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Introduce CenterCropPad, add `axes` to Pad #4190

Introduce CenterCropPad, add `axes` to Pad #4190

jantonguirao commented May 5, 2022 •

edited

Loading

p-wysocki Jun 23, 2022 •

edited

Loading

fdwr commented Sep 20, 2024 •

edited

Loading

jantonguirao commented Oct 16, 2024 •

edited

Loading