[Bug] GatherND shape conversion from ONNX is inaccurate

1. System information (version)

OpenVINO=> 2021.4.0-3839
Operating System / Platform => Ubuntu 20.04 x86_64
Compiler => GLIBC 2.31, g++ 9.3.0
Problem classification => Model Conversion
Framework: ONNX opset=12 (TensorFlow)
Model name: HITNET
ONNX, tflite, TensorFlow sample

2. Detailed description

Converting a GatherND from ONNX with batch_dims=3 set does not produce the expected output shape. Therefore, in the subsequent operation transformation operation, the dimension transformation will be invalid and the optimizer will Abort.

For example, the tool works as follows.

Op: GatherND
input[0] : shape = [ 1 64 64 320]
input[1] : shape = [ 1 64 64 1 1]
output[0]: shape = [ 4096 1]

However, the originally expected behavior is as follows.

Op: GatherND
input[0] : shape = [ 1 64 64 320]
input[1] : shape = [ 1 64 64 1 1]
output[0]: shape = [ 1 64 64 1]

Log message. (The @@@@ part was output by adding debug prints to the Model Optimizer logic by myself.)

[ 2021-09-05 17:22:08,287 ] [ DEBUG ] [ infer:116 ]  --------------------
[ 2021-09-05 17:22:08,287 ] [ DEBUG ] [ infer:117 ]  Partial infer for level0_1/level_init/GatherV2_1;level0/level_init/GatherV2_1/axis
[ 2021-09-05 17:22:08,287 ] [ DEBUG ] [ infer:118 ]  Op: GatherND
[ 2021-09-05 17:22:08,287 ] [ DEBUG ] [ infer:119 ]  Inputs:
[ 2021-09-05 17:22:08,287 ] [ DEBUG ] [ infer:19 ]  input[0]: shape = [  1  64  64 320], value = <UNKNOWN>
[ 2021-09-05 17:22:08,287 ] [ DEBUG ] [ infer:19 ]  input[1]: shape = [ 1 64 64  1  1], value = <UNKNOWN>
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:132 ]  Outputs:
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:19 ]  output[0]: shape = [4096    1], value = <UNKNOWN>
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:116 ]  --------------------
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:117 ]  Partial infer for Mul__311
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:118 ]  Op: Mul
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:119 ]  Inputs:
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:19 ]  input[0]: shape = [ 1 64 64  1], value = <UNKNOWN>
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:19 ]  input[1]: shape = [], value = 0
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:132 ]  Outputs:
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:19 ]  output[0]: shape = [ 1 64 64  1], value = <UNKNOWN>
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:116 ]  --------------------
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:117 ]  Partial infer for level0/level_init/init_to_prop/concat
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:118 ]  Op: Concat
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:119 ]  Inputs:
[ 2021-09-05 17:22:08,288 ] [ DEBUG ] [ infer:19 ]  input[0]: shape = [4096    1], value = <UNKNOWN>
[ 2021-09-05 17:22:08,289 ] [ DEBUG ] [ infer:19 ]  input[1]: shape = [ 1 64 64 48], value = <UNKNOWN>
@@@@@@@@@@@@@@@@@@@@@@@@@ shape: [4096    1]
@@@@@@@@@@@@@@@@@@@@@@@@@ not_mask: [ True False]
@@@@@@@@@@@@@@@@@@@@@@@@@ s: [ 1 64 64 48]
[ ERROR ]  Cannot infer shapes or values for node "level0/level_init/init_to_prop/concat".
[ ERROR ]  boolean index did not match indexed array along dimension 0; dimension is 4 but corresponding boolean dimension is 2
[ ERROR ]  
[ ERROR ]  It can happen due to bug in custom shape infer function <function concat_infer at 0x7fa5c296daf0>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ 2021-09-05 17:22:08,289 ] [ DEBUG ] [ infer:184 ]  Node "level0/level_init/init_to_prop/concat" attributes: {'pb': input: "level0/level_init/GatherV2_1;level0/level_init/GatherV2_1/axis"
input: "fe_shared_3"
output: "level0/level_init/init_to_prop/concat"
name: "level0/level_init/init_to_prop/concat"
op_type: "Concat"
attribute {
  name: "axis"
  i: -1
  type: INT
}
, 'kind': 'op', '_in_ports': {1: {'control_flow': False}, 0: {'control_flow': False}}, '_out_ports': {0: {'control_flow': False}}, 'name': 'level0/level_init/init_to_prop/concat', 'op': 'Concat', 'type': 'Concat', 'version': 'opset1', 'axis': 1, 'infer': <function concat_infer at 0x7fa5c296daf0>, 'out_ports_count': 1, 'dim_attrs': ['batch_dims', 'axis', 'spatial_dims', 'channel_dims'], 'shape_attrs': ['stride', 'shape', 'pad', 'output_shape', 'window'], 'IE': [('layer', [('id', <function Op.substitute_ie_attrs.<locals>.<lambda> at 0x7fa5b96fd430>), 'name', 'type', 'version'], [('data', ['axis'], []), '@ports', '@consts'])], 'is_output_reachable': True, 'is_undead': False, 'is_const_producer': False, 'is_partial_inferred': False}
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "level0/level_init/init_to_prop/concat" node. 
 For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)
[ 2021-09-05 17:22:08,290 ] [ DEBUG ] [ main:410 ]  Traceback (most recent call last):
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/middle/passes/infer.py", line 122, in partial_infer
    node.infer(node)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/front/common/partial_infer/concat.py", line 45, in concat_infer
    if np.all(shape[not_mask] == s[not_mask]):  # TODO handle -1 in a special way
IndexError: boolean index did not match indexed array along dimension 0; dimension is 4 but corresponding boolean dimension is 2

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 276, in apply_transform
    replacer.find_and_replace_pattern(graph)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/extensions/middle/PartialInfer.py", line 21, in find_and_replace_pattern
    partial_infer(graph)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/middle/passes/infer.py", line 185, in partial_infer
    raise Error('Stopped shape/value propagation at "{}" node. '.format(node.soft_get('name')) +
mo.utils.error.Error: Stopped shape/value propagation at "level0/level_init/init_to_prop/concat" node. 
 For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/intel/openvino_2021.4.582/deployment_tools/model_optimizer/mo/main.py", line 394, in main
    ret_code = driver(argv)
  File "/opt/intel/openvino_2021.4.582/deployment_tools/model_optimizer/mo/main.py", line 356, in driver
    ret_res = emit_ir(prepare_ir(argv), argv)
  File "/opt/intel/openvino_2021.4.582/deployment_tools/model_optimizer/mo/main.py", line 252, in prepare_ir
    graph = unified_pipeline(argv)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/pipeline/unified.py", line 13, in unified_pipeline
    class_registration.apply_replacements(graph, [
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 328, in apply_replacements
    apply_replacements_list(graph, replacers_order)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 314, in apply_replacements_list
    apply_transform(
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/utils/logger.py", line 111, in wrapper
    function(*args, **kwargs)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 294, in apply_transform
    raise Error('Exception occurred during running replacer "{}" ({}): {}'.format(
mo.utils.error.Error: Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "level0/level_init/init_to_prop/concat" node. 
 For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)

The operational part of ONNX where the problem occurs is shown in the figure below.

Official GatherND documentation
https://docs.openvinotoolkit.org/latest/openvino_docs_ops_movement_GatherND_5.html

3. Steps to reproduce

Download model_float32_opt.onnx.zip
Unzip the ZIP.
Convert with the following command.

${INTEL_OPENVINO_DIR}/deployment_tools/model_optimizer/mo.py \
--input_model model_float32_opt.onnx \
--data_type FP32 \
--output_dir openvino/FP32 \
--log_level=DEBUG

option: Original HITNET model URL

4. Issue submission checklist

I report the issue, it's not a question
I checked the problem with documentation, FAQ, open issues, Stack Overflow, etc and have not found solution
There is reproducer code and related data files: images, videos, models, etc.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Bug] GatherND shape conversion from ONNX is inaccurate #7379

1. System information (version)

2. Detailed description

3. Steps to reproduce

4. Issue submission checklist

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Bug] GatherND shape conversion from ONNX is inaccurate #7379

Description

1. System information (version)

2. Detailed description

3. Steps to reproduce

4. Issue submission checklist

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions