(See related issue: #5520)

In general, there are some minor inconsistencies in how scalars are handled (in the ONNX spec as well as different backends/implementations/models).

The primary recommendation of ONNX is to represent a scalar tensor as a tensor of rank zero, which is a tensor with one element and has an empty shape).

However, one common situation that arises is the use of a tensor with one element with the shape [1] (which has rank 1). This is supported/handled in some cases, but not in others (both in terms of operator specs as well as runtimes/implementations).

Recommendation going forward: I suggest that we clarify that any operator that returns a scalar-value should return a zero-dimensional tensor (which necessarily has a single-element). But for an operator that expects a scalar-input, it may be better to allow any one-element tensor. This is not a perfect solution but seems preferable to the alternative that we strictly require a zero-dimensional tensor as input (which may cause some failures in existing models).

The intuition is that of the robustness principle in specifications (disadvantages discussed there too).

This is, of course, just a general recommendation for changes/updates/clarifications going forward ... anything that is explicit already in the spec is a different matter. Unless and until it is updated, the existing specs will continue to remain the same (if they are unambiguous).

Any thoughts?