-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Description
Bug Report
Is the issue related to model conversion?
No
Describe the bug
The implementation for y_scale
for DynamicQuantizeLinear differs in two different locations
- Documentation: https://github.com/onnx/onnx/blob/main/docs/Operators.md?plain=1#L7907
- Reference: https://github.com/onnx/onnx/blob/main/onnx/reference/ops/op_dynamic_quantize_linear.py#L15-L16
The documentation has:
y_scale = (max(x) - min(x))/(qmax - qmin)
while the reference shows
maxx = np.float32(np.maximum(0, np.max(x)))
minx = np.float32(np.minimum(0, np.min(x)))
y_scale = np.float32(1.0 if maxx == minx else (maxx - minx)) / np.float32(
qmax - qmin
)
The reference appears to be based on https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/quantized/cpu/QuantUtils.h#L95-L99 which itself appears to be based on https://github.com/pytorch/FBGEMM/blob/main/include/fbgemm/QuantUtils.h
Assuming the references are correct, the documentation should be:
y_scale = (maximum(0,max(x)) - minimum(0,min(x)))/(qmax - qmin)
Side note: The documentation does have a bullet data range is adjusted to include 0.
which alludes to this. However it is easy to miss and exactly how it affects y_scale code isn't obvious so it'd be better to include it in the code snippet.
System information
NA
Reproduction instructions
Expected behavior
Implementation of documentation example snippets leads to output that matches output of reference example