I have recently shared example of hardware accelerated Realsense (D400) depth encoding to HEVC Main10 profile:

realsense-depth-to-vaapi-hevc10

The example encodes 16 bit depth Realsense data in 10 bits of HEVC Main 10 with VAAPI P010LE pixel format.

The aim is to have a purely hardware based pipeline (no depth data processing on the host). This is achieved by using Realsense Depth Units in such way that Realsense outputs P010LE compatible data with controlled precision/range trade-off.

The current limits of Realsense Depth Units are 0.0001 - 0.01, as is reported by realsense-viewer or depth_sensor.get_option_range(RS2_OPTION_DEPTH_UNITS);

Those limits, when translated to P010LE mean:

• the best precision/worst range is 6.4 mm/6.5535 m (for depth units 0.0001)
• the worst precission/best range is 64 cm/655.35 m (for depth units 0.01)
• all trade-offs in between are possible with purely hardware based encoding pipeline

My question here is:

Are the limits of Realsense Depth Units (0.0001, 0.01) based on some hardware constraints? Would it be possible to relax them? Extending lower bound to 0.00001 would allow control of precision/range trade-off starting from 0.64 mm/0.65 m (precision/range).

Here I include explanation of encoding on bit-level. This is not needed to answer the question, just for the "interested reader".

P010LE pixel format is like NV12, this means Y luminance plane followed by interleaved U/V plane half the size of Y. The color data is encoded in 16 bit words of which only 10 MSB is used. For depth encoding we are only interested in Y luminance plane, the U/V plane will be filled with constant values.

The P010LE pixel format was designed to be binary compatible with P016LE, which encodes Y data with 16 bits. P010LE may be passed in place of P016LE (6 LSB as zeroes). P016LE may be passed as P010LE (6 LSB dropped, this means dropping precision of data).

We may consider Realsense Z16 depth data as P016LE Y data. When passing it to VAAPI hardware encoder as P010LE we are dropping the 6 LSB bits of precision. This is why we get 6.4 mm/6.5535 m precision/range for depth units 0.0001.

Related to #4819, #3877, #3665, #3259

For anybody planning to use similar approach it is important that he/she understands implications of using video codec for depth encoding.

If you are not constrained by CPU and real-time requirements consider using HEVC 3D extension software encoder designed for compressing depth data or other approach.