@dkurt FYI, we might not incorporate the changes with threading. I am a bit wary of it in general.
If there's any reason I should consider it (speed?), let know.

I have one separate question for you: I'm looking into updating to Ubuntu18.04 (context: see #394 )
What should I use instead of intel-openvino-dev-ubuntu16-2020.4.287 here

Thanks!

@pichuan, I think it's only about the speed, yes. I can benchmark it on my end and let you know if there is any benefit to use thread.

That's good idea to switch to 18.04! We can also perform switch to the latest version of OpenVINO (which was not available for 16.04):

sudo curl -o GPG-PUB-KEY-INTEL-OPENVINO-2021 https://apt.repos.intel.com/openvino/2021/GPG-PUB-KEY-INTEL-OPENVINO-2021
sudo apt-key add GPG-PUB-KEY-INTEL-OPENVINO-2021
sudo echo "deb https://apt.repos.intel.com/openvino/2021 all main" | sudo tee - a /etc/apt/sources.list.d/intel-openvino-2021.list
sudo apt-get update
sudo apt-get install -y --no-install-recommends intel-openvino-dev-ubuntu18-2021.1.110
sudo ln -s /opt/intel/openvino_2021 /opt/intel/openvino

You are right, there is no motivation to move this into a separate thread due current logging is already optimal! Benchmarked master and proposed branches and there is no time difference between them. Thanks!