Update OpenCVForUnity about labassistvision HOT 2 CLOSED

Hi,

Thanks for sharing this project! I've tried to use it using the latest openCV 2.4.4 (the old version is not provided in the asset store, but I've asked support). There were some code changes needed, but it seem to work with the remaining trackers (most are legacy in the new version).

I tried the local detection using Barracuda instead of azure, but I don't see any bounding boxes or other detections. The console log does indicate detection, but the width and height are mainly set at 0, 0. As a hack I've inflated them ten times, but still no visual indication of a detection.

The local version should display bounding boxes and spatial indications on detections right? Do you, pehaps have some pointers on what could be wrong or on how to debug further?

Thanks!

from labassistvision.

Hi @rooi

Unfortunately, I do not have access to the HoloLens 2 anymore.

The local object detection using ONNX and Barracuda is an experimental feature, as I was not able to visualize the bounding boxes correctly in the given time (this feature was out of scope for the thesis). Still, I was able to retrieve the bounding boxes, but the post-processing logic is wrong (wrong scale or mirrored).

The local version should display bounding boxes and spatial indications on detections right? Do you, pehaps have some pointers on what could be wrong or on how to debug further?

This feature is not fully integrated into the application and will not show any visual output on the device. I included the experimental file into the repository as there was some interest regarding Unity & Barracuda on the HoloLens 2 from some colleagues.

• You should receive some bounding boxes (if you can simulate the pharmaceutical equipment) on line CustomVisionLocal.cs#L167.
• For visualization some postprocessing logic is required. The experimental file is not complete / correctly implemented. As a starting point, you could try training your own custom vision model and export it as ONNX. Ensure that you do not include postprocessing logic into the model, then your download file provided by custom vision will contain some sample code how to process the bounding boxes.

from labassistvision.