Feature mapping external onnx model about tribuo HOT 3 CLOSED

cc @Craigacp

from tribuo.

When loading an external model you need to map from the feature names that Tribuo uses (which are set by the data source, or however else you loaded in the data and got it into an Example) to the feature indicies that the model is expecting. In the case of a structured input like an image you need to make sure that the feature values line up as pixels in the right order for the transformation function to work (in this case org.tribuo.interop.onnx.ImageTransformer). Basically you need to make sure the feature names map to a multidimensional row major input (i.e. the first feature is pixel 0,0,0, the second feature is 0,0,1 and so on). To better work with images and other dense inputs Tribuo probably needs a dense example which will minimise the memory and other overheads as compared to using the sparse examples we currently have for dense data. We started down this path in 4.1 with adding more support for dense features in some models, but the rest of the support will be in 4.3 or later.

Now specifically in the case of object detection using yolo, I don't think Tribuo is a good fit. It doesn't have an object detection structured output type, the best you'll be able to do is a multi-label output without any bounding boxes, because Tribuo's Output implementations are not expressive enough. Adding better support for structured inputs and outputs is on our roadmap, but it's quite far away as there is substantial design effort involved and we only want to do it once.

I'd recommend if you want to do object detection in Java that you use ONNX Runtime directly, https://github.com/microsoft/onnxruntime/tree/master/java. That will let you feed a tensor in (but do it from a java.nio.FloatBuffer as multidimensional Java arrays are very slow), and get the bounding boxes, labels and scores out.

from tribuo.

@Craigacp I see hmm ok thanks!

from tribuo.