[Question] Tracking COFFEE CUP in current demo about jeelizar HOT 11 CLOSED

Hi @mrgoonie

Thank you for your feedbacks.
The cup detector is a neural network learning from a 3D renderer.
We use about 5 cups 3D models and we put random lighting (a simple phong lighting model a random number of pointlights and random material coefficients). But we don't use textures yet, that's why it can work badly with some textured cups. But it would be very doable since the input renderer is written in THREE.js.

We have also limited the perspectives of the cup. The typical use-case is like in this video: https://www.youtube.com/watch?v=9klHhWxZHoc : the AR device is looking to the cup from top, and the cup is seen vertically.

It would be perfectly doable to recognize cups with different labels, we would need to train a specific neural network. We only need the 3D model of the cup (in .OBJ, .OFF, .DAE or .GLTF or whatever 3D format compatible with THREE.JS) with the 2 textures.

Best,
Xavier

from jeelizar.

Thanks @xavierjs. That's great information.

Is the model teaching tool developed with THREE.js as well? I would love to use it in my project, can I discuss further with you in terms of pricing and feasibility via email or something? (I've tried to reach you guys via [email protected] but there is no responses so far)

from jeelizar.

Hi @mrgoonie

I am sorry if you did not receive an answer yet, we are not very numerous @jeeliz, I am just back from a week off and we are preparing the GTC Munich which occurs in a few days.
The trainer is not available in open-source, I explain some reasons in this post:

jeeliz/jeelizGlassesVTOWidget#2 (comment)

The subject of AR is still very hot for us, we are thinking about Training as a Service business model or stuffs like that. About pricing and feasibility we currently just provide development services based on our APIs (custom trained networks, 3D modelization for training, middleware atop libraries) on a time basis (with standard Eastern USA prices).

from jeelizar.

Hi @xavierjs

I do understand that the trainer tool is not in open-source, I'm not expecting to do the model training myself 😄

Let say that if I have a Coke can that needs to be trained, then how long it would take you to do it? And what is the price?

After you trained it, will it be a model data file in JSON and I only need to load it in with your API?
Then will I be able to know the position of detected area in the canvas, as well as the scale?
Something like this:

You can reached me at [email protected] as well.

Looking forward to you reply.

Much thanks,

from jeelizar.

Hi @mrgoonie

This is a raw estimation of the cost:

• $50 for the 3D model of the can (find an existing one, cleaning, converting, put the texture, do 2-3 texture versions)
• $300 for the neural network trainer script, neural network compression and tests
• $300 for the training

so around $650 for the trained neural network. It will be a JSON file that you will be able to use with this library like basic4.json

but currently the detection window is always square, so the bounding box of the can would be a square, not a rectangle.

The position and scale in pixels can be computed from detectState.positionScale (cf readme) by multiplying x and sx by the video width, y and sy by the video height.

Best,
Xavier

from jeelizar.

Hi @xavierjs

Thanks for your quick response. $650 sounds reasonable to me, square boundary is also good. But how long it would take you to do that?

Thanks,
Duy

from jeelizar.

Hello,

It will take 3 weeks from signing the contract. What's your email address to send it ?

Best,
Xavier

from jeelizar.

@xavierjs

Wow I didn't know it would take that long, I don't have that much time, would it be doable within 10 days (including the weekends)?

I mentioned my email in the previous comment, it's [email protected]

Thanks,
Duy

from jeelizar.

The longer part is the training of the network, I planned 1 week (sometimes I have to retry, or modify the network for a better performance). I can do this in 10 days with a rush fee.

Best,
Xavier

from jeelizar.

@xavierjs please send me an email regarding the contract template and your full quotation with the rush fee as well. I'm looking forward to it. Thanks mate!

from jeelizar.

Perfect, I will send it to you today.

from jeelizar.