Motion detection not working when using Tensorflow and event filter about ha-shinobi HOT 14 CLOSED

were you able to get motion triggers working in home assistance by any chance? I'm having the same problem.

from ha-shinobi.

There are several type of events resulted from detection - motion, soundChange, object, face, etc...

For the motion detection to work in HA, you will need the Shinobi Video to publish 'motion' event, in logs above I see just object, means that no motion was identified by Shinobi.

Shinobi (if configured) is sending frames to detector for furst motion detection then to all plugins (it can be forced to have first motion to allow plugins to process the frame as well - again, configuration).

Can you pls share screenshot of the motion detection and object detection?

Thanks

from ha-shinobi.

Here it is:

from ha-shinobi.

Ok, seems that the plugin is not sending the right event name, for motion it should send motion not object

from ha-shinobi.

Currently I don't see a way to send the frame for motion when object detection is enable.

from ha-shinobi.

are you trying to use tensor for motion detection or for object detection? if you want it for motion detection, can you pls share the motivation? (asking because a year ago I thought about it too)

thanks

from ha-shinobi.

Well, I actually thought that using Tensorflow for Object and Motion detection could offload the CPU as my understanding is that the built-in motion detection (Pixel Array) could spike the CPU usage when running for multiple cameras.

Now I'm wondering if you could consider the possibility of use the object detection flag as motion detection or simply add a new entity for object detection that could be used to trigger things in HA?

from ha-shinobi.

Hi, the usecase for using Tensorflow for me is the same as ronluna, to be able to benefit from the GPUs available in my system when detecting motion/objects.
Regarding the motion detection prior to object detection, then I believe that was removed as an option in the new version of Shinobi (it was at least a checkmark option in the older versions).
Hope you will be able to find a way to filter for these events anyway :o)

from ha-shinobi.

Make sense, that was my use case for exploring that option, but I stayed with the default one as I compared the results for 16 camera and the impact was minimal,
If you would like to have tensorflow supported as real motion detection you need to ask developer of Shinobi (discord) to support it or allow pixel engine (default) to work with GPU,
you can ask it over the discord channel, if he will agree I will try to assist him.

Will look at the default code to find out how can it be done using GPU, to support motion detection by external plugin, platform should notify plugin in which mode to run - as motion detection or object detection (currently the assumption is only motion detection).

Hope you find it helpful

from ha-shinobi.

Hi,
Thanks for your reply!
I fully understand your arguments, but would it, as a workaround, be possible to look for the output generated by the Tensorflow engine;
[
"f",
{
"f":"video_build_success",
"hrefNoAuth":"/videos/sC0jRzUi9E/terrasse/2023-01-07T13-07-24.mp4",
.....
"objects":"person",
"events":[
{
"f":"trigger",
"id":"terrasse",
"ke":"sC0jRzUi9E",
"details":{
"plug":"Tensorflow",
"name":"Tensorflow",
"reason":"object",
"matrices":[
{
"x":489.9746322631836,
"y":41.84635877609253,
"width":126.097412109375,
"height":218.58217477798462,
"tag":"person", ......

And use that as indicator, that a motion detection has been triggered?
Once again, I appreciate you're taking your time to reply and look into this question, thanks in advance!

from ha-shinobi.

I don't think that the integration of Shinobi in HA to be familiar with set of additional events, that's the reason I'm forwarding that event as HA event so you will be able to perform custom actions (shinobi/object).

I went trhough exactly same process and ideas as you said, but.. what would be considered as motion detection - person, cat, dog, refrigerator? Just an event of object detection? What if not objects identified, what if someone will create a plugin of objects instead object, should I support it as well?
All those questions led me to make it simple with extension point (HA event) so people would do what they want if they are using plugin and not out of the box functionality (which is also the concept of the plugin)

from ha-shinobi.

Hi,
I fully understand the uncertainty on how to make sure you're dealing with a motion detection and the thoughts you have had.
But would Tensorflow not only trigger a new event if it detect a new object in the provided frame (hence a something changed, equals motion)? Or will it keep trigger events if it keeps detects fx a refrigerator?

Another option would be, that you are able to define a "Event filter" in Shinobi, where you can specify which object types that should trigger an event. So the events triggered by Tensorflow might always be considered a motion detection?

from ha-shinobi.

no, it's hard coded as object trigger and not dynamically based on detection type, which is what should get fixed in Shinobi,
Once that will happen, this integration will know how to handle it as you expect without additional code change

from ha-shinobi.