rgbd does not broadcast about kinectron HOT 10 CLOSED

Gotit: In the drawImageToCanvas function it needs rawDepth to be true, so that it calls "return createDataUrl(inCanvas, frameType, imageType, quality);" otherwise it leaves out the "return"...

from kinectron.

After some debugging of the app code, it seems it is broadcasting an "undefined" instead of the webp. at the moment i do not understand why it is even drawing anything onto the canvas...

from kinectron.

Sorry for the bug! I've made a fix for this in the upcoming release. In the meantime if server version 0.2.0 does not have the problem. I'll post here as soon as the new release is out.

from kinectron.

This is fixed in server release 0.3.3 with a refactor of packaging and sending functionality (see commit). Please let me know if you experience other problems.

from kinectron.

Hi both, lovely work Lisa btw...

Also here with the intent of deploying the output in three.js though with a prerecorded webp output.

It would appear that whilst the issue is fixed for broadcasting - the recorded webp output doesnt contain the depth data. Im scouring the code currently to see if I can establish why but its a new project to me so I'm not exactly well orientated.

My recorded webp of the rgbd feed just has full white on the alpha channel.
(I presume its not a limitation of webp?!)

For clarity depth image is successfully recording for me as a seperate channel, just not the combined rgbd

Apologies if this is considered a seperate issue - I just felt it was related to completing this functionality in full

from kinectron.

I just wanted to add detail to my considerations here

For clarity, Im using the rgbd example in the repository as a means to see results. However, I modified it in the same fashion as the threejs.org example, inserting a

I ended up modifying kinectron-app.js in various ways to test - first recording the depth buffer to the red channel instead of the alpha in startRGBD()
i.e for (let i = 0; i < imageDataSize; i += 3) { imageDataArray[i] = frame.depthColor.buffer[i+2]; imageDataArray[i + 1] = 255;//frame.depthColor.buffer[i + 1]; imageDataArray[i + 2] = 255;//frame.depthColor.buffer[i + 2]; imageDataArray[i + 3] = 255; // set alpha channel as depth j++; }

in this scenario, modifying the three.js shader to access the r channel instead of w, the depth works perfectly - it would appear then theres no issue getting the depth information in time to write to the video file.
Very confused as to why the same data isnt saving to the alpha channel.

Assuming theres no problem for @liveprints then in using the live kinectron feed with the same example - I wonder if theres perhaps an issue recording alpha channels to a webp format?

• That said, if i write all 255's to the alpha channel, the image is as one would expect, compared to writing all 0's >> which results in the video being black.

Whats interesting too is that with the alpha channel set to all 0's during recording kinectron-app.js, you dont see anything at all (or all 50's for instance, just a very dark image)
-> this is despite the fact the shader only uses r,g & b channels to handle the colour and the alpha channel to contain the depth information.

Therefore I suspect some level of cross contamination here, but I'm unsure as to why and what can be done to avoid it.

I think the solution might have to lie recording the rgbd feed like the multi-frames format, encoding the colour into one video and the colour-depth into a seperate video.

from kinectron.

... or more likely encoding the depth into the rgb channels but along the side of the colour channels (i.e in a video twice the width or height of the source)
... this is how depthjs / depthkit.tv does it so i wonder if this was a nesscecity for them because of the same issue.

from kinectron.

@CosyStudios Apologies for the delay. I'm on parental leave and haven't been on the computer for some time. I unfortunately can't be much help without running the code.

I am booking some time in January to look at existing issues with Kinectron. In the meantime, have you found a solution or more information here? I am going to venture a guess that if depthjs / depthkit.tv is running the depth alongside the rgb image as you mention that this would be a good way to go. However, they may be doing this because they are using the raw depth, which would need 16-bits of space, rather than the 8 bits required in the RGBD feed. The RGBD feed shows the depth mapped to 0-255, whereas the raw depth shows the actual depth coming from the sensor, which is 0-3860 (The values correspond to depth in millimeters).

from kinectron.

Congratulations!

And no worries - to be fair, this is really something out fo the original scope that kinectron aims to address as a client-client comms solution so I appreciate your input, especially at this point in the year (hope youve done your shopping! :D)
Ah - interesting observation though on the two feeds, i didnt realise that.

Regardless, if we can support the 16bit rgba feed and encode it together with the colour feed, that might be a really great upgrade to all kinectron scenarios.

I for one certainly havent lost interest since it also poses a solution to me to record some volumentric video that i can use in a three.js project i have in mindand the depth.js solution is both out of price point for a sketchy demo and has way more features than I need since its really geared at the film industry.

with that in mind being the solution, ill report anything useful / acheivements here if i manage anything myself though im a bit of a newbie a canvas manipulation, im sure i can work out how to stitch the two feeds together on one canvas at least.

Appreciate & look forward to any thoughts or development you may find yourself with time for.
Thanks Lisa. have a great christmas

from kinectron.

Still not sure why the regular 8bit rgba feed though doesnt seem to be recording the deptch to the alpha channel

from kinectron.