Get error Frame Cow Too Small from frame_to_buffer about nokhwa HOT 11 CLOSED

Fixed by #27
Will be in next release.

from nokhwa.

Yep. 0.10 is soon, please standby.

from nokhwa.

What platform and camera are you running?

from nokhwa.

Same issue here; But I believe it's a bug and it's irrelevant to the platform or camera.
The issue is that in the documentation of that function, it's mentioned nowhere (I found it out through reading the code) that frame_to_buffer writes a raw frame from the camera without RGB24 decoding. Resulting in for example, a raw MJPEG frame to be written to the buffer.

To sum up:

• camera.raw_frame() returns a MJPEG buffer of size X
• trying to convert to RGBA expects the buffer to be big enough to contain width * height * channels but its capacity is the MJPEG frame's, so it fails
• Not trying to convert to RGBA results in the same issue as the min_size_buffer is much bigger than the MJPEG frame, thus dest.copy_from_slice(raw_frame) panics

That would also mean that min_buffer_size is also useless because it bases its calculations on RGB24 and not whatever the camera would return.

I have the same setup pretty much where I capture MJPEG; MJPEG buffers on average are 66% the size of a RGB24 buffer, making the calculations flawed.

What needs to be done IMO is to decode the frame in frame_to_buffer. Otherwise you should rename it to raw_frame_to_buffer but again it's impossible to use since it's impossible to know the frame size beforehand (MJPEG, YUV are all different and the buffer sizes vary with content...).

I'll try to put up a PR soon I think to solve this, but I'll need help testing it on other platforms - I only have linux at hand, but I believe that the problem is widespread as it's a buffer allocation issue outside of the platform-specific backends.

from nokhwa.

I think it would be better to kill 2 birds with one stone (solving #25 with this) by creating a custom buffertype that can decode itself.

from nokhwa.

Not sure about this approach but it might work if you simply Box the target buffer or something; Keeping the feature of decoding in-buffer is very important IMO for those kinds of libraries (my application which makes use of nokhwa does no dynamic allocations while processing frames, allowing raw webcam throughputs up to 4K60).

Unfortunately, I have some bad news: I had to fork mozjpeg-rust for this to work properly

Mozjpeg modifications: ImageOptim/mozjpeg-rust@master...OtaK:otak/read_scanlines_into#diff-c80690f4ca

And the supporting nokhwa modifications are here: senpai...OtaK:otak/buff-buffers

The big buzzkill here is that I had to disable UVC support as rust-uvc uses mozjpeg too, but an old version (I guess that's why you were targeting an old version of mozjpeg). I could make yet-another-fork of UVC that targets my mozjpeg fork but heh...rabbitholes and so on.

Also, if you're curious what I've been building with nokhwa: https://github.com/otak/hide-my-mess-rs
I made it meet with RobustVideoMatting ML inference w/ Tensorflow. Works great :)

from nokhwa.

We could extract that into a seperate crate (reimplement jpeg... oof) and then pull in UVC?
Tbh, Im not opposed to killing off uvc - the backend has been disabled due to soundness holes for many releases

from nokhwa.

Your project sounds interesting, I am considering making a VirtualCamera implementation as well...

from nokhwa.

@OtaK Do you mind if I mainline your MozJPEG fork and maintain it as a separate crate and take your YUV implementation? Could I also take it and make a PR to the MozJPEG crate to get it mainlined?

from nokhwa.

Tbh, Im not opposed to killing off uvc - the backend has been disabled due to soundness holes for many releases

Well that's good news, because it honestly seems the uvc crate is largely unmaintained, they do maintenance patches here and there and that's it. And I think the native backends are good / stable enough to get rid of one of the "cross-platform" ones.

Do you mind if I mainline your MozJPEG fork and maintain it as a separate crate and take your YUV implementation? Could I also take it and make a PR to the MozJPEG crate to get it mainlined?

I'll clean up my branch and propose a PR to mozjpeg; I'm still unsure about the soundness of what I did, I did something that made sense but welp. It does work on my machine™️

For the rest if you want I can just clean up my branch and send you a PR in here?

But anyway if you want to move fast I don't mind either way!

from nokhwa.

Running into this bug at the moment. Even though this issue is closed, it seems there has not been a release since this was fixed?

from nokhwa.