Code Monkey home page Code Monkey logo

Comments (7)

raymanfx avatar raymanfx commented on July 4, 2024 1

Yep, that looks about correct (given that the frames you get from the camera are indeed in the YUYV order).

from libv4l-rs.

raymanfx avatar raymanfx commented on July 4, 2024 1

Well this library is format agnostic. If you are asking whether ffimage will be able to convert those formats into packed RGB pixels, the answer is probably no. It should not be hard to add the conversion routines for those formats though. The existing YUV <-> RGB code should give you an idea of how to convert the image data.

There's even existing code handling YUV420p: https://github.com/raymanfx/ffimage/blob/master/ffimage-yuv/src/yuv420.rs. So if you want RGB pixels, you'll need to implement the YU12 (planar) -> YUV444 (packed) conversion step and the complete pipeline would probably look like this: YU12 -> YUV444 -> RGB.

from libv4l-rs.

raymanfx avatar raymanfx commented on July 4, 2024

So stream.next(), e.g. for the memory-mapped IO transport (see: https://github.com/raymanfx/libv4l-rs/blob/master/examples/stream_capture_mmap.rs#L35) gives you a buffer which is essentially a slice of bytes: &[u8].

It is important that you configure the v4l2 device to use a format that is supported by image-rs, e.g. RGB888. Most webcams will not support that, but only provide YUYV (aka Yuv422). I saw your comment on the image-rs repo and unfortunately it's still not something they support (AFAIK). You can use ffimage v0.10.0 from crates.io to convert a slice of YUYV bytes to a slice of RGB bytes (see: https://github.com/raymanfx/ffimage/blob/next/ffimage-yuv/benches/convert.rs#L29).

You should be able to then use ImageBuffer::from_raw() from image-rs (see: https://docs.rs/image/latest/image/struct.ImageBuffer.html#method.from_raw) and pass it those bytes.

Let me know if this helps; and sorry for the late reply!

from libv4l-rs.

sjbeskur avatar sjbeskur commented on July 4, 2024

Great question, I would really like to use this library but I too am struggling with the capture buffer conversion to an image format I can work with. The ffimage example is not clear to me .. any further guidance would be greatly appreciated.

from libv4l-rs.

sjbeskur avatar sjbeskur commented on July 4, 2024

I think I have figured it out:

use image::{ImageBuffer};
use v4l::buffer::Type;
use v4l::io::traits::CaptureStream;
use v4l::prelude::*;
use v4l::video::Capture;    
use ffimage_yuv::yuv422::Yuv422;
use ffimage_yuv::yuv::Yuv;
use ffimage::color::Rgb;
use ffimage::iter::{BytesExt, ColorConvertExt, PixelsExt};
    
// in your main function (or whatever).

let mut dev = Device::new(0).expect("Failed to open device");
let format = dev.format()?;
let (buf, meta) = stream.next().unwrap();

// unwinding the expression below in comments:

let rgb: Vec<u8>  = buf.iter()
    .copied()                           // Vec<u8> 
    .pixels::<Yuv422<u8, 0, 2, 1, 3>>() // Vec<Yuv422<u8, 0, 2, 1, 3>> // after pixels
    .colorconvert::<[Yuv<u8>; 2]>()     // Vec<[Yuv<u8>;2]>  // after colorconvert::<[Yuv<u8>; 2]>()
    .flatten()                          // Vec<Yuv<u8>> // after flatten   
    .colorconvert::<Rgb<u8>>()          // Vec<Rgb<u8>> // after colorconvert::<Rgb<u8>>
    .bytes()                            // Vec<[u8;3]> // after bytes;
    .flatten()
    .collect();

// https://github.com/raymanfx/libv4l-rs/issues/74
// NOTE: dont confuse image::Rgb with ffimage::Rgb

let img: ImageBuffer<image::Rgb<u8>, Vec<u8>> = ImageBuffer::from_raw(
    format.width, 
    format.height, 
    rgb,
)?;

img.save("test.png");


from libv4l-rs.

sjbeskur avatar sjbeskur commented on July 4, 2024

@raymanfx many thanks. This works for one of my cameras but my other camerahas a format that is Y16 (grayscale)? Seems it also supports Yu12 (YUV4:2:0) and NV12 (Y/CbCr 4:2:0) will your library work with these formats? Sorry for the dumb questions but this rather new to me.

from libv4l-rs.

sjbeskur avatar sjbeskur commented on July 4, 2024

Brilliant thank you @raymanfx . I was able to get it working wit the following snippet (replacing the critical section in the code above)

        let rgb: Vec<u8> = Yuv420p::pack(&buf, 640, 512)
            .into_iter()
            .colorconvert::<Rgb<u8>>()
            .bytes()
            .flatten()
            .collect();

The images look Ok, but this is a grayscale (IR camera) and I suspect this is not efficient. I'll study your suggestion and see what I can figure out.

from libv4l-rs.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.