Code Monkey home page Code Monkey logo

v4l2r's Introduction

Rust bindings for V4L2

LICENSE Build Status Crates.io dependency status Documentation

This is a work-in-progress library to implement safe Rust bindings and high-level interfaces for V4L2.

Currently the following is implemented:

  • Safe low-level abstractions to manage OUTPUT and CAPTURE queues, as well as buffers allocation/queueing/dequeuing for MMAP, USERPTR and DMABUF memory types,
  • High-level abstraction of the stateful video decoder interface,
  • High-level abstraction of the stateful video encoder interface,
  • C FFI for using the video decoder interface from C programs.

The library provides several levels of abstraction over V4L2:

  • At the lowest level is a very thin layer over the V4L2 ioctls, that stays as close as possible to the actual kernel API while adding extra safety and removing some of the historical baggage like the difference in format for single-planar and multi-planar queues.

  • A higher-level abstraction exposes devices, queues, and other V4L2 concepts as strongly typed objects. The goal here is to provide an nice-to-use interface that remains generic enough to be used for any kind of V4L2 device.

  • Finally, more specialized abstractions can be used by applications for performing specific tasks, like decoding a video using hardware acceleration. For these abstractions, a C FFI is usually provided so their use is not limited to Rust.

Dependencies shall be kept to a minimum: this library talks directly to the kernel using ioctls, and only depends on a few small, well-established crates.

Project Layout

lib contains the Rust library (v4l2r), including the thin ioctl abstraction, the more usable device abstraction, and task-specific modules for e.g. video decoding and encoding.

ffi contains the C FFI (v4l2r-ffi) which is currently exposed as a static library other projects can link against. A v4l2r.h header file with the public API is generated upon build.

How to use

Check lib/examples/vicodec_test/device_api.rs for a short example of how to use the device-level interface, or lib/examples/vicodec_test/ioctl_api.rs for the same example using the lower-level ioctl API. Both examples encode generated frames into the FWHT format using the vicodec kernel driver (which must be inserted beforehand, using e.g. modprobe vicodec multiplanar=1).

You can try these examples with

cargo run --example vicodec_test -- /dev/video0

for running the device API example, or

cargo run --example vicodec_test -- /dev/video0 --use_ioctl

for the ioctl example, assuming /dev/video0 is the path to the vicodec encoder.

lib/examples/fwht_encoder contains another example program implementing a higher-level vicodec encoder running in its own thread. It can be run as follows:

cargo run --example fwht_encoder -- /dev/video0 --stop_after 20 --save test_encoder.fwht

This invocation will encode 20 generated frames and save the resulting stream in test_encoder.fwht. Pass --help to the program for further options.

lib/examples/simple_decoder is a decoder example able to decode the streams produced by the fwht_encoder example above, as well as simple Annex-B H.264 streams. For instance, to decode the FWHT stream we just created above:

cargo run --example simple_decoder -- test_encoder.fwht /dev/video1 --save test_decoder.bgr

test_decoder.bgr can be checked with e.g. YUView. The format will be 640x480 BGR, as reported by the decoding program.

Finally, ffi/examples/c_fwht_decode/ contains a C program demonstrating how to use the C FFI to decode a FWHT stream. See the Makefile in that directory for build and use instructions. The program is purely for demonstration purposes of the C FII: it is hardcoded to decode the sample.fwht file in the same directory and doesn't support any other output.

v4l2r's People

Contributors

adelva1984 avatar aesteve-rh avatar andrzejtp avatar bgrzesik avatar dwlsalmeida avatar fallingsnow avatar gnurou avatar keiichiw avatar linkmauve avatar lu-zero avatar semigle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

v4l2r's Issues

ioctl functions should probably return the raw V4L2 type?

ioctls currently go through the labor of converting the ioctl result into a higher-level, safer representation. This comes with a few disadvantages:

  • Each ioctl error type must include a specific variant to handle conversion errors,
  • The returned type must be TryFrom, meaning that types that could just be From need to have their errors checked as well.

Wouldn't it be better to just return the raw V4L2 type (with the exception of v4l2_buffer and v4l2_ext_controls) and let the caller perform the conversion they want?

Bindings should be generated at build time

We currently have two generated bindings for 32 bit and 64 bit platforms, but the bindgen output is dependent on the architecture as well. Thus there doesn't seem to be a good way to provide pre-generated bindings and they should be created at build time to ensure consistency.

simple_decoder produces unreadable output on raspberry pi

I can successfully decode an annex-b h264 bitstream with the simple_decoder example on a raspberry pi:
cargo run --example simple_decoder -- test.h264 /dev/video10 --save test.yuv --input_format h264

However, the resulting decoded file doesn't display well:
ffplay -f rawvideo -pixel_format yuv420p -video_size 1920x1080 -i test.yuv

I have tested with several h264 files, and the v4l2 m2m decoder does work when I am decoding with FFmpeg:
ffmpeg -vcodec h264_v4l2m2m -i test.h264 -f rawvideo -pixel_format yuv420p test-ffmpeg.yuv
ffplay -f rawvideo -pixel_format yuv420p -video_size 1920x1080 -I test-ffmpeg.yuv

Has the simple_decoder been tested with an actual h264 input file?

Support for VIDIOC_S_CTRL?

I'm trying to implement an H264 encoder in Rust, which takes YUV420 format frames from a Raspberry Pi camera and converts it to H264. Essentially I'm trying to replicate this functionality: https://github.com/raspberrypi/libcamera-apps/blob/eca5a3c5eada4b3caa52a7ed33eafd4eb0be78c7/encoder/h264_encoder.cpp#L41

Have I interpreted the v4l2r::ioctl module correctly in that it doesn't support VIDIOC_S_CTRL to do things such as set the H264 profile and level etc?

I otherwise have a working pipeline based on the v4l2r::encoder::Encoder which is really great.

Raspberry pi never fires a EPOLLPRI event

It seems that while using V4L2 on the RPI4 no EPOLLPRI event is ever fired. The stateful decoder in v4l2r requires a EPOLLPRI to read a v4l2_event which will trigger a resolution change, in turn enabling the capture queue.

Is there a way to use the stateful decoder even if no v4l2_event is triggered? Of course you would need to manually set the output format then.

Set up Travis CI

Set up Travis CI to run cargo fmt and cargo clippy for each commit.

v4l2r::device::queue::FormatBuilder does not follow rust builder convention

I believe v4l2r::device::queue::FormatBuilder.apply() should be changed to .build() and should not set the format. Instead you should use queue.set_format().

I thought that apply was the equivalent was of build because set != apply in my mind. I think of apply and I think: ok, I'm done, give me the result (aka build). For example most people hit apply before hitting ok in a settings dialog. I didn't think that it would actually set the format.

Another option is to give a .build() and a .set() option.

That's my opinion, feel free to close if you disagree.

Transmuting between `v4l2_plane__bindgen_ty_1` and `v4l2_buffer__bindgen_ty_1` is not safe

In the code we do transmute between v4l2_plane__bindgen_ty_1 and v4l2_buffer__bindgen_ty_1, but as has been pointed out to me this is not necessarily safe because they may not be the same size on platforms for which sizeof(c_ulong) != sizeof(*mut).

It would probably be safer to move this into a helper that copies the appropriate member explicitly depending on the memory type of the buffer.

EPIPE (Broken pipe) when using stateful decoder and format changes

I have a video stream that changes from 800x600 to 640x480. I'm using the stateful decoder and I expect the set_capture_format_cb function to be called when this happens but instead I get a bunch of broken pipe errors and CPU usage goes to 100%. I'm using the raspberry pi's v4l2mem2mem hardware decoder.

This is an strace of the process when this happens.

...
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
epoll_pwait(16, [{EPOLLIN, {u32=0, u64=4294967296}}], 4, -1, NULL, 8) = 1
ioctl(14, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE}) = -1 EPIPE (Broken pipe)
^Cstrace: Process 11318 detached

Project layout

Since it is a workspace with 3 components:

  • crate library
  • ffi for the crate library
  • cli utilities

It is common to either have

  • The crate library as root crate, ffi and cli in crates/ as v4l2r-ffi and v4l2r-cli
  • Everything in crates/ as v4l2r, v4l2r-ffi, v4l2r-cli.

Are you fine with either refactor?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.