Code Monkey home page Code Monkey logo

sdroege / gst-plugin-rs Goto Github PK

View Code? Open in Web Editor NEW
116.0 11.0 39.0 13.65 MB

Rust crate for writing GStreamer plugins and various plugins - This repository moved to https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs

Home Page: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs

License: Apache License 2.0

Rust 96.00% Makefile 0.01% C 1.08% Python 0.58% Meson 0.37% Shell 0.03% PowerShell 0.02% HTML 0.20% Svelte 0.22% TypeScript 0.01% JavaScript 1.49% GLSL 0.01%
gstreamer rust multimedia codecs parsers demuxers plugins

gst-plugin-rs's Introduction

gst-plugins-rs crates.io pipeline status

Repository containing various GStreamer plugins and elements written in the Rust programming language.

The plugins build upon the GStreamer Rust bindings. Check the README.md of that repository also for details about how to set-up your development environment.

Plugins

You will find the following plugins in this repository:

  • generic

    • file: A Rust implementation of the standard filesrc and filesink elements

    • sodium: Elements to perform encryption and decryption using libsodium.

    • threadshare: Some popular threaded elements reimplemented using common thread-sharing infrastructure.

  • net

    • aws: Various elements for Amazon AWS services using the AWS SDK library

      • s3src/s3sink: A source and sink element to talk to the Amazon S3 object storage system.
      • s3hlssink: A sink element to store HLS streams on Amazon S3.
      • awstranscriber: an element wrapping the AWS Transcriber service.
      • awstranscribeparse: an element parsing the packets of the AWS Transcriber service.
    • hlssink3: An element for generating MPEG-TS HLS streams.

    • ndi: An NDI plugin containing a source, sink and device provider.

    • onvif: Various elements for parsing, RTP (de)payloading, overlaying of ONVIF timed metadata.

    • quinn: Transfer data over the network using QUIC

      • quinnquicsink/quinnquicsrc: Send and receive data using QUIC
    • raptorq: Encoder/decoder element for RaptorQ RTP FEC mechanism.

    • reqwest: An HTTP source element based on the reqwest library.

    • rtp:

      • rtpav1pay / rtpav1depay: RTP (de)payloader for the AV1 video codec.

      • rtpgccbwe: RTP bandwidth estimator based on the Google Congestion Control algorithm.

    • webrtc: WebRTC elements, with batteries included Sink elements for specific signalling protocols.

    • webrtchttp: Simple WebRTC HTTP elements (WHIP/WHEP).

  • audio

    • audiofx: Elements to apply audio effects to a stream

    • claxon: A FLAC decoder based on the Claxon library.

    • csound: A plugin to implement audio effects using the Csound library.

    • lewton: A Vorbis decoder based on the lewton library.

    • spotify: A plugin to access content from Spotify based on the librespot library.

  • video

    • cdg: A parser and renderer for CD+G karaoke data.

    • closedcaption: Plugin to deal with closed caption streams

      • ccdetect: Detects if a stream contains active Closed Captions.
      • cea608overlay: Overlay CEA-608 / EIA-608 closed captions over a video stream.
      • cea608tojson: Convert CEA-608 / EIA-608 closed captions to a JSON stream.
      • cea608tott: Convert CEA-608 / EIA-608 closed captions to timed text.
      • jsontovtt: Convert JSON to timed text.
      • mccenc: Convert CEA-608 / EIA-608 and CEA-708 / EIA-708 closed captions to the MCC format.
      • mccparse: Parse CEA-608 / EIA-608 and CEA-708 / EIA-708 closed captions from the MCC format.
      • sccenc: Convert CEA-608 / EIA-608 closed captions to the MCC format.
      • sccparse: Parse CEA-608 / EIA-608 closed captions from the MCC format.
      • transcriberbin: Convenience bin around transcriber elements like aws_transcriber.
      • tttocea608: Convert timed text to CEA-608 / EIA-608 closed captions.
      • tttojson: Convert timed text to JSON.
    • dav1d: AV1 decoder based on the dav1d library.

    • ffv1: FFV1 decoder based on the ffv1 library.

    • gif: A GIF encoder based on the gif library.

    • gtk4: A GTK4 video sink that provides a GdkPaintable for UI integration.

    • hsv: Plugin with various elements to work with video data in hue, saturation, value format

      • hsvdetector: Mark pixels that are close to a configured color in HSV format.
      • hsvfilter: Apply various transformations in the HSV colorspace.
    • png: PNG encoder based on the png library.

    • rav1e: AV1 encoder based on the rav1e library.

    • videofx: Plugin with various video filters.

      • roundedcorners: Element to make the corners of a video rounded via the alpha channel.
      • colordetect: A pass-through filter able to detect the dominant color(s) on incoming frames, using color-thief.
      • videocompare: Compare similarity of video frames. The element can use different hashing algorithms like Blockhash, DSSIM, and others.
    • webp: WebP decoder based on the libwebp-sys-2 library.

  • mux

    • flavors: FLV demuxer based on the flavors library.

    • fmp4: A fragmented MP4/ISOBMFF/CMAF muxer for generating e.g. DASH/HLS media fragments.

    • mp4: A non-fragmented MP4 muxer for generating MP4 files.

  • text

    • ahead: A plugin to display upcoming text buffers ahead.

    • json: A plugin to convert a stream of JSON objects to a higher level wrapped NDJSON output.

    • regex: A regular expression text filter plugin.

    • wrap: A plugin to perform text wrapping with hyphenation.

  • utils

    • fallbackswitch:

      • fallbackswitch: An element that allows falling back to different sink pads after a timeout based on the sink pads' priorities.
      • fallbacksrc: Element similar to urisourcebin that allows configuring a fallback audio/video if there are problems with the main source.
    • livesync: Element to maintain a continuous live stream from a potentially unstable source.

    • togglerecord: Element to enable starting and stopping multiple streams together.

    • tracers: Plugin with multiple tracers:

      • buffer-lateness: Records lateness of buffers and the reported latency for each pad in a CSV file. Contains a script for visualization.
      • pipeline-snapshot: Creates a .dot file of all pipelines in the application whenever requested.
      • queue-levels: Records queue levels for each queue in a CSV file. Contains a script for visualization.
    • uriplaylistbin: Helper bin to gaplessly play a list of URIs.

Building

gst-plugins-rs relies on cargo-c to generate shared and static C libraries. It can be installed using:

$ cargo install cargo-c

Then you can easily build and test a specific plugin:

$ cargo cbuild -p gst-plugin-cdg
$ GST_PLUGIN_PATH="target/x86_64-unknown-linux-gnu/debug:$GST_PLUGIN_PATH" gst-inspect-1.0 cdgdec

Replace x86_64-unknown-linux-gnu with your system's Rust target triple (rustc -vV).

The plugin can also be installed system-wide:

$ cargo cbuild -p gst-plugin-cdg --prefix=/usr
$ cargo cinstall -p gst-plugin-cdg --prefix=/usr

This will install the plugin to /usr/lib/gstreamer-1.0. You can use --libdir to pass a custom lib directory such as /usr/lib/x86_64-linux-gnu for example.

Note that you can also just use cargo directly to build Rust static libraries and shared C libraries. cargo-c is mostly useful to build static C libraries and generate pkg-config files.

In case cargo complains about dependency versions after a git pull, cargo update may be able to resolve those.

LICENSE

gst-plugins-rs and all crates contained in here are licensed under one of the following licenses

GStreamer itself is licensed under the Lesser General Public License version 2.1 or (at your option) any later version: https://www.gnu.org/licenses/lgpl-2.1.html

Contribution

Any kinds of contributions are welcome as a merge request.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in gst-plugins-rs by you shall be licensed under the license of the plugin it is added to.

For new plugins the MPL-2 license is preferred.

gst-plugin-rs's People

Contributors

alatiera avatar alfro avatar bilelmoussaoui avatar dpeite avatar fengalin avatar ford-prefect avatar gdesmott avatar heftig avatar jbeich avatar jyelloz avatar mathieuduponchelle avatar max-khm avatar nirbheek avatar philn avatar rafaelcaricio avatar rajneeshksoni avatar rubenrua avatar sajeerahamed15 avatar samuvlad avatar sanchayanmaity avatar sdroege avatar seungha-yang avatar thaytan avatar thiblahute avatar tkanakamalla avatar tp-m avatar vivia avatar xclaesse avatar ystreet avatar zhao-gang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gst-plugin-rs's Issues

retrieve-sender-address property in ts-udpsrc

Hi,

I am using udpsrc, I tried to replace it with ts-udpsrc, but "retrieve-sender-address" property is not there yet. I am using the property to retrieve address by adding a probe in udpsrc src pad.

struct _GstNetAddressMeta *meta = gst_buffer_get_net_address_meta(buffer);                                                                                                                    
assert(meta);                                                                                                                                                                                 
                                                                                                                                                                                              
GInetSocketAddress *sockAddr = (GInetSocketAddress *) meta->addr;                                                                                                                             
assert(sockAddr);

Is there any other way to achieve this in ts-udpsrc ?

Gstreamer does not have features: v1_14

Hi,

I’m trying to use “whipsink”, I followed the instruction mentioned here Fernando Jiménez Moreno / gst-plugins-rs · GitLab to build. When running this command cargo cbuild --manifest-path video/cdg/Cargo.tom, it produces the error

Error: CliError { error: Some(failed to select a version for `gstreamer`.
    ... required by package `gst-plugin-claxon v0.6.0 (/home/a/Desktop/x/gst-plugins-rs/audio/claxon)`
versions that meet the requirements `*` are: 0.23.0

the package `gst-plugin-claxon` depends on `gstreamer`, with features: `v1_14` but `gstreamer` does not have these features.

I have no idea how can I fix this error, I’ve searched on Google but there is no post relate to the feature of v1_14. Any help would be appreciated.

Thank you very much!

Add wrapper for GstVideoDecoder

See https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-GstVideoDecoder.html

It probably makes sense to first wrap more parts libgstvideo in gstreamer-rs so that things like GstVideoCodecFrame and GstVideoInfo/Frame can possibly be directly used without writing manual bindings. sdroege/gstreamer-rs#5

As part of this should also be a simple decoder element. Maybe just a minimal element that takes e.g. a BMP image and decodes it (or rather parses the header and throws away the header).

build issue

I tried to build this cargo.toml file

[package]
...

[dependencies]
glib = "0.4"
gstreamer = "0.10"
gstreamer-base = "0.10"
gst-plugin = "0.1"
gstreamer-video = "0.10"

[lib]
name = "gstrstutorial"
crate-type = ["cdylib"]
path = "src/lib.rs"

I'm getting this error when I try cargo build

Compiling gstreamer-video-sys v0.4.0
error: failed to run custom build command for `gstreamer-video-sys v0.4.0`
process didn't exit successfully: `/home/.../Desktop/GStreamer/plugin tests/target/debug/build/gstreamer-video-sys-0a43cf028baefe7d/build-script-build` (exit code: 1)
--- stderr
`"pkg-config" "--libs" "--cflags" "gstreamer-video-1.0 >= 1.8"` did not exit successfully: exit code: 1
--- stderr
Package gstreamer-video-1.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `gstreamer-video-1.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gstreamer-video-1.0' found

Implement a VobSub decoder

Around https://docs.rs/vobsub/0.2.3/vobsub/, probably best as GstVideoDecoder subclass once we have it... or GstElement.

Needs some changes to expose the &[u8] based API that is currently internally, and accepts elementary stream packets.

The repository also has some SRT code, might be worth adding something around that too.

CC @emk who brought this to my attention on reddit

Suggested 'mirror' gitlab.freedesktop.org/gstream doesn't respond

The proposed repo is offline: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs

Also causing build issue:
Updating git repository https://gitlab.freedesktop.org/gstreamer/gstreamer-rs warning: spurious network error (3 tries remaining): unexpected http status code: 504; class=Http (34) warning: spurious network error (2 tries remaining): unexpected http status code: 504; class=Http (34) warning: spurious network error (1 tries remaining): unexpected http status code: 504; class=Http (34) error: failed to get gstreameras a dependency of packagegst-plugin-tutorial v0.9.11 (/home/bwilma/BuildAgent/work/100b0d6fce2ffa2f/playground/gs_prestudy/webrtc/builddir/gst-plugins-rs-cross/tutorial)`

Thread panicks when filter is used inside C code

I want to use rsaudioecho filter in C code (is it even possible?), but it panicks at gst_init() if GST_PLUGIN_PATH is specified

(macOS Sierra 10.12.6, Xcode 8.3.3 (8E3004b), clang-802.0.42, rustc 1.23.0-nightly, cargo 0.24.0-nightly, GStreamer 1.12.3)

int main(int argc, char *argv[]) {
    setenv("RUST_BACKTRACE", "1", 1);
    setenv("GST_PLUGIN_PATH","/Users/username/sources/gstreamer/gst-plugin-rs",1);

    /* Initialize GStreamer */
    gst_init (&argc, &argv);

    return 0;
}

output:

thread '<unnamed>' panicked at 'assertion failed: ::types::instance_of::<Self>(ptr as *const _)', /Users/username/.cargo/git/checkouts/gstreamer-rs-8719441e7bb90fbc/2f7ee30/gstreamer/src/auto/plugin.rs:14:0
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:69
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:58
             at src/libstd/panicking.rs:381
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:397
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:577
   5: std::panicking::begin_panic
             at /Users/travis/build/rust-lang/rust/src/libstd/panicking.rs:538
   6: <gstreamer::auto::plugin::Plugin as glib::translate::FromGlibPtrBorrow<*mut gstreamer_sys::GstPlugin>>::from_glib_borrow
             at /Users/username/sources/gstreamer/gst-plugin-rs/gst-plugin-audiofx/<panic macros>:3
   7: glib::translate::from_glib_borrow
             at /Users/username/.cargo/git/checkouts/glib-928cf7b282977403/a51dc67/src/translate.rs:825
   8: gstrsaudiofx::plugin_desc::plugin_init_trampoline
             at /Users/username/sources/gstreamer/gst-plugin-rs/gst-plugin-audiofx/<plugin_define macros>:24
   9: gst_plugin_register_func
  10: _priv_gst_plugin_load_file_for_registry
  11: exchange_packets
  12: _gst_plugin_loader_client_run
  13: main
fatal runtime error: failed to initiate panic, error 5
Program ended with exit code: 9

BTW, rsaudioecho works well with gst-launch-1.0 and gst-inspect-1.0

Youtube HLS support

Would it be possible to implement streaming to Youtube over HLS ? Which is needed if you'd like to stream in H265 (HEVC)

See documentation here

webrtcsink will not connect to signaller

I have tried with both the library provided signaller as well as a custom "echo server" and running webrtcsink with gst-launch will simply never connect to the websocket. It doesn't change anything whether I provide a signaller::uri or not. Needless to say because of this, the usage instructions from the documentation do not work either, as it will not connect to the rust signaller (can be confirmed by no debugging message showing up whatsoever), so the webpage will not show the test stream. Am I doing something wrong or is this a known bug ?

Thanks!

Port over to gstreamer-rs

Currently we have lots of handcrafted bindings for various GStreamer types or call FFI functions directly. This should be moved to using the proper bindings from gstreamer-rs.

Best approach here would probably be to first do (one after another) source.rs, sink.rs, demuxer.rs and then get rid of everything in the other files.

Types like FlowReturn would be wrapped in our custom FlowError (omitting the Ok case for Result<_,_>). Whatever makes sense should become a wrapper around the gstreamer-rs types with additional functionality.

In the context of this, also see sdroege/gstreamer-rs#3 which can be done before or after (in which case we would have a custom newtype for that here for the time being).

Build issue - user error?

  1. building on Mac M1Max
  2. Newbie (<3 days old) Rust user
  3. Trying to build the gst-plugin-rs project
  4. Installed gstreamer "official" packages - devel + non-level
  5. gstreamer works fine (e.g. gst-launch-1.0 videotestsrc ! autovideosink)
  6. "cargo build" fails with this errors. I am probably missing a step or two - any pointers would be helpful 🙏

error occurred: Command "cc" "-O1" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-arch" "arm64" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include/gstreamer-1.0" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include/glib-2.0" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/lib/glib-2.0/include" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include/gstreamer-1.0" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include" "-I" "/Users/matt/Projects/cerbero-1.20/build/dist/darwin_universal/include/orc-0.4" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include/gstreamer-1.0" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/include/glib-2.0" "-I" "/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/lib/glib-2.0/include" "-Wall" "-Wextra" "-DRTPJitterBuffer=TsRTPJitterBuffer" "-DRTPJitterBufferClass=TsRTPJitterBufferClass" "-DRTPJitterBufferPrivate=TsRTPJitterBufferClass" "-o" "/Users/cgadgil/dev/src/scalers-ai/common/tmp/gst-plugin-rs/target/debug/build/gst-plugin-threadshare-6bbad1f4e13769c6/out/src/jitterbuffer/rtpjitterbuffer.o" "-c" "src/jitterbuffer/rtpjitterbuffer.c" with args "cc" did not execute successfully (status code exit status: 1).

warning: build failed, waiting for other jobs to finish...

Add wrapper for GstVideoEncoder

See https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-GstVideoEncoder.html

It probably makes sense to first wrap more parts libgstvideo in gstreamer-rs so that things like GstVideoCodecFrame and GstVideoInfo/Frame can possibly be directly used without writing manual bindings. sdroege/gstreamer-rs#5

As part of this should also be a simple decoder element. Maybe just a minimal element that takes e.g. a BMP image and encodes it (or rather adds the header).

Move panic handling for vfuncs into trait

The default implementation would just abort(), element and others could catch, report an error message and poison the object.

This probably would have to be on the ObjectType trait

Get rid of C code in demuxer

There's currently some C code for the demuxer. This should be replaced by corresponding, unsafe Rust code. Just like for Source and Sink.

It might make sense to refactor/redesign the demuxer trait at the same time. See #16

Add wrapper for GstBaseTransform

See https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-libs/html/GstBaseTransform.html
Might make sense to do this first: sdroege/gstreamer-rs#7

First step would be to come up with an API design (mostly the main trait) in the style of Source, Sink and Demuxer. That is:

  • all functions return a value and don't call into other elements. So return value would be some kind of enum
  • the general idea of the C base class would be kept (i.e. you have some kind of transform function, transform_caps, etc)
  • implementors of the trait do not have to worry about threading

As part of this should also be a simple transform element. Maybe just a minimal "volume" element, or mono to stereo converter (the latter would at least make use of caps transformations).

rtspsrc vs fallbacksrc

Hi,

We loved the idea of the fallbacksrc and very needed. But our pipeline working with rtspsrc but not wrking with fallbacksrc.

function create_rtsp_sources() { echo "Rtsp sources creating" for ((n = 0; n < $num_of_src; n++)); do src_name="SRC_${n}" src_name="${!src_name}" rtsp_sources+="rtspsrc async-handling=true location=$src_name name=source_$n message-forward=true ! \ decodebin3 ! \ queue name=custom_preprocess_q_$n leaky=no max-size-buffers=32 max-size-bytes=0 max-size-time=0 ! \ $decode_scale_elements ! videoconvert n-threads=8 ! \ video/x-raw,pixel-aspect-ratio=1/1 ! \ fun.sink_$n sid.src_$n ! \ queue name=comp_q_$n leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! \ comp.sink_$n " #echo "$n -- $rtsp_sources \n\n" done }

isnt the fallbacksrc is the same output of the rtspsrc ?
How we can replace the rtspsrc with fallbacksrc for above pipeline ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.