metartc / ffmpeg-webrtc Goto Github PK

ffmpeg-webrtc for whip and whep protocol

License: Apache License 2.0

Makefile 1.30% C 91.56% C++ 0.32% Shell 0.09% Perl 0.08% CSS 0.01% Assembly 6.44% Verilog 0.01% Objective-C 0.12% Cuda 0.04% Roff 0.01% Awk 0.01% HTML 0.01% Ruby 0.01% Python 0.04% CMake 0.01%

ffmpeg webrtc whip whep

ffmpeg-webrtc's Introduction

ffmpeg-webrtc

implement whip and whep protocol
WHIP: WebRTC HTTP Ingestion Protocol
WHEP: WebRTC HTTP Egress Protocol
集成到metaRTC到ffmpeg，使ffmpeg支持webrtc
Integration into metaRTC to ffmpeg makes ffmpeg support webRTC

metartc6 compile

cd FFmpeg-n4.3.3/metartc6/metartc6
cd libmetartccore6
mkdir build
cd build
./cmake_x64.sh
or
./cmake_android.sh
cp ./libmetartccore6.a ../../

ffmpeg compile

将编译的libmetartccore6.a和其他第三方库放入FFmpeg-n4.3.3/metartc6/目录里

./configure --enable-libx264 --enable-gpl --extra-libs='-L/path/FFmpeg-n4.3.3/metartc6 -lmetartccore6 -lpthread -lsrtp2 -lssl -lcrypto -ldl'
make -j8

推流命令 WHIP

ffmpeg ......-acodec opus -strict -2 -ar 48000 -f webrtc "url"
srs sample: whip url http://192.168.0.105:1985/rtc/v1/whip/?app=live&stream=livestream
ffmpeg ......-acodec opus -strict -2 -ar 48000 -f webrtc "http://192.168.0.105:1985/rtc/v1/whip/?app=live&stream=livestream"
ffmpeg ......-acodec opus -strict -2 -ar 48000 -f webrtc "webrtc://192.168.0.105:1985/rtc/v1/whip/?app=live&stream=livestream"
./ffmpeg -re -i /path/test.mp4 -vcodec libx264 -acodec opus -strict -2 -ar 48000 -f webrtc "http://192.168.0.105:1985/rtc/v1/whip/?app=live&stream=livestream"

拉流命令 WHEP

ffplay "webrtc://whep_url"
srs sample: whep url http://192.168.0.105:1985/rtc/v1/whip-play/?app=live&stream=livestream
ffplay "webrtc://192.168.0.105:1985/rtc/v1/whip-play/?app=live&stream=livestream"

ffmpeg-webrtc's People

Contributors

Stargazers

Forkers

asdlei99 anylee2021 cl-hello zzjs2001702 xuanniao lance-lmwang lightsun jimmyzhangcn zhangsanfeng86 carcloudfly xuchuanxin tung3110

ffmpeg-webrtc's Issues

webrtc播放部分区域出现马赛克

srs服务器rtmp推流webrtc，ffplay播放会有声画不同步和部分区域马赛克现象，编译chrome的webrtc播放没有马赛克现象

播放zlmedia的webrtc api的h265流时好时坏

使用ffplay播放zlm的webrtc流，有时是好的，有时是坏的，播放地址类似
http://127.0.0.1/index/api/webrtc?app=live&stream=test6&type=play
只有声音，没有画面。
切换回h264的流正常播放，会是什么原因？

解决windows上ffmpeg推流错误的问题

修改 ffmep/libavformat/webrtc_muxer.c 中函数 static int packet_queue_wait_start(WEBRTCContext *s, int64_t timeout)
文件中 89行的循环次数为：
int loop = 100000;
主要原因好像是因为window的pthread的等待时间和linux上的等待时间是不一样的，所以增加循环次数后就成功了。
希望能帮到各位。

python使用subprocess管道协议推送srs服务器失败

python代码

1、根据metaRTC命令参数运行推送视频时正常的
2、修改为管道推送视频流报错，没有该方法

whip 不支持https

日志如下：
/workspace/learn/ffmpeg-webrtc/FFmpeg-n4.3.3/ffmpeg_g -f video4linux2 -i /dev/video0 -g 50 -pix_fmt yuv420p -framerate 5 -c:v libx264 -profile:v baseline -tune zerolatency -preset ultrafast -bf 0 -f webrtc https://192.168.3.247:9061/index/api/whip?app=live&stream=test
ffmpeg version 681b587-patrickz Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
configuration: --prefix=/opt/ffmpeg_ubuntu --extra-libs='-lpthread -lm' --bindir=/opt/ffmpeg_ubuntu/bin --extra-cflags=-static --extra-ldflags=-static --pkg-config-flags=--static --extra-cflags=-I/opt/ffmpeg_deps/include --extra-cflags=-I/opt/ffmpeg_deps/include/fdk-aac --extra-ldflags=-L/opt/ffmpeg_deps/lib --extra-libs='-lmetartccore6 -lpthread -lsrtp2 -lssl -lcrypto -ldl' --enable-gpl --enable-libvorbis --enable-pic --enable-static --enable-nonfree --enable-libx264 --enable-libvpx --enable-libopus --disable-optimizations --disable-stripping --enable-openssl --enable-runtime-cpudetect --extra-version=patrickz
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 835.039518, bitrate: 147456 kb/s
Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 1280x720, 147456 kb/s, 10 fps, 10 tbr, 1000k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x3501d40] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
[libx264 @ 0x3501d40] profile Constrained Baseline, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x3501d40] 264 - core 164 r3106 eaa68fa - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=8 lookahead_threads=8 sliced_threads=1 slices=8 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=50 keyint_min=5 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[webrtc @ 0x35006c0] webrtc_open https://192.168.3.247:9061/index/api/whip?app=live&stream=test

webrtc init>>>>>>>>>>>>>>>>>>>https://192.168.3.247:9061/index/api/whip?app=live&stream=test

srtp init success!

avinfo->sys.whipUrl==index/api/whip?app=live&stream=test
[webrtc @ 0x35006c0] connect failed! uri= https://192.168.3.247:9061/index/api/whip?app=live&stream=test
[webrtc @ 0x35006c0] webrtc_open exit
[webrtc @ 0x35006c0] webrtc_write_header, filename https://192.168.3.247:9061/index/api/whip?app=live&stream=test

undefined reference to `x264_encoder_open_157'

make报错：

/usr/bin/ld: libavcodec/libavcodec.a(libx264.o): in function `X264_init':
/Software/Projects/ffmpeg-webrtc/FFmpeg-n4.3.3/libavcodec/libx264.c:920: undefined reference to `x264_encoder_open_157'

我系统里面没有157的libx264.so.157，我自己编出来的是libx264.so.164.
请假一下如何解决？

静态编译后需要修改端口才能推流

发现SDP查询端口需要修改才能正确推流：
文件YangZlmConnection.c, 搜索yang_zlm_query调用行，修改端口'80'为zlm服务器实际http端口；
文件yang_sdp_querySrs.c, 搜索yang_sdp_querySrs调用行，修改端口代码'session->context.streamConfig->remotePort'替换为实际srs的httpapi端口；

最后文件webrtc_muxer.c中，第152行：
int32_t mediaServer=Yang_Server_Srs;
默认采用srs服务器，修改为
int32_t mediaServer=Yang_Server_Zlm;
即采用zlm服务器。

仅供参考。

FFmpeg-Webrtc Ubuntu 20.04编译补充

FFmpeg-Webrtc Ubuntu编译

1、首先编译metartc相关包
    cd ffmpeg-webrtc/FFmpeg-n4.3.3/metartc6/metartc6
    
    cd libmetartccore6
    
    # cmake_x64会自动创建build文件夹
    ./cmake_x64.sh
    
    # 把编译成功的 libmetartccore6.a 复制到metartc6文件夹下
    cp ./libmetartccore6.a FFmpeg-n4.3.3/metartc6

    # Requires ffmpeg to be configured with --enable-gpl --enable-libx264
    sudo apt-get install libx264-dev

2、解压libsrtp-2-fit.tar.gz、openssl-1.1-fit.tar.gz并进行编译
    cd ffmpeg-webrtc/FFmpeg-n4.3.3/metartc6
    
    # 编译 srtp2
    tar zxvf libsrtp-2-fit.tar.gz
    cd libsrtp-2-fit/
    ./configure 
    make
    cp libsrtp2.a FFmpeg-n4.3.3/metartc6
    
    # 编译openssl
    tar zxvf openssl-1.1-fit.tar.gz
    cd openssl-1.1-fit/
    ./config
    make
    cp libcrypto.a libssl.a FFmpeg-n4.3.3/metartc6

3、编译FFmpeg-webrtc    
    ./configure --enable-libx264 --enable-gpl --enable-cross-compile --extra-libs='-L/home/user/ffmpeg-webrtc/FFmpeg-n4.3.3/metartc6 -lmetartccore6 -lpthread -lsrtp2 -lssl -lcrypto -ldl'
    make -j8
    
    # 编译期间会报很多错误，可以不用理会
    make install

4、目前测试结果
    1）ffplay 没有，可能是少包了，或者编译FFmpeg-webrtc时./configure中没有启用相关配置
    2）使用whip推流到SRS流媒体服务器，但是拉流失败！目前暂未找到原因
    
    3）把RTMP流推送到SRS，SRS把RTMP转成RTC，使用FFmpeg-webrtc whep拉流，成功实现延迟1秒左右

FFmpeg-Webrtc拉取 whep命令示例

# SRS的whep地址是：http://127.0.0.1:1985/rtc/v1/whep/?app=live&stream=livestream
ffmpeg -i 'webrtc://127.0.0.1:1985/rtc/v1/whep/?app=live&stream=livestream' -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video30

ffmpeg -i 'webrtc://127.0.0.1:1985/rtc/v1/whip-play/?app=live&stream=livestream' -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video30

增加ffplay、pulse配置 -》编译FFmpeg-webrtc

我把Get the Dependencies建议的包都给安装了

# opus 配置
    sudo apt-get install libopus-dev
    
# 编译FFmpeg-Webrtc
    ./configure --enable-libx264 --enable-gpl --enable-cross-compile --enable-libpulse --enable-libopus --enable-ffplay --extra-libs='-L/home/oook/user/ffmpeg-webrtc/FFmpeg-n4.3.3/metartc6 -lmetartccore6 -lpthread -lsrtp2 -lssl -lcrypto -ldl'

# ffplay 播放命令
    ffplay -i 'webrtc://127.0.0.1:1985/rtc/v1/whip-play/?app=live&stream=livestream'

webrtc推流到SRS失败

1）SRS配置：https.rtc.conf。部署在局域网内。
2）推流命令（windows）：两种写法问题相同。ffmpeg-metartc基于"msys2+msvc"和“msys2+mingw64”两种方式编译问题相同。
ffmpeg -re -i xxx.mp4 -c copy -f webrtc webrtc://192.16.15.211/live/livestream
ffmpeg -re -i xxx.mp4 -vcodec libx264 -acodec opus -strict -2 -f webrtc webrtc://192.16.15.211/live/livestream
3）错误提示：
webrtc init>>>>>>>>>>>>>>>>>>>webrtc://192.16.15.211/live/livestream
[webrtc @ 00000000003796c0] webrtc_open webrtc://192.16.15.211/live/livestream
srtp init success!
startRtc,port=19030
candidate:ip==192.16.15.211,type=host,port=8000
candidate:ip==192.16.15.211,type=host,port=8000
remoteIp=192.16.15.211,port=8000
dtls is openssl[webrtc @ 00000000003796c0] webrtc_open exit
[webrtc @ 00000000003796c0] webrtc_write_header, filename webrtc://192.16.15.211/live/livestream
u[webrtc @ 00000000003796c0] dwebrtc_write_header wait failed, Error number -138 occurred
p [webrtc @ 00000000003796c0] swebrtc_close
erver is starting,localPort=19030
webrtc disconnect
[webrtc @ 00000000003796c0] webrtc_close exit
webrtc deinit>>>>>>>>>>>>>webrtc://192.16.15.211/live/livestream
Could not write header for output file #0 (incorrect codec parameters ?): Error number -138 occurred
Error initializing output stream 0:0 --
[opus @ 0000000000330700] 129 frames left in the queue on closing
[opus @ 0000000000330700] Average Intensity Stereo band: 20.0
[opus @ 0000000000330700] Dual Stereo used: nan%
Conversion failed!

编译出错：undefined reference to `yang_create_videoEncoder'

将libmetartccore5编译动态库，然后重新配置ffmpeg-meta-main, 就提示c compiler test failed. 查看日志发现有以下错误：
/home/russ/file/ffmpeg-metartc-main/FFmpeg-n4.3.3/metartc5/libmetartccore5.so: undefined reference to yang_destroy_videoEncoder' /home/russ/file/ffmpeg-metartc-main/FFmpeg-n4.3.3/metartc5/libmetartccore5.so: undefined reference to clip255'
/home/russ/file/ffmpeg-metartc-main/FFmpeg-n4.3.3/metartc5/libmetartccore5.so: undefined reference to `yang_create_videoEncoder'

这三函数看起来确实缺少实现。不知道这里面有什么编译技巧，还是需要自己来实现一个encoder.c ？
请专家指导啊

WebRTC_Muxer.c中互斥变量没有初始化导致异常

使用ffmpeg-webrtc推流到srs，拉流时解码失败，请问哪里有问题呢

[h264 @ 0x5638f254b340] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x5638f254b340] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x5638f254b340] nal_unit_type: 5(IDR), nal_ref_idc: 0
[h264 @ 0x5638f254b340] A non-intra slice in an IDR NAL unit.
[h264 @ 0x5638f254b340] decode_slice_header error
[h264 @ 0x5638f254b340] no frame!
Error: Error sending a packet for decoding.

Segmentation fault

I use a custom WHIP proxy to publish WebRTC towards a Wowza server. It works with OBS-WHIP PR (obsproject/obs-studio#7926) without problems.

`frhb64652ds FFmpeg-n4.3.3 # ./ffmpeg -re -i /root/bbb/bbb_720p_2mbps_aac.mp4 -vcodec libx264 -acodec opus -strict -2 -ar 48000 -f webrtc "http://fr.karsiyaka.com:8080/api/whip?stream=ff_mur_test?token=85a33eea-821e-443d-a1b2-c012c8614905" -loglevel info
ffmpeg version 681b587 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-libx264 --enable-gpl --extra-libs='-L./metartc6 -lmetartccore6 -lpthread -lsrtp2 -lssl -lcrypto -ldl' --enable-openssl --enable-version3 --enable-libx264 --enable-nonfree --enable-libopus
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/root/bbb/bbb_720p_2mbps_aac.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Big Buck Bunny, Sunflower version
artist : Blender Foundation 2008, Janus Bager Kristensen 2013
composer : Sacha Goedegebure
encoder : Lavf58.76.100
comment : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
genre : Animation
Duration: 00:10:34.60, start: 0.000000, bitrate: 2240 kb/s
Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 2136 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
Metadata:
handler_name : GPAC ISO Video Handler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 96 kb/s (default)
Metadata:
handler_name : GPAC ISO Audio Handler
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #0:1 (aac (native) -> opus (native))
Press [q] to stop, [?] for help
[libx264 @ 0x55ec94141a80] using SAR=1/1
[libx264 @ 0x55ec94141a80] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x55ec94141a80] profile High, level 3.1
[libx264 @ 0x55ec94141a80] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00

webrtc init>>>>>>>>>>>>>>>>>>>http://fr.karsiyaka.com:8080/api/whip?stream=ff_mur_test?token=85a33eea-821e-443d-a1b2-c012c8614905
[webrtc @ 0x55ec94143940] webrtc_open http://fr.karsiyaka.com:8080/api/whip?stream=ff_mur_test?token=85a33eea-821e-443d-a1b2-c012c8614905

srtp init success!

avinfo->sys.whipUrl==api/whip?stream=ff_mur_test?token=85a33eea-821e-443d-a1b2-c012c8614905
session->context.avinfo->sys.whipUrl=api/whip?stream=ff_mur_test?token=85a33eea-821e-443d-a1b2-c012c8614905
startRtc,port=22341
candidate:ip==69.165.102.143,type=host,port=6978
Segmentation fault`

Incorrect build instructions

cd FFmpeg-n4.3.3/metartc6/metartc6
cd libmetartccore6
mkdir build
cd build
./cmake_x64.sh
or
./cmake_android.sh
cp ./libmetartccore6.a ../../

It should be this:

cd FFmpeg-n4.3.3/metartc6/metartc6
cd libmetartccore6
./cmake_x64.sh
or
./cmake_android.sh
cp ./build/libmetartccore6.a /usr/local/lib

Otherwise, ffmpeg can't build due to lack of libmetartccore6.a inside ldconfig folders
For debian 10, those are (default)
/usr/lib/x86_64-linux-gnu/libfakeroot
/usr/local/lib
/usr/local/lib/x86_64-linux-gnu
/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu

ffplay拉流出现只有audio没有video

ubuntu22.04

环境
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.4 LTS
Release: 22.04
Codename: jammy
6.5.0-35-generic #35~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue May 7 09:00:52 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
metartc6频繁出现只有audio没有video的情况，而且拉流缓慢(5s)，metartc5偶尔出现只有audio没有video的情况，拉流稍慢(3s)。

一直推理，在另一个客户端重复打开关闭ffplay，出现只有audio现象，如下：
有video

没有video

./ffplay webrtc://10.10.2.3/exam/2021091400479_topStream
ffplay version 18ec4fa Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
  configuration: --enable-libx264 --enable-gpl --extra-libs='-L/home/shaolin/gitee/ffmpeg-metartc/FFmpeg-n4.3.3/metartc6 -lmetartccore6 -lpthread -lsrtp2 -lssl -lcrypto -ldl'
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
[webrtc @ 0x7f158c000bc0] webrtc_read_header, filename webrtc://10.10.2.3/exam/2021091400479_topStream
[webrtc @ 0x7f158c000bc0] webrtc_open webrtc://10.10.2.3/exam/2021091400479_topStream
    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
srtp init success!

avinfo->sys.whipUrl==rtc/v1/whip/?app=%s&stream=%s
[webrtc @ 0x7f158c000bc0] connect failed! uri= webrtc://10.10.2.3/exam/2021091400479_topStream
[webrtc @ 0x7f158c000bc0] webrtc_open exit
[webrtc @ 0x7f158c000bc0] webrtc_read_header wait failed, Connection timed out
[webrtc @ 0x7f158c000bc0] webrtc_close
session->context.avinfo->sys.whipUrl=rtc/v1/whip/?app=%s&stream=%s
webrtc disconnect
[webrtc @ 0x7f158c000bc0] webrtc_close exit
webrtc://10.10.2.3/exam/2021091400479_topStream: Connection timed out
    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0

请问哪里不正确？

How to support the WebRTC playback of Tencent Cloud Live Streaming

The playback address of Tencent Cloud Live Streaming is similar to the following.
webrtc://.".$domain."/live/".$streamName or
webrtc://.".$domain."/live/".$streamName??txSecret=xxxx&txTime=xxx