Key Frames requests from peer via a webrtcbin signal ? about gstwebrtc-demos HOT 9 CLOSED

One can force a keyframe by sending a GstForceKeyUnit request upstream on the src pad of the video stream output by webrtcbin. This is converted internally by GStreamer's rtp implementation into a fir/pli rtcp message that is sent to the peer: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/blob/master/gst/rtpmanager/rtpsession.c#L4435 .

https://gstreamer.freedesktop.org/documentation/design/keyframe-force.html#upstream

from gstwebrtc-demos.

Just to document this for anyone facing the same issue. Read a bit more how events works on GStreamer:
https://gstreamer.freedesktop.org/documentation/plugin-development/advanced/events.html#upstream-events

As suggested, to request a "GstKeyFrameUnit", an event must be sent on the "video pad" of the webrtcbin SRC pad upstream.

So, once you have the "trigger" to request, simply call:

gst_pad_send_event( webrtcbin_video_src_pad, gst_event_new_custom( GST_EVENT_CUSTOM_UPSTREAM, gst_structure_new( "GstForceKeyUnit", "all-headers", G_TYPE_BOOLEAN, TRUE, NULL ) ) );

This has worked for me.

from gstwebrtc-demos.

I also experimented with gstreamer for a long time. Many different cases, but one problem - there is no keyframe.

I described the problem here:
#31
https://bugzilla.gnome.org/show_bug.cgi?id=797280

FIR in gstreamer not used. There is a NACK PLI, but apparently by the chrome graphics it does not work.

I also have a question how to solve the problem?

from gstwebrtc-demos.

Why doesn't RTP get an event GstForceKeyUnit?
What you need to do in this demo is for chrome to start sending huge frames?

from gstwebrtc-demos.

I don't understand your first question.

In order for chrome to send a key frame on request you need to send a GstForceKeyUnit event upstream on the src pad of the video output from webrtcbin.

from gstwebrtc-demos.

I don't understand your first question.

Excuse me. I from Russia. I don't speak english. :-)

When a decoder cannot decode a stream, it must request a keyframe by sending upstream events GstForceKeyUnit. RTP should get this event. Why is this not happening?

In order for chrome to send a key frame on request you need to send a GstForceKeyUnit event upstream on the src pad of the video output from webrtcbin.

I see two ways to do this:

1. Handle decoder error and send event GstForceKeyUnit. I do not know how to do that;
2. By timer, for example, every 10 seconds. I think this is not right.

Why is this demo not done this?

from gstwebrtc-demos.

Thanks for your help!
Ok, I am looking at the CAPS of the webrtcbin PAD, and on the media=video, I save it.

On the SINK element of the pipeline ( a shmsink ) when a client connects, I am pushing the event.
First, I have tried this, but nothing changes. Behavior is the same. Video flows after a minute or so.
gst_pad_push_event( pad, gst_event_new_custom( GST_EVENT_CUSTOM_DOWNSTREAM, gst_structure_new ("GstForceKeyUnit", NULL ) ) );

If instead I do like this:

gst_pad_push_event( pad, gst_event_new_custom( GST_EVENT_CUSTOM_UPSTREAM, gst_structure_new ("GstForceKeyUnit", NULL ) ) );

I get this error:

(gst_webrtc:1330): GStreamer-WARNING **: 06:32:15.307: pad webrtcbin:src_1 pushing custom-upstream event in wrong direction

Even though you said to push the event UPSTREAM, it seems to be the wrong direction.... I guess something else could be wrong.

How can I check the KeyFrame request is actually sent ?

Thanks.

from gstwebrtc-demos.

I am not sure this is relevant to this issue, but I see these "red" warnings:
0:00:09.589394552 1330 0x4894c0 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:nicesrc1:src could not send sticky events
0:00:09.590183278 1330 0xb2f10490 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:nicesrc2:src could not send sticky events
0:00:09.609341160 1330 0xb2f10580 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:nicesrc3:src could not send sticky events
0:00:09.610051930 1330 0xb2f10520 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:nicesrc4:src could not send sticky events

And, I also get somewhat frequent, these as well:

0:00:09.653312337 1330 0x489460 WARN dtlsconnection gstdtlsconnection.c:737:openssl_poll:GstDtlsConnection@0xb2f09f60 do_handshake encountered BIO error
0:00:09.655089033 1330 0x489460 WARN dtlsconnection gstdtlsconnection.c:737:openssl_poll:GstDtlsConnection@0xb2f0a448 do_handshake encountered BIO error

from gstwebrtc-demos.

I am trying to play live RTSP( H.264 Baseline profile ) with this example. And on latest chrome( Version 70.0.3538.110 (Official Build) (64-bit) ) the GStreamer pipeline gives error messages as below

0:02:05.420128000 487 0x22b90f0 INFO x264enc gstx264enc.c:2483:gst_x264_enc_encode_frame: Forcing key frame
0:02:05.423306000 487 0x22b9370 INFO GST_PADS gstpad.c:4464:gst_pad_chain_list_default:capsfilter0:sink chaining each buffer in list individually
0:02:05.448303000 487 0x22b90f0 INFO x264enc gstx264enc.c:2483:gst_x264_enc_encode_frame: Forcing key frame
0:02:05.451012000 487 0x22b9370 INFO GST_PADS gstpad.c:4464:gst_pad_chain_list_default:capsfilter0:sink chaining each buffer in list individually
0:02:05.490787000 487 0x22b90f0 INFO x264enc gstx264enc.c:2483:gst_x264_enc_encode_frame: Forcing key frame
0:02:05.493411000 487 0x22b9370 INFO GST_PADS gstpad.c:4464:gst_pad_chain_list_default:capsfilter0:sink chaining each buffer in list individually
0:02:05.517910000 487 0x22b90f0 INFO x264enc gstx264enc.c:2483:gst_x264_enc_encode_frame: Forcing key frame
0:02:05.520177000 487 0x22b9370 INFO GST_PADS gstpad.c:4464:gst_pad_chain_list_default:capsfilter0:sink chaining each buffer in list individually
0:02:05.546375000 487 0x22b90f0 INFO x264enc gstx264enc.c:2483:gst_x264_enc_encode_frame: Forcing key frame
0:02:05.548331000 487 0x22b9370 INFO GST_PADS gstpad.c:4464:gst_pad_chain_list_default:capsfilter0:sink chaining each buffer in list individually
0:02:05.587885000 487 0x22b90f0 INFO x264enc gstx264enc.c:2483:gst_x264_enc_encode_frame: Forcing key frame
0:02:05.589943000 487 0x22b9370 INFO GST_PADS gstpad.c:4464:gst_pad_chain_list_default:capsfilter0:sink chaining each buffer in list individually
0:02:05.617993000 487 0x22b90f0 INFO x264enc gstx264enc.c:2483:gst_x264_enc_encode_frame: Forcing key frame
0:02:05.620106000 487 0x22b9370 INFO GST_PADS gstpad.c:4464:gst_pad_chain_list_default:capsfilter0:sink chaining each buffer in list individually
0:02:05.647349000 487 0x22b90f0 INFO x264enc gstx264enc.c:2483:gst_x264_enc_encode_frame: Forcing key frame
0:02:05.649484000 487 0x22b9370 INFO GST_PADS gstpad.c:4464:gst_pad_chain_list_default:capsfilter0:sink chaining each buffer in list individually

My Gst launch pipeline is as below:
pipe1 = gst_parse_launch ("rtspsrc location=rtsp://192.168.1.167/stream1.mp4 latency=10 drop-on-latency=TRUE ! rtph264depay ! avdec_h264 ! x264enc ! rtph264pay ! application/x-rtp,media=video,encoding-name=H264,payload=96 ! webrtcbin name=sendrecv", &error);

SDP Exchanged as below on Chrome:
Received {"sdp":{"type":"offer","sdp":"v=0\r\no=- 8884209566387641518 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=ice-options:trickle\r\na=msid-semantic:WMS sendrecv\r\nm=video 9 UDP/TLS/RTP/SAVPF 96\r\nc=IN IP4 0.0.0.0\r\na=setup:actpass\r\na=ice-ufrag:K6V0GVfxPVYz8CPgRBsh4wmoFo97W/ud\r\na=ice-pwd:6oFosFamvKA8LF3f2+kiTo4DjDXv65bv\r\na=sendrecv\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 H264/90000\r\na=rtcp-fb:96 nack pli\r\na=framerate:29.970029970029969\r\na=fmtp:96 packetization-mode=1;profile-level-id=42c01f;sprop-parameter-sets=Z0LAH9kAUAW7AW4CAgKAAAH0gAB1MEeMGSQ=,aMuMsg==\r\na=ssrc:3804191999 msid:user1451640273@host-1eb2939 webrtctransceiver0\r\na=ssrc:3804191999 cname:user1451640273@host-1eb2939\r\na=mid:video0\r\na=fingerprint:sha-256 BB:D5:DA:53:8D:FA:04:2D:9A:8E:AE:15:3A:F2:B7:48:37:C1:F3:99:31:58:07:D6:D5:A9:DB:D2:CD:CC:A7:41\r\n"}}
webrtc.js:277 Creating RTCPeerConnection
webrtc.js:174 {"video":true,"audio":true}
webrtc.js:47 Created peer connection for call, waiting for SDP
webrtc.js:109 Received {"ice":{"candidate":"candidate:1 1 UDP 2013266431 fe80::60ac:4446:7bc8:72fb 59260 typ host","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:2 1 TCP 1015022847 fe80::60ac:4446:7bc8:72fb 9 typ host tcptype active","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:3 1 TCP 1010828543 fe80::60ac:4446:7bc8:72fb 53946 typ host tcptype passive","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:4 1 UDP 2013266430 10.142.149.203 59261 typ host","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:5 1 TCP 1015022079 10.142.149.203 9 typ host tcptype active","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:6 1 TCP 1010827775 10.142.149.203 53947 typ host tcptype passive","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:1 2 UDP 2013266430 fe80::60ac:4446:7bc8:72fb 59262 typ host","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:2 2 TCP 1015022846 fe80::60ac:4446:7bc8:72fb 9 typ host tcptype active","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:3 2 TCP 1010828542 fe80::60ac:4446:7bc8:72fb 53948 typ host tcptype passive","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:4 2 UDP 2013266429 10.142.149.203 59263 typ host","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:5 2 TCP 1015022078 10.142.149.203 9 typ host tcptype active","sdpMLineIndex":0}}
webrtc.js:109 Received {"ice":{"candidate":"candidate:6 2 TCP 1010827774 10.142.149.203 53949 typ host tcptype passive","sdpMLineIndex":0}}
webrtc.js:227 Incoming stream: 1 video tracks and 0 audio tracks
webrtc.js:47 Remote SDP set
webrtc.js:47 Got SDP offer
webrtc.js:289 Adding local stream
webrtc.js:47 Got local stream, creating answer
webrtc.js:94 Got local description: {"type":"answer","sdp":"v=0\r\no=- 8499741705278753814 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=msid-semantic: WMS 7lR7bxuA3J5mkaPiDYxiVGhxs5TBzV7gdss4\r\nm=video 9 UDP/TLS/RTP/SAVPF 96\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:BDFH\r\na=ice-pwd:7akvvd8fXdj9Re+mB8pr5589\r\na=ice-options:trickle\r\na=fingerprint:sha-256 47:C1:9B:92:A6:F4:5F:D4:18:2C:F4:E1:D1:F7:93:91:F3:7D:26:70:D2:B3:9E:AF:77:4C:10:50:79:8B:45:62\r\na=setup:active\r\na=mid:video0\r\na=sendrecv\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 H264/90000\r\na=rtcp-fb:96 nack pli\r\na=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=ssrc:882860088 cname:VfrGYuk2cyiUZ+Uw\r\na=ssrc:882860088 msid:7lR7bxuA3J5mkaPiDYxiVGhxs5TBzV7gdss4 824d9874-0554-475e-9f80-ca2ff73adcdf\r\na=ssrc:882860088 mslabel:7lR7bxuA3J5mkaPiDYxiVGhxs5TBzV7gdss4\r\na=ssrc:882860088 label:824d9874-0554-475e-9f80-ca2ff73adcdf\r\n"}
webrtc.js:47 Sending SDP answer

No video is played out on browser

from gstwebrtc-demos.