Does p2p-media-loader check segment validity? about p2p-media-loader HOT 13 CLOSED

It should be somehow supported by your Media Server, HTTP server or CDN. Otherwise, you have to implement a custom solution for that.

For example, create a program that generates hashes for the segments and stores them in the files next to the segments or injects into .m3u8 playlist files the hashes information.

from p2p-media-loader.

Looks like now this approach of hooking RTCDataChannel.send works in Firefox too

from p2p-media-loader.

Hi,

You are right. No checks. But other P2P solutions don't do the checks too (at least it was 1 year ago). It took me 2 hours to create a broken peer for Peer5 that breaks their P2P demos. So be aware of script kiddies :-)

We want to add this feature in one of the next releases. But also we want to get advice from developers about the best approach.

Any way validation information should go from the source and it is up to the source how to implement it.

I am thinking about just adding a validation callback with the following parameters:

Data (ArrayBuffer) of the downloaded segment
URL of the downloaded segment

If the callback returns false, then the segment is not valid.

A developer responsibility will be to implement the actual validator.

Examples of usage:

• You can store a hash\checksum in a segment file name (i.e. URL). Validation callback will get the hash from URL and validate the data against it.

• You can store a hash\checksum\signature\etc. in a file that is located next to segment or in a predefined location (i.e. hash for segment999.ts is stored in segment999.ts.sha). Validation callback will get the hash from the predefined location and validate the data against it.

• You can store a hash\checksum\signature\etc. in HTTP headers by the segment URL. Validation callback will get the hash from the headers and validate the data against it. For example, there is the Content-MD5 header field, as defined in RFC 1864.

• You can create a service that returns hash\checksum\signature\etc. of the segments by request. Validation callback should call the service and validate data.

from p2p-media-loader.

Another idea is in addition pass the tags of the segment from M3U playlist to the callback. hash\checksum\signature can be stored there too.

from p2p-media-loader.

It seems to be a very flexible way to that 👍

I'm a DASH/HLS newbie, but would it be possible to add a comment in the playlist that contains hashes of the following segments? Because if I understood well, the playlist is always sent by the source server 🤔

from p2p-media-loader.

@mrlika

t took me 2 hours to create a broken peer for Peer5 that breaks their P2P demos

may you share this code, to see how have you done it ?

from p2p-media-loader.

I will not publish working code here but:

RTCDataChannel.prototype._send = RTCDataChannel.prototype.send;

RTCDataChannel.prototype.send = function (data) {
    console.log(data.byteLength);
    this._send(data);
};

This is how you can easily hook RTCDataChannel.send method. Works only in Chrome, because RTCDataChannel is not an interface but a class object there. It can also be done in similar way for other browsers but requires more code.

For a peer that sends data to other peers the output will be like:

9
78
6925
16013
16013
16013
16013
16013
16013
....

They send data in messages of size 16013 where first 13 bytes are command header and next 16000 bytes are video data.

You just have to change video data (for example to zeros). Connected peers will hang or start to play video not from the source but from your modified peer :-)

The same approach breaks P2P Media Loader streamings too.

Validation, described above, can protect against such broken peers.

from p2p-media-loader.

Implemented p2pSegmentValidator setting in 696aafd.

Basically it is an ability to provide your Promise-based validation for any segment content loaded via P2P. Here is a simple example of how it can be used:

let n = 0;
let badPeer = false;

if (Hls.isSupported() && p2pml.hlsjs.Engine.isSupported()) {
    var engine = new p2pml.hlsjs.Engine({
        loader: {
            p2pSegmentValidator: async (segment, peerId) => {
                const vcId = ++n;
                console.log("VALIDATOR CALL", vcId, segment, peerId);

                // Now you have segment.data, an ArrayBuffer of received bytes from peerId.
                // As this is a promise, you can compute and/or do ajax request to check validity;
                // if segment considered valid -- you just let the handler to finish without errors;
                // otherwise -- you throw an error; in this case:
                // 1) you receive SegmentError with whatever you passed with "throw";
                // 2) you also receive PeerClose for the peer, as it is considered to be broken.

                // For sake of simplicity of this example, we just wait 1 second;
                // emulating computation or ajax request
                await new Promise(resolve => setTimeout(resolve, 1000));

                // Again, for the sake of simplicity,
                // we just say "whatever 10th validation comes from --- its broken peer"
                if (vcId === 10) {
                    badPeer = peerId;
                }

                if (badPeer !== peerId) {
                    console.log("VALIDATOR OK", vcId, segment.id, peerId);
                } else {
                    console.log("VALIDATOR FAILED", vcId, segment.id, peerId);
                    throw { message: "some error message", url: segment.url, peerId };
                }
            }
        }
    });

    document.title = engine.getDetails().loader.peerId;

    engine.on(p2pml.core.Events.PeerConnect, (peer) => {
        console.log("PEER-CONNECT", peer.id);
    });

    engine.on(p2pml.core.Events.PeerClose, (peerId) => {
        console.log("PEER-CLOSE", peerId);
    });

    engine.on(p2pml.core.Events.SegmentLoaded, (segment, peerId) => {
        console.log("SEG-LOAD", segment.id, peerId);
    });

    engine.on(p2pml.core.Events.SegmentError, (segment, error, peerId) => {
        console.log("SEG-ERR", segment.id, error, peerId);
    });

    var hls = new Hls({ liveSyncDurationCount: 7, loader: engine.createLoaderClass() });
    p2pml.hlsjs.initHlsJsPlayer(hls);
    hls.loadSource("http://wms.shared.streamshow.it/carinatv/carinatv/playlist.m3u8");
    var video = document.getElementById("video");
    hls.attachMedia(video);
} else {
    document.write("Not supported :(");
}

In the example above, we simply count p2pSegmentValidator calls; and when 10th comes -- we decide that this peer is broken (instead of actual validation algorithm).

To test this example:

1. open http://novage.com.ua/p2p-media-loader/demo.html and use http://wms.shared.streamshow.it/carinatv/carinatv/playlist.m3u8 as Video URL; make couple of peers
2. use code above with simple example, like p2p-media-loader-hlsjs/demo/hlsjs.html
   Open browser's console. When 10th call comes it will fail validation and that peer will be closed. That segment will be redownloaded from other peer (or via HTTP if none left). And that peer should not be used to download segments from ever again.

from p2p-media-loader.

Thank you very much @greenya

Do you think we could add this validation in the HTTP loader too? In PeerTube for example, any instance can duplicate a video and therefore add itself as another node in the CDN.

from p2p-media-loader.

P2P Media Loader doesn't work without a HTTP(S) source. That is why we trust the source. If a segment downloaded over P2P isn't valid we can kick the corresponding peer. But what should we do if a segment from the source isn't valid?

from p2p-media-loader.

Sorry, I should have explained it better.

In PeerTube, we have 1 source that we trust: the origin server (say server O). Other servers (say server Y and server Z) can duplicate the video to help the origin server to serve the video.

But when the web browser downloads segments from server Y and server Z, it has to check the segments validity (for example by asking server O if the segments hash are valid etc).

from p2p-media-loader.

It sounds reasonable. Implemented in 6512e29

from p2p-media-loader.

@greenya 😘

from p2p-media-loader.