This project is started by @ASoTNetworks and @darkdrgn2k to stream videos over IPFS, which overlapped with the need to live stream the Our Networks 2018 conference in Toronto. We will document here the components and processes necessary to run live streams throughout the conference and archive the video assets on the IPFS network, that is suitable for a small conference with an audience size of less than 100 people.
Here is a presentation and demo video about this project, recorded from the IPFS All Hands: November 26, 2018. Slides are available here.
We will record live streams in 720p (3 Mbps) and archive mp4 of each talk in 1080p.
- HD Video camera that supports HDMI live feed (e.g. Panasonic HC V500)
- Tripod for video camera
- Wireless microphones (e.g. presenter, handheld, audience, room tone, etc.)
- Microphone mixer (e.g. Shure M67)
- On-premise laptop or desktop running OBS Studio and other software
- Two different USB HDMI capture cards with HDMI pass-through and 3.5 mm audio input (e.g. one Elgato HD60 and one AVerMedia LGP Lite)
- 1 TB external hard disk drive
- HDMI, XLR, 3.5 mm audio, USB, ethernet, power cables
Microphones
|
(xlr / wireless)
|
v
+------------------+
| Audio Mixer | --(3.5 mm audio)----+
+------------------+ |
v
+------------------+ +---------------+ +---------------------------+
| HD Video Camera | --(hdmi)--> | Elgato HD60 | --(usb)--> | Laptop running OBS Studio |
+------------------+ +---------------+ | ↳ Streams to RTMP server |
| for HTTP & IPFS streams |
+------------------+ +---------------+ | ↳ Records mp4 files |
| Presenter Laptop | --(hdmi)--> | AVerMedia LGP | --(usb)--> | for local archiving |
+------------------+ +---------------+ +---------------------------+
| | |
(hdmi pass-through) (usb) (ethernet)
| | |
v v v
Projector 1 TB HDD Gigabit Internet
The laptop is the control centre. It has two USB capture cards, connected to separate USB buses (e.g. if it has a USB2 and USB3 interface) if possible to avoid bandwidth issues. These will be the video and audio inputs. The capture cards are of two different brands because cards like the Elgato have problems when running two in parallel. At least one card should take a 3.5 mm audio input so we can mix the audio into the stream via the audio mixer.
The laptop runs the following software:
- OBS Studio locally to stream to our servers
- OpenVPN or Yggdrasil to authenticate the streaming device
For Yggdrasil, you should compile at tag v0.3.2
, and when streaming, run it with the
configurations that will be downloaded from the streaming server at a later step.
OBS Studio is used throughout the conference to toggle between the two video feeds (i.e.
the slides and the presenter video). Using the Start Streaming
function in OBS Studio,
the stream is published at 720p to a RTMP server we will set up in the next step. Using
the Start Recording
function in OBS Studio, the operator will also record each talk as
a separate 1080p mp4 file to the external hard disk to be published after the event.
OBS Studio Website Embedded Viewer with
Source Video Player IPFS Client
| ^ ^ ^ ^
(rtmp-publish) | | | |
| (http) (ipfs) (http) (ipfs)
v | | | |
+-------------------+ +---------------------+ +---------------------+
| rtmp-server | | ipfs-server | | ipfs-mirror |
| ↳ nginx-rtmp | | ↳ ipfs with pubsub | | ↳ ipfs with pubsub |
| ↳ openvpn |<--(rtmp-pull)--| ↳ ipfs-http gateway |<--(ipfs)--| ↳ ipfs-http gateway |
| ↳ yggdrasil | | ↳ ffmpeg | |- - - - - - - - - - -|
|- - - - - - - - - -| |- - - - - - - - - - -| | Pins IPFS hashes |
| Runs RTMP server | | Encodes HLS ts+m3u8 | | learnt from IPNS id |
| publishable from | | pins on IPFS and | | of ipfs-server |
| authenticated IPs | | publishes to IPNS | +---------------------+
+-------------------+ +---------------------+
|
(rtmp-pull/push)
|
v
Other Streaming Services
The on-premise laptop running OBS Studio pushes to the rtmp-server
, which through
IP-pinning of the OpenVPN or Yggdrasil-generated IP address will allow only that device to
publish. The ipfs-server
pulls that RTMP stream, encodes ts chunks in a live m3u8 file using
ffmpeg, then IPFS adds and pins those files and uses IPNS to publish the m3u8 to its node
ID. The built-in ipfs-http gateway allow those content to be accessed via HTTP, which is
what the embedded player on the website will use. However, viewers running a IPFS client
(with pubsub enabled) can directly view the streams over IPFS. Optionally, we can run
one or more ipfs-mirror
servers that pin the live streaming content and run additional
gateways.
All the servers described above are provisioned using Terraform on Digital Ocean. In addition, the RTMP stream can be consumed by other services to provide a parallel stream that does not involve IPFS.
We will be using the following tools and services:
- Digital Ocean as the virtual machine provider
- Terraform to provision the cloud servers
- Let's Encrypt to get SSL certificates for HTTPS
The following steps assume you have a Digital Ocean account and the above listed software installed on your local machine, which can be the same device running OBS Studio.
-
Clone this repository and work from the
terraform
directory:git clone https://github.com/tomeshnet/ipfs-live-streaming.git cd ipfs-live-streaming/terraform
-
From your domain name registrar, point name servers to Digital Ocean's name servers:
ns1.digitalocean.com ns2.digitalocean.com ns3.digitalocean.com
Then store the domain name in your local environment:
echo -n YOUR_DOMAIN_NAME > .keys/domain_name
-
Set an email address to use as contact email for Let's Encrypt:
echo -n YOUR_EMAIL_ADDRESS > .keys/email_address
DEVELOPER TIP Set dryrun to use Let's Encrypt staging servers for testing
echo -n true > .keys/dryrun
-
Obtain a read-write access token from your Digital Ocean account's
API
tab, then store it in your local environment:echo -n YOUR_DIGITAL_OCEAN_ACCESS_TOKEN > .keys/do_token
-
Generate RSA keys to access your Digital Ocean VMs:
ssh-keygen -t rsa -f .keys/id_rsa
Add the SSH key to your Digital Ocean account under
Settings > Security
, then copy the SSH fingerprint to your local environment:echo -n YOUR_SSH_FINGERPRINT > .keys/ssh_fingerprint
-
Download Terraform, add it to your path. On Linux it would look something like this:
https://releases.hashicorp.com/terraform/0.11.7/terraform_0.11.7_linux_amd64.zip unzip terraform_0.11.7_linux_amd64.zip mv terraform /usr/bin
Then run initialization from our
terraform
working directory:terraform init
-
Provision the streaming servers by running:
terraform apply
By default, this will create one instance of each server type. You may choose to create multiple instances of
ipfs-mirror
by overriding themirror
variable.You may also choose to include an external HTTP (non-IPFS) stream source by adding one or more URLs to a m3u8 playlist.
For example:
terraform apply \ -var "mirror=2" \ -var "m3u8_http_urls=\'https://HLS_SOURCE_0/live.m3u8\',\'https://HLS_SOURCE_1/live.m3u8\'"
From your browser, login to your Digital Ocean dashboard and find your new VMs tagged with
ipfs-live-streaming
. -
You will find a couple new files in your
.keys
folder:client.conf (for OpenVPN on Linux) client.ovpn (for OpenVPN on MacOS and Windows) yggdrasil.conf (for Yggdrasil)
To authenticate using OpenVPN, connect with your OpenVPN client using
client.conf
orclient.ovpn
, then publish your OBS Studio stream to:rtmp://10.10.10.1:1935/live/stream1
To authenticate using Yggdrasil, start it with
yggdrasil.conf
and note the last line of output like this:sudo yggdrasil --useconf < ./keys/yggdrasil.conf ... 2018/12/14 15:16:22 Connected: 203:4bb0:9ff1:2312:e7f3:b8c4:852:[email protected] source 192.168.1.173
Then publish your OBS Studio stream to the IPv6:
rtmp://[203:4bb0:9ff1:2312:e7f3:b8c4:852:a8b1]:1935/live/stream1
-
When your streaming session is done, you can stop OpenVPN or Yggdrasil and destroy the servers with:
terraform destroy
The following streams become publicly available about 30 seconds after you start publishing to the RTMP server:
Stream | URL |
---|---|
RTMP stream | rtmp://rtmp-server.YOUR_DOMAIN_NAME/live |
HLS stream (origin) | https://YOUR_DOMAIN_NAME/live.m3u8 |
HLS stream (mirror-N) | https://ipfs-mirror-N.YOUR_DOMAIN_NAME/live.m3u8 |
IPNS HLS stream (origin) | https://ipfs-gateway.YOUR_DOMAIN_NAME/ipns/IPNS_ID (disabled) |
IPNS HLS stream (mirror-N) | https://ipfs-gateway-N.YOUR_DOMAIN_NAME/ipns/IPNS_ID (disabled) |
The origin ipfs-server
and each ipfs-mirror
also host an embedded video player, publicly available at:
Site | URL |
---|---|
Video player (origin) | https://YOUR_DOMAIN_NAME |
Video player (mirror-N) | https://ipfs-mirror-N.YOUR_DOMAIN_NAME |
The embedded video player is tested on common desktop and mobile browsers, and support the following optional URL query parameters:
Parameter | Description |
---|---|
gw |
Set IPFS gateway URL to override playback gateway |
live |
Set m3u8 file URL to override IPFS live stream |
vod |
Set IPFS content hash of mp4 file to play IPFS on-demand video stream |
The video player uses code from Video.js, graphics from ipfs/artwork, and loading animation from jxnblk/loading.