MADRID · NOV 27-28 · 2015

Realtime video streaming

MADRID · NOV 27-28 · 2015the opensource way

Iván Belmonte

MADRID · NOV 27-28 · 2015

Github: ibelmonte

SysdivisionTwitter: @sysdivisionWebsite: http://sysdivision.com

Twitter: @ibelmonte

Iván Belmonte

MADRID · NOV 27-28 · 2015

Real time weather reporting system for outdoor sports.2 main elements:- HD cameras in surfing spots (beaches)- Weather stations in surfing spots

GOAL:

Weather forecast typically fails. Spotfav gives you a window to your spot, to check the weather conditions before moving.

MADRID · NOV 27-28 · 2015

What forecast says the day before

MADRID · NOV 27-28 · 2015

What you expect when you take your car to go to the spot

MADRID · NOV 27-28 · 2015

What’s actually going on

MADRID · NOV 27-28 · 2015

IMPORTANT NOTE:

I’m a backend developer/sysadmin

I know nothing about video and audio processing

In this project I am A VICTIM

MADRID · NOV 27-28 · 2015

Video manipulation and transmission is complex

- Plenty of difficult concepts: encoding, muxing, transcoding, encapsulation, bitrates, fps…

- Different media servers, different players, some codecs may not be supported

- Several transmission protocols depending on your needs: RTP, RTSP, RTMP, HLS, HDS, Shoutcast, HTML5…

Added complexity: LIVE streaming

MADRID · NOV 27-28 · 2015

Looking for a standard setup

Most common case: H264 over RTSP

Affiliates: get this if you want to get connected to our network

MADRID · NOV 27-28 · 2015

There are several options

Commercial: Adobe media server, Wowza, Flussonic, Flumotion…

SaaS:Ustream, Justin.tv (defunct)…

Open source: ffmpeg, ffserver, libav, Live555, Red5, crtmpd, vlc, Erlyvideo…

MADRID · NOV 27-28 · 2015

OUR SCENARIO

1 Camera, H264 over RTSP

1. The goal

Embed the camera in a website and also in a smartphone app

2. Nice to have

Not permanently connected (connect on demand)Tolerant to network cuts

3. Science fiction

Be able to filter by referral (avoid other websites to embed our cameras)

MADRID · NOV 27-28 · 2015

STEP 1: WEB player (RTMP)

MADRID · NOV 27-28 · 2015

ROUND 1

A javascript serving a Directx (wtf?) script.

MADRID · NOV 27-28 · 2015

ROUND 1

Cons:

- The camera has a limited pool of connections available. Visitors exceeding this poll get rejected.

Works?

- In some browsers

MADRID · NOV 27-28 · 2015

ROUND 2

A VLC cli process reading the RTSP, transcoding and publishing an RTMP stream in a socket

MADRID · NOV 27-28 · 2015

ROUND 2

Works?

- YES…

MADRID · NOV 27-28 · 2015

ROUND 2

Works?

- YES… for 1 camera.

MADRID · NOV 27-28 · 2015

ROUND 2

Works?

- YES… for 1 camera.

What if more cameras?

- 1 VLC process for each camera.

MADRID · NOV 27-28 · 2015

ROUND 2

Round 2 version with multiple cameras

MADRID · NOV 27-28 · 2015

ROUND 2

Cons:

- A lot of processes in the server eating memory as hungry animals.

- VLC is unable to recover from network cuts.

- Each VLC process opens a socket in a different port, which in my case is a mess as I’m hosting the VLC processes inside a container and then routing the outside in.

- VLC is permanently connected to the camera

MADRID · NOV 27-28 · 2015

AND THEN ONE DAY…

Adobe updates flash player, breaks my streamings

MADRID · NOV 27-28 · 2015

BROKEN STREAMING

Two possible reasons:

1. Something is wrong with my VLC commandvlc-wrapper-vvvh,p://camera.ip.address/cgi-bin/faststream.jpg?stream=full&fps=16&size=640x480&preview"-sout'#transcode{vcodec=FLV1,vb=800,ab=32,fps=16,width=640,height=480,acodec=none}:duplicate{dst=std{access=h,p{mime=video/x-flv},mux=ffmpeg{mux=flv},dst=LINUX.BOX.IP.ADDRESS:8080/stream.flv},dst=display}'

2. Something is wrong with the new version of Flash

MADRID · NOV 27-28 · 2015

BROKEN STREAMING

Two possible reasons:

1. Something is wrong with my VLC commandvlc-wrapper-vvvh,p://camera.ip.address/cgi-bin/faststream.jpg?stream=full&fps=16&size=640x480&preview"-sout'#transcode{vcodec=FLV1,vb=800,ab=32,fps=16,width=640,height=480,acodec=none}:duplicate{dst=std{access=h,p{mime=video/x-flv},mux=ffmpeg{mux=flv},dst=LINUX.BOX.IP.ADDRESS:8080/stream.flv},dst=display}'

2. Something is wrong with the new version of Flash

Of course option 2. I contacted Adobe: - ME: your upgrade has a bug and it broke my streamings- ADOBE: …(silence)

MADRID · NOV 27-28 · 2015

ROUND 3

A super cool media server capable to encapsulate H264 RSTP into RTMP for flash players: Erlyvideo.

MADRID · NOV 27-28 · 2015

ROUND 3

Pros:

- Written in Erlang: amazingly light and fast- Connects to the source on demand

- One process, several cameras- Tolerant to network cuts (due to the previous point)

- Referrer filtering

Cons:

- Documented in russian (partially translated into english). Support forums also 80% russian.

- HLS support (for smartphone apps) only supported in its new commercial version: Flussonic.

MADRID · NOV 27-28 · 2015

STEP 2: Smartphone app (HLS)

MADRID · NOV 27-28 · 2015

Nginx RTMP module

An Nginx module capable to read streams in several formats as input, store them and then serve them encapsulated in other protocols (eg: HLS)

MADRID · NOV 27-28 · 2015

Nginx RTMP module

Works?

- YES, perfectly

Cons:

- Not very well documented

- Not tolerant to network cuts

- Unable to regain lost connectivity

MADRID · NOV 27-28 · 2015

Overall vision

MADRID · NOV 27-28 · 2015

Service schema

A video source. A separate server for each streaming format. A website visitor and an smartphone user.

MADRID · NOV 27-28 · 2015

An automated process

Using linux containers and some glue code everything can play together in an automated way

MADRID · NOV 27-28 · 2015

CONSLUSIONS

MADRID · NOV 27-28 · 2015

What I would do now if starting from scratch

1. Do it the opensource way: there are powerful tools

2. Look for a single solution for both web and mobile (probably libav, ffmpeg or Red5)

3. Spend time looking for a solution capable to serve also HTML5 formats (Mp4, Webm, Ogg)

MADRID · NOV 27-28 · 2015

THANK YOU!