COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

WebRTC Media in the Cloud

WebRTC Media in the Cloud

Chad HartChad Hart

IIT RTC Conference 2014

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 2

Why do you need a media server?

Multi-party conferencing InterworkingTranscoding

Stream processing Person-to-machineRecording

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 3

What do you care more about?

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 4

Bandwidth is not always ubiquitous or free

Client-side Server-side

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 5

CPU & processing power is expensive

Client view Provider View

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 6Image source: http://thewrestlinglifeupdate.tumblr.com/

Multi-PARTY! Video Conferencing

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 7

Easy & cheap approach to multi-party: Mesh

7

Full MeshWorks for a few

partiesNo server costLowest latency

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 8

Mesh does not scale for many video conferees

8

Full MeshClients get overloaded

Encode costs more than decodeLimited uplink bandwidth

Inconsistent performance across participants

n = number of conferees in a conference

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 9

Traditional Telephony Approach: MCU

9

Multipoint Control Unit (MCU)Good at manipulating all media

TranscodingTransizingTransratingInterworking

Server-side CPU intensiveClient can request different

conference mixesUsually client friendly

Downsizing & mixing can reduce bandwidthFewer client streams to process

MCU

n = number of conferees in a conference

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 10

Video UX: Traditional method

MCU11 22

Mixed stream 11 22

3G Device

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 11

Video UX: Traditional method

MCU11 22

Mixed stream 11 22

3G DeviceActive Talker

Media Server Chooses Layout

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 12

Client appApp ServerApp Server22

Modern UX with MCU mixing, HTML5 & JavaScript

Browser

MCU11 22

HTML5 CanvasJavascript

Mixed stream

Application-defined UX

Dimensions

Layout

11

"Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."

Benefits:Application defined UXUser specific & customizable layoutLower client BW & CPU vs. mesh

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 13

Newer approach: SFU

13

SFU

Selective Forwarding Unit (SFU) routing

Clients send one & receive manyClient can instruct SFU which

streams to sendHigh throughputCan be lots of downlink

bandwidthLow latency

n = number of conferees in a conference

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 14

New WebRTC approach: Simulcast

14

SFU

High bitrate

Low bitrate

Selective Forwarding Unit (SFU) with Simulcast

Clients send multiple streams to SFU

one high-bit rate one or more lower-bit

Client directs SFU which streams to receive

Reduces bandwidth vs. SFUSimulcast in WebRTC coming

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 15

Future Approach with VP9 – SVC?

15

SFU

MediumHigh LowLayered bitrates:

Selective Forwarding Unit (SFU) with SVC

Clients send layered stream to SFU

Varying bitrates – time, size, quality

SFU directs who gets whatComing to WebRTC eventually

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 16

Typical media server network model

Load BalancerLoad BalancerLoad BalancerLoad Balancer

ClientClient ClientClient ClientClient ClientClient ClientClient ClientClient

App ServerApp ServerApp ServerApp Server

App ServerApp Server

Media ServerMedia ServerMedia ServerMedia Server

Media ServerMedia Server

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 17

Pseudo Call-flow with a media server

AS LB MS1 MS2 Client

Call signaling

Media control

MS1 status

MS2 status

Media control

Media

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 18

Pseudo Call-flow – traditional fail-over with RE-INVITE

AS LB MS1 MS2 Client

Call signaling – REINVITE with new addresses

Media control

MS1 status

MS2 status

Media control + call states

Media

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 19

Pseudo Call-flow – fail-over the cloud way

AS LB MS1 MS2 Client

Media control

MS1 status

MS2 status

New container

Media

Memcache & Virtual IP Address

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 20

When you are serious about scale

DeclarativeConcurrencySoft real-timeRobustnessDistributionHot code loadingExternal interfacesPortability

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 21

ICE, STUN, and TURN save the day

Use of Public IP’s mitigates need for TURN serversOpen source TURN servers do reasonably wellRunning the TURN server co-resident with the media server works well when you need it

Image source: webrtcHacks- http://webrtchacks.com/an-intro-to-webrtcs-natfirewall-problem/