AstriCon 2015: WebRTC: How it Works, and How it Breaks

WebRTC How it works How it breaks How we fixed it

CAN YOU SPEAK MAGIC?

2

Ben Klang


2

Ben Klang


2

Ben Klang


3


3

WebRTC


3

WebRTCHow it works


3

WebRTCHow it worksHow it breaks


3

WebRTCHow it worksHow it breaksHow we fixed it


WHAT WEBRTC IS

4


WHAT WEBRTC IS•Javascript access to Camera and Mic

4


WHAT WEBRTC IS•Javascript access to Camera and Mic•High Quality Video & Audio Codecs

4


WHAT WEBRTC IS•Javascript access to Camera and Mic•High Quality Video & Audio Codecs•Techniques for traversing NAT

4


WHAT WEBRTC IS•Javascript access to Camera and Mic•High Quality Video & Audio Codecs•Techniques for traversing NAT•Peer-to-Peer

4


WHAT WEBRTC IS•Javascript access to Camera and Mic•High Quality Video & Audio Codecs•Techniques for traversing NAT•Peer-to-Peer•A Data Channel

4


WHAT WEBRTC IS•Javascript access to Camera and Mic•High Quality Video & Audio Codecs•Techniques for traversing NAT•Peer-to-Peer•A Data Channel•A Tool for Developers

4


WHAT WEBRTC IS NOT

5


WHAT WEBRTC IS NOT•…a Polished End-User Product

5


WHAT WEBRTC IS NOT•…a Polished End-User Product•…required to interop…but may

5


WHAT WEBRTC IS NOT•…a Polished End-User Product•…required to interop…but may•…the same thing to every application

5


WHAT WEBRTC IS NOT•…a Polished End-User Product•…required to interop…but may•…the same thing to every application•…finished.

5


WHAT WEBRTC IS NOT•…a Polished End-User Product•…required to interop…but may•…the same thing to every application•…finished.

5

But don’t sweat it


Communication Topology

6


7

Alice BobThe Trapezoid


8

Alice BobThe Triangle


9

The WebRTC TriangleAlice Bob

Signaling

Media


9

The WebRTC TriangleAlice Bob

Signaling

Media


WebRTC Infrastructure

10


11


11


11


11

http://

Signaling


11

http://

Signaling


11

http://

Signaling


11

http://

Signaling

STUN & TURNNAT Traversal


11

http://

Signaling

STUN & TURN

SFU or MCU

NAT Traversal

Larger Multiparty Video Conferences


11

http://

Signaling

Recording Conferencing PSTN Gateway

STUN & TURN

SFU or MCU

NAT Traversal

Larger Multiparty Video Conferences


WEBRTC SIGNALING

12


WEBRTC SIGNALING•Can be anything you like

12



12

http://



12

http://



12

http://



12

http://


WEBRTC SIGNALING•Can be anything you like•Select based on application requirements

12

http://


WEBRTC SIGNALING•Can be anything you like•Select based on application requirements•Integrate with existing or greenfield?

12

http://


WEBRTC SIGNALING•Can be anything you like•Select based on application requirements•Integrate with existing or greenfield?•Federate? Or Not?

12

http://


WEBRTC SIGNALING•Can be anything you like•Select based on application requirements•Integrate with existing or greenfield?•Federate? Or Not?•Anonymous or Real Names?

12

http://

CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?

13

http://


13

Alice

http://


13

Alice Bob

http://


13

Hav

e Bob

call

me h

ere!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Alice Bob

http://


13

Hav

e Bob

call

me h

ere!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Alice Bob

http://


13

Hav

e Bob

call

me h

ere!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Bob is at:

SDP:

v=0 o=bob 19915 0 IN IP4 0.0.0.0

s=- t=0 0 m=audio 61001 RTP/SAVPF 109

Alice Bob

http://


13

Hav

e Bob

call

me h

ere!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Bob is at:

SDP:

v=0 o=bob 19915 0 IN IP4 0.0.0.0


Alice Bob

http://


13

Hav

e Bob

call

me h

ere!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Bob is at:

SDP:

v=0 o=bob 19915 0 IN IP4 0.0.0.0


Alice Bob

http://


13

Hav

e Bob

call

me h

ere!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Bob is at:

SDP:

v=0 o=bob 19915 0 IN IP4 0.0.0.0


Alice Bob

http://


ICE/STUN/TURN

14


15

ICE/STUN/TURN


16

Hav

e Bob

call

me h

ere!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Bob is at:

SDP:

v=0 o=bob 19915 0 IN IP4 0.0.0.0


Alice Bob

http://


16

Hav

e Bob

call

me h

ere!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Bob is at:

SDP:

v=0 o=bob 19915 0 IN IP4 0.0.0.0


Alice Bob

SRTP

SRTP

http://


16

Hav

e Bob

call

me h

ere!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Bob is at:

SDP:

v=0 o=bob 19915 0 IN IP4 0.0.0.0


Alice Bob

SRTP

SRTP

http://


16

Hav

e Bob

call

me h

ere!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Bob is at:

SDP:

v=0 o=bob 19915 0 IN IP4 0.0.0.0


Alice Bob

SRTP

SRTP

Xhttp://


That’s how it works

17


Here’s how it breaks

18


ENVIRONMENTAL PROBLEMS

19



19

•Microphone or Speakers Muted

https://test.webrtc.org/



19

•Microphone or Speakers Muted




19

•Microphone or Speakers Muted•Too Dark or Too Backlit




19

•Microphone or Speakers Muted•Too Dark or Too Backlit•HW or Driver Issues




19

•Microphone or Speakers Muted•Too Dark or Too Backlit•HW or Driver Issues•USB Headsets




19

•Microphone or Speakers Muted•Too Dark or Too Backlit•HW or Driver Issues•USB Headsets•I’m looking at you, Windows




19

•Microphone or Speakers Muted•Too Dark or Too Backlit•HW or Driver Issues•USB Headsets•I’m looking at you, Windows•https://test.webrtc.org/



USABILITY PROBLEMS

20


USABILITY PROBLEMS•Failing to deploy TLS Cert

20


USABILITY PROBLEMS•Failing to deploy TLS Cert•Not allowing user-chosen device selection

20


USABILITY PROBLEMS•Failing to deploy TLS Cert•Not allowing user-chosen device selection•Video element rendered off-screen

20


USABILITY PROBLEMS•Failing to deploy TLS Cert•Not allowing user-chosen device selection•Video element rendered off-screen•<video> or <audio> paused

20


BROWSER PROBLEMS

21


BROWSER PROBLEMS•Spec still evolving

21

https://webrtcstandards.info/


BROWSER PROBLEMS•Spec still evolving•…though 1.0 is on the horizon

21



BROWSER PROBLEMS•Spec still evolving•…though 1.0 is on the horizon•Still no native IE or Safari :(

21



BROWSER PROBLEMS•Spec still evolving•…though 1.0 is on the horizon•Still no native IE or Safari :(•MUCH more stable than 1 year ago

21



BROWSER PROBLEMS•Spec still evolving•…though 1.0 is on the horizon•Still no native IE or Safari :(•MUCH more stable than 1 year ago•Chrome dropping non-TLS HTTP

21



BROWSER PROBLEMS•Spec still evolving•…though 1.0 is on the horizon•Still no native IE or Safari :(•MUCH more stable than 1 year ago•Chrome dropping non-TLS HTTP

21



BROWSER PROBLEMS•Spec still evolving•…though 1.0 is on the horizon•Still no native IE or Safari :(•MUCH more stable than 1 year ago•Chrome dropping non-TLS HTTP• https://webrtcstandards.info/

21



BROWSER PROBLEMS•Spec still evolving•…though 1.0 is on the horizon•Still no native IE or Safari :(•MUCH more stable than 1 year ago•Chrome dropping non-TLS HTTP• https://webrtcstandards.info/•Well, Everybody BetterRestart Their Chrome

21



SOLVING BROWSER INCOMPAT

22


SOLVING BROWSER INCOMPAT•Use a “polyfill” - Adapter.js

22

http://skylink.io/plugin/


SOLVING BROWSER INCOMPAT•Use a “polyfill” - Adapter.js•https://github.com/webrtc/adapter

22



SOLVING BROWSER INCOMPAT•Use a “polyfill” - Adapter.js•https://github.com/webrtc/adapter•IE & Safari: deploy Temasys

22



SOLVING BROWSER INCOMPAT•Use a “polyfill” - Adapter.js•https://github.com/webrtc/adapter•IE & Safari: deploy Temasys•http://skylink.io/plugin/

22



SOLVING BROWSER INCOMPAT•Use a “polyfill” - Adapter.js•https://github.com/webrtc/adapter•IE & Safari: deploy Temasys•http://skylink.io/plugin/•Pick a great WebRTC service provider

22



SOLVING BROWSER INCOMPAT•Use a “polyfill” - Adapter.js•https://github.com/webrtc/adapter•IE & Safari: deploy Temasys•http://skylink.io/plugin/•Pick a great WebRTC service provider

22



WEBRTC PROBLEMS & SOLUTIONS

23


WEBRTC PROBLEMS & SOLUTIONS•Environmental Problems

23


WEBRTC PROBLEMS & SOLUTIONS•Environmental Problems•Change the environment

23


WEBRTC PROBLEMS & SOLUTIONS•Environmental Problems•Change the environment•Teach the user

23


WEBRTC PROBLEMS & SOLUTIONS•Environmental Problems•Change the environment•Teach the user•Usability Problems

23


WEBRTC PROBLEMS & SOLUTIONS•Environmental Problems•Change the environment•Teach the user•Usability Problems•Deploy TLS Certs!!!

23


WEBRTC PROBLEMS & SOLUTIONS•Environmental Problems•Change the environment•Teach the user•Usability Problems•Deploy TLS Certs!!!•Test, test, test

23


WEBRTC PROBLEMS & SOLUTIONS•Environmental Problems•Change the environment•Teach the user•Usability Problems•Deploy TLS Certs!!!•Test, test, test•Browser Compat & Infrastructure Problems

23


WEBRTC PROBLEMS & SOLUTIONS•Environmental Problems•Change the environment•Teach the user•Usability Problems•Deploy TLS Certs!!!•Test, test, test•Browser Compat & Infrastructure Problems•Lean on someone who has already fixed it

23


24

http://mojolingo.com @MojoLingo

@bklang [email protected]

http://adhearsion.com @Adhearsion

LINKS & QUESTIONS•https://test.webrtc.org/

•https://webrtcstandards.info/

•https://github.com/webrtc/adapter

•http://skylink.io/plugin/

http://mojolingo.com

http://twitter.com/bklang

mailto:[email protected]

http://adhearsion.com


Software

AstriCon 2015: WebRTC: How it Works, and How it Breaks