November 27-29, 2012

South San Francisco Conference Center

Open Standards: Codecs, Issues and Challenges

Dan Burnett, Moderator

Director of Standards

Voxeo

November 28, 2012

Session Participants

• Justin Uberti

– Software Engineer (and WebRTC team tech lead)

– Google

• Christian Hoene

– CEO

– Symonics GmBH

November 27-29, 2012

South San Francisco Conference Center

GOOGLE

Justin Uberti

WebRTC Team Tech Lead

Open issues and challenges

• Some significant topics from this year

– Microsoft CU-RTCWEB proposal

– JSEP details

– Handling of multiple media streams

– Multiplexing of multiple media types

– SRTP keying mechanisms

Microsoft CU-RTCWEB proposal

• Microsoft made W3C proposal as an alternative to PeerConnection

– Effectively a “low-level” API

– Not compatible with current WebRTC API

– Not SDP-based

– Provided more direct control over media functionality

– Raised several good points about needed functionality

JSEP Details

• Several parts of PeerConnection behavior were underspecified

– What states are possible, and what are the valid transitions?

– What SDP does createOffer/Answer generate?

– What SDP does setLocal/setRemote accept?

– How do we surface ICE information?

– How can we report better errors?

Handling of multiple media streams

• General lack of agreement on exactly what a "media stream" is: – An m= line?

– A MediaStream?

– A MediaStreamTrack?

– An SSRC?

• And how they should be signaled: – As individual SSRCs in an m= line?

– As individual m= lines?

– Via RTP/RTCP?

• Significant issue when dealing with dozens of media streams

Multiplexing of multiple media types • Currently, each m= line is sent over a separate ICE

"connection"

• Real-world data indicates that multiplexing over a single connection would yield better results

• Several challenges here:

– What does this look like at the RTP level?

– How does this work in SDP?

– How do we do QoS with this approach?

• Relevant to the prior question as well

SRTP keying mechanisms

• Two options:

– SDES-SRTP, somewhat widely deployed, but less secure

– DTLS-SRTP, newer and less deployed, but more robust; also slightly longer call setup

• What requirements should be put on DTLS?

– MUST vs SHOULD?

– Can we easily gateway to SDES?

– Can we address the latency concerns?

Questions?

November 27-29, 2012

South San Francisco Conference Center

SYMONICS GMBH

Christian Hoene

CEO

Talk on Codecs for WebRTC

• Speech and Audio – G.711, AMR-WB,

– AAC-eLD

– Opus, eVC

• Video – VP8, H.264s

– VPnext, H.265

• Selecting MTI Codecs – Audio

– Video

• Patents – License Fees

– Risks

– Wars

Soundcheck and Video-Test

IPR and Codecs

• Codecs are complex algorithms – Compress and decompress multimedia signal

– Time consuming to developed

– Non-trival

• Codecs are well patentable – Technical problem

– Global consensus on patent-eligiblity

• Most codecs are protected by IPRs – Most codec patents require paying license fees

Are IPR royalty-free good?

Pro

• Manufactors of „boxes“ (phones, GWs, software) – Cisco, Apple, Nokia, Ericsson …

• Payment for development work

done – Paid for R&D

• IPR fees add to a non-free

product – Technical products and patents

• Deployment only to paying

customers – Mobile phone model

Contra

• Providers of services and open source software – Google, Mozilla …

• Money/fame is earned by other

means – Paid for ads with donations or

• Free software and patents would change that – Software product and patents

• Global, wide-scale deployment

easy as everbody can use it – Internet

Speech Coding

• Reference (CD Quality)

• Analog Telephone (60s-70s)

• ISDN – G.711 (80s)

• Cellular Phone (90s)

• 2000s: VoIP is not better

• Skype Silk (24kHz, 24kb/s)

Audio Coding

• Reference

• GSM (FR, 8 kbits)

• Opus (stereo, 64 kHz, 64 kbits)

© Symonics GmbH 2012 18

IETF Opus Speech and Audio Codec

• Opus consists of two coding algorithms – Based on CELT and Silk

• Opus supports three different operational modes – Silk only – Hybrid (Silk and CELT) – CELT only

• IETF RFC 6716: Audio Codec call Opus – 6-512kbps – 2.5 to 60ms – Dynamic changeable modes – Support for concealment of time adjustment – by JM. Valin, K. Vos, T. Terriberry – September 2012

• Status – Mandatory for WebRTC – Royality free? (Qualcomm has filed a IPR statement)

© Symonics GmbH 2012 19

IPR Risks = Costs (and angst)

• Codec may covered by unknown IPR

• IPR fees are subject of negotiation

• Patent usage may be forbitten

Substantial risks for codec users

IPR Risk Management

• Cross licensing, patent pools

• Careful codec development

• Ask for IP insurance

• Munich Re, 10 M€ cover, Premium 300k€, Propability 1:40

Comparision Features vs. Costs

AMR-WB

AAC-LD

G.711Opus

EVC (2014)

Cost

s (z

ero

to

hig

h)

Features (few to many)

VP8

• Some specs

H.264

• Some specs

Video Codec for WebRTC

• Status: Unsettled

• Many different proposals, no consensus – draft-alvestrand-rtcweb-vp8-00

• Google In favor of VP8 – reasonable draft – draft-burman-rtcweb-h264-proposal-00

• Ericsson, Nokia, MS, RIM: H.264/AVC Constrained Baseline Profile Level 1.2 – reasonable draft

– draft-dbenham-webrtc-videomti-00

• Cisco, Apple: In favor of H264/AVC – propaganda – draft-marjou-rtcweb-video-codec-00

• France Telecom: In favor of H264/AVC –short

• Interviews: Many different opinions

Comparision

• Todo: Two videos side by side

Technical Comparision

H.264

• Quality vs. Power vs. Bandwidth

• Well deployed (since many years)

• Interop (Media gateways)

• No of implementations

• Hardware support

• Rather equal

Vp8

• Quality vs. Power vs. Bandwidth

• Rather new

• Few implementation

• Hardware support just coming

Development perspectives

H265

• What ever codec is selected now, other codecs will be used in (near) future.

VPnext

Royalties and IPR Risks

H.264

• long tradition – MPEG LA. Cost for both codec

and content

– Up to 100.000 users free

– Internet usages are free

– but Motorola vs. Microsoft case (Seattle)

– 0,20 cent per license with caps

VP8

• : So far it is royalty free – No costs

– But MPEG LA is searching for patents in VP8

• (anti trust case is going on).

Patent War: Important court cases

• Microsoft vs. Motorola

– On H.264

• Google vs. MPEG LA

– Anti Trust in respect to VP8 pool

Reasonable Outcome

• Google and MPEG LA agree on

– Google joins MPEG H264 pool

– MPEG LA opens VP8 pool

– Some H264/AVC profile become royalty free

• H.264 is selected for WebRTC

Alternative Outcome

• No agreement

– Law suits will continue

• Google will provide VP8 insurance

– VP8 will be used in WebRTC

Summary

• For speech and audio codecs: Opus is the best choise

• Opus will dominate the market

– Despite lack of mobile and multi-channel support

• As for video codecs, we have to wait for legal courts to decise

– If all behave reasonable, H.264 will be chosen

• Whatever the decision, in a few year other video codecs will be used…

Questions?

November 27-29, 2012

South San Francisco Conference Center

Thank You

Dan Burnett, Moderator

Voxeo

dburnett at voxeo dot com