Media Processing in Video Conferences for Cooperating Over the Top and Operator Based Networks

Syed Safi Ali ShahSupervisor: Jörg OttInstructor: Kaisa Kettunen03-April-2012Thesis work conducted at Oy LM Ericsson Ab, Finland

Agenda

• Background– What is an OTT– What is a network operator

• Research Objective• Grounds for competition between OTTs and network operators• Case Study: Video Conferencing

– Bandwidth and CPU demands for Video Conferences – statistics– How operator based networks handle conference session demands– How OTT applications handle conference session demands– Limitations / Room for improvement

• Proposed Solution– Technical Evaluation and Challenges– Business/Contractual Challenges

• Conclusion and Future Work

Background

• Network Operators:– Organizations which own and administer networks such as Telecom Operators, ISPs

(Internet Service Provider)– Network Operators aim to provide network hosted services to users of their networks, and

tend to charge their customers on the basis of the service being used (Service Aware Charging)

• Over The Top (OTT):– Applications and services running on end devices using the network for transferring data,

but not using any other network hosted intelligence– Such OTT services use the underlying network as a mere bit-pipe.– The underlying network finds it difficult to distinguish which data belongs to what type of

OTT service• Some complex methods are available such as packet inspection but are not preferred for large

volumes of data throughput due to high cost (for example in terms of packet delays while trying to inspect encrypted or tunneled OTT traffic)

Research Objective

• Study and analyze the factors driving competition between OTT service providers & network operators

• Taking video conferencing as a case study:

– Identify the various methods OTT applications employ to work while avoiding the use of network hosted intelligence, and point out any limitations that might exist in such methods

– In the light of current competition, evaluate whether it is still feasible for network operators to invest in network intelligence or should network operators accept their role as bit pipes and leave the service offerings to the OTT industry

Grounds for Competition between OTT service providers and network operators

– Because of OTT applications, operator can not charge for the service but can only charge for data utilization

• Competition from alternate internet access providers is forcing operators to offer flat rate data packages

• Flat rate data plans result in increased OTT traffic (operators term it as “revenue-less traffic”)

• To sustain increased traffic, operators need to invest more in their network capacity (more OPEX and CAPEX for the operator)

• This results is a decoupling of traffic and revenue

– OTT applications have less or no regard of the underlying network topology

• Overlay routes are not calculated based on network layer routing information

• Hard for operators to optimize and plan their networks according to OTT traffic

– OTT services do not use network hosted intelligence• Services/intelligence hosted inside the network remain

under-utilized and operator revenue drops

Case Study: Video Conferencing

• Video conferencing services are being offered by both– Network Operators– OTT service providers

• Video conferencing taken as case study to understand competition between OTTs and network operators

• Video conferencing involves many participants communicating simultaneously• Different Architectures:

Central conference server End system mixing Mesh or multi-unicast multicast

Video Conference: Demands• Video Conferencing poses demands on the system

– Some node(s) has to receive and mix the media streams from all parties, optimize the mixed media stream for best display and deliver the media streams to all parties

• Requires processing power

• Requires Bandwidth

• Media mixing is a complex process. • Mixing = Decoding + other transforms (if needed) + Encoding !!!

Media mixing node in a centralized conference architecture (Media Processor)

Statistics - Processor Demands (Standard Defination Video)

H.264 Video SD @ 25fps

0

1000

2000

3000

4000

5000

6000

3 4 5 6

Number of Participants

CP

U C

ycle

s p

er s

eco

nd

(x1

0^6)

The maximum CPU cycles/sec of smart phones available today

The CPU cycles per sec of a high-end personal computer today

H264 decoding SD video 640×360 @ 25fps = 400 Mega cycles/secondH264 encoding SD video 640×360 @ 25fps = 550 Mega cycles/second

Processor load on node performing media mixing in a conference session:

Statistics - Processor Demands (High Defination Video)

H.264 Video HD @ 25fps

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

3 4 5 6

Number of Participants

CP

U C

ycle

s p

er s

eco

nd

(x1

0^6)

The CPU cycles per second available in a high-end personal computer today

H264 decoding HD video 1280×720 @ 25fps = 900 Mega cycles/secondH264 encoding HD video 1280×720 @ 25fps = 2000 Mega cycles/second

Processor load on node performing the media mixing tasks:

Statistics - Bandwidth Demands

Datarate demands at the Mixer

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

CIF (kbps) SD (kbps) HD (kbps)

Video resolutions @ 25fps

Kb

ps

3 participants

4 participants

5 participants

6 participants

Resolution Bitrate

HD (1280×720) 1.5 Mbps

SD (640×360) 512 Kbps

CIF (352×288) 256 Kbps

Bandwidth demands on node performing media mixing

Average data rates for different qualities of video streams

How operators handle demands

• Operators generally follow the centralized conference server architecture

• Operators deploy specialized nodes for media processing and mixing in their networks.

– Media Gateways– Session Border Controllers– Application Servers

• For example: MRFP (Media Resource Function Processor) in IMS (IP Multimedia Subsystem)

• These nodes are generally:– Reliable– Robust– Always available in the network– Expensive

How OTTs handle demands

• OTT services generally implement one of the following conference architectures:

– End system mixing• Single peer chosen as mixer• Multiple peers share mixer task (co-operative mixing)

– Mesh (multi-unicast)

• OTT based video conferencing applications try to handle media mixing/processing demands on end user devices

– because there is generally no central conference server available in the OTT P2P overlay network

• Such end devices are typically• Low on processing power• Limited energy (mobile devices)• Bandwidth constrained and network fluctuations• Cheap

Limitations in OTT based video conferencing solutions• End devices today might be capable enough (in terms of network

bandwidth, cpu power) to handle the requirements of simple point to point calls

• But in more complex scenarios such as video conferencing, in the absence of any support from the network, end devices struggle to meet the network bandwidth and processor demands.

– This results in unreliable conference sessions with unsatisfactory end user experience

• The exact demands of a video conference session depends upon:• Quality/resolution of video streams.• Number of participants in the session• How the participants are arranged in the network (do they belong to same or different

networks/ISPs)• What type of devices are involved (mobile wireless devices Vs stationary wired

machines)

Proposed Solution – Cooperation between network operators and

OTTs

How can the network help OTT services

• The underlying network can lend a helping hand in many ways to OTT applications– The network can provide various QoS guarantees to data streams

belonging to specific service/application– Media processors can be hosted in the network, to take the media

processing load away from end devices– Frequently accessed content can be cached in the network nodes close

to the users– NAT traversal/proxy servers can be hosted in the network which can be

used by applications running over it– The network operator can open other interfaces with which third party

applications can target their services better to end users, for example the network operator can provide

• User location information• User profiling data

What’s in it for the operator

• Better control of services running over the network– Optimize network according to demands/requirements of

applications/service being offered over it (better traffic management, less congestion on links)

• Bring the lost customer base back– Attract new users to the network by improving the operator’s

service portfolio– Possibility to charge users for service usage rather than plain

data

Technical Requirements

• For network operator– Open external interfaces

• Network operators can open external interfaces from media processing servers hosted inside their networks so external users/applications can access them (publically accessible URIs?)

– Security• Better network security at network edges to guard the external interfaces

– Service Discovery Mechanisms• Make it possible for external users to locate and find services in operator network

(dynamic DNS?)– Signaling Gateways

• Deployed at network edges to translate protocols incase cooperating networks use different signaling protocols for session establishment/teardown etc

• For OTT service provider– Proxy Peers

• Nominate some peers in the overlay network which serve as gateways connecting internal users to external services

Contractual Challenges

• To allow OTT applications/users to use network hosted services, network operator needs to charge & authenticate the OTT users

• User authentication models are different in different domains– Operator based networks: typically strong authentication schemes

• Users of operator networks are generally authenticated with hardware modules e.g SIM cards

– OTT services: typically weak authentication schemes:• Authenticated against email address etc• Not authenticated at all sometimes

Suggested solutions to contractual challenges• For OTTs having centrally administered authentication information

e.g Skype, GoogleTalk, Apple Facetime– Standard AAA interfaces opened on the OTT login servers– Operator nodes can then query OTT login servers to verify if a request

comes from a registered OTT user and grant a temporary identity which can be used for billing the OTT for the service usage

– OTT can in turn charge the user directly or generate the required revenue through advertisements

• For OTTs without central authentication information– Web services can be launched where users can pay online for

operator’s service usage. A key/password can be issued to the user.

• Not very straight forward…

Suggested solutions to contractual challenges

OTT user gets authentication credentials from a web service before she can use operator network hosted services

Conclusion / Future work

• There is need and benefits for both players (OTTs, network operators) to have mutual cooperative contracts– OTTs can use operator’s network as a smart pipe instead of a bit pipe

• There are no major technical challenges that are in the way of such a cooperation

• The business models associated with such cooperative contracts will most probably be based around revenue sharing but the exact specifics are hard to predict– Depend on many market dynamics: competition between operators,

end user behaviors etc

– More research on such market trends and whether they favour cooperation between operators and OTTs can be taken up as a future study

Example call case

MCU Query

MCU Respoce (mcu@ims.org, ...)

INVITE To: mcu@ims.org

(contains recipient list)

INVITE, To: mcu@ims.org (contains recipient list)

Alice Proxy PeerProxy

(CSCF)MCUDNS

DNS Query (ims.org)

DNS Response (Address of CSCF)

407 (Proxy Authentication Required)

INVITE (forwarded)

200 OK

INVITE To: mcu@ims.org

(contains recipient list)

OTT domain(p2psip.org)

Operator Domain(ims.org)

407

INVITE To: mcu@ims.org

(contains recipient list) (with valid credentials)

200 OK200 OK

100 Trying

100 Trying100 Trying

Chooses one

MCU

Starts sending out

invites to other

participants

Media Stream

Alice wants to establish a video conference with her friends.Her friends are distributed in different OTT networks(logged into their different client applications), and someare in the operator network.

1. Alice uses her overlay’s service discovery mechanisms to locate an MCU, which can handle the conference’s signaling and media mixing/processing requirements.

2. Alice dials into the MCU (through the proxy peer in her overlay). The SIP Invite from Alice contains the URIs of all participants she wishes to invite to the conference session. 3. The proxy peer in Alice’s domain resolves the URI of the MCU. The proxy peer forwards the Invite request to the CSCF.

4. The CSCF will act as a proxy, and may choose to authenticate Alice. This can be based upon the SIP usage of HTTP digest authentication mechanism.

5. Once the authentication is complete, the CSCF forwards the Invite to the MCU. On receipt of the Invite message the MCU will start inviting the requested participants to the conference session.

Example call case – cont…

INVITETo: darth@p2psip-b.org

INVITE To: darth@p2psip-b.org

Darth Proxy PeerProxy

(CSCF)MCUDNS

DNS Query (p2psip-b.org)

DNS Response (Address of Proxy Peer)

180 Ringing

INVITETo: darth@p2psip-b.org

OTT domain(p2psip-b.org)

Operator Domain(ims.org)

180 Ringing 180 Ringing

100 Trying

100 Trying

Locates Darth within the overlay

Media Stream

200 OK 200 OK 200 OK

100 Trying

6. To invite a user outside its domain, for example Darth, the MCU will send a query to DNS through its local proxy (P-CSCF) to resolve the domain name of p2psip-b.org. In response, the DNS will provide the public address of the proxy@p2psip-b.org.

7. The MCU will then send the SIP invite request to the proxy peer which will use the P2P client protocol to locate Darth within the overlay network.

8. Once the proxy peer has located Darth, it would forward the SIP Invite to the him. Otherwise, if Darth can not be located or is offline/disconnected, a 404 (Not Found) response will be sent back from the proxy peer to the MCU.

Thanks

Questions/Comments…?