An Overlay Network Providing Application-Aware Multimedia

Services

Maarten WijnantsBart Cornelissen

Wim LamotteBart De Vleeschauwer

10/10/2006 AAA-IDEA2006 2

Problem Statement

• Increased use of real-time streaming of multimedia content in networked apps– Efficient user communication– In the future: also more “enhanced” uses

• Consequence: growing demand for services that can be applied on multimedia streams– Continuously diversifying market of end-user

computation devices– Sometimes also requested by applications

10/10/2006 AAA-IDEA2006 3

Problem Statement

• What is the optimal location to provide/ implement these multimedia services?– At sender-side?– At receiver-side?– Inside the transportation network, in the

delivery path between sender and receiver

• Benefits of in-network service provision– Senders need to transmit only 1 version/

quality of a multimedia stream– Destinations receive (near-)optimal stream

10/10/2006 AAA-IDEA2006 4

Our Overlay Network

• Our overlay network consists of interconnected proxy servers

• Goals– Concurrently provide powerful services for a

wide variety of networked applications– Extensibility

• This is achieved by combining– A generic base layer – A plug-in mechanism which allows deployment

of application-aware services

10/10/2006 AAA-IDEA2006 5

Outline

• Introduction– Problem Statement– Our Overlay Network

• Software Architecture• Example Overlay Services

– Sound Mixer Service– Face Detection Service– Performance Evaluation

• Conclusions• Future Work

10/10/2006 AAA-IDEA2006 6

Software Architecture

• Proxy servers run on the GNU/Linux OS– Implemented as a userspace process– Uses iptables/netfilter to transfer packets from

kernel- to userspace and vice versa

• Proxy process consists of 2 subsystems– Generic base layer

• Application independent

– Plug-in manager through which additional functionality can be added to the proxy

• Plug-ins are application-aware• Each multimedia service is implemented as plug-in

10/10/2006 AAA-IDEA2006 7

Software Architecture

• Generic base layer– Basic user management functionality– Moves packets between kernel- and userspace

• Plug-in manager– Enables/manages the installation of plug-ins– Plug-ins can register interest for certain

network streams– Intercepted network packets are handed over

to the plug-ins which have registered interest• In the order in which they have registered interest

– Checks the packet verdict returned by plug-ins

10/10/2006 AAA-IDEA2006 8

Software Architecture

10/10/2006 AAA-IDEA2006 9

Sound Mixer Service

• EDM NVE framework– Supports real-time audio (and video)

communication between users– Originally developed for the PC platform

• Sufficient processing power• PC users receive all audio streams sent out by clients

currently located in their Area of Interest (AoI)

– Recently also ported to mobile devices• Processing requirements associated with receiving

and decoding multiple audio streams simultaneously and mixing them together locally is too high

• Decision to limit number of audio streams a mobile client can receive to one

10/10/2006 AAA-IDEA2006 10

Sound Mixer Service

• Substantial difference between audio experience available to desktop and mobile NVE users

• Sound mixer service bridges this gap– Maintains separate mixing unit for every client– Ensures proxy receives necessary input audio streams– Incoming audio packets are decoded and subsequently

transferred to the correct mixing units– Also considers positional information to generate a 3D

spatialized output signal

10/10/2006 AAA-IDEA2006 11

Sound Mixer Service

+

+

=

• Advantages– Improves the audio

experience provided to mobile NVE users

– No additional bandwidth or processing requirements are introduced at client-side

• Drawback– Sound mixer service

introduces a small amount of additional delay

10/10/2006 AAA-IDEA2006 12

Face Detection Service

• iConnect meeting system– Goal = enable both collocated and

geographically dispersed participants to meet in an efficient manner

– Computer-augmented meeting room containing a touch-sensitive digital whiteboard

• Acts as shared workspace (display documents, …)• Each participant is represented as an avatar

– Remotely connecting to iConnect also possible• Shared workspace is displayed on the remote screen• Same collaboration possibilities

– Audio and video conferencing support

10/10/2006 AAA-IDEA2006 13

Face Detection Service

• Video conferencing streams of remote participants need to be displayed on whiteboard– Can consume a lot of screen space!

• Face detection service– Intercepts video conferencing streams– Decodes these streams and applies face

detection– Result is encoded at much lower resolution and

framerate and transmitted to the shared whiteboard as “video” avatar

10/10/2006 AAA-IDEA2006 14

Face Detection Service

• Need for displaying video stream of remote users is eliminated in many cases– Feedback provided by video

avatars suffices to guarantee meeting effectiveness

• Saves screen space on the shared whiteboard– Scarce resource!– Freed up screen space can

be used to display other content

10/10/2006 AAA-IDEA2006 15

Performance Evaluation

Average CPU Utilization Maximum CPU Utilization

Sound Mixer Service 1,5 % 4 %

Face Detection Service 7,5 % 20 %

Both services simultaneously

9,5 % 22 %

• Performance of the 2 overlay services was tested on a single proxy server– Intel Pentium D 3 Ghz– Linux kernel 2.6.14

• Simultaneous use of services introduces nearly no overhead!

10/10/2006 AAA-IDEA2006 16

Conclusions

• I have presented our overlay network consisting of interconnected proxy servers– Generic base layer– Plug-in mechanism enables installation of application-

aware servicesOverlay network can provide powerful and

efficient services for a wide variety of networked applications

Need for deploying and managing a separate overlay network for each distinct networked application is eliminated– Is confirmed by the experimental results produced by

the two presented overlay services

10/10/2006 AAA-IDEA2006 17

Future Work

• Improve the efficiency of the face detection service

• Implementation of an overlay routing plug-in– Would enable proxy servers to route packets

around failing or congested network links– Proxy-to-proxy communication would become

more resilient to network errors• Beneficial for end-user

• Continue integration in iConnect system

Questions?