54
Scalable Service Oriented Architecture for Audio/Video Conferencing By Ahmet Uyar Wednesday, March 23, 2005

Scalable Service Oriented Architecture for Audio/Video

Embed Size (px)

Citation preview

Page 1: Scalable Service Oriented Architecture for Audio/Video

Scalable Service Oriented Architecture for Audio/Video Conferencing

By

Ahmet Uyar

Wednesday, March 23, 2005

Page 2: Scalable Service Oriented Architecture for Audio/Video

Outline

Research Issues

Criteria for videoconferencing systems

Overview of current videoconferencing systems

Overview of GlobalMMCS architecture

NaradaBrokering overview and additions

Performance tests for audio/video delivery

Service oriented architecture for videoconferencing

Conclusion

Page 3: Scalable Service Oriented Architecture for Audio/Video

Research IssuesWe investigate the question of how to develop scalable and universally accessible videoconferencing systems over Internet.We propose using publish/subscribe event broker systems for the distribution of real-time audio and video streams in videoconferencing sessions and investigate the issues pertaining to scalability, performance, data representation, meeting management and media processing services.Since real-time audio/video delivery requires low latency and high bandwidth, we investigate the performance and the scalability of this software based messaging middleware extensively. We propose service oriented architecture for videoconferencing. We identify the tasks performed in videoconferencing sessions and provide independently scalable components for each task. We identified three main tasks in videoconferencing sessions:

audio/video distributionmedia processing meeting management.

Page 4: Scalable Service Oriented Architecture for Audio/Video

Criteria for Videoconferencing Systems

We identified the following criteria for videoconferencing systems:

Scalability

Security

Traversing through firewalls, proxies and NAT

Supporting heterogeneous clients

Easy to develop, maintain and use

Support for data conferencing

Page 5: Scalable Service Oriented Architecture for Audio/Video

Videoconferencing Standards and Systems

Multicast based systemsAccessGrid

H.323 based systemsPolycom

CUseeMe

VRVS

Page 6: Scalable Service Oriented Architecture for Audio/Video

Multicast Based Systems

AccessGrid is the most commonly used room based videoconferencing system for group communications.

Scales well.

Difficult to provide security services. No authority to manage multicast IP numbers. Vulnerable to denial-of-service attacks.

No support for going through firewalls and proxies.

Low end users can not join meetings. No media processing is provided.

Easy to use and understand.

Third party data conferencing applications can be used.

Page 7: Scalable Service Oriented Architecture for Audio/Video

H.323 Based Systems

There are many companies that provide H.323 based videoconferencing systems. Polycom, FVC, etc.

Does not scale well.

H.235 defines the security mechanisms but most H.323 based systems do not implement it yet.

H.323 based systems are not firewall friendly. It requires almost all ports to be open.

Limited number of heterogeneous clients can be supported.

Not very easy to understand and develop services.

T.120 define data conferencing: whiteboard sharing, file transfer and application sharing.

Page 8: Scalable Service Oriented Architecture for Audio/Video

A B CMC

D E F

AudioVideo

Multicast or unicast bus

A

B

D

EF

MCU

C

AudioVideo

H.323 Centralized

Multipoint Conferencing H.323 Decentralized

Multipoint Conferencing

H.323 MCU cascading architecture

Page 9: Scalable Service Oriented Architecture for Audio/Video

VRVS (Virtual Rooms Videoconferencing System)

Uses software reflectors to distribute audio/video streams.

Not open source. No details available.

Can go through firewalls, NATs and proxies.

Page 10: Scalable Service Oriented Architecture for Audio/Video

GlobalMMCS Overview

Videoconferencing Tasks:Audio/Video Distribution

Media processing

Meeting management

MeetingManagement

Unit

NaradaBrokeringMessaging Network

Media Processors

Media Processing Unit

user

useruser user

user

Page 11: Scalable Service Oriented Architecture for Audio/Video

Evaluation of GlobalMMCS

Scalability: Provides scalability by separating media processing from media delivery.

Security: NB provides all security services. It also takes precautions against denial of service and replay attacks.

Traversing through firewalls, proxies and NAT

Supporting heterogeneous clients: Since we provide a scalable media processing framework and many transport protocols, we can support a diverse set of end points.

Easy to develop, maintain and use

Support for data conferencing

Page 12: Scalable Service Oriented Architecture for Audio/Video

Media Distribution Middleware(NaradaBrokering)

Requirements for Media Delivery High Bandwidth Low latency Tolerate Package Loss

NaradaBrokeringNB organizes brokers in a hierarchical cluster-based architecture.NB supports dynamic broker and link additions/removals.Messages are routed only to those brokers that have at least one subscription NB has a flexible transport mechanism NB is JMS compliant and supports reliable message delivery. NB provides performance monitoring service

Page 13: Scalable Service Oriented Architecture for Audio/Video

SSC-A

SC-1

SC-2

SC-3

e

g

c4 56

b

f

d

aSSC-C

SC-7

SC-8

SC-9

s

u

o

q

t

r

p

SSC-DSC-11

y

z

SC-10w

x

v

SSC-BSC-4

SC-5

SC-6

l

n

i

j

m

k

h

NaradaBrokering broker organization

Page 14: Scalable Service Oriented Architecture for Audio/Video

Incorporating Support for Audio/Video Delivery in NaradaBrokering

Adding support for an unreliable transport protocol, UDP

Implementing a distributed topic number generation mechanism

Designing a Unique ID Generation Mechanism

Designing a new compact event

Adding support for legacy RTP clients

Some improvements in the routing algorithm

Page 15: Scalable Service Oriented Architecture for Audio/Video

Implementing Distributed Topic Number Generation Mechanism

The requirements for topic number generation:spatial independence of a topic generator temporal independence of a topic generator Acceptable size

One topic number generator runs in every broker20 bytes topic generator id guarantees spatial independence

220 = 1,048,576 topic number generators44 bytes timestamp provides temporal independence

244=17592186044416 distinct timestamp values 557 years with one millisecond resolution.

UUID solves a similar problem with 16 bytes This mechanism can also be used to generate unique ids.

20 bits 44 bits

Topic NumberGenerator ID Timestamp

Page 16: Scalable Service Oriented Architecture for Audio/Video

Designing a New Event

In publish/subscribe messaging systems, messages tend to have many headers.A message in JMS API has at least 10 headers. These headers take around 200 bytes when they are serialized to transfer over the network.A ULAW audio package for 20 ms has a size of 172 bytes and entails 64 kbps network bandwidth. Padding an extra 200 bytes of header to each audio package results in the bandwidth requirement of 148 kbps.It is also more costly to serialize/de-serialize more headers. RTPEvent has four headers and 14 bytes long.

Event and Media headers are 1 byte eachTopic Name is 8 bytesSource info is 4 bytes.

Used to route messagesintelligenty in system

Eliminates echo problem

Identifies the mediatype of the Event

EventHeader

MediaHeader

Topic Name Source Info RTP Payload

IdentifiesEvent as

RTPEventRTP Payload

RTP Header(12 bytes)

Audio or Video Data

Page 17: Scalable Service Oriented Architecture for Audio/Video

Supporting Legacy RTP Clients

RTPLinks receive raw RTP packages over UDP or Multicast from legacy systems, wrap them in RTPEvents and propagate them through the broker node. It also receives RTPEvents from the broker node and sent them as raw RTP packages to clients. Each RTPLink starts two sockets: one for RTP and the other for RTCP. Similarly, it subscribes to two topics: one for RTP and the other for RTCP.Some RTP sessions might have more than one media stream, in that case, each stream might be published to a different topic. RTPLinks can either be managed by statically of dynamically.

Page 18: Scalable Service Oriented Architecture for Audio/Video

Performance Tests of NaradaBrokering

The Characteristics of Audio and Video Streams

Quality Assessment of Media Delivery

Performance Tests for One Broker

Performance Tests for Distributed Brokers

Wide-Area Media Delivery Tests

Page 19: Scalable Service Oriented Architecture for Audio/Video

Characteristics of Audio and Video Streams

Audio streams are composed of fixed size packages with regular intervals. We chose 64 kbps ULAW audio stream to be used in the tests:

One audio package is sent every 30ms. Each audio package is 252 bytes.There are 4100 packages in total, during 2 min.

Video codecs also encode frames periodically. However, each frame may have multiple video packages. Full picture update frames have much more packages. We chose H.263 video format, avrg. bandwidth 280kbps, for 2 min:

15 frames are encoded every second. One frame every 66ms.1800 frames and 5610 packages in total. On avrg. 3.1 packages per frame.One full picture update every 60 frames or 4 seconds.

0

4

8

12

16

20

0 200 400 600 800 1000 1200 1400 1600 1800

video frame number

nu

mb

er

of

pa

ck

ag

es

Page 20: Scalable Service Oriented Architecture for Audio/Video

Quality Assessment of Media DeliveryThere are three important factors: latency, jitter and package lossITU recommends that the mouth-to-ear latency of audio should be

Less than 400ms for acceptable qualityLess than 300ms for good qualityLess than 150ms for excellent quality.

The total latency is the combination of:Processing at sender and receiver Transmission latency Routing latency by the broker network

We limit the routing latency to 100ms at most. The packages that take more than 100ms are labeled as late arrivals.We limit the jitter caused by routing to 10msWe limit the loss rate to 1.0%

Page 21: Scalable Service Oriented Architecture for Audio/Video

Performance Tests for One Broker

Single Meeting TestsSingle audio meeting tests

Single video meeting tests

Audio + Video meeting tests

Multiple Meeting TestsMultiple audio meeting tests

Multiple video meeting tests

Multiple Audio + Video meeting tests

Page 22: Scalable Service Oriented Architecture for Audio/Video

Single Meeting TestsOne transmitter and 12 measuring receivers. Other receivers are passive. Tests are conducted in a Linux cluster with 8 identical machines. These machines had Double Intel Xeon 2.4GHz CPUs, 2GB of memory with Linux 2.4.22 kernel. All programs are written in Java. There is gigabit connection among the cluster nodes.

NB Broker

Machine 1

PassiveReceivers

PassiveReceivers

PassiveReceivers

Machine 2

Machine 3

Machine 4

Machine 8

Audio/VideoTransmitters

MeasuringReceivers

Page 23: Scalable Service Oriented Architecture for Audio/Video

Single Audio Meeting Tests INumberOfClients

L(1)(ms)

L(N)(ms)

L(av)(ms)

J(av)(ms)

LA(av)(%)

12 0.5 0.7 0.6 0.18 0

100 0.5 2.3 1.4 0.15 0

200 0.5 4.1 2.3 0.18 0

400 0.5 7.9 4.2 0.21 0

800 0.5 15.5 8 0.18 0

1200 0.5 22.6 11.6 0.22 0

1400 0.5 26.5 13.5 0.26 0

1500 3.3 32.3 17.8 0.44 0.25

1600 2260 2290 2275 1.2 100

0

5

10

15

20

25

30

35

40

0 500 1000 1500 2000

number of participants

Ave

rage

late

ncy

in m

s

first userlast user

Average

Page 24: Scalable Service Oriented Architecture for Audio/Video

Single Audio Meeting Tests IIThe latency of first user is constant and does not depend on the number of users in a meetingEach audio package is independent of others. The routing of each package is completed before the next one arrives. All audio packages in the audio stream takes almost the same amount of time to arrive to a client.The broker saturates when the latency of the last user is more than 30ms. 1500 users can be supported in an audio meeting

Audio Package Latencies for the Last User in Single Audio M eeting w ith 800 Participants

0

5

10

15

20

25

30

35

40

0 500 1000 1500 2000 2500 3000 3500 4000

Package Num ber

Lat

ency

in m

s

Page 25: Scalable Service Oriented Architecture for Audio/Video

Broker saturation in single audio meeting

Latency values for the middle user in single audio meeting with 1600 participants.

0

1000

2000

3000

4000

5000

0 1000 2000 3000 4000

Package Number

Lat

ency

in

ms

Page 26: Scalable Service Oriented Architecture for Audio/Video

Single Video Meeting Tests INumberOfClients

L(1)(ms)

L(N)(ms)

L(av)(ms)

J(av)(ms)

LA(N) (%)

12 1 1.3 1.2 0.44 0

100 3.1 5 4 2 0

200 6.3 10.2 8.3 4.7 0

300 10.2 16.2 13.2 7.8 0

400 13.4 21.2 17.3 10.1 0.75

500 18.2 28.5 23.4 13.2 3.0

600 22.6 34.5 28.6 15.5 5.1

700 29.8 43.7 36.8 18.1 8.4

800 45.6 61.6 53.6 21.3 17.6

900 93.7 111.7 102.7 23.8 40.8

1000 1599 1619 1609 27.8 99

0

20

40

60

80

100

120

0 500 1000 1500

number of participantA

vera

ge

Lat

ency

in

ms

f irst

last

average

Page 27: Scalable Service Oriented Architecture for Audio/Video

Single Video Meeting Tests IILatency values for the last receiver in single video meeting with 400 participants.Peaks correspond to full picture update frames. One broker can support at most 400 participants because of late arriving packages. Although the broker is saturated when there are 1000 participants.The main reason for the late arriving packages are the full picture updates.

0

20

40

60

80

100

120

140

0 1000 2000 3000 4000 5000 6000

Package Number

La

ten

cy

in

ms

Page 28: Scalable Service Oriented Architecture for Audio/Video

Audio and Video Combined Meeting Tests

Each one affects the other. Our initial tests showed that the impact of video meeting on the performance of an audio meeting is significant. Therefore, we gave priority to audio routing at the broker. There are two queues at the broker: audio and non-audio. If an audio package arrives, it is routed first as long as the routing of the currently routed package is over. When there are 600 participants, there is only 5ms difference. Therefore, the impact of the video meeting is not significant on the performance of the audio meeting

0

3

6

9

12

0 200 400 600 800 1000

number of participants

Av

era

ge

la

ten

cy

in

ms

audio + video

audio only

Page 29: Scalable Service Oriented Architecture for Audio/Video

Comparison of single video meetings and audio + video meetings

This test shows that the impact of an audio meeting on the performance of a video meeting is not significant.

In audio and video combined meetings, the broker supports almost the same number of participants as in the case of single video meetings. The main reason for this is the better utilization of broker resources when there are two concurrent meetings.

0

10

20

30

40

50

60

0 200 400 600 800 1000

number of clients

Av

rg.

late

nc

y i

n m

s

audio + video

video only

Page 30: Scalable Service Oriented Architecture for Audio/Video

Multiple Video Meeting Tests

Total users

# of Meetings

L(av) (ms)

J(av)(ms) LA(av)

100 5 2.25 0.68 0

200 10 2.74 0.85 0

300 15 3.17 0.86 0

400 20 4.54 1.1 0

500 25 5.94 1.3 0

600 30 6.8 1.37 0

700 35 10.6 1.52 % 0.7

800 40 81.1 1.8 % 19

900 45 2787 3.3 % 98

0

20

40

60

80

100

120

0 200 400 600 800 1000

Total Number of ReceiversA

vrg

. L

aten

cy i

n m

s

m ulti-m eetings ingle m eeting

Page 31: Scalable Service Oriented Architecture for Audio/Video

Latency values for each video package when there are 30 meetings with 600 participants.

This graph shows that there are no peaks in latency values for full picture update frames as it was the case in the single video meeting case.

0

5

10

15

20

25

30

35

40

0 1000 2000 3000 4000 5000

package number

late

ncy

in

ms

Page 32: Scalable Service Oriented Architecture for Audio/Video

Summary of Single Broker Tests

1500 participants are supported in one audio meeting

400 participants are supported in one video meeting

Up to 400 audio participants and 400 video participants are supported in audio + video meetings.

700 participants can be supported in 35 video meetings each having 20 participants

1300 participants can be supported in 65 audio meetings each having 20 participants

20 audio and 20 video meetings can be supported each having 20 participants.

Page 33: Scalable Service Oriented Architecture for Audio/Video

Performance Tests for Distributed Brokers

We have given priority to inter-broker package delivery over local client deliveries. This lets packages to travel many brokers with very little overhead. It lets the broker network to scale. It also eliminates cases where one overloaded broker severely affects the performance of other brokers.

FirstQueue

InterBrokerrouter

ToBrokers

SecondQueue

Tolocal

clients

Thread 1

Thread 2clientrouter

Page 34: Scalable Service Oriented Architecture for Audio/Video

First User (ms)

MidUser (ms)

Last User (ms)

Avr. (ms)

Broker 1 400 users 15.8 20.2 24.5 20.2

Broker 2(6 users) 16.1 16.1 16.2 16.1

First User (ms)

Mid User (ms)

Last User (ms)

Avr. (ms)

Broker 1400 users 16.1 20.5 24.9 20.5

Broker 2(6 users) 1.4 1.5 1.6 1.5

Test results with single and double queuing

Machine 1

VideoTransmitter

MeasuringReceivers

Broker 1

Machine 2

Broker 2

Machine 3

VideoReceivers

Machine 4

Page 35: Scalable Service Oriented Architecture for Audio/Video

Single Video Meeting Tests for Distributed Brokers

There are equal number of participants in each broker.

We gather results from first and last user from each broker.

Machine 1

Broker 1Broker 2Broker 3Broker 4

VideoTransmitter

VideoReceivers

VideoReceivers

VideoReceivers

VideoReceivers

MeasuringReceivers

Linux Cluster 1Linux Cluster 2

Page 36: Scalable Service Oriented Architecture for Audio/Video

Latencies from 4 brokersBroker1 and Broker2 have very similar latency values. Broker3 and Broker4 have similar and slightly better latency values. Going through multiple brokers does not introduce considerable overhead. Scalability of the system can be increased almost linearly by adding new brokers.

0

30

60

90

120

150

180

0 200 400 600 800 1000

Number of receivers

Av

rg.

La

ten

cy

in

ms

broker1

broker2

broker3

broker4

Page 37: Scalable Service Oriented Architecture for Audio/Video

Multiple Meeting Tests for Distributed Brokers

The same setting as the single video meeting tests. However, all broker were running at cluster 2. The behavior of the broker network is more complex when there are multiple concurrent meetings compared to having a single meeting. Having multiple meetings provide both opportunities and challenges. Conducting multiple concurrent meetings on the broker network can increase both the quality of the service provided and the number of supported users as long as the size of these meetings and the distribution of clients among brokers are managed appropriately. The best broker utilization is achieved when there are multiple streams coming to a broker and each incoming stream is delivered to many receivers. If all brokers are utilized fully in this fashion, multi broker network provides better services to higher number of participants.

Page 38: Scalable Service Oriented Architecture for Audio/Video

Multiple Video Meeting Tests4 brokers can support 48 meetings with 1920 users in total with excellent quality.This number is higher than the single video meeting tests in which four brokers supported up to 1600 users.When we repeated the same test with meeting size 20, 1400 participants can be supported.

Number of Meetings

Totalusers

Broker1(ms)

Broker2(ms)

Broker3(ms)

Broker4(ms)

40 1600 3.34 6.93 8.43 8.37

48 1920 3.93 8.46 14.62 10.59

60 2400 9.04 170.04 89.97 25.83

Number of Meetings

Total users

Broker1(%)

Broker2(%)

Broker3(%)

Broker3(%)

40 1600 0.00 0.00 0.00 0.00

48 1920 0.12 0.29 0.50 0.50

60 2400 0.16 1.30 2.51 2.82

Latency values and loss rates for meeting size 40

Page 39: Scalable Service Oriented Architecture for Audio/Video

Wide-Area Media Delivery TestsWe tested with five distant sites:

Syracuse, NY, Tallahassee, Florida, Cardiff, UKTwo sites at Bloomington, IN

We tested two cases:

single broker at Indiana

one broker at each site

Page 40: Scalable Service Oriented Architecture for Audio/Video

Summary of Wide-Area TestsRunning brokers at distributed locations has many benefits:

Saves bandwidth, and eliminates bandwidth limitations.Transferring smaller number of streams yields better transmission services with smaller latency, jitter and loss rates. Load is distributed to many brokers, more users can be served with better quality services. sender-to-receiver transmission latency can be reduced considerably by running brokers at geographically distant locations.

The networks that we used provided excellent services with very small loss rates, latency and jitter values.The network connections need to be checked for high quality. Cardiff site was not even able to support 10 video streams (3Mbps), way below its full capacity (10Mbps).

Page 41: Scalable Service Oriented Architecture for Audio/Video

Meeting Management Architecture and Services

There are three main components.Meeting Management Unit starts/ends meetings, handles user joins and leaves. Media Processing Unit provides; audio mixing, video mixing and image grabbing. A unified framework is provided to distribute service providers and to manage the interactions among system components.

user

useruser

user

RLM Broker 2RLM Broker 1

RLM Broker N

RTP Link Manager

Meeting Management Unit

MeetingSchedulers

Meeting Managers

Video Session

Audio Session

NaradaBrokering Media andContent Distribution Network

MediaServerManager

Media Processing Unit

Video MixerServers

Image GrabberServers

Audio MixerServers

MediaServers

Page 42: Scalable Service Oriented Architecture for Audio/Video

Messaging Among System Components

Although some messages are sent to a group of destinations, many messages are destined to one target. Therefore, an efficient message exchange mechanism should be designed.

We use reliable JMS messages to provide communications among various components in the system.

This simplifies building a scalable solution, since messages can be delivered to multiple destinations without explicit knowledge of the publisher.

Messaging Semantics

Request/Response messaging

Group messaging

Event based messaging

Page 43: Scalable Service Oriented Architecture for Audio/Video

Topic Naming ConventionsTwo types of topics are needed; group topics and unique component topics All topic names start with a common root, GlobalMMCS.Group topic names are constructed by adding the component name to the root

GlobalMMCS/AudioSessionGlobalMMCS/AudioMixerServer

Unique component topic names are constructed by adding the unique ids:GlobalMMCS/AudioSession/<sessionID> GlobalMMCS/AudioMixerServer/<serverID>

Sometimes a component communicates with many different components; in that case, there is one more layer to distinguish these communication channels

GlobalMMCS/AudioSession/<sessionID>/AudioMixerServer GlobalMMCS/AudioSession/<sessionID>/RtpLinkManager

Page 44: Scalable Service Oriented Architecture for Audio/Video

Service Distribution Framework

A unified framework to distribute many types of service providersAddressing: Each service provider and consumer is identified by a unique topic name.Service Discovery: Dynamic discovery mechanism. Inquiry & ServiceDescription messages.Service Selection: the consumer selects the best service provider.Service Execution: the consumer executes the service by sending an Request message.

Broker Network Service Provider 3

Service Provider 2

Service Provider 1

Service Provider N

Consumer 3

Consumer 2

Consumer 1

Consumer M

•Advantages:

• Fault tolerant

• Scalable

• Location independent

Page 45: Scalable Service Oriented Architecture for Audio/Video

Session Management

Audio and video sessions are managed separately.

AudioSession objects manage audio sessions and VideoSession objects manage video sessions

MeetingManager objects act as factories for session objects. They initialize and end them.

AudioSession and VideoSession objects provide session management services to participants, such as user joins and leaves. While handling these requests, they usually talk to other system components, such as media processing units and RTP link managers.

Page 46: Scalable Service Oriented Architecture for Audio/Video

Broker Network

user A

RLM Broker A

GlobalMMCS/AudioMixerServer/<serverID>

Audio Mixer Server

Audio Session

GlobalMMCS/AudioSession/<sessionID>/AudioMixerServer

RTP Link Manager

GlobalMMCS/AudioSession/Participant/<userID>

GlobalMMCS/AudioSession/<sessionID>/Participant

GlobalMMCS/RtpLinkManager/<brokerID>

GlobalMMCS/AudioSession/<sessionID>/RtpLinkManager

GlobalMMCS/AudioSession/<sessionID>/RtpEventMonitor

JMS message paths for an AudioSession

Page 47: Scalable Service Oriented Architecture for Audio/Video

Audio Mixing & Performance Tests

D R PQ

AudioMixer

Stream 1

D R PQStream 2

S E

Multiple Streams Speaker

D R PQSpeaker 1

D R PQSpeaker N

ETo listeners

S ETo Speaker 1

S ETo Speaker N

S: SubtracterE: Encoder

D: DecoderR: RepacketizerPQ: Package Queue

To MultipleStreamsSpeaker

NumberOfmixers

CPU usage

% Quality

5 12 No loss

10 24 No loss

15 34 No loss

20 46Negl.Loss

6 speakers in each mixer. Two of them were continually talking.One more audio stream constructed with the mixed stream of all speakers. All streams were 64kbps ULAW. The machine: WinXP, 512 MB memory, 2.5 GHz Intel Pentium 4 CPU. This machine can support around 20 audio mixing sessions

Page 48: Scalable Service Oriented Architecture for Audio/Video

JMS message paths for a VideoSession

Broker Network

user 1

RLM Broker A

GlobalMMCS/ImageGrabberServer/<serverID>

Image GrabberServer

Video Session

GlobalMMCS/VideoSession/<sessionID>/ImageGrabberServer

RTP Link Manager

GlobalMMCS/VideoSession/Participant/<userID>

GlobalMMCS/VideoSession/<sessionID>/Participant

GlobalMMCS/RtpLinkManager/<brokerID>

GlobalMMCS/AudioSession/<sessionID>/RtpLinkManager

GlobalMMCS/VideoSession/<sessionID>/RtpEventMonitor

Video MixerServer

GlobalMMCS/VideoSession/<sessionID>/VideoMixerServer

GlobalMMCS/VideoMixerServer/<serverID>

Page 49: Scalable Service Oriented Architecture for Audio/Video

Video Mixing & Performance TestsFour video streams are mixed into one video stream. Incoming video streams were 150 kbps H.261 stream.Mixed video stream was H.263 with 18 fps.Linux machine with 1 GB memory and 1.8GHz Dual Intel Xeon CPU. Only 3 video mixers are served.

VideoMixer

Encoder

Mixedvideo

DecoderStream 1

Resizer

DecoderStream 2

Resizer

DecoderStream 3

Resizer

DecoderStream 4

ResizerBuffer

Buffer

Buffer

Buffer

Number of Video Mixers

CPU usage %

1 20

2 42

3 68

4 94

Page 50: Scalable Service Oriented Architecture for Audio/Video

Mixed video streams in various media players

Page 51: Scalable Service Oriented Architecture for Audio/Video

Image Grabbing & Performance Tests

The purpose of image grabbing is to provide users with a meaningful video stream list in a session.

Generated images can either be published on topics on the broker network or can be saved to files.

An image is saved every 60sec to the disk in JPEG format.

The video stream was an H.261 stream with an av. bw of 150 kbps.

50 image grabbers can be supported on this machine.

The same machine as video mixing test.

EncodedImage

Decoder

VideoStream

Resizer EncoderBuffer LoopingThread

Numberof IG

CPU usage %

10 15

20 35

30 50

40 60

50 70

Page 52: Scalable Service Oriented Architecture for Audio/Video

Media Processing Service Distribution

MediaServers act as factories for service providers. They start/stop/advertise them.

Each MediaServer is independent of others. New ones can be added dynamically .

Service providers can either be started from command line when starting the service container, or they can be started by using the MediaServerManager.

New services are assigned to least loaded media processing units.

NaradaBrokeringBroker Network

JMS Messages

SP: ServiceProvider

MediaServerManager 2

SP 1 SP 2

SP N

MediaServer K

MediaServerManager M

MediaServerManager 1

JMS Messages

SP 1 SP 2

SP N

MediaServer 1

SP 1 SP 2

SP N

MediaServer 2

Page 53: Scalable Service Oriented Architecture for Audio/Video

Conclusion Main Contributions:

Proposing a new architecture for scalable videoconferencing that separates media distribution, media delivery and meeting managementInvestigating the issues related to using publish/subscribe systems for real-time audio/video deliveryAnalyzing the performance and the scalability of NaradaBrokering broker network for the distribution of real-time audio and video streams in videoconferencing sessions. Implementing a meeting management and media processing service distribution mechanism based on publish/subscribe middleware.

Future WorksMedia processing service distribution algorithms may be developed for large scale deploymentsAudio/video stream delivery through firewalls need to be investigatedMore performance tests can be conducted with higher number of brokers

Page 54: Scalable Service Oriented Architecture for Audio/Video

PublicationsAhmet Uyar, Wenjun Wu, Geoffrey Fox. “Service-Oriented Architecture for a Scalable Videoconferencing System”. Submitted to IEEE International Conference on Pervasive Services 2005 (ICPS'05) 11-14 July 2005, Santorini, Greece.A. Uyar, G. Fox. “Investigating the Performance of Audio/Video Service Architecture I: Single Broker”. To be presented at The International Symposium on Collaborative Technologies and Systems. May 2005, Missouri, USA.A. Uyar, G. Fox. “Investigating the Performance of Audio/Video Service Architecture II: Broker Network.” To be presented at The International Symposium on Collaborative Technologies and Systems. May 2005, Missouri, USA.Ahmet Uyar, Shrideep Pallickara, Geoffrey Fox. “Towards an Architecture for Audio/Video Conferencing in Distributed Brokering Systems”. The proceedings of The 2003 International Conference on Communications in Computing, June 23 - 26, Las Vegas, Nevada, USA.