Scalable Service Oriented Architecture for Audio/Video Conferencing
By
Ahmet Uyar
Wednesday, March 23, 2005
Outline
Research Issues
Criteria for videoconferencing systems
Overview of current videoconferencing systems
Overview of GlobalMMCS architecture
NaradaBrokering overview and additions
Performance tests for audio/video delivery
Service oriented architecture for videoconferencing
Conclusion
Research IssuesWe investigate the question of how to develop scalable and universally accessible videoconferencing systems over Internet.We propose using publish/subscribe event broker systems for the distribution of real-time audio and video streams in videoconferencing sessions and investigate the issues pertaining to scalability, performance, data representation, meeting management and media processing services.Since real-time audio/video delivery requires low latency and high bandwidth, we investigate the performance and the scalability of this software based messaging middleware extensively. We propose service oriented architecture for videoconferencing. We identify the tasks performed in videoconferencing sessions and provide independently scalable components for each task. We identified three main tasks in videoconferencing sessions:
audio/video distributionmedia processing meeting management.
Criteria for Videoconferencing Systems
We identified the following criteria for videoconferencing systems:
Scalability
Security
Traversing through firewalls, proxies and NAT
Supporting heterogeneous clients
Easy to develop, maintain and use
Support for data conferencing
Videoconferencing Standards and Systems
Multicast based systemsAccessGrid
H.323 based systemsPolycom
CUseeMe
VRVS
Multicast Based Systems
AccessGrid is the most commonly used room based videoconferencing system for group communications.
Scales well.
Difficult to provide security services. No authority to manage multicast IP numbers. Vulnerable to denial-of-service attacks.
No support for going through firewalls and proxies.
Low end users can not join meetings. No media processing is provided.
Easy to use and understand.
Third party data conferencing applications can be used.
H.323 Based Systems
There are many companies that provide H.323 based videoconferencing systems. Polycom, FVC, etc.
Does not scale well.
H.235 defines the security mechanisms but most H.323 based systems do not implement it yet.
H.323 based systems are not firewall friendly. It requires almost all ports to be open.
Limited number of heterogeneous clients can be supported.
Not very easy to understand and develop services.
T.120 define data conferencing: whiteboard sharing, file transfer and application sharing.
A B CMC
D E F
AudioVideo
Multicast or unicast bus
A
B
D
EF
MCU
C
AudioVideo
H.323 Centralized
Multipoint Conferencing H.323 Decentralized
Multipoint Conferencing
H.323 MCU cascading architecture
VRVS (Virtual Rooms Videoconferencing System)
Uses software reflectors to distribute audio/video streams.
Not open source. No details available.
Can go through firewalls, NATs and proxies.
GlobalMMCS Overview
Videoconferencing Tasks:Audio/Video Distribution
Media processing
Meeting management
MeetingManagement
Unit
NaradaBrokeringMessaging Network
Media Processors
Media Processing Unit
user
useruser user
user
Evaluation of GlobalMMCS
Scalability: Provides scalability by separating media processing from media delivery.
Security: NB provides all security services. It also takes precautions against denial of service and replay attacks.
Traversing through firewalls, proxies and NAT
Supporting heterogeneous clients: Since we provide a scalable media processing framework and many transport protocols, we can support a diverse set of end points.
Easy to develop, maintain and use
Support for data conferencing
Media Distribution Middleware(NaradaBrokering)
Requirements for Media Delivery High Bandwidth Low latency Tolerate Package Loss
NaradaBrokeringNB organizes brokers in a hierarchical cluster-based architecture.NB supports dynamic broker and link additions/removals.Messages are routed only to those brokers that have at least one subscription NB has a flexible transport mechanism NB is JMS compliant and supports reliable message delivery. NB provides performance monitoring service
SSC-A
SC-1
SC-2
SC-3
e
g
c4 56
b
f
d
aSSC-C
SC-7
SC-8
SC-9
s
u
o
q
t
r
p
SSC-DSC-11
y
z
SC-10w
x
v
SSC-BSC-4
SC-5
SC-6
l
n
i
j
m
k
h
NaradaBrokering broker organization
Incorporating Support for Audio/Video Delivery in NaradaBrokering
Adding support for an unreliable transport protocol, UDP
Implementing a distributed topic number generation mechanism
Designing a Unique ID Generation Mechanism
Designing a new compact event
Adding support for legacy RTP clients
Some improvements in the routing algorithm
Implementing Distributed Topic Number Generation Mechanism
The requirements for topic number generation:spatial independence of a topic generator temporal independence of a topic generator Acceptable size
One topic number generator runs in every broker20 bytes topic generator id guarantees spatial independence
220 = 1,048,576 topic number generators44 bytes timestamp provides temporal independence
244=17592186044416 distinct timestamp values 557 years with one millisecond resolution.
UUID solves a similar problem with 16 bytes This mechanism can also be used to generate unique ids.
20 bits 44 bits
Topic NumberGenerator ID Timestamp
Designing a New Event
In publish/subscribe messaging systems, messages tend to have many headers.A message in JMS API has at least 10 headers. These headers take around 200 bytes when they are serialized to transfer over the network.A ULAW audio package for 20 ms has a size of 172 bytes and entails 64 kbps network bandwidth. Padding an extra 200 bytes of header to each audio package results in the bandwidth requirement of 148 kbps.It is also more costly to serialize/de-serialize more headers. RTPEvent has four headers and 14 bytes long.
Event and Media headers are 1 byte eachTopic Name is 8 bytesSource info is 4 bytes.
Used to route messagesintelligenty in system
Eliminates echo problem
Identifies the mediatype of the Event
EventHeader
MediaHeader
Topic Name Source Info RTP Payload
IdentifiesEvent as
RTPEventRTP Payload
RTP Header(12 bytes)
Audio or Video Data
Supporting Legacy RTP Clients
RTPLinks receive raw RTP packages over UDP or Multicast from legacy systems, wrap them in RTPEvents and propagate them through the broker node. It also receives RTPEvents from the broker node and sent them as raw RTP packages to clients. Each RTPLink starts two sockets: one for RTP and the other for RTCP. Similarly, it subscribes to two topics: one for RTP and the other for RTCP.Some RTP sessions might have more than one media stream, in that case, each stream might be published to a different topic. RTPLinks can either be managed by statically of dynamically.
Performance Tests of NaradaBrokering
The Characteristics of Audio and Video Streams
Quality Assessment of Media Delivery
Performance Tests for One Broker
Performance Tests for Distributed Brokers
Wide-Area Media Delivery Tests
Characteristics of Audio and Video Streams
Audio streams are composed of fixed size packages with regular intervals. We chose 64 kbps ULAW audio stream to be used in the tests:
One audio package is sent every 30ms. Each audio package is 252 bytes.There are 4100 packages in total, during 2 min.
Video codecs also encode frames periodically. However, each frame may have multiple video packages. Full picture update frames have much more packages. We chose H.263 video format, avrg. bandwidth 280kbps, for 2 min:
15 frames are encoded every second. One frame every 66ms.1800 frames and 5610 packages in total. On avrg. 3.1 packages per frame.One full picture update every 60 frames or 4 seconds.
0
4
8
12
16
20
0 200 400 600 800 1000 1200 1400 1600 1800
video frame number
nu
mb
er
of
pa
ck
ag
es
Quality Assessment of Media DeliveryThere are three important factors: latency, jitter and package lossITU recommends that the mouth-to-ear latency of audio should be
Less than 400ms for acceptable qualityLess than 300ms for good qualityLess than 150ms for excellent quality.
The total latency is the combination of:Processing at sender and receiver Transmission latency Routing latency by the broker network
We limit the routing latency to 100ms at most. The packages that take more than 100ms are labeled as late arrivals.We limit the jitter caused by routing to 10msWe limit the loss rate to 1.0%
Performance Tests for One Broker
Single Meeting TestsSingle audio meeting tests
Single video meeting tests
Audio + Video meeting tests
Multiple Meeting TestsMultiple audio meeting tests
Multiple video meeting tests
Multiple Audio + Video meeting tests
Single Meeting TestsOne transmitter and 12 measuring receivers. Other receivers are passive. Tests are conducted in a Linux cluster with 8 identical machines. These machines had Double Intel Xeon 2.4GHz CPUs, 2GB of memory with Linux 2.4.22 kernel. All programs are written in Java. There is gigabit connection among the cluster nodes.
NB Broker
Machine 1
PassiveReceivers
PassiveReceivers
PassiveReceivers
Machine 2
Machine 3
Machine 4
Machine 8
Audio/VideoTransmitters
MeasuringReceivers
Single Audio Meeting Tests INumberOfClients
L(1)(ms)
L(N)(ms)
L(av)(ms)
J(av)(ms)
LA(av)(%)
12 0.5 0.7 0.6 0.18 0
100 0.5 2.3 1.4 0.15 0
200 0.5 4.1 2.3 0.18 0
400 0.5 7.9 4.2 0.21 0
800 0.5 15.5 8 0.18 0
1200 0.5 22.6 11.6 0.22 0
1400 0.5 26.5 13.5 0.26 0
1500 3.3 32.3 17.8 0.44 0.25
1600 2260 2290 2275 1.2 100
0
5
10
15
20
25
30
35
40
0 500 1000 1500 2000
number of participants
Ave
rage
late
ncy
in m
s
first userlast user
Average
Single Audio Meeting Tests IIThe latency of first user is constant and does not depend on the number of users in a meetingEach audio package is independent of others. The routing of each package is completed before the next one arrives. All audio packages in the audio stream takes almost the same amount of time to arrive to a client.The broker saturates when the latency of the last user is more than 30ms. 1500 users can be supported in an audio meeting
Audio Package Latencies for the Last User in Single Audio M eeting w ith 800 Participants
0
5
10
15
20
25
30
35
40
0 500 1000 1500 2000 2500 3000 3500 4000
Package Num ber
Lat
ency
in m
s
Broker saturation in single audio meeting
Latency values for the middle user in single audio meeting with 1600 participants.
0
1000
2000
3000
4000
5000
0 1000 2000 3000 4000
Package Number
Lat
ency
in
ms
Single Video Meeting Tests INumberOfClients
L(1)(ms)
L(N)(ms)
L(av)(ms)
J(av)(ms)
LA(N) (%)
12 1 1.3 1.2 0.44 0
100 3.1 5 4 2 0
200 6.3 10.2 8.3 4.7 0
300 10.2 16.2 13.2 7.8 0
400 13.4 21.2 17.3 10.1 0.75
500 18.2 28.5 23.4 13.2 3.0
600 22.6 34.5 28.6 15.5 5.1
700 29.8 43.7 36.8 18.1 8.4
800 45.6 61.6 53.6 21.3 17.6
900 93.7 111.7 102.7 23.8 40.8
1000 1599 1619 1609 27.8 99
0
20
40
60
80
100
120
0 500 1000 1500
number of participantA
vera
ge
Lat
ency
in
ms
f irst
last
average
Single Video Meeting Tests IILatency values for the last receiver in single video meeting with 400 participants.Peaks correspond to full picture update frames. One broker can support at most 400 participants because of late arriving packages. Although the broker is saturated when there are 1000 participants.The main reason for the late arriving packages are the full picture updates.
0
20
40
60
80
100
120
140
0 1000 2000 3000 4000 5000 6000
Package Number
La
ten
cy
in
ms
Audio and Video Combined Meeting Tests
Each one affects the other. Our initial tests showed that the impact of video meeting on the performance of an audio meeting is significant. Therefore, we gave priority to audio routing at the broker. There are two queues at the broker: audio and non-audio. If an audio package arrives, it is routed first as long as the routing of the currently routed package is over. When there are 600 participants, there is only 5ms difference. Therefore, the impact of the video meeting is not significant on the performance of the audio meeting
0
3
6
9
12
0 200 400 600 800 1000
number of participants
Av
era
ge
la
ten
cy
in
ms
audio + video
audio only
Comparison of single video meetings and audio + video meetings
This test shows that the impact of an audio meeting on the performance of a video meeting is not significant.
In audio and video combined meetings, the broker supports almost the same number of participants as in the case of single video meetings. The main reason for this is the better utilization of broker resources when there are two concurrent meetings.
0
10
20
30
40
50
60
0 200 400 600 800 1000
number of clients
Av
rg.
late
nc
y i
n m
s
audio + video
video only
Multiple Video Meeting Tests
Total users
# of Meetings
L(av) (ms)
J(av)(ms) LA(av)
100 5 2.25 0.68 0
200 10 2.74 0.85 0
300 15 3.17 0.86 0
400 20 4.54 1.1 0
500 25 5.94 1.3 0
600 30 6.8 1.37 0
700 35 10.6 1.52 % 0.7
800 40 81.1 1.8 % 19
900 45 2787 3.3 % 98
0
20
40
60
80
100
120
0 200 400 600 800 1000
Total Number of ReceiversA
vrg
. L
aten
cy i
n m
s
m ulti-m eetings ingle m eeting
Latency values for each video package when there are 30 meetings with 600 participants.
This graph shows that there are no peaks in latency values for full picture update frames as it was the case in the single video meeting case.
0
5
10
15
20
25
30
35
40
0 1000 2000 3000 4000 5000
package number
late
ncy
in
ms
Summary of Single Broker Tests
1500 participants are supported in one audio meeting
400 participants are supported in one video meeting
Up to 400 audio participants and 400 video participants are supported in audio + video meetings.
700 participants can be supported in 35 video meetings each having 20 participants
1300 participants can be supported in 65 audio meetings each having 20 participants
20 audio and 20 video meetings can be supported each having 20 participants.
Performance Tests for Distributed Brokers
We have given priority to inter-broker package delivery over local client deliveries. This lets packages to travel many brokers with very little overhead. It lets the broker network to scale. It also eliminates cases where one overloaded broker severely affects the performance of other brokers.
FirstQueue
InterBrokerrouter
ToBrokers
SecondQueue
Tolocal
clients
Thread 1
Thread 2clientrouter
First User (ms)
MidUser (ms)
Last User (ms)
Avr. (ms)
Broker 1 400 users 15.8 20.2 24.5 20.2
Broker 2(6 users) 16.1 16.1 16.2 16.1
First User (ms)
Mid User (ms)
Last User (ms)
Avr. (ms)
Broker 1400 users 16.1 20.5 24.9 20.5
Broker 2(6 users) 1.4 1.5 1.6 1.5
Test results with single and double queuing
Machine 1
VideoTransmitter
MeasuringReceivers
Broker 1
Machine 2
Broker 2
Machine 3
VideoReceivers
Machine 4
Single Video Meeting Tests for Distributed Brokers
There are equal number of participants in each broker.
We gather results from first and last user from each broker.
Machine 1
Broker 1Broker 2Broker 3Broker 4
VideoTransmitter
VideoReceivers
VideoReceivers
VideoReceivers
VideoReceivers
MeasuringReceivers
Linux Cluster 1Linux Cluster 2
Latencies from 4 brokersBroker1 and Broker2 have very similar latency values. Broker3 and Broker4 have similar and slightly better latency values. Going through multiple brokers does not introduce considerable overhead. Scalability of the system can be increased almost linearly by adding new brokers.
0
30
60
90
120
150
180
0 200 400 600 800 1000
Number of receivers
Av
rg.
La
ten
cy
in
ms
broker1
broker2
broker3
broker4
Multiple Meeting Tests for Distributed Brokers
The same setting as the single video meeting tests. However, all broker were running at cluster 2. The behavior of the broker network is more complex when there are multiple concurrent meetings compared to having a single meeting. Having multiple meetings provide both opportunities and challenges. Conducting multiple concurrent meetings on the broker network can increase both the quality of the service provided and the number of supported users as long as the size of these meetings and the distribution of clients among brokers are managed appropriately. The best broker utilization is achieved when there are multiple streams coming to a broker and each incoming stream is delivered to many receivers. If all brokers are utilized fully in this fashion, multi broker network provides better services to higher number of participants.
Multiple Video Meeting Tests4 brokers can support 48 meetings with 1920 users in total with excellent quality.This number is higher than the single video meeting tests in which four brokers supported up to 1600 users.When we repeated the same test with meeting size 20, 1400 participants can be supported.
Number of Meetings
Totalusers
Broker1(ms)
Broker2(ms)
Broker3(ms)
Broker4(ms)
40 1600 3.34 6.93 8.43 8.37
48 1920 3.93 8.46 14.62 10.59
60 2400 9.04 170.04 89.97 25.83
Number of Meetings
Total users
Broker1(%)
Broker2(%)
Broker3(%)
Broker3(%)
40 1600 0.00 0.00 0.00 0.00
48 1920 0.12 0.29 0.50 0.50
60 2400 0.16 1.30 2.51 2.82
Latency values and loss rates for meeting size 40
Wide-Area Media Delivery TestsWe tested with five distant sites:
Syracuse, NY, Tallahassee, Florida, Cardiff, UKTwo sites at Bloomington, IN
We tested two cases:
single broker at Indiana
one broker at each site
Summary of Wide-Area TestsRunning brokers at distributed locations has many benefits:
Saves bandwidth, and eliminates bandwidth limitations.Transferring smaller number of streams yields better transmission services with smaller latency, jitter and loss rates. Load is distributed to many brokers, more users can be served with better quality services. sender-to-receiver transmission latency can be reduced considerably by running brokers at geographically distant locations.
The networks that we used provided excellent services with very small loss rates, latency and jitter values.The network connections need to be checked for high quality. Cardiff site was not even able to support 10 video streams (3Mbps), way below its full capacity (10Mbps).
Meeting Management Architecture and Services
There are three main components.Meeting Management Unit starts/ends meetings, handles user joins and leaves. Media Processing Unit provides; audio mixing, video mixing and image grabbing. A unified framework is provided to distribute service providers and to manage the interactions among system components.
user
useruser
user
RLM Broker 2RLM Broker 1
RLM Broker N
RTP Link Manager
Meeting Management Unit
MeetingSchedulers
Meeting Managers
Video Session
Audio Session
NaradaBrokering Media andContent Distribution Network
MediaServerManager
Media Processing Unit
Video MixerServers
Image GrabberServers
Audio MixerServers
MediaServers
Messaging Among System Components
Although some messages are sent to a group of destinations, many messages are destined to one target. Therefore, an efficient message exchange mechanism should be designed.
We use reliable JMS messages to provide communications among various components in the system.
This simplifies building a scalable solution, since messages can be delivered to multiple destinations without explicit knowledge of the publisher.
Messaging Semantics
Request/Response messaging
Group messaging
Event based messaging
Topic Naming ConventionsTwo types of topics are needed; group topics and unique component topics All topic names start with a common root, GlobalMMCS.Group topic names are constructed by adding the component name to the root
GlobalMMCS/AudioSessionGlobalMMCS/AudioMixerServer
Unique component topic names are constructed by adding the unique ids:GlobalMMCS/AudioSession/<sessionID> GlobalMMCS/AudioMixerServer/<serverID>
Sometimes a component communicates with many different components; in that case, there is one more layer to distinguish these communication channels
GlobalMMCS/AudioSession/<sessionID>/AudioMixerServer GlobalMMCS/AudioSession/<sessionID>/RtpLinkManager
Service Distribution Framework
A unified framework to distribute many types of service providersAddressing: Each service provider and consumer is identified by a unique topic name.Service Discovery: Dynamic discovery mechanism. Inquiry & ServiceDescription messages.Service Selection: the consumer selects the best service provider.Service Execution: the consumer executes the service by sending an Request message.
Broker Network Service Provider 3
Service Provider 2
Service Provider 1
Service Provider N
Consumer 3
Consumer 2
Consumer 1
Consumer M
•Advantages:
• Fault tolerant
• Scalable
• Location independent
Session Management
Audio and video sessions are managed separately.
AudioSession objects manage audio sessions and VideoSession objects manage video sessions
MeetingManager objects act as factories for session objects. They initialize and end them.
AudioSession and VideoSession objects provide session management services to participants, such as user joins and leaves. While handling these requests, they usually talk to other system components, such as media processing units and RTP link managers.
Broker Network
user A
RLM Broker A
GlobalMMCS/AudioMixerServer/<serverID>
Audio Mixer Server
Audio Session
GlobalMMCS/AudioSession/<sessionID>/AudioMixerServer
RTP Link Manager
GlobalMMCS/AudioSession/Participant/<userID>
GlobalMMCS/AudioSession/<sessionID>/Participant
GlobalMMCS/RtpLinkManager/<brokerID>
GlobalMMCS/AudioSession/<sessionID>/RtpLinkManager
GlobalMMCS/AudioSession/<sessionID>/RtpEventMonitor
JMS message paths for an AudioSession
Audio Mixing & Performance Tests
D R PQ
AudioMixer
Stream 1
D R PQStream 2
S E
Multiple Streams Speaker
D R PQSpeaker 1
D R PQSpeaker N
ETo listeners
S ETo Speaker 1
S ETo Speaker N
S: SubtracterE: Encoder
D: DecoderR: RepacketizerPQ: Package Queue
To MultipleStreamsSpeaker
NumberOfmixers
CPU usage
% Quality
5 12 No loss
10 24 No loss
15 34 No loss
20 46Negl.Loss
6 speakers in each mixer. Two of them were continually talking.One more audio stream constructed with the mixed stream of all speakers. All streams were 64kbps ULAW. The machine: WinXP, 512 MB memory, 2.5 GHz Intel Pentium 4 CPU. This machine can support around 20 audio mixing sessions
JMS message paths for a VideoSession
Broker Network
user 1
RLM Broker A
GlobalMMCS/ImageGrabberServer/<serverID>
Image GrabberServer
Video Session
GlobalMMCS/VideoSession/<sessionID>/ImageGrabberServer
RTP Link Manager
GlobalMMCS/VideoSession/Participant/<userID>
GlobalMMCS/VideoSession/<sessionID>/Participant
GlobalMMCS/RtpLinkManager/<brokerID>
GlobalMMCS/AudioSession/<sessionID>/RtpLinkManager
GlobalMMCS/VideoSession/<sessionID>/RtpEventMonitor
Video MixerServer
GlobalMMCS/VideoSession/<sessionID>/VideoMixerServer
GlobalMMCS/VideoMixerServer/<serverID>
Video Mixing & Performance TestsFour video streams are mixed into one video stream. Incoming video streams were 150 kbps H.261 stream.Mixed video stream was H.263 with 18 fps.Linux machine with 1 GB memory and 1.8GHz Dual Intel Xeon CPU. Only 3 video mixers are served.
VideoMixer
Encoder
Mixedvideo
DecoderStream 1
Resizer
DecoderStream 2
Resizer
DecoderStream 3
Resizer
DecoderStream 4
ResizerBuffer
Buffer
Buffer
Buffer
Number of Video Mixers
CPU usage %
1 20
2 42
3 68
4 94
Mixed video streams in various media players
Image Grabbing & Performance Tests
The purpose of image grabbing is to provide users with a meaningful video stream list in a session.
Generated images can either be published on topics on the broker network or can be saved to files.
An image is saved every 60sec to the disk in JPEG format.
The video stream was an H.261 stream with an av. bw of 150 kbps.
50 image grabbers can be supported on this machine.
The same machine as video mixing test.
EncodedImage
Decoder
VideoStream
Resizer EncoderBuffer LoopingThread
Numberof IG
CPU usage %
10 15
20 35
30 50
40 60
50 70
Media Processing Service Distribution
MediaServers act as factories for service providers. They start/stop/advertise them.
Each MediaServer is independent of others. New ones can be added dynamically .
Service providers can either be started from command line when starting the service container, or they can be started by using the MediaServerManager.
New services are assigned to least loaded media processing units.
NaradaBrokeringBroker Network
JMS Messages
SP: ServiceProvider
MediaServerManager 2
SP 1 SP 2
SP N
MediaServer K
MediaServerManager M
MediaServerManager 1
JMS Messages
SP 1 SP 2
SP N
MediaServer 1
SP 1 SP 2
SP N
MediaServer 2
Conclusion Main Contributions:
Proposing a new architecture for scalable videoconferencing that separates media distribution, media delivery and meeting managementInvestigating the issues related to using publish/subscribe systems for real-time audio/video deliveryAnalyzing the performance and the scalability of NaradaBrokering broker network for the distribution of real-time audio and video streams in videoconferencing sessions. Implementing a meeting management and media processing service distribution mechanism based on publish/subscribe middleware.
Future WorksMedia processing service distribution algorithms may be developed for large scale deploymentsAudio/video stream delivery through firewalls need to be investigatedMore performance tests can be conducted with higher number of brokers
PublicationsAhmet Uyar, Wenjun Wu, Geoffrey Fox. “Service-Oriented Architecture for a Scalable Videoconferencing System”. Submitted to IEEE International Conference on Pervasive Services 2005 (ICPS'05) 11-14 July 2005, Santorini, Greece.A. Uyar, G. Fox. “Investigating the Performance of Audio/Video Service Architecture I: Single Broker”. To be presented at The International Symposium on Collaborative Technologies and Systems. May 2005, Missouri, USA.A. Uyar, G. Fox. “Investigating the Performance of Audio/Video Service Architecture II: Broker Network.” To be presented at The International Symposium on Collaborative Technologies and Systems. May 2005, Missouri, USA.Ahmet Uyar, Shrideep Pallickara, Geoffrey Fox. “Towards an Architecture for Audio/Video Conferencing in Distributed Brokering Systems”. The proceedings of The 2003 International Conference on Communications in Computing, June 23 - 26, Las Vegas, Nevada, USA.