Embed Size (px)
Citation preview
DISTRIBUTED RESOURCE ALLOCATION AND PERFORMANCE
OPTIMIZATION FOR VIDEO COMMUNICATION OVER
MESH NETWORKS BASED ON SWARM INTELLIGENCE
A Dissertation
presented to
the Faculty of the Graduate School
University of Missouri-Columbia
In Partial Fulfillment
Of the Requirements for the Degree
Doctor of Philosophy
by
BO WANG
Dr. Zhihai He, Dissertation Supervisor
DECEMBER 2007
The undersigned, appointed by the Dean of the Graduate School, have examined the
dissertation entitled
DISTRIBUTED RESOURCE ALLOCATION AND PERFORMANCE
OPTIMIZATION FOR VIDEO COMMUNICATION OVER
MESH NETWORKS BASED ON SWARM INTELLIGENCE
presented by Bo Wang,
a candidate for the degree of Doctor of Philosophy,
and hereby certify that in their opinion it is worth acceptance.
Dr. Zhihai He
Dr. Curt H. Davis
Dr. Guilherme DeSouza
Dr. Justin Legarsky
Dr. Wenjun Zeng
To my loving parents Pingxin Wang and Xianju Cheng,
my caring sister Li Wang,
and my precious wife Xiaoning Lu
ACKNOWLEDGMENTS
On completion of my dissertation, I wish to acknowledge the following persons who
helped me during my long path:
First, I would like to express my sincere gratitude to my dissertation advisor, Dr. Zhihai
He. His intellectual guidance, keen insight, motivation and encouragement have been the
great support for me throughout the whole work during my research. I would also like to
express my gratitude to Dr. Curt H. Davis, Dr. Guilherme DeSouza, Dr. Justin Legarsky,
and Dr. Wenjun Zeng, for agreeing to serve as members of my guidance and doctoral
committees, and for their time, consideration and suggestion to help improve the quality of
my dissertation.
I would like to extend my appreciation to my current labmates at Video Processing and
Networking Lab, Xi Chen, York Chung, Xiwen Zhao, and Jay Eggert for their technical
assistance and wonderful friendship. I would also like to thank all my friends at Columbia,
Missouri, who have made my life at University of Missouri-Columbia such an unforgettable
experience.
I am also thankful to the following former and current staff at University of Missouri-
Columbia, for their various forms of support during my graduate study: Jim Fischer, Shirley
Holdmeier, Betty Barfield, and Kelly Scott.
Last, I would like to acknowledge my family for all their love and support. I especially
would like to acknowledge my parents Pingxin Wang, Xianju Cheng and my sister Li Wang
for their love, support and encouragement and understanding in dealing with all the chal-
lenges I have faced in my life. I would specially acknowledge my wife, Xiaoning Lu, for her
ii
support, encouragement, understanding and unwavering love in the past ten years of my
life. Without their love and support, I could not go through so far. Nothing in a simple
paragraph can express the love I have for the four of you.
iii
TABLE OF CONTENTS
ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
CHAPTER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 This Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Major Contributions of the Research . . . . . . . . . . . . . . . . . . . . . . 5
1.3.1 Convex Mapping of High-dimensional Resource Constraints . . . . . . 5
1.3.2 Distributed Optimization over Wireless Sensor Networks . . . . . . . 6
1.3.3 Distributed Rate Allocation for Video Mesh Networks . . . . . . . . . 7
1.3.4 Distributed Resource Allocation for Wireless Video Sensor Networks . 7
1.3.5 Evaluation of PSO Algorithm . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Dissertation Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Background and Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1 Network Resource Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.2 Resource Allocation Over Mesh Networks . . . . . . . . . . . . . . . . 14
2.2 Previous Optimization Techniques . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.1 Optimization Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 16
iv
2.2.2 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3 Particle Swarm Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.1 PSO Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.2 Social Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3 Particle Swarm Optimization over Wireless Sensor Networks . . . . . . . 30
3.1 Convex Mapping of High-dimensional Resource Constraints . . . . . . . . . . 31
3.1.1 WVSN Operation Models . . . . . . . . . . . . . . . . . . . . . . . . 32
3.1.2 WVSN Performance Optimization . . . . . . . . . . . . . . . . . . . . 33
3.1.3 Transform of the Solution Space . . . . . . . . . . . . . . . . . . . . . 34
3.1.4 Performance Optimization Using PSO . . . . . . . . . . . . . . . . . 36
3.1.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.1.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2 Distributed Optimization over Wireless Sensor Networks . . . . . . . . . . . 40
3.2.1 Optimization Problems Using Decentralized PSO . . . . . . . . . . . 40
3.2.2 Source Localization Application . . . . . . . . . . . . . . . . . . . . . 42
3.2.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2.4 Comparison with Gradient Search Algorithms . . . . . . . . . . . . . 45
3.2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4 Distributed Rate Allocation for Video Mesh Networks . . . . . . . . . . . 49
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.1.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.1.2 Major Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.2 Resource Allocation for Video Mesh Networks . . . . . . . . . . . . . . . . . 53
4.2.1 Formulation of Generic Resource Allocation Problems . . . . . . . . . 53
4.2.2 Basic Framework for Distributed Resource Allocation . . . . . . . . . 55
v
4.3 Distributed Rate Allocation for Video Mesh Network . . . . . . . . . . . . . 56
4.3.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.3.2 Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.4 Distributed and Asynchronous Particle Swarm Optimization . . . . . . . . . 60
4.4.1 Original Optimization Problem Decomposition . . . . . . . . . . . . . 61
4.4.2 In-Network Fusion and Particle Migration . . . . . . . . . . . . . . . 62
4.4.3 Handling Network Bottleneck Issue Using Collaborative Resource Con-
trol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.4.4 Algorithm Description of DAPSO . . . . . . . . . . . . . . . . . . . . 66
4.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.5.1 Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.5.2 Convex Distributed Rate Allocation and Performance Optimization
Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.5.3 Nonconvex Distributed Rate Allocation and Performance Optimiza-
tion Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.5.4 Comparison with Gradient-Based Lagrangian Dual Algorithms . . . . 74
4.6 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5 Distributed Resource Allocation for Wireless Video Sensor Networks . 86
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.1.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.1.2 Major Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.2 Energy Efficient Resource Allocation and Performance Optimization . . . . . 90
5.2.1 Channel Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.2.2 Power-Rate-Distortion Module and Transmission Behavior Analysis . 92
5.3 Resource Allocation and Performance Optimization Using Particle Swarm
Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.3.1 Optimization Problem Formulation . . . . . . . . . . . . . . . . . . . 94
vi
5.3.2 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
5.4 Energy Efficient Distributed and Asynchronous Particle Swarm Optimization 97
5.4.1 Decomposition of Original Optimization Problem . . . . . . . . . . . 98
5.4.2 Algorithm Design of EEDAPSO . . . . . . . . . . . . . . . . . . . . . 101
5.4.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.5 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6 Evaluation of PSO Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.1 Convergence Analysis of PSO Algorithm . . . . . . . . . . . . . . . . . . . . 110
6.1.1 Random Search Techniques . . . . . . . . . . . . . . . . . . . . . . . 110
6.1.2 PSO Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . 114
6.2 Optimization Algorithms Evaluation and Comparison . . . . . . . . . . . . . 119
6.2.1 Optimization Algorithms Introduction . . . . . . . . . . . . . . . . . 119
6.2.2 Test Problem Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 122
6.2.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
VITA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
vii
LIST OF TABLES
Table
4.1 Total distortion and communication cost for DAPSO and centralized opti-
mization algorithm under different topologies. . . . . . . . . . . . . . . . . . 84
5.1 Configuration of the channel model parameters. . . . . . . . . . . . . . . . . 96
6.1 PSO algorithm parameters used in the test. . . . . . . . . . . . . . . . . . . 118
6.2 Simulation Results for PSO, GA, and BFGS Algorithms. . . . . . . . . . . . 124
viii
LIST OF FIGURES
Figure
1.1 Illustration of mesh networks. . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1 Video Streaming over Ad hoc networks. . . . . . . . . . . . . . . . . . . . . . 16
2.2 Decomposition of the original optimization problem. . . . . . . . . . . . . . . 18
2.3 Illustration of particle swarm optimization. . . . . . . . . . . . . . . . . . . . 25
2.4 Pseudo code for the basic PSO algorithm. . . . . . . . . . . . . . . . . . . . 26
3.1 Performance optimization with swarm intelligence and convex projection. . . 36
3.2 The performance metric decrease as the particles update their positions of
PSO with convex mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.3 The traces of the all particles moving in the solution space of PSO with
convex mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.4 A WSN topology with 20 sensors and 24 links. . . . . . . . . . . . . . . . . . 44
3.5 The performance function value decrease as the particles update their posi-
tions of decentralized PSO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.6 The traces of the particles moving in the sensor field of decentralized PSO. . 46
3.7 Optimization convergence of WSN with 20 sensors wake up and different link
number in decentralized PSO. . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.8 Comparison of decentralized PSO and subgradient search algorithm on opti-
mization convergence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.1 Illustration of video communication over mesh networks. . . . . . . . . . . . 54
4.2 Illustration of distributed asynchronous optimization. . . . . . . . . . . . . . 57
4.3 Distributed and asynchronous PSO algorithm. . . . . . . . . . . . . . . . . . 62
4.4 An example of bottleneck link in a multi-hop mesh network. . . . . . . . . . 64
ix
4.5 Illustration of particle migration. . . . . . . . . . . . . . . . . . . . . . . . . 67
4.6 A randomly generated video mesh network with 16 nodes, 15 links and 6
video sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.7 Convergence of network utility functions of DAPSO to the global optima of
convex function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.8 The traces of the particles moving in critical link A of convex function. . . . 72
4.9 The traces of the particles moving in critical link B of convex function. . . . 73
4.10 The traces of the particles moving in critical link C of convex function. . . . 74
4.11 Convergence of network utility function of DAPSO to the global optima of
nonconvex function: w1=1 and w2=0.1 . . . . . . . . . . . . . . . . . . . . . 75
4.12 The traces of the particles moving in critical link A of nonconvex function:
w1=1 and w2=0.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.13 The traces of the particles moving in critical link B of nonconvex function:
w1=1 and w2=0.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.14 The traces of the particles moving in critical link C of nonconvex function:
w1=1 and w2=0.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.15 Convergence of network utility function of DAPSO to the global optima of
nonconvex function: w1=1 and w2=1 . . . . . . . . . . . . . . . . . . . . . . 79
4.16 The traces of the particles moving in critical link A of nonconvex function:
w1=1 and w2=1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.17 The traces of the particles moving in critical link B of nonconvex function:
w1=1 and w2=1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.18 The traces of the particles moving in critical link C of nonconvex function:
w1=1 and w2=1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.19 Comparison between DAPSO and distributed gradient search with different
starting price setting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.20 The total source rate beyond link capacity during iteration. . . . . . . . . . . 82
4.21 Response of DAPSO to video content change. . . . . . . . . . . . . . . . . . 83
x
4.22 Different wireless sensor network topologies with different number of sessions. 83
4.23 (a) An example video mesh network; and (b) convergence of network utility
function with DAPSO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.24 (a) An example video mesh network; and (b) convergence of network utility
function with DAPSO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.25 (a) An example video mesh network; and (b) convergence of network utility
function with DAPSO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.1 A WVSN topology with 9 sensor nodes, 8 links and 4 video sessions. . . . . . 98
5.2 The average overall video distortion decrease as the particles update their
positions under different PSO size. . . . . . . . . . . . . . . . . . . . . . . . 99
5.3 The average source rate for each video stream updates during PSO iteration. 100
5.4 The average encoding power for each video stream updates during PSO iter-
ation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.5 The average packet loss probability for each video stream decreases during
PSO iteration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.6 The average video distortion decreases during EEDAPSO search process. . . 103
5.7 The average video distortion decreases during EEDAPSO search process. . . 105
5.8 The source rate for every video stream on critical link A updates during
EEDAPSO search process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.9 The source rate for every video stream on critical link B updates during
EEDAPSO search process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.10 The encoding power for every video stream updates during EEDAPSO search
process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.11 The transmission power on every link updates during EEDAPSO search process.108
5.12 The packet loss probability for each video stream decreases during EEDAPSO
search process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.13 (a) An example WVSN; and (b) convergence of network utility function with
EEDAPSO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
xi
5.14 (a) An example WVSN; and (b) convergence of network utility function with
EEDAPSO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.1 Plot for a nonconvex objective function. . . . . . . . . . . . . . . . . . . . . 117
6.2 Convergence probability under different initial random solution. . . . . . . . 118
6.3 (a) Convergence of function value for problem H1; and (b) convergence of
function value for problem H2. . . . . . . . . . . . . . . . . . . . . . . . . . . 124
6.4 (a) Convergence of function value for problem H3 (n=4); and (b) convergence
of function value for problem H3 (n=8). . . . . . . . . . . . . . . . . . . . . 125
xii
ABSTRACT
Mesh networking technologies allow a system of communication devices to communi-
cate with each other over a dynamic and self-organizing wired or wireless network from
everywhere at anytime. Important examples of mesh networking include wireless sensor
networks, multimedia communication over community networks and Internet, peer-to-peer
video streaming, etc. Large-scale mesh communication networks involve a large number of
heterogeneous devices, each with different on-board computation speeds, energy supplies,
and communication capabilities, communicating over the dynamic and unreliable networks.
How to coordinate the resource utilization behaviors of these devices in a large-scale mesh
network such that each of them operates in a contributive fashion to maximize the overall
performance of the system as a whole remains a challenging task.
Network resource allocation and performance optimization can be formulated as a net-
work utility maximization (NUM) problem under resource constraints. In many mesh net-
working applications, especially in video communication over mesh networks, network utility
maximization is often a high-dimensional, nonlinear, constrained optimization problem. An
effective solution to this type of problems needs to meet the following three requirements:
distributed, asynchronous, and non-convex.
In this work, based on swarm intelligence principles, we develop a set of distributed
and asynchronous schemes for resource allocation and performance optimization for a wide
range of mesh networking-based applications. To successfully apply the swarm intelligence
principle in distributed resource allocation and performance optimization in large-scale mesh
networks, there are three important issues that need to be carefully investigated.
First, existing PSO schemes are not able to efficiently handle constraints, especially
xiii
constraints in a high-dimensional space. However, in large-scale video mesh networks, the
resource constraints are often represented by a convex region embedded in a very high di-
mensional space. To address this issue, we propose to transform the solution space defined
by resource constraints into a convex region in a low-dimensional space. We then merge
the convex condition with the swarm intelligence principle to guide the movement of each
particle to efficiently search for the optimum solution. Second, distributed optimization
requires decomposition of centralized network utility function and resource constraints into
local ones. However, in video mesh networking, the resource utilization behavior of neigh-
boring network nodes are highly coupled and interwound. In this work, we propose various
methods and approaches for decomposition of network utility function and interwound re-
source constraints. Third, one of the key challenges in resource allocation and performance
optimization is to handle critical / bottleneck links which have very limited resource however
are shared by multiple video communication sessions. We observe that the resource alloca-
tion results at these critical links have direct impact on the overall system performance. To
address this issue, we propose various schemes to fuse the resource allocation information
of neighboring optimization modules, propagate and share the resource allocation results at
critical links, use this external information to guide the of movements of particles in each
local optimization module to efficiently search for the optimum solution.
Our extensive experimental results in distributed resource allocation and performance
optimization demonstrate that the proposed schemes work efficiently and robustly. Com-
pared to existing algorithms, including gradient search and Lagrange optimization, the pro-
posed approach had the advantage of faster convergence and the ability to handle generic
network utility functions. Compared to centralized performance optimization schemes, the
proposed approach significantly reduces communication overhead while achieving similar
performance. The distributed algorithms for resource allocation and performance optimiza-
tion provide analytical insights and important guidelines for practical design of large-scale
video mesh networks.
xiv
Chapter 1
Introduction
Mesh networking technologies allow a system of communication devices to communi-
cate with each other over a dynamic and self-organizing wired or wireless network from
everywhere at anytime. Important examples of mesh networking include wireless sensor
networks, multimedia communication over community networks and Internet, peer-to-peer
video streaming, etc. Large-scale mesh communication networks involve a large number of
heterogeneous devices, each with different on-board computation speeds, energy supplies,
and communication capabilities, communicating over the dynamic and unreliable networks.
How to coordinate the resource utilization behaviors of these devices in a large-scale mesh
network such that each of them operates in a contributive fashion to maximize the overall
performance of the system as a whole remains a challenging task.
1.1 Overview
Currently there has been considerable research interest in the mesh networks, as illus-
trated in Fig. 1.1. Mesh network consists of nodes that communicate to each other and are
capable of hopping radio messages to a base station where they are passed to a PC or other
client. Each network node also acts as a router, forwarding data packets to other nodes. All
wireless mesh networking systems share a set of common requirements, include low power
consumption, ease of use, scalability, responsiveness, and range. Because of their ad hoc
nature, mesh networks can respond to conditions much more quickly and reliably than static
1
networks. If a node in a mesh network fails, the other nodes in the mesh will notice the
failure and adjust their routing accordingly without human intervention. The resilience,
flexibility, and decentralized administration make the mesh networks more attractive than
the traditional networking systems.
Figure 1.1: Illustration of mesh networks.
Network resource allocation and performance optimization can be formulated as a net-
work utility maximization (NUM) problem under resource constraints. In many mesh net-
working applications, especially in video communication over mesh networks, network utility
maximization is often a high-dimensional, nonlinear, constrained optimization problem. An
effective solution to this type of problems needs to meet the following three requirements:
(1) Distributed. Lack of a centralized powerful node for computation, mesh networks are
often not able to support centralized computation. In addition, centralized performance
optimization introduces significant communication overhead, becomes extremely costly or
even infeasible in large-scale mesh networks. The decomposability structure of network util-
ity maximization leads to the most appropriate distributed algorithm for a given network
resource allocation problem. Decomposition theory naturally provides the mathematical
method to build the foundation for the distributed control of networks. This helps us ob-
tain the most appropriate distributed algorithm for a given network resource allocation
problem, such as distributed routing, scheduling to power control and congestion control.
(2)Asynchronous. Network nodes are communicating with each other in an asynchronous
and on-demand manner. This requires the distributed performance optimization to also
2
operate in an asynchronous fashion. (3)Non-convex. As the network utility functions in
many video mesh networking applications are highly nonlinear and non-convex, an effective
solution should be able to handle generic non-convex objective functions.
Based on Kelly’s work [1] in 1998, the framework of NUM has found many applica-
tions in network resource allocation algorithms and network congestion control algorithms
[2, 3, 4, 5], such as Internat rate allocation and TCP congestion control. These work inter-
pret source rates as primal variables and link congestion prices as dual variables to solve an
implicit global utility optimization problem or lagrange dual problem. Previous research use
price-based strategy [1, 3, 6, 7, 8], where prices are computed to reflect relations between re-
source demands and supplies, to coordinate the resource allocations at multiple hops. Their
results show that the price is effective as a method to arbitrate resource allocation. Most
of the papers in the vast related on network resource allocation use a standard lagrange
dual based distributed algorithm [1, 3]. While it is well known that dual based distributed
algorithm needs utility function to be convex or concave. This is the major drawback of
dual base distributed algorithm when the application is inelastic or utility functions are
nonconvex, which leads to divergence of congestion control. Several existing distributed
optimization algorithms based on incremental sub-gradient search [9, 10] assume that the
objective function to be additive and convex. And the drawbacks of the gradient or subgra-
dient based distributed algorithm are that it is sensitive to local optima or saddle points,
and it also suffers from slow convergence speed.
1.2 This Work
In this work, based on swarm intelligence principles, we develop a set of distributed
and asynchronous schemes for resource allocation and performance optimization for a wide
range of mesh networking-based applications.
Particle swarm optimization (PSO), developed by Eberhart and Kennedy in 1995[11,
12], is a population based stochastic optimization technique which is inspired by social
3
behavior of bird flocking or fish schooling. PSO shares many similarities with evolutionary
computation techniques, such as Genetic Algorithms (GAs). The main advantage of PSO
over other global optimization strategies is that the large number of random solutions of
PSO algorithm make the technique avoid dropping in the local minimum of the optimization
problem, so that the PSO algorithm does not need to have requirement for the objective
function of the optimization problem. In the past several years, PSO algorithm has been
successfully applied in many research and application areas. It is demonstrated that PSO
gets better results in a faster, cheaper way compared with many other methods.
To successfully apply the swarm intelligence principle in distributed resource allocation
and performance optimization in large-scale mesh networks, there are three important issues
that need to be carefully investigated.
First, existing PSO schemes are not able to efficiently handle constraints. To address
this issue, we propose to transform the solution space defined by resource constraints into
a convex region in a low-dimensional space. We then merge the convex condition with the
swarm intelligence principle to guide the movement of each particle to efficiently search
for the optimum solution. Second, distributed optimization requires decomposition of
centralized network utility function and resource constraints into local ones. However, in
video mesh networking, the resource utilization behavior of neighboring network nodes are
highly coupled and interwound. In this work, we propose various methods and approaches
for decomposition of network utility function and interwound resource constraints. Third,
one of the key challenges in resource allocation and performance optimization is to handle
critical / bottleneck links which have very limited resource however are shared by multiple
video communication sessions. To address this issue, we propose various schemes to fuse the
resource allocation information of neighboring optimization modules, propagate and share
the resource allocation results at critical links, use this external information to guide the
of movements of particles in each local optimization module to efficiently search for the
optimum solution.
Our extensive experimental results in distribution resource allocation and performance
4
optimization demonstrate that the proposed schemes work efficiently and robustly. Com-
pared to existing algorithms, including gradient search and Lagrange optimization, the pro-
posed approach had the advantage of faster convergence and the ability to handle generic
network utility functions. Compared to centralized performance optimization schemes, the
proposed approach significantly reduces communication overhead while achieving similar
performance. The distributed algorithms for resource allocation and performance optimiza-
tion provide analytical insights and important guidelines for practical design of large-scale
video mesh networks.
1.3 Major Contributions of the Research
This section presents several evolutionary schemes based on swarm intelligence which
solve the different performance optimization problems over wireless sensor networks, includ-
ing the nonlinear nonconvex optimization problems which can not be solved by the lagrange
dual algorithm, meanwhile the proposed algorithms have fast convergence speed.
1.3.1 Convex Mapping of High-dimensional Resource Constraints
There is a significant body of research work on performance optimization of wireless
sensor networks, such as energy minimization, rate allocation and topology control [13, 14].
These results and algorithms are generic in their nature, and could be used to improve the
performance of WVSNs. However, they have not considered the unique characteristics of
WVSN, such as the complex nonlinear resource utilization behavior of each sensor node
function, which will potentially render these analysis and algorithms inefficient or even
impractical.
In this dissertation, we first consider the unique characteristics of WVSN and develop an
evolutionary optimization scheme using a swarm intelligence principle to solve the WVSN
performance optimization problem. We transform the solution space set by the flow balance
and energy constraints to a convex region in a low-dimensional space. Our analysis shows
5
that this transform can reduce the computational complexity and remove the interdepen-
dence between the control variables. We then merge the convex property of the new solution
space with the original swarm intelligence principle to guide the movement of each particle
which automatically satisfies the constraints during the evolutionary optimization process.
Finally our experimental results demonstrate that the proposed performance optimization
scheme is very efficient.
1.3.2 Distributed Optimization over Wireless Sensor Networks
Recently, several distributed optimization algorithms based on gradient search have been
proposed in the literature [9, 15, 10]. Most of existing approaches assume that the objective
function to be additive and convex. Otherwise, it will be very difficult to assure convergence
of the distributed gradient search algorithm, and will be sensitive to the local optima or
saddle points. In addition, existing algorithms also suffer from slow convergence speed
problem.
In this dissertation, we develop an evolutionary distributed optimization scheme using
swarm intelligence principle [16], called decentralized particle swarm optimization (DPSO),
to solve the distributed WSN optimization problem. Based on the particle swarm intelli-
gence principle, sensor nodes share information with each other through local information
exchange and communication to solve a joint estimation or optimization problem. The
proposed DPSO scheme has low communication energy cost and assures fast convergence.
In addition, the objective function does not need to be convex. We use source location
as an example to demonstrate the efficiency and evaluate the performance of the proposed
DPSO algorithm. The proposed DPSO algorithm is a distributed algorithm with the most
advantage that it will not be sensitive to the local optima or saddle points, and has very fast
convergence speed compared with distributed gradient search algorithm. Our experimental
results demonstrate that the proposed optimization scheme is very efficient and outperforms
the existing distributed gradient-based optimization schemes.
6
1.3.3 Distributed Rate Allocation for Video Mesh Networks
To successfully deploy the video mesh networking technology, there are a number of issues
that need to be carefully investigated, including packet routing, flow control, Quality of
Service guarantee, resource allocation, and performance optimization. Within the context of
large-scale mesh networks, a distributed and asynchronous solution to the resource allocation
and performance optimization is highly desired.
This dissertation presents a distributed and asynchronous particle swarm optimization
(DAPSO) evolutionary technique to optimize the network utility maximization problems.
Unlike many network rate allocation and performance optimization algorithms which can
only handle convex network utility functions, the proposed scheme is able to handle generic
nonlinear noncovex network utility functions, and has very fast convergence speed compared
with gradient based lagrange dual algorithm. For the DAPSO algorithm, we need to take two
steps: (1) decompose the global resource parameters and network utility function; and (2)
handle inter-dependent resource constraints, such as bottle neck links. We will use a specific
rate allocation and performance optimization problem in wireless video sensor network as an
example to demonstrate the efficiency and performance of the proposed scheme and compare
its performance with other algorithms, such as gradient based lagrange dual algorithm. Our
simulation results demonstrate that the proposed performance optimization scheme is very
efficient and can significantly enhance the network utility.
1.3.4 Distributed Resource Allocation for Wireless Video Sensor
Networks
In the WVSNs, the two major operations on each video sensor are video compression and
wireless transmission. And the wireless transmission is also restricted to the transmission
bandwidth. In this dissertation, we will analyze the power-rate-distortion (P-R-D) module
of the video encoding and its distortion performance, and will also analyze the transmis-
sion power consumption for the wireless video communication and its impact for the video
7
quality. Finally, based on these modules, we will approach a distributed and asynchronous
algorithm for the energy efficient resource allocation and performance optimization over
wireless video sensor networks.
In this dissertation, we present an energy efficient distributed and asynchronous PSO
(EEDAPSO) algorithm to solve the resource allocation and performance optimization prob-
lem over wireless video sensor network. We first decompose the original optimization prob-
lem into several sub-optimization problems, and next design the proper algorithm to handle
the interdependent constraints and do the performance optimization. Compared with the
centralized algorithm, our simulation results demonstrate that this proposed distributed
and asynchronous resource allocation and performance optimization scheme is very efficient
and it only needs very low communication cost.
1.3.5 Evaluation of PSO Algorithm
Random Search Techniques are convergent algorithms for constrained nonlinear prob-
lems. Based on this, we provide the mathematical convergence analysis for PSO algorithm.
We proved that PSO algorithm is a local convergence algorithm, which means that after
predefined number of function iteration, all the solutions will converge to an optimum so-
lution which is no guaranteed a global optimum. However, the experimental results based
on many difficult optimization problems show that the large number of random solutions of
the PSO algorithm that make up this technique converges to its global optimum in a good
opportunity. Meanwhile, we compare PSO algorithm with genetic algorithm (GA), and
Broydon-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton algorithm for their global search
capabilities based on a suite of difficult analytical optimization problems, the experimental
results show that the PSO algorithm has the better convergence probability to the global
optimum.
8
1.4 Dissertation Organization
Following the Introduction, the dissertation is organized as follows.
Chapter 2 first presents a background introduction and related works for network
resource allocation optimization problems. Next introduces several previous optimization
algorithms mostly used in the resource allocation optimization problems, including convex
optimization, lagrange duality, gradient or subgradient methods. Meanwhile a brief dis-
cussion about these algorithms is given. An algorithm based on swarm intelligence, called
particle swarm optimization (PSO), is also introduced. For PSO algorithm which has many
attractive features including ease of implementation and the fact that no gradient informa-
tion requirement, its social behavior and applications are also discussed.
Chapter 3 first presents an evolutionary optimization scheme using swarm intelligence
principle to solve the WVSN performance optimization problem. The algorithm transforms
the original high-dimensional constrained optimization problem to a problem in the low
dimension without constraints which can be solved by PSO algorithm. Next proposes a
distributed algorithm, called decentralized PSO (DPSO), to solve the generic parameter
estimation or performance optimization problems over WSNs. For the proposed DPSO
scheme, there is no requirement for the objective function, and the scheme will not be sensi-
tive to the local optima or saddle points. The source localization application demonstrates
the efficiency of the DPSO algorithm.
Chapter 4 presents a distributed and asynchronous particle swarm optimization (DAPSO)
evolutionary technique to optimize the network utility maximization problems. The pro-
posed DAPSO algorithm can solve the rate allocation and performance optimization prob-
lem for video communication over mesh networks very efficiently. The DAPSO algorithm
is easy and powerful, has fast convergence speed and can also solve the generic nonconvex
optimization problems efficiently which can’t be solved by the traditional lagrange dual
algorithm.
Chapter 5 presents an energy efficient distributed asynchronous PSO (EEDAPSO)
9
algorithm to solve the resource allocation and performance optimization problem over wire-
less video sensor networks, including rate allocation and power management. The proposed
EEDAPSO algorithm considers both encoding distortion and transmission distortion for
the whole video quality over the WVSN, and solves the original optimization problem in a
distributed and asynchronous way.
Chapter 6 presents the convergence analysis for PSO algorithm based on random search
techniques. PSO algorithm has been proved that it is a local convergence algorithm, but the
large number of random solutions of the PSO algorithm make this technique converge to its
global optimum in a good opportunity. Meanwhile, PSO algorithm is compared with genetic
algorithm (GA), and Broydon-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton algorithm
for their global search capabilities based on a suite of difficult analytical optimization prob-
lems.
Finally, Chapter 7 summarizes the major contributions of the research and provides
some directions of possible future works.
10
Chapter 2
Background and Related Work
This chapter presents a background introduction for network resource allocation op-
timization problems and discusses several optimization algorithms and their applications.
First we present a brief introduction for network utility maximization (NUM), which is used
for most of network resource allocation problems, and review the currently applications of
NUM and also the resource allocation optimal problems in wireless mesh networks. Next we
introduce several optimization algorithms mostly used in the resource allocation optimiza-
tion problems, such as convex optimization, lagrange duality, and gradient and subgradient
methods. And the flow control problem solved by using these kinds of algorithms is also
discussed. An algorithm based on swarm intelligence, called particle swarm optimization
(PSO), is introduced later. For PSO algorithm which has many attractive features includ-
ing ease of implementation and the fact that no gradient information requirement, its social
behavior and applications are also discussed.
2.1 Network Resource Allocation
Most of the network resource allocation problems can be formulated as a constrained
optimization problems of some network utility functions. Normally, there are three levels
methods to efficiently solve a network resource allocation optimization problem. First is
on theoretical properties, such as global optimality. It is well known as a convex optimiza-
tion. Second is on computational properties, such as centralized algorithms. Third is on
11
decomposable properties, such as dual optimization. [17, 18]
2.1.1 Overview
Network utility maximization (NUM) problems provide an important approach to con-
duct network resource allocation, such as rate allocation or power management. The decom-
posability structure of network utility maximization leads to the most appropriate distrib-
uted algorithm for a given network resource allocation problem. The distributed solutions
are particularly attractive in large scale networks, where the centralized solutions are infea-
sible, too costly, too fragile, and inapplicable [17].
Decomposition theory naturally provides the mathematical method to build the foun-
dation for the distributed control of networks. This helps us obtain the most appropriate
distributed algorithm for a given network resource allocation problem, such as distributed
routing, scheduling to power control and congestion control.
Based on Kelly’s work [1] in 1998, the framework of NUM has found many applications in
network resource allocation algorithms and network congestion control algorithms [2, 3, 4, 5],
such as Internat rate allocation and TCP congestion control. Traffic from such applications
is elastic, which is a typical example in TCP traffic over Internet. Other examples include the
available bit rate (ABR) service, which enables maximal link utilization over ATM networks
[19, 20, 21]. These work interpret source rates as primal variables and link congestion prices
as dual variables to solve an implicit global utility optimization problem or lagrange dual
problem. Some network flow control problems have been used in the context of congestion
avoidance in multihop networks by using max-flow min cost theorems [22, 23]. Furthermore,
allocation of limited network resources, such as network bandwidth, power control can also
be formulated as basic NUM problems.
Meanwhile, the framework of NUM has also been substantially extended from internet
congestion control to a general approach of understanding interactions across layers. An
optimization framework of layered structure is introduced in [24]. The cross-layer optimiza-
tion can be applied to enable a clean-slate design of the protocol stack. The algorithm
12
obtains relations to different layers of protocol stack, and couples them through a limited
amount of information passed back and forth. Different layers iterate on different subsets
at different time scales using local information to achieve individual optimization. These
local algorithms collectively achieve a global optimization objective.
Many networks resource allocation problems can be formulated as a constrained max-
imization of some utility function. The key problem here is how the available bandwidth
within the network should be shared between each controllable traffic. A distributed and
asynchronous optimization scheme is particularly attractive in large-scale broadband net-
works where a centralized optimization scheme is infeasible, too fragile and too costly. There
are many research work on network utility performance optimization on many applications
using distributed optimization scheme. Primal and lagrangian dual algorithm [1, 25, 3, 4, 26]
based on distributed scheme have been proposed to many applications to solve the utility
function optimization problem. Standard textbook [27], [28] and [29] summarize the math-
ematics distributed computation of optimization techniques.
In the dual based distributed algorithm, the lagrange dual variables can be interpreted
as shadow prices for network resource allocation. Here, each link l calculates a “price” pl for
a unit of bandwidth at link l based on the local source rate, and the source s is controlled
by the total price ps =∑
pl, where the sum is taken over all links that source s pathes.
Then based on these, the source s chooses a transmission source rate xs to maximize its own
benefit Us(xs)− ps · xs. These kinds of algorithm require the utility function to be strictly
convex or concave, where the convexity property readily leads to a distributed algorithm
that converges to a globally optimal resource allocation problem. Normally, the video data
processing utility function is nonconvex, these kinds of optimization problems can not be
handled properly by previous distributed optimization algorithm. In addition, existing
algorithms based on gradient projection also suffer from slow convergence speed problem.
Most of the papers in the vast related on network resource allocation use a standard
lagrange dual based distributed algorithm. While it is well known that dual based distrib-
uted algorithm needs utility function to be convex or concave. This is the major drawback
13
of dual base distributed algorithm when the application is inelastic or utility functions are
nonconvex, which leads to divergence of congestion control.
2.1.2 Resource Allocation Over Mesh Networks
Wireless Mesh Networks (WMNs) are formed by dynamically self-organized and self-
configured wireless nodes that use multi-hop wireless relaying. WMNs have emerged as a
key technology for next generation wireless networking. In the recent past, based on many
advantages it has: (1) inexpensive network infrastructure, (2) easiness of implementation
the network, and (3) broadband data support. WMNs attract significant research and
inspiring numerous applications, including community meshes, vehicular platoons, and home
entertainment networks [30, 31]. However, there are significant challenges in the deployment
and optimization for efficient video streaming transmission over such wireless mesh network
due to the network resources and dynamics.
A wireless mesh network consists of a large number of wireless nodes spreading across a
geographical area without the help of a fixed infrastructure. Each node in the network for-
wards packets for its peer nodes and each end-to-end flow traverses multiple hops of wireless
links from a source to a destination. Because of its ad hoc nature, wireless mesh network
can respond to conditions much more quickly and reliably than static networks. Ad hoc
networks inherit the traditional problems of communication networks, such as bandwidth
optimization, power control, and transmission quality enhancement.
In wired networks, the flows only contend at the router that performs the flow scheduling.
Compared with this, the multihop wireless networks show that the flow also contend at
shared channel if they are within the interference ranges of each other. This presents the
problem of optimizing resource allocation respecting to both resource utilization and fair
across contending multihop flows [32].
Previous research use price-based strategy [1, 3, 6, 7, 8], where prices are computed to
reflect relations between resource demands and supplies, to coordinate the resource alloca-
tions at multiple hops, and the objective functions here are needed to be strictly convex.
14
Their results show that the price is effective as a method to arbitrate resource allocation.
The basic idea is to set prices on mutually contending links based on their congestion, and
the goal is to allocate the transmission rates in such a way that whole networks’ utility
is maximized. Here, the shadow price is associated with the links to reflect the relations
between the traffic load of individual link and its bandwidth capacity, the utility is asso-
ciated with an end-to-end flow to reflect its resource requirement. Transmission rates are
chosen to respond to the aggregated price along every flow such that the whole benefits of
flows are maximized. A distributed algorithm is obtained by using the lagrangian dual of
the optimization problem and hence decomposing the problem. Some other research use
gradient and subgradient search algorithm to handle the distributed optimization problem
in sensor networks [9, 10].
Since deployment of such an ad hoc network is fast and flexible, it is very attractive
to support real time media streaming over an ad hoc network. Recently there are many
research interesting on video streaming over ad hoc networks [33, 34, 35, 36, 37, 38, 39].
When there are multiple video streaming in the network, these streams share and compete
for the common resources, such as bandwidth and transmission power. Rate allocation
in this case serves the purpose of resource allocation among these streams (see Fig.2.1).
A video stream utilizes more network resources and achieves better video quality when it
reaches a higher video source rate. The rate allocation algorithm should be fair and efficient
among the streams. A distributed algorithm will be desirable and efficient because the
whole computational burden is shared by all participating nodes, and the solution can be
easily adjusted when the network conditions are fluctuant.
2.2 Previous Optimization Techniques
In this section, we present several optimization algorithms mostly used in the resource
allocation optimization problems, such as convex optimization, lagrange duality, gradient
and subgradient methods, and a brief discussion is given.
15
Figure 2.1: Video Streaming over Ad hoc networks.
2.2.1 Optimization Algorithms
I. Convex Optimization
Since 1940s, a large effort has been into developing algorithms for solving various
classes of optimization problems, analyzing their properties, and developing implementa-
tions. Mathematical optimization is used for many applications, such as an aid to a human
decision maker, system designer, or system operator [40].
Convex optimization methods are widely used in the design and analysis of communi-
cation networks and signal processing algorithms. A convex optimization problem can be
formulated as below [40]:
min . f0(x) (2.1)
s.t. fi(x) ≤ 0, i = 1, ..., m
where the functions f0, ..., fm : Rn → R are convex, satisfy
fi(αx + βy) ≤ αfi(x) + βfi(y) (2.2)
for all x, y ∈ Rn and α, β ∈ R with α + β = 1, α ≥ 0, and β ≥ 0.
16
Convex optimization techniques are very important in engineering application, because
a local optimum is also the global optimum [40]. This is the theoretical foundation for
gradient search and lagrange dual algorithms used in distributed optimization.
Convex optimization problems are the largest subset of optimization problems which are
efficiently solvable, whereas nonconvex optimization problems are generally difficult. Cur-
rently there are many softwares which can generate accurate and reliable solutions with-
out the headaches of initialization, step size selection, and the risks of trapping in a local
optimum[41]. Once an engineering problem can be formulated into a convex optimization
problem, it is reasonable to consider it solvable.
II. Lagrange Duality
Convex optimization techniques have useful Lagrange duality properties. The original
optimization problem in Eq.(2.1) can be formulated to a lagrangian function as follow:
L(x, λ) = f0(x) +m∑
i=1
λifi(x) (2.3)
where λi is dual variable.
The original optimization problem is referred as primal optimization problem, the so-
called dual function g(λ) is defined as the minimum of the Lagrangian function
g(λ) = minx∈Rn
L(x, λ) (2.4)
Notice that, even if the original optimization problem is not convex, the dual function
g(λ) is always concave because it is a pointwise minimum of a family of linear functions[41].
The dual variable λ is dual feasible if λ ≥ 0.
For any primal feasible x and dual feasible λ, it turns out f0(x) ≥ g(λ). This means
the dual function value g(λ) always is the lower bound of the primal function f0(x). The
maximization lower bound can be obtained on the optimal value f ∗ of the original problem
by solving the dual optimization problem:
max . g(λ) (2.5)
s.t. λ ≥ 0
17
Here the optimal value f ∗ is achieved at an optimal solution x∗, f ∗ = f0(x∗).
The basic idea of decomposability structures which lead to distributed algorithms are to
decompose the original large optimization problem into distributively solvable subproblems
which are coordinated by a high-level master problem by means of some kinds of signaling
[27, 28, 29, 42], as illustrated in Fig. 2.2. Most of the existing decomposition techniques can
be classified into primal decomposition and dual decomposition [17].
Figure 2.2: Decomposition of the original optimization problem.
A primal decomposition is a approach since the original problem allocates the existing
resources by directly giving each subproblem the resources that it can use. Dual decomposi-
tion method is that the original problem sets the price for the resources to each subproblem
which can decide the amount of resources to be used.
When the original problem has a coupling constraints, the dual decomposition can be
formulated as follows:
max .∑
i
fi(xi) (2.6)
s.t.∑
i
hi(xi) ≤ b, xi ∈ Xi
where Xi is the range xi should be inside. If the constraints in (2.6) are absent, then the
problem is the primal decomposition problem. After using lagrangian duality properties,
18
the problem in (2.6) is changed to
max .∑
i
fi(xi)− λT (∑
i
hi(xi)− b) (2.7)
s.t. xi ∈ Xi
Then the dual problem is solved by updating the dual variable λ
min . g(λ) =∑
i
gi(λ) + λTb (2.8)
s.t. λ > 0
where gi(λ) is the maximization value obtained from dual function by using lagrangian for
a given λ. If the dual function g(λ) is differentiable, the dual problem in (2.8) can be solved
by using gradient methods mentioned in the following section.
III. Gradient and Subgradient Methods
Lagrange duality properties can lead to decomposability structures [17]. After perform-
ing a decomposition, the objective function of the optimization problem may or may not
be differentiable. Gradient and subgradient methods are a popular technique for iteratively
optimization problems of differentiable and nondifferentiable functions. These methods are
distinguished and very convenient because of their simplicity, little requirements of memory
usage, and amenability for parallel implementation [43, 28].
Below is a general concave maximization problem over a convex constraint set:
max . f0(x) (2.9)
s.t. x ∈ S
Both the gradient and subgradient projection methods generate a sequence feasible points
{x(t)} as follow:
x(t + 1) = [x(t) + α(t)p(t)]S (2.10)
where p(t) is a gradient of subgradient of f0 at x(t), α(t) is a positive step size, and [·]Sdenotes the projection onto a feasible convex set S. However, there is a difference between
19
gradient and subgradient methods. Each iteration of the subgradient method may not
improve the objective value as happens with a gradient method. For sufficiently small step
size α(t), the distance of the current point x(t) to the optimal solution x∗ decrease makes
the subgradient method converge.
There are many results on convergence of the gradient and subgradient methods with
different choices of step size. For a constant step size α, more convenient for distributed
algorithms, the gradient method converges to the optimal value provided that step size
is sufficiently small, whereas for the subgradient method, it is guaranteed to converge to
within some range of the optimal value, in other words, the subgradient method finds an
ε suboptimal point within a finite number of steps. The major drawbacks for the gradient
and subgradient methods are that they have slow convergence speed and are sensitive to the
local optimum and saddle points if the objective function is not strictly convex or concave.
2.2.2 Applications
I. Basic Model of Flow Control
Let us consider a generic flow control optimization problem over a large-scale communi-
cation network with a set L = {1, ..., L} logical links, each link with a capacity of cl, l ∈ L.
The communication network can be wired or wireless. The whole network is shared by a set
S = {1, ..., S} sources. Each source s emits one flow and transmits at a source rate xs which
satisfies ms ≤ xs ≤ Ms (ms ≥ 0 and Ms < ∞ are the minimum and maximum transmission
rates), uses a set L(s) ⊆ L of links in its path, and has a utility function Us(xs). Each link
l is shared by a set S(l) = {s ∈ S|l ∈ L(s)} of sources.
The flow control optimization problem can be formulated as the problem of maximizing
the network utility∑
s Us(xs), over the S sources, subject to linear network flow constraints∑
s∈S(l) xs ≤ cl. Using centralized scheme to solve this optimization problem would require
to know all utility functions, which is not reality in many applications. The existing dis-
tributed optimization algorithms based on incremental subgradient method and traditional
20
lagrange dual distributed algorithm requires the utility function U(x) to be strictly concave
or convex, which can not be satisfied to many reality applications. These kinds of algorithms
need critical convexity properties to prove convergence to global optimum.
The NUM problem optimization model can be formulated as
max .∑
s∈S(l)
Us(xs) (2.11)
s.t.∑
s∈S(l)
xs ≤ cl, l = 1, ..., L
where U(·) is the utility functions. In general, it is a nonlinear, nonconvex function. Existing
distributed optimization algorithms based on incremental sub-gradient search [9, 10] assume
that the objective function U(x) is additive and convex, and the algorithms based on price-
based lagrangian dual flow control optimization [1, 25, 3] assume that the objective function
is increasing and strictly concave.
II. Dual Optimization
A dual optimization is appropriate when the problem has a coupling constraint such
that the optimization problem can be decomposed into several subproblems. Normally,
dual optimization uses Lagrange duality properties. The basic idea in Lagrange duality is
to relax the original optimization problem in(2.11) by transferring the constraints to the
objective in the form of a weighted sum. Define the Lagrangian as:
L(x, p) =∑
s
Us(xs)−∑
l
λl(∑
s∈S(l)
xs − cl)
=∑
s
(Us(xs)− xs
∑
l∈L(S)
λl) +∑
l
λlcl
(2.12)
The optimization problem of the lagrangian dual problem is
maxx
L(x, p) =∑
s
(Us(xs)− xsλs) +
∑
l
λlcl (2.13)
s.t. xs ∈ M, s = 1, ..., S
where M = [ms,Ms] denotes the range which xs must lie in, and λs =∑
l∈L(s)
λl.
21
The dual problem is solved by updating the dual variable λs:
minλ
D(λ) =∑
s
Ds(λs) +
∑
l
λlcl (2.14)
s.t. λ ≥ 0
where Ds(λs) is the dual function obtained as the maximum value of the lagrangian solved
in (2.13) for a given λ. This approach will have appropriate results only if strong duality
holds, this means the original optimization problem should be convex.
Here we can interpret λl as the price per unit bandwidth at link l, then the λs is the total
price per unit bandwidth for all links in the path of source s, xsλs represents the bandwidth
cost to source s, and Ds(λs) represents the maximum benefit for source s can achieve at the
given price λs [3]. The total price λs for source s summarizes all the congestion information
that s needs to know. For a given λ, a unique maximizer exists since original objective
function is strictly convex. The important point here is for a given λ, individual source s
can solve problem in (2.14) separately without the need to know the information of other
sources.
If the dual function is differentiable, then the dual problem in (2.14) can be solved by
using the gradient projection method. The link price λl is adjusted as follows:
λl(t + 1) = [λl(t)− α∂D
∂λl
(x(t), λ(t))]∗ (2.15)
where α is a parameter for step size, [x]∗=max{x, β}, and β is the link price lower bound.
D is continuously differentiable
∂D
∂λl
(x(t), λ(t)) =∑
s∈S(l)
xs(t)− cl = xl(t)− cl (2.16)
where xl is the aggregate source rate at link l. Then substituting (2.16) to (2.15), we will
get the price adjustment equation for link l ∈ L.
λl(t + 1) = [λl(t) + α(xl(t)− cl)]∗ (2.17)
From the demand and supply, if the demand for aggregate source rate at link l is less
than the supply cl, then reduce price λl, otherwise increase the price λl. The adjustment for
22
price is completely distributed and can be implemented by only using local information. In
each iteration, source s solves problem in (2.14) individually and propagates its result xs(λ)
to all the links on its path. Meanwhile link l will update its price λl based on current source
rate demand and propagates the new price to sources path this link. The dual variable
(price) λ will converge to the dual optimal after finite iterations and the primal variable
(source rate) x will also converge to the optimal solution.
The basic dual optimization algorithm can be summarized as follows:
• Step 1: Initializes λl(0) to some positive value for all links.
• Step 2: Each source s locally solves problem when receives the total price λs in its
path and propagates this new source rate xs to the whole network.
• Step 3: Each link l updates its price λl when receives all rates xs that go through
this link and propagates the new price to all sources that path this link.
• Step 4: Go through Step 2 to Step 3 until the termination criterion is satisfied.
In Low’s work, he has already proof that when the following condition are satisfied: the
original utility function is strictly convex, twice continuously differentiable, the curvatures
of utility function are bounded away from zero and the time between consecutive updates is
also bounded, then starting from any initial rates inside the range M = [ms,Ms] and prices
λ(0) ≥ 0, every accumulation point (x∗, λ∗) of the sequence (x(t), λ(t)) generated by the
basic dual optimization algorithm mentioned above are primal optimal. Moreover, for all
sources s, the error in price estimation and rate calculation both converge to zero, and the
gradient estimation error is also converges to zero [3].
2.3 Particle Swarm Optimization
Particle swarm optimization (PSO), developed by Kennedy and Eberhart in 1995 [11,
12, 16], is a promising population-based new optimization technique which models the set of
23
potential problem solutions as a swarm of particles moving about in a virtual search space.
Some of the attractive features of the PSO include the ease of implementation and the fact
that no gradient information is required. It can be widely used to solve many different
optimization problems, including most of the problems that can be solved by using Genetic
Algorithms (GAs).
Many popular optimization algorithms are deterministic, like the gradient-based al-
gorithms. Compared with gradient-based algorithms, the PSO algorithm is a stochastic
algorithm that does not need any gradient information. This allows the PSO algorithm to
be used on many functions where the gradient algorithm is either unavailable or computa-
tionally to obtain. In the past several years, PSO algorithm has been successfully applied
in many research and application areas. PSO is also attractive because that there are few
parameters to adjust. It is demonstrated that PSO gets better results in a faster, cheaper
way compared with many other methods.
2.3.1 PSO Algorithm
The original PSO algorithm was based on the sociological behavior associated with
birds flock. The method of PSO is inspired by the movement of flocking birds and their
interactions with their neighbors in the group. Instead of using evolutionary operators to
manipulate the individuals, like in other evolutionary computational algorithms, each indi-
vidual in PSO in the search space with a velocity which is dynamically adjusted according
to its own flying experience and its companions’ flying experience [44].
The high-level idea of PSO can be summarized as follows. To find the minimum of an
objective function f(x) (x is a vector) within a solution space, the PSO algorithm starts
with a set of candidate solutions called particles, {xp|1 ≤ p ≤ P} distributed in the solution
space. A typical value of P is between 20 and 50 [16]. During the optimization process,
each particle xp moves within the solution space in search for the minimum of f(x) , and
the corresponding movement path is denoted by xp(t), where t represents time. At each
24
time step, the movement of particle xp is given by
xp(t + 1) = xp(t) + v (2.18)
where
v = w · v + c1Θ1[xsp − xp(t)] + c2Θ2[x
g − xp(t)] (2.19)
Here, w, c1 and c2 are weighting factors, Θ1 and Θ2 are two random numbers, these parame-
ters can influence the maximum step size that a particle can reach in a single iteration. xsp
is the best solution that the particle itself has found so far, and xg is the best solution that
all particles have found so far, the value of v should be clamped in the range [−vmax,vmax]
to reduce the likelihood that the particle might leave the search space. Fig. 2.3 gives the
illustration of PSO algorithm. Each particle, when determining its next move in search for
the global optimum, always balances the behaviors of its own and the group [16].
Figure 2.3: Illustration of particle swarm optimization.
The PSO algorithm consists of repeated applications of the particle updated equation
represented in Eq. (2.18). Fig. 2.4 gives a pseudo code example for the basic PSO algorithm.
Note that the two if statements are the equivalent of applying Eq. (2.18) and Eq. (2.19)
respectively.
The initialization mentioned in the PSO algorithm includes the following: (1). Initial-
ization of each particle xm (xm is a n-dimensional vector). Each coordinate particle xm
25
should be initialized to a value drawn from the uniform random distribution on the search
space. This distributes the initial position of all the particles randomly throughput the
search space. (2). Initialization of each vm (vm is a n-dimensional vector) of each particle
xm. Initializes each vm to a value drown from the uniform random distribution on the
interval [−vmax,vmax]. Alternatively, the vm of each particle could be initialized to 0, since
the starting positions of each particle are already randomize. The stop condition mentioned
in the PSO algorithm depends on the type of problems being solved. Normally, the PSO
algorithm runs for a fixed number of iterations or until a specified error bound is reached.
Figure 2.4: Pseudo code for the basic PSO algorithm.
2.3.2 Social Behavior
A number of scientists have created computer simulations of various interpretations of
the movement of a bird flock or fish school. It does not seem a large leap of logic to suppose
that some same rules underlie animal social behavior, including herds, school, flocks and that
of humans[12]. Sociobiologist E. O. Wilson states that individual members of the school can
26
profit from the discoveries and previous experience of all other members of the school during
the search for food, this advantage can become decisive, outweighing the disadvantages of
competition for food item, whenever the resource is unpredictably distributed in patches[45].
We can achieve that social sharing of information offers an evolutionary advantage from
this statement. This hypothesis is the fundamental to the development of particle swarm
optimization[12].
The psychological assumptions of PSO theory are general and noncontroversial. In the
search of consistent cognitions, individuals will tend to retain their own best beliefs, and
will also consider the beliefs of other colleagues. Adaptive change happens when individuals
perceive that others’ beliefs are better than their own. This concept is not new, what is new
is that these simple concepts taken together create an evolutionary information processing
technique which is powerful enough to manage the huge amount of information comprising
human knowledge[46].
2.3.3 Applications
Currently, PSO has been applied to solve many problems. One of the first applications
of PSO is in neural network introduced by J. Kennedy and R. Eberhart in 1995. One of
the authors’ first experiments involved training weights for a three layers neural network
solving the XOR problem [12]. Their results show that the PSO algorithm performs very
well on this problem. Most of PSO applications are involved in neural networks. Currently
there are many modified PSO approaches used on many aspects in neural network[47, 48,
49], including fuzzy neural networks, B-Spline for nonlinear system identification, feed-
forward neural networks with weight decay, and etc. Furthermore, paper[50] combines PSO
algorithm and fuzzy neural network together to handle the financial risk early warning
problem, which is the foundation of the effective risk management.
Genovesi [51] uses PSO to handle the electromagnetic optimization problems. In this lit-
erature, the problem of synthesis of frequency selective surfaces is handled by simultaneously
27
optimizing both the real and binary parameters. PSO algorithm is used for design optimiza-
tions of electromagnetic devices in [52]. A modified PSO (Quantum PSO) algorithm[53] is
applied to linear array antenna synthesis, which is one of the standard problems used by
antenna engineers.
PSO algorithm is also applied to the parameter extraction of an equivalent circuit model
[54] recently. And PSO combined with adaptive simulated annealing with tunneling (ASAT)
are applied to the RF circuit synthesis techniques in [55].
Another application for PSO algorithm is on sensor networks [56, 57, 58, 59, 60]. A
sink node placement optimization problem is discussed in [56]. Target location estimation
problem is discussed in [57]. Paper [58] uses PSO algorithm for cluster formation in wireless
sensor networks, and paper [60] uses PSO algorithm for clustering node problems in ad
hoc sensor networks. An optimization problem of multicast routing in sensor networks in
discussed in [59]. And in [61], PSO algorithm is used to solve programming problems in
MIMO-based cross layer design for sensor networks.
2.4 Summary
Network resource allocation optimization has been widely used in Internet, TCP traffic
and ATM performance optimizations. Most of the network resource allocation problems
can be formulated as a constrained optimization problems of some network utility func-
tions. Network utility maximization (NUM) has been used for most of network resource
allocation problems, and there are many applications currently, such as resource allocation
optimization problems over wireless mesh networks. Meanwhile, the framework of NUM
has also been substantially extended from internet congestion control to a general approach
of understanding interactions across layers.
There are several basic optimization concepts most used in the resource allocation opti-
mization problems, such as convex optimization, lagrange duality, gradient and subgradient
methods. The basic network flow control problem has already been successfully solved by
28
lagrangian dual algorithm. The mainly drawback for this algorithm is that it needs the
objective function to be strictly convex or concave, otherwise the optimization problem can
not be handled properly. And for the gradient search algorithm, it also suffers from the
slow convergence speed.
For the PSO algorithm, it has many attractive features including ease of implementation
and the fact of no gradient information requirement. The main advantage of PSO over other
global optimization strategies is that the large number of random solutions that make up
the PSO make the technique avoid dropping in the local optima or saddle points of the
optimization problem. Meanwhile, unlike Genetic Algorithm (GA), PSO has no evolution
operators such as crossover and mutation and also has fast convergence speed.
29
Chapter 3
Particle Swarm Optimization over
Wireless Sensor Networks
A wireless sensor network (WSN) is a system of spatially distributed sensor nodes to col-
lect important information about the target environment. In this Chapter, we first consider
the unique characteristics of WVSN and develop an evolutionary optimization scheme using
a swarm intelligence principle to solve the WVSN performance optimization problem [62].
We transform the solution space set by the flow balance and energy constraints to a convex
region in a low-dimensional space without constraints. We then merge the convex condition
with the swarm intelligence principle to guide the movement of each particle during the evo-
lutionary optimization process. Our experimental results demonstrate that the proposed
performance optimization scheme is very efficient. Next, we develop an evolutionary distrib-
uted optimization scheme using swarm intelligence principle, called decentralized particle
swarm optimization (DPSO), to solve the generic distributed WSN parameter estimation or
optimization problem [63]. Based on the particle swarm intelligence principle, sensor nodes
share information with each other through local information exchange and communication
to solve a joint estimation or optimization problem. We use the swarm intelligence princi-
ple to guide the movement of each particle during the evolutionary optimization process.
The proposed distributed optimization scheme has low communication energy cost and as-
sures fast convergence. In addition, the objective function is not required to be convex.
30
We use source location as an example to demonstrate the efficiency of the proposed DPSO
scheme. Our experimental results demonstrate that the proposed optimization scheme is
very efficient and outperforms the existing distributed gradient-based optimization schemes.
3.1 Convex Mapping of High-dimensional Resource Con-
straints
WSNs have been envisioned for a wide range of applications, such as battlefield in-
telligence, environmental tracking, and emergency response. Each sensor node has limited
computational capacity, battery supply and communication capability [13]. A wireless video
sensor network (WVSN) is a system of spatially distributed video sensors which capture,
process and transmit video information over a wireless ad hoc network. In WVSN, each
sensor, equipped with a video camera, is able to capture, process and transmit visual infor-
mation about the circumstance activities.
Compared to other conventional sensor networks, GPS and temperature sensor networks,
the WVSN has the following unique characteristics: (1) The video sensor data is volumi-
nous. (2) Video data compression is computationally intensive and energy consuming, which
consumes about 60-80% of the total energy [64]. Therefore, the WVSN operates under se-
vere bit and energy constraints. Performance optimization of a large-scale WVSN under
resource constraints is a nonlinear, high-dimension, constrained optimization problem.
There is a significant body of research work on performance optimization of wireless
sensor networks, such as energy minimization, rate allocation and topology control [13, 14].
These results and algorithms are generic in their nature, and could be used to improve the
performance of WVSNs. However, they have not considered the unique characteristics of
WVSN, such as the complex nonlinear resource utilization behavior of each sensor node
function, which will potentially render these analysis and algorithms inefficient or even
impractical.
31
3.1.1 WVSN Operation Models
We assume that the WVSN has K video sensor nodes {Vi}, and has L communication
links. Each node compresses the video sensor data and let the output bit rate be Ri. Let Pi
be the energy consumption used in video compression. According to our previous work on
energy consumption modeling and power-rate-distortion analysis for video compression [64],
the coding distortion of the compressed video data is give by D(Ri, Pi). Our experimental
studies [64] suggest that the P-R-D behavior can be well captured by the following model
D(Ri, Pi) = σ2i 2−λRi·g(Pi) (3.1)
Here, σ2i represents the picture variance at node Vi and λ represents the resource utilization
efficiency of the video encoder. g(P ) is the power consumption model of the microprocessor.
The analysis in [64] suggests that g(P ) = P1γ , 1 ≤ γ ≤ 3. We assume that the performance
of the whole WVSN is measured by the overall video distortion
D =K∑
i=1
D(Ri, Pi). (3.2)
The power consumption in wireless transmission between nodes can be modeled as Pt(i, j) =
ci,j × ri,j, where Pt(i, j) is the power dissipated at node i when it is transmitting to node
j, ri,j is the bit rate transmitted from node i to node j, and ci,j is the power consumption
cost of radio link from node i to node j. The cost can be computed by
ci,j = α + β × dmij , (3.3)
where α is the distance-independent constant term, β is coefficient term associated with the
distance-independent term, dij is the distance between node i and j, and m is the path loss
index. The power dissipation at a receiver can be modeled as [14]:
Pr(i) = ρ×∑
j 6=i
rji (3.4)
where∑j 6=i
rji is the total bit rate of the received video data at node i and ρ is a constant.
32
3.1.2 WVSN Performance Optimization
The WVSN performance optimization has to satisfy the flow balance and energy con-
straints. Let Ei be the energy supply at node Vi, and T the operational lifetime of the
sensor network. Mathematically, the optimization problem can be formulated as
min{Ri,Pi},{rij}
D =K∑
i=1
D(Ri, Pi)
s.t. riB +∑j 6=i
rij −∑j 6=i
rji = Ri,
Pi · T +∑k 6=i
ρ · rki · T +∑j 6=i
cij · rij · T + ciB · riB · T ≤ Ei,
0 ≤ riB, rij ≤ rmax, 0 ≤ Ri ≤ Rmax, 0 ≤ Pi ≤ Pmax (3.5)
where rij and riB are data rates transmitted from node i to node j and from node i to
base-station B, rmax, respectively [14]. Rmax and Pmax are the maximum possible values of
rates and power. The first constraint is the flow balance constraint, stating that the total
data transmitted from node i should be equal to total data received from other nodes plus
the data generated by itself. The second constraint is the power consumption constraint,
representing that the total energy used for data processing should not exceed the energy
constraint.
In the following, we develop a matrix representation of the performance optimization
problem in (3.5). We assemble all the link rate variables {rij} into a link rate vector r, and
all the node bit rates {Ri} into vector R, all the power consumption in video compression
{Pi} at each node into vector P. Let X = [r,R,P], which represents all the control variables
that need to be determined by the performance optimization. Note that the flow balance
and energy constraints are both linear. Therefore, the performance optimization problem
in (3.5) can be written in the following matrix form
minX
J(X)
s.t. M ·X = 0
N ·X ≤ E0
0 ≤ X ≤ Xmax, (3.6)
33
where M represents the topology of the network, and N represents the energy cost for video
transmission over the network. E0 represents the initial power supply. We can see that X is
a (2K+L)-dimension vector, M and N are both K×(2K+L) matrices. This is a nonlinear,
very high-dimensional, constrained optimization problem for a large-scale WVSN.
3.1.3 Transform of the Solution Space
Handling the constraints is one of the central challenges in population-based evolutionary
optimization, such as PSO. One of the existing approaches to handle constraints is as follows:
move the particle (or compute the next generation) according to the swarm intelligence
principle; check if the new position of the particle satisfies the constraints; if not, this
movement of the particle is canceled and the particle stays in its current position.
We observe that handling the constraints in the original space of vector X = [r,R,P] is
very inefficient. First, in the original space, variables are interdependent. For example, if we
change the coding bit rate of node i, then the transmission rates of its associated links, as
well as the bit rates of its neighboring nodes will be affected, because they have to satisfy the
flow balance constraints. Second, according to our preliminary studies, the probability of a
solution vector [r,R,P] generated by swarm intelligence principle satisfying the constraints
is extremely small. This will render the optimization algorithm very inefficient.
In this paper, we use a linear representation of the solution space to define a linear
transform, which is able to map the high-dimensional constrained original solution space
into a convex region in a low-dimensional space without constraints. We then develop a
new evolutionary optimization scheme using the swarm intelligence principle and the convex
properties to solve the performance optimization in this low-dimensional convex space.
I. Linear Representation of the Solution Space
First, we consider the flow balance constraint M ·X = 0. This constraint implies that
34
the vector X must be in the null space of matrix M. Therefore, X can be represented by
X =
n1∑i=1
yi · gi = GY, (3.7)
where G = [g1,g2, ...,gn1 ] are the orthonormal basis vectors of the null space and Y =
[y1, y2, ..., yn1 ]t are coefficients. We can see that n1 = (2K + L)− rank(M).
Second, we consider the energy constraint. We assume that the optimum performance
is achieved when all energy is used, i.e, N ·X = E0. According to (3.7), we have
N ·G ·Y = E0. (3.8)
We suppose X0 is one specific solution (not necessary the minimum solution) which satisfies
the flow balance and energy constraints. Therefore, X0 must be in the null space of M. Let
X0 = GY0 Since N ·G ·Y0 = E0, Therefore,
N ·G · (Y −Y0) = 0. (3.9)
This implies that Y −Y0 is in the null space of matrix N ·G. Let H = [h1,h2, ...,hn2 ] be
the orthonomal basis vectors of the null space. Therefore, we have
Y −Y0 =
n2∑i=1
zi · hi = HZ. (3.10)
II. Transform of the Solution Space
Eqs. (3.7) and (3.10) tell us that if a vector X satisfies the flow balance and energy
constraints, then it can be represented in a low-dimensional space by vector Z. Since G
and H are both orthonormal matrices, from Eqs. (3.7) and (3.10), we can see that
Z = Ht ·Gt · [X−X0]. (3.11)
This defines a transform from the original solution space S = {X|X satisfies all the con−straints in (3.6)} to a new space S′:
F : X 7−→ Z = Ht ·Gt · [X−X0]. (3.12)
35
The new solution space is given by
S′ = {Z | 0 ≤ F−1(Z) ≤ Xmax}. (3.13)
This transform approach is very important and has the following advantages.
Lemma 1 The dimension of the new space S′ is much lower than the original space S.
The transform is able to reduce number of dimensions by up to 2K, where K is the number
of sensor nodes.
Proof The matrices G and H can be obtained from the singular value decomposition of
matrices M and NG. Therefore, n1 = (2K+L)−rank(M) and n2 = (2K+L)−rank(M)−rank(NG). Therefore, the number of reduced dimensions is rank(M) + rank(NG). If the
sensor network is not ill-conditioned, such as there is no orphan nodes, rank(M)+rank(NG)
should be close or equal to 2K. A detailed proof of these arguments is omitted here due to
page limitation.
Lemma 2 The new space S′ is convex.
Proof Notice that the transform F is linear, and the original space S is also convex.
Therefore, S′ is also convex.
Figure 3.1: Performance optimization with swarm intelligence and convex projection.
3.1.4 Performance Optimization Using PSO
36
Based on the swarm intelligence principle and the unique properties given by Lemmas 1
and 2, we are going to develop a new evolutionary optimization scheme for WVSN perfor-
mance optimization. The new swarm optimization will operate in the new solution space
S′. According to Lemma 1, this will significantly reduce the computational complexity.
In addition, in the new space, since the flow balance and energy constraints are implicitly
satisfied, the particles can move freely in S′. To make sure that each particle move within
the convex region of S′, we can use the property that S′ is convex. This implies that if two
particles X1 and X2 are in S′, then
λX1 + (1− λ)X2 ∈ S′, 0 ≤ λ ≤ 1. (3.14)
In PSO, the movement of each particle is defined by (2.18) and (2.19). If we choose the
weights such that
w + c1Θ1 + c2Θ2 = 1, (3.15)
then (2.19) can be written as
xm(t + 1) = w · x′m + c1Θ1x∗m + c2Θ2x
∗. (3.16)
Here,
x−m = xm(t) + [xm(t)− xm(t− 1)], (3.17)
is the predicted position of xm(t) according to the inertial velocity. In (3.21), we know that
x∗m and x∗ are in S′ because they are the best positions that have been found so far by the
particle itself and the whole group. However, x−m is not necessarily in S′. If not, we find the
convex projection for x′m as follows. Let {xi|1 ≤ i ≤ I} be the set of all particles, including
the initial particles and their moving histories. We know that these particles are all in S′.
Therefore, according to the convex condition, their weighted sum is also in S′. Let
[p′1, p′2, ...p
′I ] = arg min
{pi}||
I∑i
pixi − x−m||2, (3.18)
where 0 ≤ pi ≤ 1 andI∑i
pi = 1. Then, x′m =I∑i
p′ixi ∈ S′ which is the convex projection of
x−m. In (3.21), we replace x−m by x′m, then the new position of the m-th particle xm(t + 1)
37
must be in S′ according to the convex condition. Using this convex projection method, we
can make sure that each particle moves within the solution space.
3.1.5 Experimental Results
We test the proposed performance optimization scheme with different WVSN topology.
In the following experiment, the network has K = 10 nodes and L = 15 links. The particle
population size is 20, and the maximum generations is 2000. The other parameters are set as
follows: λ = 0.729 and γ = 0.5, α = 50, β = 0.0013, m = 2, ρ = 5, λk = 0.023 and σk2 = 10.
Fig. 3.2 shows that the overall video distortion metric quickly reaches to its minimum after
several generations. Fig. 3.6 shows the movement path of each particle in the solution space
(projected onto a 2-D plane for illustration purpose). After all the particles converge, an
optimum solution is found. We can see that the proposed performance optimization works
very efficiently. Our experimental results with other settings of WVSN yield similar results.
0 5 10 15 20 251.5
2
2.5
3
3.5
4
4.5
5
Iteration Number
Min
imum
Fun
catio
n V
alue
Figure 3.2: The performance metric decrease as the particles update their positions of PSOwith convex mapping.
38
−14 −13.5 −13 −12.5 −12 −11.5 −11 −10.5 −10 −9.50
10
20
30
40
50
60
70
80
Dimension 1
Dim
ensi
on 2
Figure 3.3: The traces of the all particles moving in the solution space of PSO with convexmapping.
3.1.6 Summary
In this section, based on the unique properties of WVSN and the swarm intelligence
principle, we have developed an evolutionary optimization scheme to solve the nonlinear
constrained performance optimization problem in WVSN. This section focuses on the an-
alytic development of the performance optimization method. The most challenge for PSO
algorithm is handling constraints. Our analysis shows that we can transform the solution
space into a convex region in a low-dimension space to reduce the computational complex-
ity and remove the interdependence between the control variables. The convex property of
the new solution space can be incorporated in the original swarm intelligence principle to
guide the movement of particles such that each particle in the new generation automatically
satisfies the constraints.
39
3.2 Distributed Optimization over Wireless Sensor Net-
works
A major challenge in WSN system design is data collection and analysis. Since each
node has very limited resources for communication and computation, it is inefficient or
even impractical to transmit all sensor data to a central location for information processing,
such as estimation and optimization. Therefore, distributed in-network signal processing
and optimization are highly desired, especially in large-scale WSN systems [9]. Recently,
several distributed optimization algorithms based on gradient search have been proposed
in the literature [9, 10]. Most of existing approaches assume that the objective function
to be additive and convex. Otherwise, it will be very difficult to assure convergence of the
distributed gradient search algorithm. In addition, existing algorithms also suffer from slow
convergence speed problem. In this section we present a new distributed scheme, called
decentralized PSO [63], which will not be sensitive to local optimum or saddle points and
has fast convergence speed.
3.2.1 Optimization Problems Using Decentralized PSO
Let us consider a generic parameter estimation or optimization problem over a WSN
with N sensors and each sensor taking M measurements about the target phenomenon. The
problem is to estimate a p× 1 vector parameter θ ∈ Rp from M independent measurements
x collected by n distributed selected sensors. The model is
min f(x, θ)
s.t. θ ∈ Rp (3.19)
where f(·) is a cost or utility functions. In general, it is a nonlinear function. Existing
distributed optimization algorithms based on incremental sub-gradient search assume that
the objective function f(x, θ) is additive and convex. However, in this work, there are no
such requirements and f(x, θ) can be a generic nonlinear function.
40
Our major idea in DPSO can be summarized as follows. Suppose there are K sensor
nodes which are involved in the distributed optimization or estimation task. We partition
the swarm particles (typically, in the range of 20-50 particles) into K sub-groups with each
node managing a sub-group of particles. More specifically, it is tasked to evaluate the
objective function for each particle in its sub-group and manage their movements based on
the swarm intelligence principle described in Eq. (2.19). After each iteration, the sensor
node finds the minimum solution within its sub-group and shares this sub-group minimum
with its neighboring nodes. Upon receiving the external information from its neighbors
about their sub-group minimum, each sensor node uses these external information to guide
the particles inside its sub-group so as to facilitate the search and optimization process.
The decentralized PSO algorithm can be stated as follows:
For each particle cluster {For each particle {
Repeat initialized particle until it satisfies
all the constraints}}Do {
Compare the received other cluster best solution with
local cluster best solution, update the local solution
with the best one.
For each particle {Use normal PSO algorithm to find particle
itself best solution and find new cluster
best solution among all cluster particles.}
Transmit new cluster best solution to other
particle clusters.
}While max iterations or convergence criteria is met.
41
In each particle cluster, the movement of each particle is defined by (2.18) and (2.19).
Here we choose the weights such that
w + c1Θ1 + c2Θ2 = 1 (3.20)
then (2.19) can be written as
xm(t + 1) = w · x∗m + c1Θ1xp + c2Θ2xg (3.21)
where,
x∗m = xm(t) + [xm(t)− xm(t− 1)] (3.22)
is the predicted position of xm(t) according to the inertial velocity.
3.2.2 Source Localization Application
In this section, we will use source location as an example to evaluate the performance of
the proposed DPSO algorithm. Location estimation of an acoustic source is an important
problem in both environmental and military applications [65]. Source localization problem
has been traditionally solved through nonlinear least-squares estimation, which is equivalent
to the maximum likelihood estimation when the observation noise is modeled as a white
gaussian noise [66]. In this source localization scenario, sensors near the source or target
are waken up and each sensor collects M measurements about the source location using
received signal strength. In the sensor network field, the acoustic source is located at an
unknown position θ. We assume that each sensor knows its own location ri, i=1,...,n. And
the received signal strength measurement model for the j-th measurement, j=1,...,M , at
node i is given by the following signal propagation model [65]
xi,j =A
‖θ − ri‖β+ ωi,j (3.23)
where A is a constant, β is the signal energy decay exponent and ωi,j are independent
Gaussian noise at each sensor i for every measurement j.
42
A maximum likelihood estimate θ is the global minimum formulated as
θ = arg minθ
1
n ·Mn∑
i=1
M∑j=1
(xi,j − A
‖θ − ri‖β)2 (3.24)
Sheng’s work uses Maximum likelihood estimator to solve the acoustic source localization
problem in sensor network [67], but this approach requires sensors to transmit all their data
to a fusion center for processing, which is impractical because of requiring massive amount
energy and bandwidth for a large network. In Rabbat and Nowak’s work [9, 15], they
propose gradient based distributed algorithm to solve the problem. The advantage of this
distributed algorithm is that it dose not need all the data transmitted to a central point for
processing compared with algorithm provided in [67]. But the drawback of this distributed
algorithm is that it is sensitive to local optimum and saddle points.
In the decentralized PSO algorithm, we assume each sensor knows all other sensors’
measurements and positions information after initial communication. In each particle clus-
ter, it finds its own cluster best estimation parameter. And during the estimation process,
each sensor node only transmits local best estimation parameter through communication
links. After communicating with other particle clusters and updating each cluster’s best
estimation parameter, all particles will find the best solution when all the particles converge.
Compared with gradient based distributed algorithm, the decentralized PSO algorithm will
not suffer from local optimum or saddle points.
3.2.3 Experimental Results
We test the proposed distributed optimization scheme using decentralized PSO with
different WSN topologies. In the following experiment, a sensor network was constructed
by placing 100 sensors uniformly distributed in a sensor field. Here in our experiment, one
sensor field has a size of 100×100 unit square. Sensor location is denoted by ri = (x, y). We
randomly select n sensors to wake up to collect measurements and estimate source location.
In total, there are K links between these sensors. These links should let all the selected
sensors to join the communication. The other parameters are set as follows: M = 10,
43
A = 100, and β = 2. The particle cluster size selects 2, we only assign 2 particles for each
sensor. All sensors make measurements at a signal-to-noise ratio (SNR) of 3 dB.
Fig. 3.4 depicts an example network topology using 20 sensors and 22 links. Fig. 3.5
shows that the overall minimum objective function quickly reaches to its minimum after
several iterations with DPSO. Fig. 3.6 shows the movement path of each particle in the
sensor field. Here in each particle sub-group, we only trace one particle’s moving path.
From these experimental results, we can see that the proposed scheme works very efficiently.
Our experimental results with other settings of WSN with different wake up sensor number
and link number yield similar results. Due to page limitation, they are omitted here.
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
X Position
Y Po
sitio
n
Sleep SensorWake Up SensorTargetParticle
Figure 3.4: A WSN topology with 20 sensors and 24 links.
In the sensor network which has fixed number of wake up sensors, when we increase the
number of communication links in the network, we will reach faster convergence. Fig. 3.7
shows the experimental results for one sensor network with 20 sensors waken up. As we
can see that as the number of communication links increases, the sensor nodes are able to
share information more efficiently, therefore speed up the DPSO convergence process. As a
44
0 2 4 6 8 10 12 14 16 18 200
20
40
60
80
100
120
140
Iteration Number
Min
imum
Fun
ctio
n Va
lue
Figure 3.5: The performance function value decrease as the particles update their positionsof decentralized PSO.
result, the number of iterations that are needed for reaching the minimum reduces.
3.2.4 Comparison with Gradient Search Algorithms
In this section, we compare the proposed DPSO with distributed gradient search algo-
rithms proposed in the literature [9, 15]. We will first give a short review about distributed
incremental gradient algorithm [9, 15]. This distributed incremental algorithm is a cycled
parameter estimation. On the k-th cycle, sensor i (i = 1, ..., n) receives an estimation ψki−1
from its neighbour and updates as following:
ψki = ψk
i−1 − α∇fi(ψki−1) (3.25)
where α is a positive step size, and ∇fi(ψki−1) represents the gradient of fi at ψk
i−1
fi(ψ) =1
m
m∑j=1
(xi,j − A
‖ψ − ri‖β)2 (3.26)
At the beginning, sets arbitrary initial condition ψ00 = θ0, and we will have θk = ψk
n after a
45
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
X Position
Y Po
sitio
n
Figure 3.6: The traces of the particles moving in the sensor field of decentralized PSO.
complete cycle through the network. Each subiteration only focuses on optimizing a single
function fi, which only depends on local data at sensor i.
The minimum objective function values found by the decentralized PSO and gradient
search algorithms were recorded as a function of the iteration number of the search process.
The comparisons are based on the same sensor network topology with the same number
of waken-up sensors and communication links. The experimental results are based on the
average of 100 experiments Fig 3.8 shows the comparison results of DPSO and gradient
search algorithms on optimization convergence. From the experimental results, we can see
that the decentralized PSO scheme is more efficient than the sub-gradient search algorithm.
3.2.5 Summary
In this section, we have developed an evolutionary optimization scheme, called decen-
tralized PSO (DPSO), to solve the WSN distributed generic parameter estimation or op-
timization problem based on swarm intelligence principles. The basic operation involves
46
20 22 24 26 28 30 32 34 36 38 4010
12
14
16
18
20
22
24
26
28
30
Communication Link Number
Itera
tion
Num
ber
Figure 3.7: Optimization convergence of WSN with 20 sensors wake up and different linknumber in decentralized PSO.
parameter estimation in each sensor and shares its own local results with its neighbors. The
proposed DPSO algorithm does not have requirement for the objective function, is not sen-
sitive to the local optimum or saddle points, and has very fast convergence speed compared
with gradient search method. Simulation results show that our evolutionary optimization
scheme is very efficient for different network topology.
47
0 5 10 15 20 25 300
20
40
60
80
100
120
140
160
180
200
Iteration Number
Min
imum
Fun
ctio
n V
alue
Decentralized PSOSubgradient Search
Figure 3.8: Comparison of decentralized PSO and subgradient search algorithm on opti-mization convergence.
48
Chapter 4
Distributed Rate Allocation for Video
Mesh Networks
Rate allocation and performance optimization problem is the most important aspect
in multi-hop video mesh networks. Video communication over mesh networks has found
many important applications, including webcam video communication over Internet, video
surveillance, and wireless vision sensor networks. We consider a large number of video ses-
sions sharing a mesh network. Since video streaming is bandwidth-demanding while the
network has a limited bandwidth resource, it is important for us to optimally allocate this
critical network resource among video sessions to maximize the overall system performance
or network utility. This optimization is often a high-dimensional nonlinear constrained opti-
mization problem. For large-scale networks, a centralized and synchronous solution to this
rate allocation and performance optimization problem is too costly, non-scalable, fragile,
and even infeasible in many cases. In this chapter, based on a swarm intelligence principle,
we develop a distributed and asynchronous particle swarm optimization (DAPSO) scheme
for distributed rate allocation and performance optimization [68]. Specifically, we study
the problem of decomposition of a network utility function with global resource parame-
ters and inter-dependent resource constraints into local optimization problems, each being
solved by particle swarm optimization. We develop in-network fusion and particle migra-
tion schemes to exchange information between neighboring PSO modules and coordinate
49
their search behaviors and optimization processes. We also develop a collaborative resource
control scheme to efficiently handle network bottleneck issues. Our extensive simulation re-
sults demonstrate that the proposed DAPSO algorithm is very effective for distributed rate
allocation and performance optimization. The proposed optimization framework is generic
and can be extended to other network resource allocation and performance optimization
problems with slight modification.
4.1 Introduction
A wireless mesh network (WMN) is a set of fixed and/or mobile nodes that self-assemble
into a dynamic multi-hop ad hoc network [69]. In this work, we study collaborative video
communication over a large-scale mesh network where a large number of sender devices
transmit compressed video data, either storage (pre-compressed) or live video data, to a large
number of receivers through multi-hop transmission with packet relay. The communication
link between two neighboring network nodes can be either wired or wireless. This type of
video mesh networking technology is found in many important applications, such as webcam
video communication over Internet / enterprise / community networks, video surveillance,
and image/video sensor networks [69, 70, 30].
To successfully deploy the video mesh networking technology, there are a number of
issues that need to be carefully investigated, including packet routing, flow control, Quality
of Service guarantee, resource allocation, and performance optimization [69]. In this work,
we focus on rate allocation and performance optimization. Video streaming is bandwidth-
demanding while the network has a limited bandwidth resource, it is important for us to
optimally allocate this critical network resource among different video sessions so as to
maximize the overall system performance or network utility. This optimization is often
a high-dimensional nonlinear constrained optimization problem. We propose to develop a
distributed asynchronous optimization scheme to solve the rate allocation and performance
optimization problem. The proposed optimization framework is generic and can be extended
50
to other network resource allocation and performance optimization problems with slight
modification.
4.1.1 Related Work
Rate allocation has been extensively studied in rate-distortion analysis and quality op-
timization for point-to-point video communication [71, 72]. It has also been considered in
the broad context of resource allocation [14]. Although a number of cross-layer resource al-
location and performance optimization schemes have been developed, they mostly focus on
centralized performance optimization for single-stream video communication over networks
with relatively simple topologies (e.g. a chain topology) [70, 73].
Within the context of large-scale mesh networks, especially video mesh networks, the
network resource allocation and performance optimization is often a high-dimensional con-
strained nonlinear optimization problem with a large number of resource parameters (op-
timization variables) [17]. In this case, a distributed and asynchronous solution to the
resource allocation and performance optimization is highly desired [17]. This is because,
first, in large-scale mesh networks, communicating the information that is required by each
step of the optimization algorithm from every network node to a central location often in-
volves a significant communication overhead and a large communication delay. However,
a distributed solution only requires local information exchanges. Second, because of the
large communication delay in gathering global information, a centralized rate allocation
algorithm is often not able to quickly respond to local changes in network conditions and
time-varying video data characteristics. However, a distributed approach has the advantage
of quick response to local changes. Third, a distributed solution is scalable. The resource
allocation and performance optimization procedure can be easily extended when new nodes
are added into the mesh network. Therefore, a distributed and asynchronous optimization
scheme is particularly attractive in large-scale networks [74, 17].
A number of distributed network utility optimization schemes based on prime and La-
grangian dual and gradient search have been developed in the literature [1, 9, 10, 25, 3, 26,
51
38, 75]. For those dual-based distributed algorithms, the Lagrange dual variables can be
considered as prices for network resource allocation [1, 25, 3]. Most of existing approaches,
especially those based on incremental sub-gradient search [9], assume that the objective
utility function to be additive and convex. Convex functions have unique properties. A
local minimum is also a global minimum and the duality gap between the prime and dual
optimization problems is zero [17]. For nonconvex objective functions, it will be very dif-
ficult to assure convergence of the distributed optimization process. However, in many
scenarios of resource allocation and performance, especially in video communication over
mesh networks, the utility function is often non-convex. In some cases, the utility function
even has no explicit expression due to the inherent difficulty in its mathematical modeling
and we can only obtain its function value for a given set of independent variables. How to
develop a distributed and asynchronous scheme to optimize a generic nonlinear objective
function over large-scale networks remains an open and challenging problem. In addition,
with the context of rate allocation and performance optimization for large-scale video mesh
networks, some important issues, such as network bottleneck links and their impact to the
overall video streaming and fast response to time-varying video characteristics, have not
been sufficiently addressed.
However, within the context of video communication over networks, the relationship
between the video quality of service metric (or system performance metric) and resource
utilization parameters are often nonlinear and complex. Therefore, there is a need to develop
a distributed asynchronous optimization algorithm which is able to handle generic nonlinear
network utility functions. In general, the rate allocation and performance optimization for
video communication over network problem is a nonlinear, nonconvex optimization problem.
Existing distributed optimization algorithms based on incremental sub-gradient search [9]
[10] assume that the objective function U(x) is additive and convex, and the algorithms
based on price-based lagrangian dual flow control optimization [1] [25] [3] assume that
the objective function is increasing and strictly concave. Distributed optimization scheme
proposed in [38] for multiple video streams in networks can still only handle the convex
52
optimization problem.
4.1.2 Major Contributions
In this paper, based on a swarm intelligence principle [16], we develop a distributed
asynchronous particle swarm optimization (DAPSO) scheme to solve the rate allocation
and performance optimization problem for large-scale video mesh networks. The major
contributions of this work include: (1) we explore the idea of particle swarm optimization
which provides a natural and ideal platform for distributed resource allocation, collabora-
tive video communication, and network performance optimization. (2) We develop simple
yet efficient schemes for local DAPSO modules to exchange information which enable the
fast convergence of DAPSO. (3) We develop a simple yet efficient scheme to address the
network bottleneck issue in distributed rate allocation and performance optimization. (4)
Unlike many network resource allocation performance optimization algorithms in the litera-
tures which are able to handle convex network utility functions, the proposed optimization
framework is generic, is able to handle nonconvex network utility functions, even does not
require their specific expressions, and can be extended to other network resource allocation
and performance optimization problems with slight modification.
4.2 Resource Allocation for Video Mesh Networks
In this section, we formulate the resource allocation problem for video mesh networks
and then discuss a generic distributed solution for this type of problems.
4.2.1 Formulation of Generic Resource Allocation Problems
In this work, we model the mesh network as a graph with V network nodes V =
{1, 2, · · ·V } and L logical links L = {1, 2, · · ·L}. The mesh network is shared by a set of
video transmission sessions (or streams), denoted by S = {1, 2, · · · , S}, as illustrated in
Fig. 4.1. In a large-scale mesh network, for example, a community or enterprize network
53
supporting online video chatting or conference services, there could be a large number of
simultaneous video sessions crossing over the mesh network and sharing communication
links.
Figure 4.1: Illustration of video communication over mesh networks.
In performance optimization of video mesh networks under resource constraints, all
network nodes need to collaborate in video streaming and network resource utilization
so as to maximize the overall system performance under resource constraints. Let X =
{x1, x2, · · · , xN} be the set of resource parameters. Example resource parameters include
encoding bit rate of a video stream, link transmission rate, transmission power, and delay
bound 1[70, 76]. There are two types of resource parameters, local and global resource pa-
rameters. A local parameter is associated with a specific network node or link, or a local
neighborhood of nodes or links. For example, data processing and transmission power are
two local resource parameters. A global parameter involves a set of networks nodes or links
that spatially distribute over the network. For example, the encoding bit rate of a video
streaming is a global parameter since a video stream involve a series of network nodes and
link along its transmission path.
The network operates under resource constraints. There are two types of constraints:
1In video communication over multi-hop networks, delay is also an important resource since, with alarge delay bound, the network has more flexibility in scheduling so as to improve its data communicationefficiency.
54
independent constraints, such as the energy constraint, which are uniquely associated with
one specific node or link; and inter-dependent constraints, such as flow balance constraints,
which are often associated with a neighborhood of network nodes or links and different
constraints are inter-dependent of each other.
As in the literature, a network utility function, denoted by U, is used to describe the
overall system performance. Since the overall system performance depends on the config-
uration of resource parameters, therefore, U is a function of {x1, x2, · · · , xN}, denoted by
U(x1, x2, · · · , xN). Now, the problem performance optimization under resource constraints
can be formulated as
max U(x1, x2, · · · , xN) (4.1)
s.t. independent constraints
interdependent constraints.
In the following, we will use rate allocation as an example to show how a practical
resource allocation problem could be fitted into the formulation in (4.1).
4.2.2 Basic Framework for Distributed Resource Allocation
Distribution of computation implies decomposition. More specifically, we need to de-
compose the global optimization problem with a global network utility function (objective
function) and all resource constraints into a set of local optimization problems with a lo-
cal objective function and local resource constraints, as illustrated in Fig. 4.2. There are
two basic decomposition methods, primal and dual decomposition. The former is based
on decomposing the original primal problem, whereas the latter is based on decomposing
the Lagrangian dual problem. In dual decomposition, a pricing method is often used to
coordinate the resource utilization between different optimization modules [17].
After decomposition, neighboring optimization modules are allowed to exchange status
55
information through local communication. In optimization over networks, it is highly desir-
able that the optimization modules operate in an asynchronous manner. More specifically,
each local optimization module immediately moves to its next optimization step / itera-
tion using the available status information received so far from neighboring nodes instead
of waiting for synchronous status information exchange with other nodes. If a distributed
asynchronous optimization scheme is effective, after a number of rounds of local optimization
and information exchange, the overall system performance should approach the optimum
obtained by global optimization. If not possible, at least a sub-optimum should be achieved.
From (4.1), we can see that the major task in decomposing a global constrained op-
timization problem into a set of local optimization modules is to decompose the network
utility function with global resource parameters and handle those inter-dependent resource
constraints. To decompose the global utility function, our basic idea is to introduce a set of
local resource parameters to replace the global ones and decompose the network utility func-
tion into local ones which only have local resource parameters. In doing so, we need to make
sure theoretically and/or experimentally that the local network utility functions converge
to the global optimum during distributed optimization. To handle those inter-dependent
resource constraints, our basic idea is to use local communication between optimization
modules to exchange information and let them negotiate with each other to make sure that
the inter-dependent resource constraints are satisfied during distributed optimization.
In the following sections, based on this observation and a swarm intelligence principle
[16], we will develop a distributed asynchronous scheme called DAPSO for rate allocation
and performance optimization for video mesh networks.
4.3 Distributed Rate Allocation for Video Mesh Net-
work
In this section, we first formulate the rate allocation problem within the basic framework
of (4.1). We discuss how this problem can be decomposed into local optimization problems
56
Figure 4.2: Illustration of distributed asynchronous optimization.
to be solved using DAPSO.
4.3.1 Problem Formulation
In rate allocation, the major resource constraint is in the form of link capacity and
the resource parameters are the bit rate of each video stream. While determining the bit
rates of video streams in order to maximize the overall video communication performance,
we need to make sure that the aggregated bit rate of all video streams on every link of the
network does not exceed its link capacity [2, 75]. More specifically, suppose we have N video
streams (sessions) that are sharing communication links and crossing over the network. Let
xn ∈ X = {x1, x2, · · · , xN} be the bit rate of video stream n. Note that xn is a global
resource parameter. Let
XL[l] = {xn ∈ X|video stream n uses link l}. (4.2)
which is the subset of video sessions that use link l.
Let Cl be the link capacity. Here, the link capacity can be considered as the average
number of information bits per second that can be successfully transmitted over the link.
A number of factors, including SIR (signal-interference ratio), transmission distance, com-
munication protocols, forward error correction (FEC) schemes, and packet re-transmission,
57
contribute to this link capacity [77]. According the link capacity constraint, we have
∑
xn∈XL[l]
xn ≤ Cl, 1 ≤ l ≤ L. (4.3)
In the following, we define the network utility function for video communication over
mesh networks. The ultimate goal in video communication system design is to provide users
with the best-quality videos. Therefore, the system performance should be measured by the
overall video quality. A commonly used metric for measuring the quality of a single video
stream is the encoding distortion. As suggested by the literature on rate-distortion (R-D)
modeling for video coding [72], we use the following R-D model
D(xn) = σ2n × 2−λxn , (4.4)
to describe the relationship between video coding distortion D and source rate xn for video
stream n. Here, σ2n represents the picture variance. More specifically, within the context of
motion prediction based video coding, it is the variance of difference picture after motion
compensation. λ is an encoder-related parameter. It should be noted that in this work
we just use the R-D model in (4.4) as an example to demonstrate the proposed DAPSO
algorithm. Certainly, this model can be replaced by any other more accurate R-D model de-
veloped in the literature [72, 78]. As we can see from the following sections, the optimization
procedure of the proposed DAPSO algorithm does not depend on the specific expression of
the R-D model.
Within the context of video communication over mesh networks with a large number of
simultaneous video streams, we need establish a performance metric function, or a network
utility function, to characterize the overall system performance. One commonly used mea-
sure to describe the overall video quality of multiple video streams is the aggregated video
distortion, i.e.,
U(x1, x2, · · · , xN) =N∑
n=1
D(xn) =N∑
n=1
σ2n × 2−λxn . (4.5)
At this moment, we consider stationary video sources and assume that σ2n is constant.
As noted by a number of researchers, when characterizing the overall quality over multiple
58
video streams, besides minimizing the aggregated video distortion, we also need to minimize
the quality variation between different video streams. From a user’s perspective, minimizing
the quality variation is also a very important part of maintaining the fairness among users
and different video services. By incorporating the quality variation into the network utility
function, we have
U(x1, x2, · · · , xN) =N∑
n=1
D(xn) +N∑
n=1
|D(xn)− D|, D =1
N
N∑n=1
D(xn). (4.6)
Now, the rate allocation and performance optimization can be formulated as
min U(x1, x2, · · · , xN) =N∑
n=1
[D(xn) + |D(xn)− 1
N
N∑n=1
D(xn)|]
(4.7)
s.t.∑
xn∈XL[l]
xn ≤ Cl, 1 ≤ l ≤ L. (4.8)
xn ≥ 0, 1 ≤ n ≤ N. (4.9)
Here, the resource constraints in (4.8) are inter-dependent since they involve global resource
parameters in X, while those in (4.9) are independent.
From this rate allocation example we can see that the network utility function in video
communication over networks U(x1, x2, · · · , xN) is often a nonlinear nonconvex (or noncon-
cave) function. For large-scale networks, the performance optimization problems in (4.1) and
(4.7) are often a high-dimensional nonlinear nonconvex constrained optimization problems.
Existing methods developed for convex optimization and existing algorithms for distributed
rate allocation, flow control, and resource allocation, such as distributed gradient based
lagrange dual algorithm, cannot be applied. In this work, based on a swarm intelligence
principle, we develop a distributed and asynchronous particle swarm optimization (DAPSO)
algorithm to solve the constrained nonlinear rate allocation problem in (4.7).
4.3.2 Decomposition
We propose to decompose the global optimization problem in (4.7) into a set of local
optimization modules, each of which is associated with a communication link. More specif-
ically, let L = {1, 2, · · ·L} be the set of links that are involved in the multiple-session video
59
communication, as illustrated in Fig. 4.1. In total, we have L local optimization modules.
For each link, which hosts a local optimization module, we define a set of local resource para-
meters, Xl = {xnl}, where xnl represents the bit rate of video session n at local optimization
module l. We define the l-th local optimization module to be
maxXl
Ul(Xl) =∑
n∈XL[l]
D(xnl) +∑
n∈XL[l]
|D(xnl)− D|, D =1
N
∑
n∈XL[l]
D(xnl).
s.t.∑
n∈XL[l]
xnl ≤ Cl. (4.10)
xnl ≥ 0, n ∈ XL[l].
Note that, for a video session, its bit rates at all links along the transmission path should
be the same. In other words,
xnl = xnk, ∀ l, k ∈ LS[n], (4.11)
where LS[n] is the set of links used by video session n. This is an inter-dependent flow
balance constraint [14]. We can see that the original global optimization problem has been
decomposed into L local optimization modules in (4.12) plus a set of inter-dependent re-
source constraints in (4.13), which will be solved by the proposed DAPSO algorithm to
be presented in the following section. It should be noted that the specific decomposition
procedure depends on the actual problem formulation. However, we observe that the rate al-
location problem in (4.7) is quite representative and many other network resource allocation
problems share similar formulation, often having a network utility function as an optimiza-
tion objective plus inter-dependent resource constraints (e.g. flow balance constraints) and
several local resource constraints (e.g. link capacity and energy constraints) [17].
4.4 Distributed and Asynchronous Particle Swarm Op-
timization
In the proposed DAPSO scheme, each local optimization problem is solved by parti-
cle swarm optimization (PSO). Through local communication, neighboring PSO modules
60
share important information in a distributed and asynchronous manner to expedite the
search process and make sure that the inter-dependent flow balance constraints in (4.13)
are satisfied. To do this, we propose to explore two major ideas: (1) in-network fusion
and particle migration to handle inter-dependent resource constraints and (2) collaborative
resource management to handle network bottleneck issues. In the following sections, we
discuss these design issues in more detail.
4.4.1 Original Optimization Problem Decomposition
In rate allocation, we propose to decompose the global optimization problem in (4.7)
into a set of local optimization modules, each of which is associated with a communication
link. More specifically, let L = {1, 2, · · ·L} be the set of links that are involved in the
multiple-session video communication, as illustrated in Fig. 4.1. In total, we have L local
optimization modules. For each link, which corresponds to a local optimization module,
we introduce a set of local resource parameters, Xl = {xnl}, where xnl represents the bit
rate of video stream n at local optimization module l. We define the l-th local optimization
module to be
min Ul(Xl) = w1 ×∑
n∈XL[l]
D(xnl) + w2 ×∑
n∈XL[l]
|D(xnl)− D∗|, D∗ =1
N∗∑
n∈XL[l]
D(xnl).
s.t.∑
n∈XL[l]
xnl ≤ Cl. (4.12)
xnl ≥ 0, n ∈ XL[l].
Here w1 and w2 are weight parameters for overall video distortion and distortion difference,
and N∗ is the total number of the video streams path link l. Note that, for a video stream,
its bit rate on each link along the transmission path should be the same. In other words,
xnl = xnk, ∀ l, k ∈ LS[n], (4.13)
where LS[n] is the set of links used by video stream n. This is an inter-dependent flow
balance constraint. We can see that the original global optimization problem has been de-
composed into L local optimization modules in (4.12) plus a set of inter-dependent resource
61
constraints in (4.13), which will be solved by the following DAPSO algorithm.
4.4.2 In-Network Fusion and Particle Migration
In the proposed DAPSO algorithm, each local constrained optimization in (4.12) is
solved by the PSO algorithm discussed in Section 2.3.1. Each local PSO module has a set
of local particles moving around in the solution space defined by local constraints in (4.12)
and searching for optimum solution for the local optimization module. Neighboring PSO
modules share status information about their “group-best”, denoted by Xgl , and then use
this external information to guide the movement of their internal particles so as to meet the
inter-dependent resource constraints in (4.13) in a collaborative manner. This is achieved
by two major operations: in-network fusion and particle migration, as illustrated in Fig. 4.3.
Figure 4.3: Distributed and asynchronous PSO algorithm.
More specifically, during in-network fusion, neighboring local PSO modules exchange and
fuse information about their group-best particles so as to generate new group-best particles
which has the following two properties: (1) they still satisfy the independent constraints
in each local PSO module. In other words, they are still in the solution space. (2) They
should satisfy the inter-dependent resource constraints better than each group-best particle.
For example, consider two neighboring PSO modules l and k, 1 ≤ l, k ≤ L. Suppose video
stream n is optimized by both modules xgnl and xg
nk which are the bit rates of video stream
62
n in the group-best particles found by local PSO modules l and k, respectively. During
in-network fusion, these two modules need to negotiate with each other to determine what
is a better bit rate for each, denoted by xgnl and xg
nk, such that the inter-dependent resource
constraint is better satisfied. The specific fusion rule depends on the actual formulation of
the constraint. For example, in the rate allocation case, the following fusion rule can be
used
xnk = xnl = min(xgnk, x
gnl), (4.14)
which are the new group-best particles in local PSO modules l and k. The major reason that
we choose the “min” operation in (4.14) is that those new group-best particles still satisfy
the link capacity constraints in both DAPSO modules. Certainly, with this fusion rule
applied to all local PSO modules and for all video sessions, the new generation of group-best
particles of both modules will satisfy the inter-dependent flow balance constraint better.
Once the group-best particle in the local DAPSO module is updated, guided by the swarm
intelligence principle in (2.19), it will attract the rest particles towards this new location
in the subsequent local search. We call this operation as particle migration because these
local particles move towards a new location due to external factors.
4.4.3 Handling Network Bottleneck Issues Using Collaborative
Resource Control
One of the major challenges in network resource allocation is to deal with bottleneck
links which have very limited bandwidth resources however are shared by a large number
of video sessions. This network bottleneck issue has not be adequately addressed in the
literature. This problem becomes even more difficult in distributed resource allocation
since local optimization modules may not be aware of the bottleneck links in some remote
places of the network. Let us consider the example in Fig. 4.4. The network has four video
sessions. Video sessions s1 and s2 share link l1 whose link capacity is 300 kbps (kilo bits
per second). Sessions s1, s3, and s4 share link l2 whose capacity is only 180 kbps. Let xn,li ,
63
1 ≤ n ≤ 3 and i = 1, 2, be the local bit rate of session sn at local optimization module li.
We have the following two link capacity constraints:
Constraint l1 : x1,l1 + x2,l1 ≤ 300, (4.15)
Constraint l2 : x1,l2 + x3,l2 + x4,l2 ≤ 180. (4.16)
Suppose all video sessions have similar scene statistics. Therefore, in the local optimization
module l1, video session s1 is allocated for 300 / 2 = 150 kbps. However, in module l2, it is
only allocated for 180 / 3 = 60 kbps due to resource competition from video sessions s3 and
s4. s1 has to choose 60 kbps since constraints l1 and l2 have to be both satisfied. In this case,
link l2 becomes the bottleneck link. From this simple example, we can see that bottleneck
links play a critical role in network resource allocation; the overall system performance
is controlled by the resource constraints at bottleneck links; and resource constraints at
non-bottleneck links, such as that in (4.15), become less critical or even useless.
Figure 4.4: An example of bottleneck link in a multi-hop mesh network.
To effectively handle bottleneck link in distributed rate allocation, there are two major
issues that need to be carefully addressed. First, the information of bottleneck links needs to
be properly propagated and shared with other local optimization modules. Second, we need
to develop a collaborative resource management scheme to coordinate the resource utiliza-
tion between video sessions. Video sessions should yield their resources at non-bottleneck
64
links to those with bottleneck links. For example, in the example of Fig. 4.4, at link l1,
the local DAPSO module allocates 150 kbps for session s1. However, s1 can not use this
much bandwidth resource because of its bottleneck at another link l2. In this case, within
the local DAPSO module of l1, s1 should yield part of its unused bandwidth resource to
s2. To this end, we need to develop a scheme to intelligently control and manage the re-
source utilization at each link in a collaborative manner. Otherwise, the video sessions at
non-bottleneck links will keep pushing their own resource allocation toward their capacity
limits, which might significantly slow down the convergence process.
It should be noted that the bottleneck is session-specific. In other words, a link is a
bottleneck for one video session but may not be the bottleneck for another one. For example,
session s3 may have a bottleneck at link l3 if the link capacity of l3 is very small. On the
other hand, a non-bottleneck link for one video session might be a bottleneck for another.
For example, link l1 is a non-bottleneck link for session s1 but it can be a bottleneck link for
session s2, as illustrated in Fig. 4.4. In addition, since many video sessions are competing
for the network resources, as the network resource allocation are gradually adjusted during
distributed resource allocation, the bottleneck for a video session might shift from one link to
another. This session-specific and dynamic bottleneck issue presents a significant challenge
in distributed resource allocation.
To address this bottleneck issue, we propose a method called collaborative resource con-
trol. Our basic idea is to introduce a resource budget window for each resource parameter
at each local DAPSO module. For example, the resource parameter xnl is the bit rate of
video session n in the l-th PSO module. We impose the following resource budget window
on xnl:
C−nl ≤ xnl ≤ C+
nl, (4.17)
where C−nl and C+
nl will be adjusted by the following particle migration procedure. The
window size is denoted by L = C+nl − C−
nl. Initially, we can set C−nl to be 0 and C+
nl to the
link capacity so that the resource budget window does not affect the original link capacity
65
constraints in (4.12). During distributed rate allocation, we propose to use the linearly-
increase and multiplicative-decrease rule which is used in TCP network control [79] to
adjust the resource budget window. More specifically, in the proposed scheme, a bottleneck
link is determined based on the following condition: if
xgn,∗ = min
k∈LS [n]xg
n,k = xgn,l, (4.18)
where xgn,k is the bit rate of video session sn in the group-best particle at PSO module k,
we claim link l is now the bottleneck for sn. In this case, we linearly increase the window
size L by a fixed amount ∆L. (In this work, ∆L is empirically chosen.) This is because, by
increasing the resource budget window size L for video session sn on this bottleneck link,
the constraint on its bit rate xn,l is further relaxed, thus video session sn has more flexibility
in competing for more system resource with other video sessions on this link during the
local optimization process. On the other hand, if link l is not the bottleneck link for session
sn according to the criteria in (4.18), we reduce the budget window size L by half and
center the resource window around xgn,∗. Note that link l is not the bottleneck link for video
session sn, which implies that the resource constraint on the bit rate xn,l is too loose as that
in (4.15). In this case, we need to reduce its resource budget window so that it can yield
part of its resource to other competing video sessions during the local optimization process.
Once the resource budget window is modified, it will serve as a constraint to regulate
the particle movement during PSO at link l. It is well known that the linearly-increase
and multiplicative-decrease rule in TCP is able to converge to fair bandwidth allocation
between different network communication sessions [79]. During our experiments, we also
observe that it has a similar behavior in DAPSO, However, at this moment, we are not able
to theoretically justify this convergence behavior since the whole DAPSO process is much
more complicated than network transmission control.
4.4.4 Algorithm Description of DAPSO
66
Figure 4.5: Illustration of particle migration.
In this section, we summarize the DAPSO algorithm. As discussed in Section 4.3.2, we
decompose the centralized rate allocation problem in (4.7) into a set of local optimization
problems in (4.12). Each local optimization problem is solved with PSO. The proposed
DAPSO operates at each link, performing the following three major tasks: (A) allocating
the link bandwidth among video sessions that share this link; (B) transmitting its group-
best to its neighboring DAPSO modules; (C) receiving the group-best information from
its neighboring DAPSO modules and using them to guide its own particle movement and
adjusting the resource budget window, as explained in Sections 4.4.2 and 4.4.3. In the
following, we outline the major actions performed by each DAPSO module.
Step 1 Initialization. Initialize the local PSO module by randomly generating a set of
particles in the solution space [16]. The total number of particles in each PSO
module can be adjusted. In our simulations, we set it to be 10. Also initialize the
resource budget window as explained in Section 4.4.3.
Step 2 Local optimization. Each local DAPSO module moves its own particles in the
solution space based on the swarm intelligence principle in (2.19) and records its
group-best particle.
Step 3 Sharing group-best information. If the group-best particle has been updated, the
67
DAPSO module transmits this group-best information to its neighboring DAPSO
modules through local communication. To control the amount of communication
overhead, this information can be transmitted only when the amount of change in
group-best is larger than a given threshold. It should be noted that this information
sharing is asynchronous: it is shared only when it is updated and different DAPSO
modules may update their group-best information at different time instances.
Step 4 Particle migration and fusion. Upon receiving the group-best information from its
neighbors, each DAPSO module updates its group-best based on the fusion rule in
(4.14). It also updates its resource budget window as explained in Section 4.4.3.
Step 5 Iteration and exit. Repeat Steps 2-4 until the change in local utility value (e.g.
total video distortion) is below a given threshold.
In Section 3.2.3, we will present an extensive set of simulation results to demonstrate the
efficiency of the proposed DAPSO algorithm and experimentally show that the algorithm
converges very fast.
4.5 Experimental Results
In this section, we evaluate the proposed DAPSO algorithm with different scenarios of
video streaming over mesh networks. We will experimentally study its convergence behav-
ior and response to video content change. We will compare the DAPSO algorithm with
distributed gradient search, which is extensively used in the literature for network utility
maximization [17].
4.5.1 Simulation Setup
We simulate various scenarios of video streaming over wired mesh networks, for exam-
ple, webcam video communication over community networks or Internet. We randomly
generate a network of video communication nodes, as well as a set of video sessions with
68
randomly selected source and destination nodes. We then use existing shortest-path routing
algorithm to determine the routing path of each video session [80]. During each simulation,
we randomly set the link capacity of each link to a value between 100 kbps and 500 kbps.
For each video session, we also randomly set its picture variance σ2 in (4.4) to be a value
between 100 and 800. Using MATLAB, we write a discrete event simulator to simulate the
DAPSO algorithm outlined in Section 4.4.4 at each link and the asynchronous local commu-
nication between neighboring DAPSO modules for information sharing. The whole DAPSO
procedure consists a sequence of iterations. Within each iteration, the local DAPSO module
moves its particles based on the swarm intelligence principle in (2.19). In our simulations,
we assume that the one-hop communication delay is fixed at one time unit (e.g., 5 millisec-
onds). The DAPSO algorithm will perform distributed rate allocation and determine the
bit rate for each video session. It should be noted that the purpose of these simulations is to
test the performance of the DAPSO algorithm instead of validating a video mesh network
system. Certainly, the simulation can be further extended to incorporate more video encod-
ing and network communication features. However, we believe that our current simulation
setup is sufficient for demonstrating the performance of the proposed DAPSO algorithm.
4.5.2 Convex Distributed Rate Allocation and Performance Op-
timization Application
Fig. 4.6 depicts an example network topology with 16 nodes and 15 links. Here, we have
6 video sessions communicating over the network.
The DAPSO WVSN distortion optimization problem [81] can be formulated as
D =L∑
l=1
∑
n∈S(l)
D(xn) (4.19)
where L is the total link number and S(l) includes the sources s ∈ {1, ..., S} path link l.
The other parameters used in DAPSO are set as follows: link capacity cl = 100, particle
size = 20, PSO update iteration = 10, initial window size = 0.3× cl, decrease window step
69
size = 0.7, and increase window step size = 2. The parameters used in WVSN are set as
follows: λ = 0.023, and σ21 = 100, σ2
2 = σ26 = 200, σ2
3 = σ25 = 400, σ2
4 = 800.
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
X Position
Y P
ositi
onDestination Node Source
A
B
C
Figure 4.6: A randomly generated video mesh network with 16 nodes, 15 links, and 6 videosessions.
At the end of each iteration, we compute the average, minimum, and maximum utility
values of 6 video sessions at all local DAPSO modules. We obtain the global optimum
solution using brute-force search. We compare DAPSO algorithm solution with centralized
algorithm solution which is theoretical optimal solution and is impractical in the real net-
work. Fig. 4.7 shows that the average, minimum, and maximum utility values all converge
to the global optima. When the DAPSO solution converges, all video sessions have simi-
lar quality as expected in the objective function in (4.7). This implies that the proposed
DAPSO is quite efficient.
To show more detail about the distributed rate allocation and interactions between
neighboring DAPSO modules, especially those at bottleneck links, we choose three links,
A, B, and C, as shown in Fig. 4.6 and plot the local bit rates of those video sessions that
pass through these three links in Figs. 4.8, 4.9, and 4.10, respectively. Here, we only
70
show the local bit rate from the group-best particle. We can clearly see that link A is a
bottleneck link for session 5 and more system resource (bandwidth) of link C is shifted
from session 5 to session 6 during the information sharing and resource negotiation process.
When the DAPSO solution converges, all video sessions have similar quality as expected in
the objective function in (4.7). This implies that the proposed DAPSO is quite efficient.
0 10 20 30 40 50 60 70 80 90 1000
5
10
15
20
25
30
35
Iteration Number
Util
ity F
unct
ion
Val
ue
DAPSO Utility Function Ave ValueDAPSO Utility Function Min ValueDAPSO Utility Function Max Value
Theoretical Utility Function Value
Figure 4.7: Convergence of network utility functions of DAPSO to the global optima ofconvex function.
4.5.3 Nonconvex Distributed Rate Allocation and Performance
Optimization Application
The system model used in the previous experiments is still a convex utility function, next
we change the system model to a nonconvex utility function to do the similar experiments
again.
We use the same R-D behavior analysis model for the video compression. We assume that
the performance of the whole WVSN is not only measured by the overall video distortion,
71
0 10 20 30 40 50 60 70 80 90 10018
20
22
24
26
28
30
Iteration Number
Sou
rce
Rat
eSource 1Source 2Source 3Source 5
Figure 4.8: The traces of the particles moving in critical link A of convex function.
but also by the difference between every source. The system model [82] can be changed as
follows:
D = w1×L∑
l=1
∑
n∈S(l)
D(xn) + w2×L∑
l=1
∑
n∈S(l)
|D(xn)−Dl| (4.20)
Here w1 and w2 are weight parameters for overall video distortion and distortion difference.
Dl is the average video distortion on link l in the network.
We test the proposed distributed and asynchronous optimization scheme using DAPSO
with different WVSN topologies. In the following experiment, the sensor network topology
is same as the previous experiment. The old parameters used in DAPSO are same as the
previous experiment, and new parameters w1 and w2 are set as follows: w1 = 1, w2 = 0.1.
Similar as the previous experiment, we compute the average, minimum and maximum
nonconvex utility function value of each video session at all local DAPSO modules. Fig. 4.11
shows the average, minimum, and maximum utility values all converge to the global optima.
Similarly, to show more detail about the distributed rate allocation and interactions between
neighboring DAPSO modules, especially at bottleneck links A, B, and C which we mentioned
72
0 10 20 30 40 50 60 70 80 90 10015
20
25
30
35
40
45
50
55
60
Iteration Number
Sou
rce
Rat
e
Source 1Source 2Source 4
Figure 4.9: The traces of the particles moving in critical link B of convex function.
before. Fig. 4.12, 4.13, and 4.14 show the local bit rates of those video sessions that pass
through these three links respectively. Here, we only show the local bit rate from the group-
best particle. And the bottleneck links’ capacity usage percentage for link A, B and C are
near 100%, 84% and 65% when the whole network is balanced.
The weight parameters here will control the optimization results, when we change the
parameters to: w1 = 1 and w2 = 1, Fig. 4.15 shows the average, minimum, and maximum
utility values all converge to the global optima. And Fig. 4.16, 4.17, and 4.18 show the
local bit rates of those video sessions that pass through these three bottleneck links, A, B,
and C respectively. Here we only show the local bit rate from the group-best particle. And
the bottleneck links’ capacity usage percentage for link A, B and C are near 68%, 52% and
36% when the whole network is balanced. We can see that when we want the weight of
source rate difference equals to the weight of source rate distortion, the link capacity usage
percentage drops very quickly.
73
0 10 20 30 40 50 60 70 80 90 10025
30
35
40
45
50
55
60
65
70
75
Iteration Number
Sou
rce
Rat
eSource 5Source 6
Figure 4.10: The traces of the particles moving in critical link C of convex function.
4.5.4 Comparison with Gradient-Based Lagrangian Dual Algo-
rithms
Many existing methods for network utility maximization are based Lagrangian dual and
distributed gradient or sub-gradient search [17]. They often assume that the utility function
is convex. This is because for convex functions, a local minimum is also a global minimum
and the duality gap between the prime and dual optimization problems is zero. In this
way, the decomposition of the centralized optimization problem can be performed on the
Lagrangian dual using a pricing approach where a resource price is set for each subproblem
to coordinate the resource utilization behaviors of local optimization modules [17].
In this section, we compare the proposed DAPSO algorithm with gradient-based La-
grangian dual algorithms on convex network utility functions. To this end, we remove
the video quality smoothness term∑N
n=1 |D(xn) − D| from the network utility function
U(x1, x2, · · · , xn) in (4.6). With this new convex network utility function, we apply the
gradient-based Lagrangian dual approach [17] to solve rate allocation problem in (4.7) and
74
0 10 20 30 40 50 60 70 80 90 1000
5
10
15
20
25
30
35
Iteration Number
Non
−con
vex
Util
ity F
unct
ion
Val
ue
DAPSO Non−convex Utility Function Ave ValueDAPSO Non−convex Utility Function Min ValueDAPSO Non−convex Utility Function Max Value
Figure 4.11: Convergence of network utility functions of DAPSO to the global optima ofnonconvex function: w1=1 and w2=0.1
compare it with the proposed DAPSO algorithm.
The Lagrangian algorithm is defined as follows
L(R,p) =L∑
l=1
∑
i∈S(l)
Di +L∑
l=1
pl(∑
i∈S(l)
Ri − cl) (4.21)
where pl is the price for link l. Using the gradient-based method [27, 28, 29], the link price
pl can be adjusted as follows
pl(t + 1) = [pl(t) + α∂L
∂pl
(R(t),p(t))]∗ (4.22)
where α is a parameter for step size, [x]∗=max{x, β}, and β is the lower bound of link price.
Since Di is strictly convex and L(R,p) is continuously differentiable, we have
∂L
∂pl
(R(t),p(t)) =∑
i∈S(l)
Ri − cl = Rl − cl (4.23)
where Rl is the aggregate source rate at link l. Substituting (4.23) to (4.22), we have the
price adjustment scheme for link l ∈ L:
pl(t + 1) = [pl(t) + α(Rl − cl)]∗ (4.24)
75
0 10 20 30 40 50 60 70 80 90 10018
20
22
24
26
28
30
Iteration Number
Sou
rce
Rat
eSource 1Source 2Source 3Source 5
Figure 4.12: The traces of the particles moving in critical link A of nonconvex function:w1=1 and w2=0.1
According to the optimum condition, ∂L/∂Ri = 0, we can update the bit rate as follows:
Ri(t + 1) =ln( pi(t)
λσ2i ln 2
)
ln 2· 1
−λ(4.25)
where
pi(t) =∑
l∈L(i)
pl(t) (4.26)
is the overall price for video session i on its transmission path. The above procedure
is repeated until the overall network utility function converges to the global optima. It
should be noted that, in distributed gradient search, the price information pl(t) should be
propagated along the video transmission path so that the overall price pi(t) in (4.26) can be
computed. 2 Furthermore, the convergence behavior of distributed gradient search depends
on the initial settings of starting point and link pricing.
2In essence, this is similar to the in-network fusion in the DAPSO algorithm because all effective dis-tributed performance optimization schemes require local information sharing and propagation.
76
0 10 20 30 40 50 60 70 80 90 10015
20
25
30
35
40
45
50
55
Iteration Number
Sou
rce
Rat
e Source 1Source 2Source 4
Figure 4.13: The traces of the particles moving in critical link B of nonconvex function:w1=1 and w2=0.1
We compare the distributed gradient search against the DAPSO algorithm on the ex-
ample in Fig. 4.6 with a convex network utility function as discussed in the above. For
distribution gradient search, we set α = 0.5× 10−3, β = 1.5× 10−4, and the initial prices to
be 0.2, 0.5 and 0.8 in different experiments. Fig. 4.19 shows the convergence behaviors of
DAPSO and distributed gradient search with initial price setting at 02, 0.5, and 0.8. It can
be seen that the DAPSO converges to the global minimum much faster than the distributed
gradient search. It should be noted that, in distributed gradient search, the link capacity
constraint is enforced through link pricing (or penalty) [17]. Therefore, when the link price
is low (e.g. 0.2), video sessions attempt to use higher bit rates (link bandwidth). In this
way, the overall bit rate will exceed the link capacity and the overall network utility (video
distortion) is even lower than the optimum case, as we can see in Fig. 4.19. Fig. 4.20 show
the average link bandwidth that has been over used by video sessions. The saw effect is
caused by link price update.
77
0 10 20 30 40 50 60 70 80 90 10028
30
32
34
36
38
40
42
44
Iteration Number
Sou
rce
Rat
e
Source 5Source 6
Figure 4.14: The traces of the particles moving in critical link C of nonconvex function:w1=1 and w2=0.1
4.6 Discussion and Conclusion
In this paper, based on the swarm intelligence principle, we have developed a distributed
asynchronous scheme for rate allocation and performance optimization of video mesh net-
works. We have studied the problem of decomposition of objective functions with global re-
source parameters and inter-dependent resource constraints. We have developed in-network
fusion and particle migration schemes for neighboring DAPSO modules to exchange their
group-best information to guide their local particle movements in search for the optimum
solution. By adjusting the resource budget window, we have developed a scheme to ad-
dress the network bottleneck issue. Our extensive simulation results demonstrate that the
proposed DAPSO algorithm is very efficient in distributed rate allocation and performance
optimization.
To test the response of DAPSO to time-varying video content, we double the variance
of video session 3 from 120 to 240 in the middle of simulation. Fig. 4.21 shows how the
78
0 50 100 15030
40
50
60
70
80
90
100
110
120
Iteration Number
Non
−con
vex
Util
ity F
unct
ion
Val
ue
DAPSO Non−convex Utility Function Ave ValueDAPSO Non−convex Utility Function Min ValueDAPSO Non−convex Utility Function Max Value
Figure 4.15: Convergence of network utility functions of DAPSO to the global optima ofnonconvex function: w1=1 and w2=1
DAPSO adjusts its local resource allocation and re-converge to another optimum point. We
test DAPSO algorithm under different network topologies. Fig. 4.22 shows different wireless
sensor network topologies with different number of sessions, there are 3, 4, 5, 6, 7, 8 sessions
in sequence for our test. And Table.4.1. gives the total video distortion and communication
cost for DAPSO algorithm and centralized optimization algorithm using PSO under these
different topologies. We can easily see that the DAPSO algorithm achieves better result
and has very low communication cost compared with the centralized optimization algorithm.
Centralized optimization algorithm using PSO is that we assume there exist one node which
knows the whole network information, this algorithm is impractical and too cost in the actual
networks. Figs. 4.23 to 4.25 show more convergence results for DAPSO on other examples
of random network topologies.
Compared with other methods, DAPSO has the following advantages:
(1) the algorithm is simple; (2) the algorithm is powerful, and DAPSO’s convergence speed
is very fast; (3) there is no predefined limitation on the objective function, not same as
79
0 50 100 1505
10
15
20
25
30
Iteration Number
Sou
rce
Rat
e
Source 1Source 2Source 3Source 5
Figure 4.16: The traces of the particles moving in critical link A of nonconvex function:w1=1 and w2=1
the other primal-dual based optimization algorithm ; and (4) the algorithm is asynchronous
because each local module only checks its neighbor communication information at special
time.
80
0 25 50 75 100 125 15010
12
14
16
18
20
22
24
26
Iteration Number
Sou
rce
Rat
eSource 1Source 2Source 4
Figure 4.17: The traces of the particles moving in critical link B of nonconvex function:w1=1 and w2=1
0 25 50 75 100 125 15014
16
18
20
22
24
26
28
30
32
34
Iteration Number
Sou
rce
Rat
e
Source 5Source 6
Figure 4.18: The traces of the particles moving in critical link C of nonconvex function:w1=1 and w2=1
81
0 5 10 15 20 25 30 35 40 45 50 55 0 5101520250
10
20
30
40
50
60
70
80
90
100
110
120
Iteration Number
Util
ity F
unct
ion
Val
ue
Lagrangian Dual (0.2)Lagrangian Dual (0.5)Lagrangian Dual (0.8)DAPSO
Figure 4.19: Comparison between DAPSO and distributed gradient search with differentstarting price setting.
0 10 20 30 40 50 60 70 80 90 1000
1
2
3
4
5
6
7
8
9
Iteration Number
Beyo
nd L
ink
Cap
acity
Rat
e
Figure 4.20: The total source rate beyond link capacity during iteration.
82
0 20 40 60 80 100 120 140 160 180 2000
5
10
15
20
25
30
35
Iteration Number
Util
ity F
unct
ion
Val
ue
DAPSO Utility Function Ave ValueDAPSO Utility Function Min ValueDAPSO Utility Function Max Value
Changes in videovariance
Figure 4.21: Response of DAPSO to video content change.
0 50 1000
20
40
60
80
100
0 50 1000
20
40
60
80
100
0 50 1000
20
40
60
80
100
0 50 1000
20
40
60
80
100
0 50 1000
20
40
60
80
100
0 50 1000
20
40
60
80
100
Figure 4.22: Different wireless sensor network topologies with different number of sessions.
83
Table 4.1: Total distortion and communication cost for DAPSO and centralized optimizationalgorithm under different topologies.
DAPSO Centralized PSOTotal Distortion Comm. Cost Total Distortion Comm. Cost
1.8602 70 1.8764 1681.9635 80 2.3979 2644.0408 110 4.1624 3004.2106 168 5.7492 45614.6613 112 20.7778 21015.0269 192 23.1137 528
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
X Position
Y P
osi
tion
0 10 20 30 40 50 60 70 80 90 100 1100
10
20
30
40
50
60
70
80
90
100
Iteration Number
Util
ity F
unct
ion V
alu
e
DAPSO Utility Function Ave ValueDAPSO Utility Function Min ValueDAPSO Utility Function Max Value
(a) (b)
Figure 4.23: (a) An example video mesh network; and (b) convergence of network utilityfunction with DAPSO.
84
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
X Position
Y P
osi
tion
0 10 20 30 40 50 60 70 80 90 100 1100
50
100
150
200
250
300
350
Iteration Number
Util
ity F
unct
ion V
alu
e
DAPSO Utility Function Ave ValueDAPSO Utility Function Min ValueDAPSO Utility Function Max Value
(a) (b)
Figure 4.24: (a) An example video mesh network; and (b) convergence of network utilityfunction with DAPSO.
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
X Position
Y P
osi
tion
0 10 20 30 40 50 60 70 80 90 100 1100
50
100
150
200
250
300
350
Iteration Number
Util
ity F
unct
ion V
alu
e
DAPSO Utility Function Ave ValueDAPSO Utility Function Min ValueDAPSO Utility Function Max Value
(a) (b)
Figure 4.25: (a) An example video mesh network; and (b) convergence of network utilityfunction with DAPSO.
85
Chapter 5
Distributed Resource Allocation for
Wireless Video Sensor Networks
Resource allocation is an important research topic in wireless networks. Bandwidth
and energy are two principle wireless sensor network resources, and the main challenge
in designing wireless sensor networks is to use network resources as efficiently as possible
while providing the Quality-of-Service required by the users [83]. For wireless video sensor
networks (WVSNs), the video sensor data is voluminous and the video data compression is
computationally intensive and energy consuming, which consumes about 60-80% of the total
energy supply. Therefore, the performance optimization problem of a large-scale WVSN
under the severe bit and energy resource constraints is a nonlinear, constrained optimization
problem. In this chapter, we present an energy efficient distributed and asynchronous
PSO (EEDAPSO) algorithm to solve the resource allocation and performance optimization
problem over wireless video sensor network. We first decompose the original optimization
problem into several sub-optimization problems, and next design the properly algorithm to
handle the interdependent constraints and do the performance optimization. Compared with
the centralized algorithm, our simulation results demonstrate that this proposed distributed
and asynchronous resource allocation and performance optimization scheme is very efficient
and it only needs very low communication cost.
86
5.1 Introduction
Energy-Efficient is a key concern in wireless sensor networks [84]. For wireless video sen-
sor networks (WVSNs), there are many challenges of wireless video transmission, including
higher error rate, wireless channel bandwidth limitations, and battery energy constraints.
And future wireless sensor networks are expected to support a variety of services with di-
verse quality-of-service (QoS) requirements. Bandwidth and energy are the two principal
wireless sensor network resources, and the main challenge in designing wireless networks is
to use network resources as efficiently as possible while providing the QoS required by the
users [83].
Wireless sensor networks have been envisioned for a wide range of applications, such as
battlefield intelligence, environmental tracking, and emergency response[13]. Each sensor
node has limited computational capacity, battery supply and communication capability.
Compared with the traditional cellular networks, the wireless sensor networks are self-
organized, highly dynamic, with each communication node serving as both servers and
routers for data transmission and have following advantages, such as: system setup and
maintenance is remarkably reduced, data processing and interpretation can be distributed
across the sensor nodes, and systems become more fault tolerant. But there are also some
limitations should be considered: communication bandwidth available on the nodes are
limited, and each nodes has restricted battery life.
In this work, we focus on resource allocation and performance optimization based on
bandwidth and energy. Video streaming compression and transmission are bandwidth and
energy demanding while the network has a limited bandwidth resource and each video
sensor has a limited energy resource. It is important for us to optimally allocate these crit-
ical network resources among different video sessions so as to maximize the overall system
performance or network utility. We propose to develop an energy-efficient distributed asyn-
chronous optimization scheme to solve the resource allocation and performance optimiza-
tion problem over wireless video sensor networks. The proposed optimization framework is
87
generic and is able to handle nonconvex network utility functions.
5.1.1 Related Work
Currently, compressed video data represented the dominant source of internet traffic.
Truly portable communications drive much of the development of wireless technology. The
end users require a compact interface to the network in the form of a pocket-sized, and
battery-powered terminal. As such, power control [85] is a critical issue in the design of
wireless system. The resource allocation including power control, joint source and channel
coding for video transmission over wireless networks has been studied in [86]. And a cross-
layer packet scheduling scheme that streams pre-encoded video over wireless downlink packet
access networks to multiple users is presented in [87].
Wireless sensor networks recently have received extensive research interests, and have
been shown in many different areas, including environment monitoring[88, 89], protocol
design[90, 91], minimum energy routing[92], and power management topology control[93,
94]. Each of these algorithms is developed to improve the efficiency of the WSN. The
implementation of WVSN for the environment monitoring applications have been shown
in [88, 89]. Paper[90] presents a mobile ad hoc protocol design, its routing algorithm is
quite suitable for a dynamic self starting network, as required by users wishing to utilize
ad-hoc networks. Paper[91] highlights the research challenges in video sensor networks
and the protocols development of specific network layer and MAC layer. The work in [92]
presents the algorithms to select the routes and the corresponding power levels such that
the network lifetime is maximized. And the work presented in [93, 94] focus on the topology
control process for optimal base station location under topological network lifetime.
The framework to study radio resource management in a variety of wireless networks
with different service criteria based on game-theoretic approaches has been presented in
[83]. This approach focuses on infrastructure networks where users transmit to a common
concentration and distributed algorithms with emphasis on energy efficiency. For game-
theoretic approaches, the choice of the utility function has a great impact on the nature of
88
the resource allocation and performance optimization. Paper[95] represents a frame work of
power management for energy efficient wireless video communication, and the algorithms
are based on Lagrangian relaxation. However, they still require the objective function to
be convex, and use lagrangian relaxation to handle the constrained optimization problem.
Paper [96] studies the problem of energy minimization for data gathering over a multiple-
sources single-sink communication substrate in wireless sensor networks by exploring the
energy-latency tradeoffs using rate adaptation techniques. But the optimization problem
in this article is still formulated as a convex programming problem which is solvable in
polynomial time by using general optimization tools. The work presented in [97] studies
the utility-based maximization for wireless resource allocation in the downlink direction
of a wireless networks with a central control system, such as a cellular base station or
access point. And the resource allocation and outage control for solar-powered WLAN
mesh networks is considered in [98].
5.1.2 Major Contribution
The ultimate goal for communication system design is to optimize the whole system
performance under given networks resources. For wireless video sensor networks, the goal
is to utilize the limited resources of the network, such as transmission bandwidth, energy
supply and computational capability of each sensor node, in the highest efficient way to
reach the video quality as good as possible. In the traditional video communication systems,
the rate-distortion (R-D) model has been applied to analyze the performance of the video
communication system under bandwidth constraints. In general, the energy consumption
issue is not a major concern because the video processing can be done offline with additional
power supply. In WVSNs, we apply the power-rate-distortion (P-R-D) model[99] to analyze
the signal processing and communication behavior of the video communication system under
bandwidth and energy supply constraints. In the WVSNs, the two major operations on each
video sensor are video compression and wireless transmission. The energy supply of each
video sensor node is mainly used by video compression and wireless transmission. And the
89
wireless transmission is also restricted to the transmission bandwidth.
In this chapter, based on a swarm intelligence principle [16], we develop a distributed
asynchronous energy efficient particle swarm optimization (EEDAPSO) scheme to solve the
resource allocation and performance optimization problem for large-scale video mesh net-
works based on analyzing the power-rate-distortion (P-R-D) model of the video encoding,
video distortion, and transmission power consumption for the wireless video communica-
tion. The major contributions of this work include: (1) we explore the ideal of particle
swarm optimization which provides a natural and idea platform for distributed resource
allocation, collaborative video communication, and network performance optimization. (2)
We decompose the original resource allocation and optimization problem into several sub-
optimization problems, and develop simple yet efficient schemes based on PSO for each local
optimization modules to exchange information which enable the fast convergence. (3) We
also develop a simple yet efficient scheme to address the network bottleneck resource issue
in distributed resource allocation and performance optimization. (4) Compared with many
other network resource allocation performance optimization algorithms in the literatures
which are only able to handle convex network utility functions, the proposed optimization
framework is generic, is able to handle nonconvex network utility functions, even does not
require their specific expressions, and can be extended to other network resource allocation
and performance optimization problems with some modification.
5.2 Energy Efficient Resource Allocation and Perfor-
mance Optimization
In wireless video sensor networks, the raw video data captured by a video sensor is
extremely high. Without the efficient compression before transmission, the requirement
of network bandwidth and power consumption for wireless transmission will be enormous.
Experimental studies in [100, 101] show that video encoding consumes about 60-80% of
the total energy supply, even for small picture sizes, such as QCIF (176 × 144) videos.
90
Therefore, for WVSNs, the major role of the video sensor is to efficiently compress the
raw video sensor data and send out the compressed video stream. Here, a commonly used
measurement for video quality is the end-to-end distortion D which is the mean square error
(MSE) between the original picture captured by the sensor node and the received picture at
the transmission destination node [99]. In video transmission over WVSNs, the end-to-end
distortion D consists of two parts, source encoding distortion Ds and transmission distortion
Dt. Ds is caused by lossy video compression, and Dt is caused by transmission errors.
Intuitively, the source encoding distortion Ds is a function of video encoding power Ps
and source coding bit rate Rs, denoted by Ds(Ps, Rs). The transmission distortion Dt is
a function of transmission power Pt, transmission bit rate Rt and transmission distance d,
denoted by Dt(Pt, Rt, d). Therefore, the energy efficient resource allocation and performance
optimization of one wireless video stream can be formulated as follows:
min D = Ds(Ps, Rs) + Dt(Pt, Rt, d) (5.1)
s.t. Rt ≤ Cl
Ps + Pt ≤ P0
where P0 is the initial power supply of the sensor node, Cl is the link capacity of link l in
the video stream transmission path. Here we do not consider channel coding, so Rs =
Rt. To handle this kind of optimization problem, we need to analyze the power-rate-
distortion (P-R-D) behavior of the video encoding distortion Ds and transmission error
behavior of transmission distortion Dt. We first provide the wireless channel model for video
transmission over WVSNs, and next give the analysis of the P-R-D model and transmission
error behavior.
5.2.1 Channel Model
In the wireless video sensor networks, we consider the flat Rayleigh fading channels
process. It can be shown that for coherent binary phase shift keying (BPSK) modulation,
91
the bit error rate (BER) pb at the receiver is given by [77]
pb =1
2(1−
√Γ
1 + Γ) (5.2)
where Γ =Er
b
N0α2. Here Er
b is the received energy per bit at receiver sensor node, N0 is
the noise power spectrum density, and α2 is the expected value of the square of Rayleigh
distributed channel gain.
We use the free space propagation model to predict received signal strength. The free
space power received is given by [77]
Pr =GtGrλ
2
(4π)2κ× d−2 × Pt (5.3)
where Pt is the transmitted power, Pr is the received power, Gt is the transmitter antenna
gain, Gr is the receiver antenna gain, λ is the wavelength, κ is the system loss factor not
related to propagation (κ ≥ 1), and d is the distance between transmitter and receiver
sensor nodes. Bit energy is decided by the transmission power and rate, it can be written
as Eb = P/R.
In wireless channels, we only consider packet loss due to unrecoverable bit errors. A
packet will be treated as lost only if the corrupted bits in the packet can not all be recovered.
Suppose the bit errors of a video packet are independent, the packet loss probability (PLP)
is given by
p = 1− (1− pb)BN (5.4)
where BN is the number of bits in the video packet. Overall, the packet loss probability
depends on the state of the radio link, and the transmission power used.
5.2.2 Power-Rate-Distortion Model and Transmission Behavior
Analysis
92
The end-to-end distortion is given by D = Ds+Dt, where Ds is video encoding distortion,
and Dt is transmission error distortion. This end-to-end distortion D describes the complex
behavior of errors in the video encoding approach and transmission system.
The P-R-D model for power-scalable video encoding is given as follows:
Ds = σ2e−λ·Rs·g(Ps) (5.5)
where σ2 is the variance of the source data , λ is a model parameter related to encoding
efficiency, and g(.) is the inverse function of the power consumption model Φ(.) of the
microprocessor [99]. Here we choose the power consumption model as Ps = Φ(Cs) = C32s
(Cs represents the encoder complexity) or g(Ps) = P23
s [102]. Comparing the P-R-D model
with the R-D model mentioned in [103], we can see that the P-R-D model considers the
encoding efficiency of the video encoder.
The transmission distortion analyzes the behavior of transmission errors. Therefore, the
whole system transmission distortion model needs to consider the complexity of error prop-
agation in video transmission over lossy channel [104, 105], and specific channel conditions
such as channel bandwidth and BER. It also needs to consider the complex data represen-
tation and coding scheme employed by the video encoder, error resilience and concealment
methods, as well as the operating mechanism of the video decoder [106].
The average transmission distortion Dt of a sequence of video frames is given by [99]
Dt = θp
1− pFd(n) =
a
1− b + bβ· p
1− pFd(n) (5.6)
where a is a constant describing the energy loss ratio of the encoder filter, b is a constant
describing the motion randomness of the video scene, and β is the intra refreshing rate.
Here p is packet loss probability, and Fd(n) is the average frame difference over the whole
video sequence [106].
93
5.3 Resource Allocation and Performance Optimiza-
tion Using Particle Swarm Optimization
In this section, we develop the basic framework to measure, control and optimize the sys-
tem performance of the WVSN under bandwidth constraints and energy constraints. Here
we use the Particle Swarm Optimization (PSO) algorithm to perform resource allocation
and optimize the whole system performance [107].
5.3.1 Optimization Problem Formulation
We assume that the WVSN has N video streams which are sharing communication links
and crossing over the sensor network. The whole network has V video sensor nodes, and
L communication links, each link with a capacity of Cl (l ∈ L). The link capacity can be
considered as the maximum information bits per second that can be successfully transmitted
over the link. The video stream starting node compresses the sensor raw video data and let
the encoding bit rate be Rs and transmission bit rate be Rt. Here we don’t consider channel
coding, so Rs = Rt = R. Let Ps be the power consumption used in video compression and
Pt be the transmission power. Each video stream i uses a set L(i) ⊆ L of links on its path,
each link l is shared by a set N(l) = {i ∈ N | video stream i pathes link l} of video streams,
and K(k) means all the transmission happened on the sensor node k.
Under link capacity constraints, for determining the bit rates of every video stream in
order to maximize the overall system performance, we need to make sure that the aggregated
bit rate of all video streams on every link of the network does not exceeds its link capacity.
Let Ri ∈ R = {R1, R2, · · · , RN} be the bit rate of video stream i. According to the link
capacity constraints, we have
∑
Ri∈N(l)
Ri ≤ Cl, 1 ≤ l ≤ L. (5.7)
Under power consumption constraints, the total power for encoding power P is and trans-
mission power P it of video stream i should be no greater than the initial power P i
0 of this
94
video stream source sensor node. For other sensors which join the video communication
over the network, the total transmission power∑
j∈K(k)
P jt happened on this sensor k should
be no greater than its initial power P k0 . According to the power consumption constraints,
we have
P is + P i
t ≤ P i0,
∑
j∈K(k)
P jt ≤ P k
0 (5.8)
In the following, we define the network utility function for video transmission over wire-
less video sensor networks. The ultimate goal for video communication system is to provide
users with the best quality videos under given network resource constraints. Therefore, the
system performance should be measured by the overall video streams quality. We use the
end-to-end distortion to measure the video quality of a single video stream. The aggregated
video distortion for the whole network can be formulated as
D =∑
i
(Ds(Ri, Pis) + Dt(Ri, P
it ))
=∑
i
(σ2i e−λi·Ri·g(P i
s) + θipi
1− pi
F id(n))
(5.9)
where pi is the total packet loss probability (PLP) for video stream i on its all path L(i),
and at this moment, we consider stationary video streams and assume that σ2i , λi, θi and
F id(n) are constant. pi (PLP for video stream i) is given by
pi = 1−∏
l∈L(i)
(1− pli) (5.10)
here pli is the packet loss probability for video stream i on its path link l, and this packet loss
probability happened on link l can be obtained based on transmission power P lt , aggregate
source rate Rl on this link, and the distance dl between the link starting sensor node and
ending sensor node.
We assemble all the source rate variables Ri into a source rate vector R, all the encoding
power P is of each video stream into vector Ps, and all the transmission power P l
t on each
transmission node into vector Pt. Let X = [R,Ps,Pt], which represents all the control
variables that need to be determined by the performance optimization. Note that the link
capacity constraints and power consumption constraints are both linear. Therefore, the
95
Table 5.1: Configuration of the channel model parameters.Parameter Description Value
Gt Transmitter antenna gain 1Gr Receiver antenna gain 1λ Wavelength 1/8κ System loss factor 1N0 Noise power spectrum density 10−12
α2 Rayleigh gain 2BN Packet size 512
energy efficient resource allocation and performance optimization problem over WVSN can
be formulated as
minX
D(X) =∑
i
(Dis + Di
t) (5.11)
s.t.∑
Ri∈N(l)
Ri ≤ Cl, 1 ≤ l ≤ L
P is + P i
t ≤ P i0,
∑
j∈K(k)
P jt ≤ P k
0
Normally, for the particle swarm optimization algorithm, there are no requirements for
the objective function D(X). And the objective function D(X) can be a generic nonlinear,
nonconvex function on X.
5.3.2 Experimental Results
We test the proposed resource allocation and performance optimization scheme using
PSO with different wireless video sensor networks (WVSN) topologies. We randomly gener-
ate a network of video communication nodes, as well as a set of video sessions with randomly
selected source and destination nodes. We then use existing shortest-path routing algorithm
to determine the routing path of each video session [80]. In the following experiment, a wire-
less video sensor network has 4 video sessions (N = 4) and 8 links (L = 8) was selected.
The parameters used in the flat Rayleigh fading channel and the free space propagation
model are set in Table.5.1. The other parameters used in resource allocation and perfor-
mance optimization scheme are set as follows: link capacity Cl = 100, particle size = 20,
96
λi = 0.023, σ21 = σ2
3 = 30, σ22 = 20, σ2
4 = 50 θi = 0.8, and Fd(n) = 200.
Fig. 5.1 depicts an example network topology which has 4 sources and 8 links, link A,
and B are two bottleneck links here. There are three video streams path link A, and two
video streams path link B. Fig. 5.2 shows that the average overall video distortion decreases
as the particles update their positions during optimization process under different PSO size.
When the PSO size increases, it will find better optimal results, but when the PSO size is
bigger than 30, the advantage is very small. Bigger particle size means more computation
complexity, normally, we use 20 particles for our simulation.
To show more detail about the resource allocation and performance optimization, we
plot the average source rate, average encoding power and average package loss probability
for each video session during optimization process. Fig. 5.3 shows average source rate
for each video stream updates during PSO iteration. Fig. 5.4 shows average encoding
power for each video stream updates during PSO iteration. And Fig. 5.5 shows average
packet loss probability for each video stream decreases during PSO iteration. After all the
particles converge, this means an optimal result has been found, but we can see that there
are still some system resources (bandwidth, and energy) left. This means the centralized
optimization algorithm is not very efficient, and meanwhile it is too costly and impractical.
Next, we will develop a distributed optimization algorithm to make the result more efficient.
5.4 Energy Efficient Distributed and Asynchronous Par-
ticle Swarm Optimization
In the previous section, we introduced a centralized optimization algorithm using PSO
to handle the resource allocation and performance optimization problem over WVSN. This
centralized algorithm is too costly and impractical, because it needs to know the whole
network information to do the optimization. In this section, we will introduce an energy
efficient distributed and asynchronous particle swarm optimization (EEDAPSO) scheme
to solve the original optimization problem over WVSN. To design this distributed and
97
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
X Position
Y P
ositi
on
Destination Node Source
A
B
Figure 5.1: A WVSN topology with 9 sensor nodes, 8 links and 4 video sessions.
asynchronous optimization algorithm, we need to take two steps: (1) decompose the original
optimization problem into several sub-optimization problems; and (2) handle the interlaced
information appropriately.
5.4.1 Decomposition of Original Optimization Problem
In resource allocation for WVSN, we first need to decompose the global optimization
problem in (5.11) into several local optimization problems. Every local optimization model is
associated with a transmission sensor node, every transmission link happened from this node
should be considered of this local model. Fig. 5.6 shows a sketch figure of the decomposition
for the same network topology we used in Fig. 5.1.
Eq. (5.10) defines the packet loss probability (pi) for video stream i, which is decided
by every packet loss probability pli on link l in video stream i’s path. Because 1 − pi =
∏l∈L(i)
(1− pli), we first need to decompose this into an additive modality. For different video
streams path same link l, the packet loss probabilities should be same pli = pl
j because the
98
0 3 6 9 12 15130
135
140
145
150
155
160
165
170
175
180
Iteration Number
Ave
rage
Fun
ctio
n V
alue
PSO size = 10PSO size = 20PSO size = 30
Figure 5.2: The average overall video distortion decrease as the particles update theirpositions under different PSO size.
channel and transmission power are the same. Furthermore, we know that
1∏l
(1− pl)≈ 1
Nl
∑
l
1
1− pl
(5.12)
when pl is small enough (pl < 0.1). Here Nl is the number of links on video stream l′s path.
Let K = {1, 2, · · ·K} be the set of transmission nodes which are involved in the multiple-
session wireless video communications, as illustrated in Fig. 5.6, then we have K local op-
timization modules in total. For every transmission node, which corresponds to a local
optimization module, there are two kinds of resource parameters. Local resource parame-
ters, and global resource parameters. For local resource parameters, they are only affected
by the local optimization process, but for the global resource parameters, they are not only
affected by the local optimization process, but also need to be shared with other optimization
modules which include these kinds of parameters. The original energy efficient resource al-
location and performance optimization problem in (5.11) can be represented approximately
99
0 3 6 9 12 155
10
15
20
25
30
35
40
Iteration Number
Ave.
Sou
rce
Rat
eSource Rate 1Source Rate 2Source Rate 3Source Rate 4
Figure 5.3: The average source rate for each video stream updates during PSO iteration.
as follows:
min D =K∑
k=1
[N∑
i=1
ai · ( 1
Li
Dis) +
L∑
l=1
bl(N∑
i=1
1
Li
· θF id(n)
1
1− pl
)] (5.13)
s.t.∑
Ri∈N(l)
Ri ≤ Cl, 1 ≤ l ≤ L
P is + P i
t ≤ P i0,
∑
j∈K(k)
P jt ≤ P k
0
where Li represents the total number of link on video stream i′s path, ai and bl are para-
meters. ai = 1 means video stream i paths this transmission node, and bl = 1 means link l
is one transmission link of this transmission node.
Here for each local optimization module, we introduce a set of global resource para-
meters Rk = {Rik|i ∈ K(k)}, where Rik represents the bit rate of video stream i at local
optimization module k, and this module k can include several transmission links. And we
also introduce a set of resource parameters Pk = {P isk, P
jtk|i, j ∈ K(k)}, where P i
sk represents
the encoding power of video stream i if there exists encoding process in this local module,
otherwise no encoding power in this local optimization module, which is the global resource
100
0 3 6 9 12 15
0.4
0.5
0.6
Iteration Number
Ave.
Enc
odin
g Po
wer Enc Power for VS1
Enc Power for VS2Enc Power for VS3Enc Power for VS4
Figure 5.4: The average encoding power for each video stream updates during PSO iteration.
parameter, and P jtk represents the transmission power j at local optimization module k,
which is the local resource parameter.
Note that, for a video stream, its bit rate on each link along its transmission path should
be the same. In other words,
Rim = Rin, ∀ m,n ∈ LS(i), (5.14)
where LS(i) is the set of links used by video stream i. This is an interdependent flow
balance constraint. We can see that the original global optimization problem has been
decomposed into K local optimization modules in (5.13) plus a set of interdependent flow
balance constraints and each local module’s own power consumption constraints, which will
be solved by the following EEDAPSO algorithm.
5.4.2 Algorithm Design of EEDAPSO
In the proposed Energy Efficient Distributed and Asynchronous PSO (EEDAPSO) al-
gorithm, each local constrained optimization problem in (5.13) can be solved using the PSO
101
0 3 6 9 12 150.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
0.13
Iteration Number
Ave
. Pac
ket L
oss
Pro
babi
lity
Ave. PLP for VS1Ave. PLP for VS2Ave. PLP for VS3Ave. PLP for VS4
Figure 5.5: The average packet loss probability for each video stream decreases during PSOiteration.
algorithm discussed in Section 2.3.1. Each local PSO module has a group of local particles
moving around the solution space defined by the local constraints for the local optimum.
Neighboring PSO modules share status information about their “group-best” particle, this
external information is used to guide the movements of the internal particles inside of this
local module. For neighboring PSO modules, they need to meet the interdependent flow
balance constraints in (5.14) by using in-network fusion and particle migration which are
already introduced in Section 4.4.3.
Here we choose minimum of both source rates as in-network fusion rule, and the source
rates part of the particle to do the particle migration, called rate particle migration here.
The bottleneck link issue also can be properly handled by rate particle migration method
by using a resource budget window which is used in DAPSO algorithm. One simple rule
for adjusting resource budget window is used as follows: at one check time during the
optimization procedure for one local PSO module, if this module gets a new group-best
rate particle, it will generate a new resource budget window to replace the old one, and at
102
Figure 5.6: The average video distortion decreases during EEDAPSO search process.
the same time, it will also check if the current local group-best particle is inside the new
group-best rate particle. If the local group-best particle is inside the new group-best rate
particle, this means this video stream source rate maybe can still reach more link bandwidth
resource, then we need to increase its resource budget window size step by step, otherwise,
the resource budget window size will be decreased step by step. Meanwhile, when we do
rate particle migration, we also need to share encoding power from the video stream starting
node to all the path of this video stream. Here we choose the average encoding power from
all the particles in this local module to share with other modules. Because all the particles
will converge at the end of the optimization procedure, how to choose the encoding power
to share will not affect the optimization procedure greatly.
The energy efficient distributed and asynchronous PSO (EEDAPSO) algorithm can be
stated as follows:
Step 1 Initialization. Initialize the local PSO module by randomly generating a set of
particles on every transmission node k (1 ≤ k ≤ K). The total number of particles
in each PSO module can be adjusted. In our simulations, we set it to be 20. Also
103
initialize the resource budget window.
Step 2 Local optimization. Each local EEDAPSO module moves its own particles in the
solution space based on the swarm intelligence principle in (2.19) and records its
group-best particle.
Step 3 Sharing group-best information. If the group-best particle has been updated, the
EEDAPSO module transmits this group-best information to its neighboring mod-
ules and shares average encoding power on each video stream’s whole path through
local communication. To control the amount of communication overhead, this infor-
mation can be transmitted only when the amount of change in group-best is larger
than a given threshold. It should be noted that this information sharing is asyn-
chronous: it is shared only when it is updated and different EEDAPSO modules
may update their group-best information at different time instances.
Step 4 Particle migration and fusion. Upon receiving the group-best information from its
neighbors, each EEDAPSO module updates its group-best based on the fusion rule
in (4.14). It also updates its resource budget window as explained in Section 4.4.3.
Step 5 Iteration and exit. Repeat Steps 2-4 until the changes in local utility value (e.g.
total video distortion) are below a given threshold.
In each local PSO module, the movement of every particle is defined by (2.18) and (2.19).
Here we choose the weights as the same requirement introduced in Section. 4.2.2.
5.4.3 Experimental Results
Fig. 5.7 shows that average video distortion decreases as the particles update their
positions during EEDAPSO optimization process. Fig. 5.8, and Fig. 5.9 show the source
rate for every video stream on several bottleneck links updates during PSO search process.
Here in each cluster PSO module, we only trace the cluster best particle’s moving path.
104
Fig. 5.10 shows encoding power for every video stream updates during PSO search
process, and Fig. 5.11 shows transmission power on every link updates during PSO search
process. From these, we can see that the EEDAPSO algorithm is much more energy efficient
than the centralized optimization algorithm. Fig. 5.12 shows packet loss probability for each
video stream decreases during EEDAPSO search process.
0 10 20 30 40 50 60 70 80100
110
120
130
140
150
160
170
Iteration Number
Ave
. Util
ity F
unct
ion
Val
ue
Figure 5.7: The average video distortion decreases during EEDAPSO search process.
5.5 Discussion and Conclusion
In this chapter, based on the video compression, wireless transmission behavior of
WVSN and the swarm intelligence principle, we have developed an evolutionary optimiza-
tion scheme to solve the energy efficient distributed and asynchronous optimization problem
of video communication. The basic operation involves (1). decomposition of the original
optimization problem with global resource parameters and inter-dependent resource con-
straints into several sub-optimization problems of each transmission node; (2). in-network
fusion and particle migration schemes for neighboring EEDAPSO modules to exchange their
105
0 10 20 30 40 50 60 705
10
15
20
25
30
35
40
45
50
55
Iteration Number
Vide
o St
ream
Sou
rce
Rat
e on
Lin
k A
VS1 Source RateVS2 Source RateVS3 Source Rate
Figure 5.8: The source rate for every video stream on critical link A updates duringEEDAPSO search process.
group-best information to guide their local particle movements in search for the optimum
solution; and (3). resource budget window scheme to address the network bottleneck issue.
The outstanding contribution of this algorithm is there is no convex or concave requirement
for the utility function, which is required in other traditional distributed optimization algo-
rithms, and the proposed EEDAPSO algorithm is very efficient. Figs. 5.13 and 5.14 show
the convergence results for EEDAPSO on some examples of random network topologies.
Compared with other methods, EEDAPSO has the following advantages:
(1) the algorithm is simple, which will have low computation complexity; (2) the algorithm
is powerful, and EEDAPSO’s convergence speed is very fast; (3) there is no predefined limi-
tation on the objective utility function, such as convex requirement for the utility function;
and (4) the algorithm is asynchronous, each local module only checks its neighbor sharing
communication information at special time.
106
0 10 20 30 40 50 60 700
10
20
30
40
50
60
70
80
90
100
Iteration Number
Vid
eo S
tream
Sou
rce
Rat
e on
Lin
k B
VS2 Source RateVS3 Source RateVS4 Source Rate
Figure 5.9: The source rate for every video stream on critical link B updates duringEEDAPSO search process.
0 10 20 30 40 50 60 700.35
0.4
0.45
0.5
0.55
Iteration Number
Enc
odin
g P
ower
Pe 1Pe 2Pe 3Pe 4
Figure 5.10: The encoding power for every video stream updates during EEDAPSO searchprocess.
107
0 10 20 30 40 50 60 70
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Iteration Number
Tran
smis
sion
Pow
er
Pt 1Pt 2Pt 3Pt 4Pt 5Pt 6Pt 7Pt 8
Figure 5.11: The transmission power on every link updates during EEDAPSO searchprocess.
0 10 20 30 40 50 600.04
0.06
0.08
0.1
0.12
0.14
0.16
Iteration Number
Pac
ket L
oss
Pro
babi
lity
in E
ED
AP
SO
PLP for VS1PLP for VS2PLP for VS3PLP for VS4
Figure 5.12: The packet loss probability for each video stream decreases during EEDAPSOsearch process.
108
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
X Position
Y P
osi
tion
0 20 40 60 80 100 120140
150
160
170
180
190
200
210
220
230
240
Iteration Number
Ave
. U
tility
Funct
ion V
alu
e
(a) (b)
Figure 5.13: (a) An example WVSN; and (b) convergence of network utility function withEEDAPSO.
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
X Position
Y P
osi
tion
0 20 40 60 80 100 120150
200
250
300
350
400
Iteration Number
Ave
. U
tility
Funct
ion V
alu
e
(a) (b)
Figure 5.14: (a) An example WVSN; and (b) convergence of network utility function withEEDAPSO.
109
Chapter 6
Evaluation of PSO Algorithm
From the previous chapters, the global search capabilities of PSO algorithm make it
can handle a suite of difficult non-convex optimization problems. This chapter provides the
mathematical analysis of PSO algorithm and gives the convergence analysis. Meanwhile,
the PSO algorithm is compared with other previous published optimization algorithms, such
as genetic algorithm (GA) and quasi-Newton algorithm (BFGS).
6.1 Convergence Analysis of PSO Algorithm
First Random Search Techniques, which are convergent algorithms for constrained non-
linear problems, are introduced. Next a proof of guaranteed convergence PSO is developed.
The analysis is given finally.
6.1.1 Random Search Techniques
The convergence of random search techniques have been studied by F. J. Solis and R.
Wets in 1981[108]. In their paper, the criteria under which algorithms can be considered
to be global search algorithm or local search algorithms is provided. For some convenience,
the relevant definitions from [108] are presented as follows.
Definition 1 Let f : Rn → R and S ⊆ Rn. Seek a solution x in S which minimizes the
function f on the search space S or at least which yields an acceptable approximation of
the infimum of f on S.
110
The basic random search technique is outlined as follows:
• Step 1: Initialization. Find x0 ∈ S, and set k := 0.
• Step 2: Generate a vector ξk ∈ Rn from distribution µk.
• Step 3: Set xk+1 = D(xk, ξk), choose µk+1, set k := k + 1 and go to step 1.
In the kth iteration, this algorithm requires a probability space (Rn, µk), where µk is a
probability measure corresponding to a distribution on Rn. D is a function that constructs
a solution to the problem. This solution carries the guarantee that the newly constructed
solution will not worse than the current solution.
The sufficient conditions for the convergence is as follows:
(H1) D subject to {f(xk)}∞k=0 nonincreasing: f(D(x, ξ)) ≤ f(x), and if ξ ∈ S then
f(D(x, ξ)) ≤ min{f(x), f(ξ)}.(H2) Any A ⊆ S with v(A) > 0,
∞∏k=0
(1− µk(A)) = 0.
Global convergence means that there exists one infimum of function f on the search
space S that the sequence {f(xk)}∞k=0 converges to. For a Essential Infimum α of f on S,
we have
α = inf(t|v(x ∈ S|f(x) < t) > 0) (6.1)
where v denotes a n-dimensional volume or Lebesgue measure[109]. This means that there
must exist more than one solution in the subset A of S arbitrarily close to α. This avoids
some pathological case which a function has a minimum consisting of a single discontinuous
point, such as below:
f =
2x2 + 1, x 6= 1
−6, x = 1
Meanwhile, Optimality region is given by:
Rε,M =
{x ∈ S|f(x) < α + ε}, αfinite
{x ∈ S|f(x) < −M, α = −∞(6.2)
Here M > 0 and big enough. If the algorithm finds a solution in the optimality region, then
it has found an acceptable approximation to the global minimum of the function.
111
For random search algorithm, a local search method has µk with bounded support Mk
that v(S∩Mk) < v(S), and a global search method has µk with support Mk that v(S∩Mk) =
v(S). In (H2), µk(A) is the probability of A generated by µk. This means that for any
subset A of S with positive measurement v, the sampling strategy given by µk can not
consistently ignore this subset A.
Below is Global Search Convergence Theorem from Solis and Wets’s work[108]:
Theorem 1. Suppose function f is measurable, S ⊆ Rn is measurable, (H1) and (H2)
are satisfied. Let {xk}∞k=0 be a sequence generated by the random search algorithm, then
limk→∞
P (xk ∈ Rε,M) = 1
where P (xk ∈ Rε,M) is the probability at step k, the solution xk generated in Rε,M .
Proof : By (H1), at step k xk /∈ Rε,M implies that xl /∈ Rε,M for any l < k. Thus
P (xk ∈ S \Rε,M) ≤k−1∏
l=0
(1− µl(Rε,M))
where S \Rε,M means that the set S with Rε,M removed. Hence
P (xk ∈ Rε,M) = 1− P (xk ∈ S \Rε,M) ≥ 1−k−1∏
l=0
(1− µl(Rε,M))
.
By (H2) we know that∞∏
k=0
(1− µk(A)) = 0, thus
1 ≥ limk→∞
P (xk ∈ Rε,M) ≥ 1− limk→∞
k−1∏
l=0
(1− µl(Rε,M)) = 1
From this Global Search Theorem, we know that an algorithm satisfying (H1) and (H2)
is a global optimization algorithm.
For some algorithms that fail to satisfy the condition (H2), a local search condition
(H3) is defined as below. The local search algorithm has better rates of convergence, but
no guarantee that the global optimum will be found.
(H3) Any x0 ∈ S, L0 = {x ∈ S|f(x) ≤ f(x0)} is the compact set, ∃γ > 0 and η ∈ (0, 1]
such that for all k and all x in the L0,
µk((D(x, ξ) ∈ Rε,M) ∪ (dist(D(x, ξ), Rε,M) < dist(x, Rε,M)− γ)) ≥ η
112
where dist(x, A) is the distance between a solution x and a set A.
dist(x, A) = infb∈A
dist(x, b)
From conditions (H1) and (H3), the Local Search Convergence Theorem[108] can be
represented as below:
Theorem 2. Suppose function f is measurable, S ⊆ Rn is measurable, (H1) and (H3)
are satisfied. Let {xk}∞k=0 be a sequence generated by the random search algorithm, then
limk→∞
P (xk ∈ Rε,M) = 1
where P (xk ∈ Rε,M) is the probability at step k, the solution xk generated in Rε,M .
Proof : Let x0 be the initial iterate used in the search algorithm. By (H1), all future
iterates in L0 ⊇ Rε,M . Since L0 is compact, there always exists an integer p such that
γp > dist(a, b), ∀a, b ∈ L0
By applying Bayes rule and (H3), we have
P (xl+p ∈ Rε,M |xl /∈ Rε,M) =P (xl+p ∈ Rε,M ,xl /∈ Rε,M)
P (xl+p ∈ Rε,M)
≥ P (xl+p ∈ Rε,M ,xl /∈ Rε,M)
≥ P (xl /∈ Rε,M , dist(xk, Rε,M) ≤ γ(p− (k − l)), k = l, ..., l + p)
≥ ηp
The claim here is
P (xkp /∈ Rε,M) ≤ (1− ηp)k, ∀k ∈ {1, 2, ...}
By induction, for k = 1 applying Bayes rules and (H3) p times
P (xp ∈ Rε,M) ≥ P (xp ∈ Rε,M ,x0 /∈ Rε,M) ≥ ηp
For general k applying Bayes rules and (H3) another p times
P (xkp /∈ Rε,M) = P (xkp /∈ Rε,M |x(k−1)p /∈ Rε,M)P (x(k−1)p /∈ Rε,M)
≤ [1− P (xkp ∈ Rε,M |x(k−1)p /∈ Rε,M)](1− ηp)(k−1)
≤ (1− ηp)k
113
Hence
P (xkp+l ∈ Rε,M) ≥ P (xkp ∈ Rε,M ≥ 1− (1− ηp)k
where l = 0, 1, ..., p− 1. This finishes the proof since limk→∞(1− ηp)k = 0.
6.1.2 PSO Convergence Analysis
The stochastic nature of the PSO makes it more difficult to prove its convergence prop-
erty. Here we use Random Search Technique to do the PSO convergence analysis [110].
Here we have
F0 = {x ∈ S|f(x) ≤ f(x0)} (6.3)
where x0 = argmax{f(xi)}, i ∈ 1, ..., S. xi represents the particle i and x0 represents the
worst particle which has the largest function f value of all the particles.
From equations 2.18 and 2.19, a function D can be defined as consisting of a single
discontinuous point, such as below:
D(xg,k,xki ) =
xg,k, f(h(xki )) ≥ f(xg,k)
h(xki ), f(h(xk
i )) < f(xg,k)
where h(xki ) represents the application function which performs the PSO updates, xk
i rep-
resents the particle i at time step k and xg,k represents the best solution found so far from
all the particles at time step k. Since the sequence of {xg,i}ki=0 is nonincreasing, the above
definition of function D complies with (H1).
From equation 2.18, we have
h(xki ) = xk+1
i = xki + vk
i (6.4)
By the definition of F0 and assumption that all the particles are initialized in F0, we have
xg,0,x0i ∈ F0.
Let hN(xki ) denote N continuous applications of h on xk
i , then we have the following
conclusion.
Lemma 1 There exist a value 1 ≤ N ≤∝ so that ‖ hn(xki ) − hn+1(xk
i ) ‖< ε, ∀n ≥ N ,
and ε > 0.
114
Proof From the velocity and position update equations for PSO, by substituting equa-
tion 2.19 into equation 2.18, we have
xi(t + 1) = xi(t) + wvi(t) + c1Θ1[xsi (t)− xi(t)] + c2Θ2[x
g(t)− xi(t)]
= (1− η1 − η2)xi(t) + η1xsi (t) + η2x
g(t) + wvi(t)
We also know that
xi(t) = xi(t− 1) + vi(t)
Thus
vi(t) = xi(t)− xi(t− 1)
Finally we have
xi(t + 1) = (1 + w − η1 − η2)xi(t)− wxi(t− 1) + η1xsi (t) + η2x
g(t)
This is a non-homogeneous recurrence relation problem which can be resolved by using
standard techniques introduced in [111]. This recurrence relation can be written in a matrix
vector product as following
xi(t + 1)
xi(t)
1
=
1 + w − η1 − η2 −w η1xsi (t) + η2x
g(t)
1 0 0
0 0 1
xi(t)
xi(t− 1)
1
The characteristic polynomial of this matrix is
(1− λ)[w − λ(1 + w − η1 − η2) + λ2]
which has three solutions
λ = 1
α = 1+w−η1−η2+γ2
β = 1+w−η1−η2−γ2
where
γ =√
(1 + w − η1 − η2)2 − 4w
Here the α and β are eigenvalues of the matrix, and the explicit form of the recurrence
relation is given by
xi(t) = k1 + k2αt + k3β
t
115
where k1, k2 and k3 are constants determined by the initial conditions of the system [111].
A system of three equations can be constructed with these initial conditions.
From three initial conditions of PSO, the system is given by
xi(0)
xi(1)
xi(2)
=
1 1 1
1 α β
1 α2 β2
k1
k2
k3
The third condition of the system can be solved by using the recurrence relation from xi(0)
and xi(1). The system can be solved by using Gaussian elimination method, then we have
k1 = αβxi(0)−xi(1)(α+β)+xi(2)(α−1)(β−1)
k2 = β(xi(0)−xi(1))−xi(1)+xi(2)(α−β)(α−1)
k3 = α(xi(1)−xi(0))+xi(1)−xi(2)(α−β)(β−1)
Because α− β = γ, we can further simplify these equations to
k1 =η1xs
i (t)+η2xg(t)
η1+η2
k2 = β(xi(0)−xi(1))−xi(1)+xi(2)γ(α−1)
k3 = α(xi(1)−xi(0))+xi(1)−xi(2)γ(β−1)
Here both xsi (t) and xg(t) are dependent on the time step t. They maybe change with every
time step, depending on the objective function and the particles. Whenever either xsi (t) or
xg(t) changes, the value of k1, k2 and k3 must be recomputed.
From vi(t + 1) = xi(t + 1)− xi(t), we have
limt→+∝
vi(t + 1) = limt→+∝
[xi(t + 1)− xi(t)]
= limt→+∝
[k2αt(α− 1) + k3β
t(β − 1)]
= 0
when ‖ α ‖, ‖ β ‖< 1.
Meanwhile, we have
limt→+∝
xi(t) = limt→+∝
[k1 + k2αt + k3β
t] =η1x
si (t) + η2x
g(t)
η1 + η2
116
when ‖ α ‖, ‖ β ‖< 1, and
xi(t + 1) = xi(t) + vi(t + 1)
= xi(t) + wvi(t)− (η1 + η2)xi(t) + η1xsi (t) + η2x
g(t)
Therefore, we can see that in the limit time t, we have xi(t + 1) = xi(t) and the system will
converge but no guarantee it will reach the global optimum. In summary, the PSO algorithm
can start at any initial states which will lead to a stagnant state in a finite number of time
steps. By using a large number of particles, the probability of becoming trapped in such a
stagnant state is reduced dramatically [110].
We have already proved that the PSO algorithm is only a local convergence algorithm,
this means that after finite iterations, all the solutions will converge to an optimum solution
which is no guaranteed a global optimum. However, the large number of random solutions
of the PSO algorithm make up this technique converges to its global optimum in a good
opportunity. We give a simple experiment of nonconvex optimization problem here to show
this.
010
2030
4050
0
10
20
30
40
5010
15
20
25
30
35
40
45
50
Figure 6.1: Plot for a nonconvex objective function.
The nonconvex objective function here is
min . f(x, y) = 50− {15e−0.01[(x−10)2+(y−10)2] + 20e−0.01[(x−35)2+(y−10)2]
+ 35e−0.01[(x−10)2+(y−35)2] + 30e−0.01[(x−35)2+(y−35)2]}(6.5)
117
14 16 18 20 22 24 26 28 300
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Objective Function Value
Prob
abilit
y
Figure 6.2: Convergence probability under different initial random solution.
Table 6.1: PSO algorithm parameters used in the test.
Parameter Description ValueN Particle Size 20c1 Learning Rate 1.5c2 Learning Rate 1.5w Inertia Weight 0.729
vini Initial Velocity 0.1
We can easily find that the global optimum is at point A = [10, 35], and the optimum
objective function value is around 15. And also there are three local optimum points, which
are B = [10, 10], C = [35, 10] and D = [35, 35]. Fig.6.1 gives the plot for the nonconvex
objective function.
Table 6.1 shows the parameters of PSO algorithm used in the experiment test. We
finished 500 time simulations with different initial random solutions. Our experimental
results show that the PSO algorithm will find the global optimum in 90% opportunity. See
Fig. 6.2 for details about convergence probability.
118
6.2 Optimization Algorithms Evaluation and Compar-
ison
For the optimization problems which process multiple local minima, normally it is dif-
ficult to find the global best solution. This section evaluates genetic algorithm (GA),
Broydon-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton algorithm, and particle swarm
optimization (PSO) algorithm for their global search capabilities using a suite of difficult
analytical test problems.
6.2.1 Optimization Algorithms Introduction
This section gives the brief introduction for the Genetic Algorithms and Broyden-
Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton algorithms.
I. Introduction of Genetic Algorithm
Genetic algorithms are based on a biological metaphor: They view learning as a com-
petition among a population of evolving candidate problem solutions. A fitness function
evaluates each solution to decide whether it will contribute to the next generation of so-
lutions. Then, through operations analogous to gene transfer in sexual reproduction, the
algorithm creates a new population of candidate solutions[112].
In the 1950s, computer scientists studied evolutionary systems as optimization tools,
and introduced the basics of the evolutionary computing. In 1975[113], Holland defined the
basic concept of the GA. Based on the Schema theorem[113], Genetic Algorithm techniques
have a solid theoretical foundation[114].
Genetic algorithms are implemented as a computer simulation in which a population
of abstract representations, called chromosomes, of candidate solutions to an optimization
problem evolves toward better solutions. Traditionally, solutions are represented in binary
as strings of 0s and 1s, but other encodings are also possible. The algorithm then evaluates
from a population of randomly generated individuals and happens in generations. In each
119
generation, the algorithm evaluates the fitness of every individual, and multiple individuals
are randomly selected from the current population (based on their fitness), and modified to
generate a new population. The new population is then used in the next iteration of the
algorithm. Commonly, the algorithm will be terminated when either a maximum number of
generations has been produced, or a satisfactory fitness condition has been reached for the
population. If the algorithm has been terminated due to a maximum number of generations,
a satisfactory solution may or may not have been reached.
The operation of the Genetic Algorithm can be represented as follows:
• Step 1: Choose initial population N , the crossover probability Pc, and the mutation
probability Pm.
• Step 2: Evaluate the fitness of each individual in the population.
• Step 3: Select best ranking individuals based on the probability related to their fitness
to reproduce.
• Step 4: Generate offspring by the new generation through crossover and mutation.
• Step 5: Evaluate the individual fitness of the generated offspring.
• Step 6: Replace worst ranking individuals in the population with the generated
offspring.
• Step 7: Go through Step 3 to Step 6 until the termination criterion is satisfied.
Genetic algorithm is a very effective way to quickly find a optimal solution to a complex
problem. In a search space for which little is known, Genetic algorithm is also effective.
Genetic algorithms find many applications in computer science, engineering, economics,
chemistry, manufacturing, mathematics, physics and other fields[114].
II. Introduction of BFGS Algorithm
120
Quasi Newton methods play an important role in the numerical solution of uncon-
strained optimization problems. The Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-
Newton method is derived from the Quasi Newton method in optimization to solve the
unconstrained nonlinear optimization problems. Quasi Newton method assumes that the
function can be locally approximated as a quadratic in the region around the optimum, and
use the first and second derivatives to find the stationary point.
In Quasi Newton methods, the Hessian matrix of second derivatives of the function to be
minimized does not need to be computed at any stage, where the Hessian matrix is the square
matrix of second-order partial derivatives of a function. Instead, the Hessian is updated
by analyzing successive gradient vectors. Quasi-Newton methods are a generalization of
the secant method to find the root of the first derivative for multidimensional problems.
In multi-dimensions the secant equation is under-determined, and quasi-Newton methods
differ in how they constrain the solution. The BFGS method is one of the most successful
members of this class[115].
BFGS Quasi Newton methods are used when the Hessian matrix is difficult or time-
consuming to evaluate. Instead of obtaining an estimate of the Hessian matrix at a single
point, this method gradually builds up an approximate Hessian matrix by using gradient
information from some or all of the previous iterates visited by the algorithm. The general
algorithm of the BFGS Quasi Newton algorithm can be represented as follows:
• Step 1: For the optimization problem inf {f(x); x ∈ Rm}, choose a starting point x0,
an m×m approximate Hessian matrix H0, and a positive scalar ε. Set k = 0.
• Step 2: Evaluate the gradient ∇f(xk).
• Step 3: If ‖∇f(xk)‖ < ε, stop.
• Step 4: Set Hkdk = −∇f(xk), dk is a generated direction.
• Step 5: Update solution xk+1 = xk + θkdk, θk > 0 by means of an approximate
minimization f(xk+1) ≈ minθ≥0
f(xk + θdk).
121
• Step 6: Compute δk = xk+1 − xk, and γk = ∇f(xk+1)−∇f(xk).
• Step 7: Compute Hk+1 = Hk + (1 +γT
k Hkγk
δTk γk
)− δkγTk Hk+HkγkδT
k
δTk γk
.
• Step 8: Update k = k + 1, and go to Step 3.
6.2.2 Test Problems Analysis
A suite of difficult analytical test problems previously published by Soest and Casius
[116]. We choose several test problems from [116] for PSO algorithm evaluation compared
with other optimization algorithms. The test problems we choose for the evaluation can be
briefly described as follows:
Problem 1: One 2-dimensional function [116] which has several local maxima and a
global maximum of 2 at the coordinates (8.6998, 6.7665).
H1(x1, x2) =sin2(x1 − x2
8) + sin2(x2 + x1
8)
d + 1(6.6)
where x1, x2 ∈ [−100, 100], and d =√
(x1 − 8.6998)2 + (x2 − 6.7665)2.
Problem 2: One 2-dimensions function used in [117] has several local maxima around
the global maximum of 1.0 at (0,0).
H2(x1, x2) = 0.5− sin2(√
x21 + x2
2)− 0.5
(1 + 0.001(x21 + x2
2))2
(6.7)
where x1, x2 ∈ [−100, 100].
Problem 3: The test function from [118] is used with dimensionality n = 4, 8. The
function contains a large number of local minima with a global minimum of 0 at |xi| < 0.05.
H3(x1, ..., xn) =n∑
i=1
(t · sgn(zi) + zi)2 · c · di, if |xi − zi| < t
di · x2i , otherwise
(6.8)
where xi ∈ [−1000, 1000], zi = b|xi
s|+ 0.49999c · sgn(xi) · s, c = 0.15, s = 0.2, t = 0.05, and
122
di =
1, i = 1, 5
1000, i = 2, 6
10, i = 3, 7
100, i = 4, 8
(6.9)
The use of the floor function in Eq.(6.8) makes the search space for this problem the
most discrete of all problems tested.
6.2.3 Experimental Results
Three off-the-shelf optimization algorithms: GA, BFGS, and PSO, were applied to all
the test problems for comparison purpose. For each optimization algorithm, one thousand
optimization runs were performed with each optimizer starting from random initial guesses
and using standard optimization algorithm parameters. Each run will be terminated based
on a predefined number of function iteration for the particular problem being solved or
predefined termination condition being met.
For all of the test problems, an algorithm will be considered success only if it converges to
within 10−3 of the know optimum cost function value within a specified number of function
evaluations [116]. For PSO algorithm, number of function iteration for each run is set to
100. Parameters of PSO algorithm used in the experiment test is the same as shown in Table
6.1. And Table 6.2 shows the simulation results from PSO, GA, and BFGS algorithms for
the difficult optimization problems introduce in 6.2.2. 1
From the simulation results, we can see that PSO algorithm is more robust than GA and
BFGS algorithms for these test problems. PSO converges to the correct global optimum
nearly 100% for problem H1, 52% for problem H2, and 45% for problem H3 with n=4.
But PSO algorithm does not converge to the correct global optimum for problem H3 with
n=8 within 10−3. In contrast, none of other algorithms converge more than 32% for any
test problems. Meanwhile, PSO algorithm has the faster convergence speed than other
1the simulation results for GA and BFGS algorithms are coming from Schutte’s work[119].
123
Table 6.2: Simulation Results for PSO, GA, and BFGS Algorithms.
Algorithm H1 H2 H3(n=4) (n=8)
PSO 0.999 0.518 0.443 0.000GA 0.000 0.034 0.000 0.000
BFGS 0.00 0.32 0.00 0.00
algorithms. 2 Fig.6.3 and Fig.6.4 show function value convergence during particle iteration
for all the test problems.
0 10 20 30 40 50 60 70 80 90 1000
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Function Iteration Number
Fun
ctio
n V
alue
Problem H1
PSO Algorithm
0 10 20 30 40 50 60 70 80 90 1000.75
0.8
0.85
0.9
0.95
1
Function Iteration Number
Funct
ion V
alu
e
Problem H2
PSO Algorithm
(a) (b)
Figure 6.3: (a) Convergence of function value for problem H1; and (b) convergence offunction value for problem H2.
6.3 Summary
This chapter first provides the mathematical analysis of PSO algorithm and gives the
convergence analysis. PSO algorithm has been proved that it is a local convergence algo-
rithm, which means that after predefined number of function iteration, all the solutions
2In Schutte’s work[119], the function iteration number for H1 is 10000, for H2 is 20000, for H3 (n=4) is50000, and for H3 (n=8) is 100000.
124
0 10 20 30 40 50 60 70 80 90 1000
0.5
1
1.5
2
2.5
3
3.5
4x 10
6
Function Iteration Number
Fun
ctio
n V
alue
Problem H3 (n=4)
PSO Algorithm
0 10 20 30 40 50 60 70 80 90 1000
0.5
1
1.5
2
2.5x 10
7
Function Iteration Number
Fu
nct
ion
Va
lue
Problem H3 (n=8)
PSO Algorithm
(a) (b)
Figure 6.4: (a) Convergence of function value for problem H3 (n=4); and (b) convergenceof function value for problem H3 (n=8).
will converge to an optimum solution which is no guaranteed a global optimum. How-
ever, from our experimental results coming from some difficult optimization problems, we
can see that the large number of random solutions of the PSO algorithm make up this
technique converges to its global optimum in a good opportunity. Meanwhile, we compare
PSO algorithm, genetic algorithm (GA), and Broydon-Fletcher-Goldfarb-Shanno (BFGS)
quasi-Newton algorithm for their global search capabilities based on a suite of difficult ana-
lytical optimization problems. The experimental results show that the PSO algorithm has
the better convergence probability to the global optimum compared with the other two
algorithms.
125
Chapter 7
Conclusion and Future Work
This chapter summaries the major contributions of the research work and also gives a
discussion about the future topics on the research work.
7.1 Conclusion
This dissertation discusses the resource allocation and performance optimization prob-
lems over ad hoc networks, especially the video mesh network. In this dissertation, four opti-
mization algorithms based on swarm intelligence (PSO with convex mapping, decentralized
PSO, distributed and asynchronous PSO, and energy efficient distributed and asynchronous
PSO) are developed for different optimization problems over wireless sensor networks. As
a summary, the most contributions of this dissertation are that we develop several evo-
lutionary optimization schemes using swarm intelligence to solve the nonlinear resource
allocation and performance optimization problems based on the properties of different net-
works. Meanwhile, compared with others’ research work, our algorithms do not need the
objective function to be convex or concave, and have very fast convergence speed.
I. PSO with Convex Mapping
Based on the unique characteristics of WVSNs and swarm intelligence principle, we
develop a new PSO algorithm with convex mapping. This algorithm transforms the solution
126
space set by the flow balance constraints and energy constraints to a convex region in a low-
dimensional space, and merges the convex condition with the swarm intelligence principle
to guide the movement of each particle during the evolutionary optimization process. We
have already proved that the dimension of the new space S′ is much lower than the original
space S. The transform is able to reduce number of dimensions by up to 2K, where K is
the number of sensor nodes. This transformation can reduce the computational complexity
and remove the interdependence between the control variables. Meanwhile we proved that
the new space S′ is also convex. We use this convex property to reduce the PSO algorithm
computation, so that we can omit the solution space boundary check during the algorithm
iteration.
II. Decentralized PSO
The WSN distributed optimization problems can be solved by an evolutionary opti-
mization scheme, called decentralized PSO (DPSO), which is based on swarm intelligence
principles. In this algorithm, sensor nodes share information with each other through local
information exchange and communication to solve a joint estimation or optimization prob-
lem. The basic operation involves parameter estimation in each sensor and transmits its own
local results to the network. The proposed DPSO scheme has low communication energy
cost and assures fast convergence. In addition, the objective function is not required to be
convex. We use source localization as an example to demonstrate the efficiency of the pro-
posed DPSO scheme. Simulation results show that our evolutionary optimization scheme is
very efficient for different network topology and has fast convergence speed compared with
gradient search algorithm.
III. Distributed and Asynchronous PSO
We use an evolutionary distributed and asynchronous PSO (DAPSO) technique to op-
timize the resource allocation and performance optimization problems over video mesh net-
works. The basic operation involves utility function maximization optimization in each
127
local link and in-network fusion and sharing. Unlike many network resource allocation per-
formance optimization algorithms in the literatures which are only able to handle convex
network utility functions, the outstanding contribution of this evolutionary algorithm is
there is no convex or concave requirement for the utility function. The proposed DAPSO
scheme is able to handle generic nonlinear nonconvex network utility functions. We use a
specific rate allocation and quality optimization problem for an example to demonstrate the
efficiency of the proposed scheme and compare its performance with other algorithms, such
as distributed lagrange dual algorithm.
Compared with other methods, DAPSO has the following advantages: (1) the algorithm
is simple; (2) the algorithm is powerful, and DAPSO’s convergence speed is very fast; (3)
there is no predefined limitation on the objective function, not same as the other primal-
dual based optimization algorithm ; and (4) the algorithm is asynchronous because each
local module only checks its neighbor communication information at special time.
IV. Energy Efficient DAPSO
Resource allocation is an important research topic in wireless networks. Bandwidth and
energy are two principle wireless sensor network resources, and the main challenge in de-
signing wireless sensor networks is to use network resources as efficiently as possible while
providing the Quality-of-Service required by the users. For wireless video sensor networks,
the ultimate goal is to utilize the limited resources of the network, such as transmission
bandwidth, energy supply and computational capability of each sensor node, in the highest
efficient way to reach the video quality as good as possible. We approach a distributed and
asynchronous algorithm, called EEDAPSO, for the energy-efficient resource allocation and
performance optimization over wireless video sensor networks based on the P-R-D module
of video encoding and transmission power consumption for the video communication. The
proposed EEDAPSO algorithm considers both encoding distortion and transmission distor-
tion for whole video quality, and solves the original optimization problem in a distributed
and asynchronous way.
128
IV. Evaluation of PSO Algorithm
The convergence analysis for PSO algorithm is given based on the random search tech-
niques, which are convergent algorithms for constrained nonlinear problems. PSO algorithm
has been proved that it is a local convergence algorithm, which means that after predefined
number of function iteration, all the solutions will converge to an optimum solution which
is no guaranteed a global optimum. However, our experimental results show that the large
number of random solutions of the PSO algorithm make this technique converge to its global
optimum in a good opportunity. Meanwhile, comparing with genetic algorithm (GA), and
Broydon-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton algorithm for their global search
capabilities based on a suite of difficult analytical optimization problems, we can see the
PSO algorithm has the better convergence probability to the global optimum.
7.2 Future Work
The following aspects of the work will be explored to make this research more complete:
(1). Protocol summary. We will have protocol design and performance evaluation using
discrete event simulation. (2). PSO algorithm improvement. During the final stage opti-
mization, the small step size causes the high cost in terms of function iteration, which is a
common trait among global search algorithm. How to improve this will be a challenge in
the future study.
129
Bibliography
[1] F. Kelly, A. Maulloo, and D. Tan, “Rate control for communication networks: Shadow
prices, proportional fairness and stability,” Journal of the Oper. Res. Soc., vol. 49,
no. 3, pp. 237–252, Mar. 1998.
[2] R. La and V. Anantharam, “Utility-based rate control in the internet for elastic traf-
fic,” IEEE/ACM Trans. on Networking, vol. 10, no. 3, pp. 272–286, Apr. 2002.
[3] S. Low and D. Lapsley, “Optimization flow control-i: Basic algorithm and conver-
gence,” IEEE/ACM Trans. Netw., vol. 7, no. 6, pp. 861–874, Dec. 1999.
[4] S. Low, “A duality model of tcp and queue management algorithms,” IEEE/ACM
Trans. Netw., vol. 24, no. 4, pp. 525–536, Aug. 2003.
[5] S. Kunniyur and R. Srikant, “End-to-end congestion control: utility functions, random
losses and ecn marks,” IEEE/ACM Trans. on Networking, vol. 10, no. 5, pp. 689–702,
Oct. 2003.
[6] Y. Qiu and P. Marbach, “Bandwidth allocation in ad hoc networks: a price-based
approach,” in INFOCOM 2003. 22nd Annual Joint Conference of the IEEE Computer
and Communications Societies, vol. 2, Apr. 2003, pp. 797–807.
[7] M. Luthi, S. Nadjm-Tehrani, and C. Curescu, “Comparative study of price-based re-
source allocation algorithms for ad hoc networks,” in 20th Intl. Parallel and Distributed
Processing Symposium, Apr. 2006.
130
[8] K. Kar, S. Sarkar, and L. Tassiulas, “A simple rate control algorithm for maximizing
total user utility,” in Proc. 23rd Joint Conf. of IEEE Computer and Communications
Societies (INFOCOMM), Anchorage, AK, USA, Apr. 2001, pp. 133–141.
[9] M. Rabbat and R. Nowak, “Distributed optimization in sensor networks,” in Proc. of
the 3rd Intl. symposium on Information processing in sensor networks, Berkeley, CA,
US, Apr. 2004, pp. 20–27.
[10] S. Son, M. Chiang, S. Kulkarni, and S. Schwartz, “Clustering in distrib-
uted incremental estimation in wireless sensor networks.” [Online]. Available:
www.princeton.edu/ chiangm/cluster.pdf
[11] R. Eberhart and J. Kennedy, “A new optimizer using particle swarm theory,” in Proc.
Sixth Intl. Symposium on Micromachine and Human Science, Nagoya, Japan, Oct.
1995, pp. 39–43.
[12] J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Porc. of IEEE Intl.
Conf. on Neural Networks, Perth, Australia, 1995, pp. 1942–1948.
[13] I. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, “A survey on sensor
networks,” IEEE Commun. Mag., pp. 102–114, Aug. 2002.
[14] Y. Hou, Y. Shi, and H. Sherali, “Rate allocation in wireless sensor networks with
network lifetime requirement,” in Proc. of the 5th ACM Intl. Symposium on Mobile
Ad Hoc Networking and Computing (Mobi-Hoc), Tokyo, Japan, May 2004, pp. 67–77.
[15] M. Rabbat and R. Nowak, “Decentralized source localization and tracking,” in Proc.
of IEEE Intl. Conf. on Acoustics, Speech, and Signal Processing (ICASSP), May 2004,
pp. 17–21.
[16] R. Eberhart, Y. Shi, and J. Kennedy, Swarm Intelligence. Morgan Kaufmann, 2001.
131
[17] D. Palomar and M. Chiang, “A tutorial on decomposition methods for network utility
maximization,” IEEE Journal of Sel. Areas in Commun., vol. 24, no. 8, pp. 1439–1451,
Aug. 2006.
[18] ——, “Alternative decompositions and distributed algorithms for network utility max-
imization,” IEEE Global Telecommunications Conference (GLOBECOM 2005), vol. 5,
Dec. 2005.
[19] Y. Choi, S. Kang, and S. Chong, “Efficient abr service engine for atm networks,” IEE
Proc. on Circuits, Devices and Systems, vol. 148, no. 6, pp. 323–327, Dec. 2001.
[20] S. Chong, S. Lee, and S. Kang, “A simple, scalable, and stable explicit rate allocation
algorithmfor max-min flow control with minimum rate guarantee,” IEEE/ACM Trans.
on Networking, vol. 9, no. 3, pp. 322–335, Jun. 2001.
[21] N. Ghani and J. W. Mark, “Enhanced distributed explicit rate allocation for abr
services in atm networks,” IEEE/ACM Trans. on Networking, vol. 8, no. 1, pp. 71–
86, Feb. 2000.
[22] T. Leighton and S. Rao, “Multicommodity max-flow min-cut theorems and their use in
designing approximation algorithms,” Journal of the ACM, vol. 46, no. 6, pp. 787–832,
Nov. 1999.
[23] R. Ahlswede, N. Cai, S. Li, and R. Yeung, “Network information flow,” IEEE Trans.
on Information Theory, vol. 46, no. 4, pp. 1204–1216, Jul. 2000.
[24] X. Lin, N. B. Shroff, and R. Srikant, “A tutorial on cross-layer optimization in wireless
networks,” IEEE Journal on Selected Areas in Commun., vol. 24, no. 8, pp. 1452–1463,
Aug. 2006.
[25] J. Lee, M. Chiang, and R. Calderbank, “Price-based distributed algorithm for optimal
rate-reliability tradeoff in network utility maximization,” IEEE Journal of Sel. Areas
in Commun., vol. 24, no. 5, pp. 962–976, May. 2006.
132
[26] C. Tan, D. Palomar, and M. Chiang, “Distributed optimization of coupled systems
with applications to network utility maximization,” in IEEE Proc. Int. conf. on
Acoust., Speech, Signal Process. (ICASSP), vol. 5, Toulouse, France, May 2006, pp.
981–984.
[27] D. Bertsekas and J. Tsitsiklis, Parallel and Distributed Computation: Numerical Meth-
ods. Englewood Cliffs, NJ: Prentice-Hall, 1989.
[28] D. Bertsekas, Nonlinear Programming, 2nd ed. Belmont, MA: Athena Scientific,
1999.
[29] D. Luenberger, Linear and Nonlinear Programming, 2nd ed. Reading, MA: Addison-
Wesley, 1989.
[30] I. Akyildiz and X. Wang, “A survey on wireless mesh networks,” IEEE Communica-
tions Magazine, vol. 43, no. 9, pp. S23–S30, Sep. 2005.
[31] B. Braunstein, T. Trimble, R. Mishra, B. Manoj, and R. Rao, “On the traffic behavior
of distributed wireless mesh networks,” in Proc. of 2006 Intl. Symposium on a World
of Wireless, Mobile and Multimedia Networks, Jun. 2006, pp. 26–29.
[32] Y. Xue, B. Li, and K. Nahrstedt, “Optimal resource allocation in wireless ad hoc
networks: a price-based approach,” IEEE Trans. on Mobile Computing, vol. 5, no. 4,
pp. 347–364, Apr. 2006.
[33] T. Nguyen and A. Zakhor, “Multiple sender distributed video streaming,” IEEE
Trans. on Multimedia, vol. 6, no. 2, pp. 315–326, Apr. 2004.
[34] M. Chen and A. Zakhor, “Rate control for streaming video over wireless,” in Twenty-
third AnnualJoint Conf. of the IEEE Computer and Communications Societies, IN-
FOCOM 2004, vol. 2, Hongkong, China, Mar. 2004, pp. 1181–1190.
133
[35] X. Zhu, E. Setton, and B. Girod, “Rate allocation for multi-camera surveillance over
an ad hoc wireless network,” in Proc. Picture Coding Symposium, (PCS-04), San
Francisco, CA, US, Dec. 2004.
[36] X. Zhu and B. Girod, “Distributed rate allocation for multi-stream video transmission
over ad hoc networks,” in IEEE Intl. Conf. on Image Processing, Genoa, Italy, Sep.
2005, pp. 157–160.
[37] X. Zhu, E. Setton, and B. Girod, “Congestion-distortion optimized video transmission
over ad hoc networks,” EURASIP Journal of Signal Processing: Image Communica-
tions, vol. 20, pp. 773–783, Sep. 2005.
[38] J. S. X. Zhu and B. Girod, “Joint routing and rate allocation for multiple video streams
in ad-hoc wireless networks,” Journal of Zhejiang University, Science A, vol. 7, no. 5,
pp. 727–736, May 2006.
[39] X. Zhu and B. Girod, “Distributed rate allocation for multi-stream video transmission
over ad hoc networks,” in IEEE Intl. Conf. on Image Processing (ICIP), Sep. 2005,
pp. 57–60.
[40] S. Boyed and L. Vandenberghe, Convex Optimization. Cambridge, UK: Cambridge
University Press, 2004.
[41] Z. Luo and W. Yu, “An introduction to convex optimization for communications and
signal processing,” IEEE Journal on Selected Areas in Commun., vol. 24, no. 8, pp.
1426–1438, Aug. 2006.
[42] L. S. Lasdon, Optimization Theory for Large Systems. Courier Dover, 2002.
[43] N. Z. Shor, Minimization methods for non-differentiable functions. Berlin, Germany:
Springer-Verlag, 1985.
[44] Y. Shi and R. Eberhart, “Empirical study of particle swarm optimization,” in Proc.
of the 1999 Congress on Evolutionary Computation, Washington, DC, US, Jul. 1999.
134
[45] E. O. Wilson, Sociobiology: The new synthesis. Cambridge, MA: Belknap Press,
1975.
[46] J. Kennedy, “The particle swarm: social adaptation of knowledge,” in Proc. of IEEE
Intl. Conf. on Evolutionary Computation, Indianapolis, IN, US, Apr. 1997, pp. 303–
308.
[47] A. Chatterjee, K. Pulasinghe, K. Watanabe, and K. Izumi, “A particle-swarm-
optimized fuzzy-neural network for voice-controlled robot systems,” IEEE Trans. on
Industrial Electronics, vol. 52, no. 6, pp. 1478–1489, Dec. 2005.
[48] L. dos Santos Coelho and R. A. Krohling, “Nonlinear system identification based
on b-spline neural network and modified particle swarm optimization,” in Proc. of
2006 IEEE Intl. Joint Conf. on Neural Networks, Vancouver, Canada, Jul. 2006, pp.
3748–3753.
[49] M. Carvalho and T. B. Ludermir, “Particle swarm optimization of feed-forward neural
networks with weight decay,” in Proc. of 6th Intl. Conf. on Hybrid Intelligent Systems,
Dec. 2006, pp. 5–8.
[50] F. Huang, R. Li, H. Liu, and R. Li, “A modified particle swarm algorithm combined
with fuzzy neural network with application to financial risk early warning,” in Proc. of
2006 IEEE Asia-Pacific Conf. on Services Computing (APSCC), Guangzhou, China,
Dec. 2006, pp. 168–173.
[51] S. Genovesi, R. Mittra, A. Monorchio, and G. Manara, “Particle swarm optimization
for the design of frequency selective surfaces,” IEEE Antennas and Wireless Propa-
gation Letters, vol. 5, pp. 277–279, Dec. 2006.
[52] S. L. Ho, S. Yang, G. Ni, and H. C. Wong, “A particle swarm optimization method with
enhanced global search ability for design optimizations of electromagnetic devices,”
IEEE Trans. on Magnetics, vol. 42, no. 4, pp. 1107–1110, Apr. 2006.
135
[53] S. M. Mikki and A. A. Kishk, “Quantum particle swarm optimization for electromag-
netics,” IEEE Trans. on Antennas and Propagation, vol. 54, no. 10, pp. 2764–2775,
Oct. 2006.
[54] X. Shi, K. S. Yeo, J.-G. Ma, M. A. Do, and E. Li, “Scalable model of on-wafer
interconnects for high-speed cmos ics,” IEEE Trans. on Advanced Packaging, vol. 29,
no. 4, pp. 770–776, Nov. 2006.
[55] J. Park, K. Choi, and D. J. Allstot, “Parasitic-aware rf circuit design and optimiza-
tion,” IEEE Trans. on Circuits and Systems-I, vol. 51, no. 10, pp. 1953–1966, Oct.
2004.
[56] C. Mendis, S. M. Guru, S. Halgamuge, and S. Fernando, “Optimized sink node path
using particle swarm optimization,” in Proc. of 20th Intl. Conf. on Advanced Infor-
mation Networking and Applications, Apr. 2006, pp. 5–10.
[57] M. M. Noel, P. P. Joshi, and T. C. Jannett, “Improved maximum likelihood estimation
of target position in wireless sensor networks using particle swarm optimization,” in
Proc. of 3rd Intl. Conf. on Information Technology: New Generations, Apr. 2006, pp.
274–279.
[58] S. M. Guru, S. K. Halgamuge, and S. Fernando, “Particle swarm optimisers for cluster
formation in wireless sensor networks,” in Proc. of Intl. Conf. on Intelligent Sensors,
Sensor Networks and Information Processing Conf., Dec. 2005, pp. 319–324.
[59] P. Yuan, C. Ji, Y. Zhang, and Y. Wang, “Optimal multicast routing in wireless ad
hoc sensor networks,” in Proc. of Intl. Conf. on Networking, Sensing, and Control,
Mar. 2004, pp. 367–371.
[60] J. Tillett, R. Rao, and F. Sahin, “Cluster-head identification in ad hoc sensor net-
works using particle swarm optimization,” in Proc. of Intl. Conf. on Personal Wireless
Communications, Dec. 2002, pp. 201–205.
136
[61] Y. Yuan, Z. He, and M. Chen, “Virtual mimo-based cross-layer design for wireless
sensor networks,” IEEE Trans. on Vehicular Technology, vol. 55, no. 3, pp. 856–864,
May 2006.
[62] B. Wang and Z. He, “Performance optimization of wireless video sensor networks using
swarm optimization with convex mapping,” in IEEE Intl. Conf. on Image Processing
(ICIP) 2006, Atlanta, GA, Oct. 2006, pp. 1293–1296.
[63] ——, “Distributed optimization over wireless sensor networks using swarm intelli-
gence,” in IEEE Intl. Symposium on Circuits and Systems (ISCAS) 2007, New Or-
leans, LA, May 2007.
[64] Z. He, Y. Liang, L. Chen, I. Ahmad, and D. Wu, “Power-rate-distortion analysis for
wireless video communication under energy constraint,” IEEE Trans. on Circuits and
System for Video Technology, vol. 15, no. 5, pp. 645–658, May 2005.
[65] J. Chen, K. Yao, and R. Hudson, “Source localization and beamforming,” IEEE Sig.
Proc. Magazine, vol. 19, no. 2, pp. 30–39, Mar. 2002.
[66] D. Blatt and A. O. Hero, “Energy-based sensor network source localization via pro-
jection onto convex sets,” IEEE Trans. on Signal Processing, vol. 54, no. 9, pp. 3614–
3619, Sep. 2006.
[67] X. Sheng and Y. H. Hu, “Energy based acoustic source localization,” in Information
Processing in Sensor Networks (IPSN), Palo Alto, CA, USA, Apr. 2003.
[68] B. Wang and Z. He, “Distributed asynchronous rate allocation and performance op-
timization for video communication over mesh networks,” To be Submitted to IEEE
Transactions on Circuits and Systems for Video Technology (CSVT), 2007.
[69] I. F. Akyildiz and X. Wang, “A survey on wireless mesh networks,” IEEE Communi-
cation Magazine, vol. 43, no. 9, pp. S23–S30, Sep. 2005.
137
[70] Y. Andreopoulos, N. Mastronarde, and M. van der Schaar, “Cross-layer optimized
video streaming over wireless multihop mesh networks,” IEEE Journal on Selected
Areas in Comm., vol. 24, no. 11, pp. 2104–2115, Nov. 2006.
[71] K. Ramchandran and M. Vetterli, “Rate-distortion optimal fast thresholding with
complete jpeg/mpeg decoder compatibility,” IEEE Trans. Image Processing, vol. 3,
pp. 700–704, Sep. 1994.
[72] Z. He and S. K. Mitra, “A unified rate-distortion analysis framework for transform
coding,” IEEE Transactions on Circuits and System for Video Technology, vol. 11,
pp. 1221–1236, Dec. 2001.
[73] M. van Der Schaar and N. S. Shankar, “Cross-layer wireless multimedia transmission:
challenges, principles, and new paradigms,” IEEE Wireless Communications, vol. 12,
no. 4, pp. 50–58, Aug. 2005.
[74] J. Xiao, A. Ribeiro, Z. Luo, and G. Giannakis, “Distributed compression-estimation
using wireless sensor networks,” IEEE Signal Proc. Mag., vol. 23, pp. 27–41, Jul. 2006.
[75] X. Zhu and B. Girod, “Distributed rate allocation for video streaming over wireless
networks with heterogeneous link speeds,” in Proc. International Symposium on Mul-
timedia over Wireless, Honolulu, Hawaii, Aug. 2007.
[76] Z. He and D. Wu, “Resource allocation and cross-layer optimization for real-time
wireless video communication over portable devices,” IEEE Transactions on Circuits
and System for Video Technology, vol. 16, no. 5, pp. 590–599, May. 2006.
[77] T. S. Rappaport, Wireless Communications: Principles and Practices. New Jersey:
Prentice Hall, 1996.
[78] L.-J. Lin and A. Ortega, “Bit-rate control using piecewise approximated rate-
distortion characteristics,” IEEE Transactions on Circuits and Systems for Video
Technology, vol. 38, pp. 82–93, Jan. 1990.
138
[79] C. M. Kozierok, The TCP/IP Guide: A Comprehensive, Illustrated Internet Protocols
Reference. No Starch Press, 2005.
[80] D. A. Dunn, W. D. Grover, and M. H. MacGregor, “Comparison of k-shortest paths
and maximum flow routing for network facility restoration,” IEEE Journal on Selected
Areas in Communications, vol. 12, no. 1, pp. 88–99, Jan. 1994.
[81] B. Wang and Z. He, “Distributed and asynchronous optimization over networks us-
ing swarm intelligence,” in Accepted by Fourth Intl. Symposium on Neural Networks
(ISNN) 2007, Nanjing, China, 2007.
[82] ——, “Nonconvex optimization problems over video mesh network using distrib-
uted and asynchronous particle swarm optimization,” in IEEE Intl. Conf. on Image
Processing (ICIP) 2007, San Antonio, TX., Sep. 2007.
[83] F. Meshkati, H. V. Poor, and S. C. Schwartz, “Energy-efficient resource allocation in
wireless networks,” IEEE Signal Processing Magzine, pp. 58–68, May 2007.
[84] V. Raghunathan, C. Schurgers, S. Parks, and M. B. Srivastava, “Energy-aware wireless
microsensor networks,” IEEE Signal Processing Magzine, vol. 19, no. 2, pp. 40–50,
Mar. 2002.
[85] J. Rulnick and N. Bambos, “Mobile power management for wireless communication
networks,” Wireless Networks, vol. 3, pp. 3–14, 1997.
[86] S. Zhao, Z. Xiong, and X. Wang, “Optimal resource allocation for wireless video
over cdma networks,” IEEE Trans. on Mobile Computing, vol. 4, no. 1, pp. 56–67,
Jan./Feb. 2005.
[87] P. Pahalawatta, R. Berry, T. Pappas, and A. Katsaggelos, “Content-aware resource
allocation and packet scheduling for video transmission over wireless networks,” IEEE
Journal on Selected Areas in Communications, vol. 25, no. 4, pp. 749–759, May 2007.
139
[88] R. Holman, J. Stanley, and T. Ozkan-Haller, “Applying video sensor networks to
nearshore environment monitoring,” IEEE Pervasive Computing, vol. 2, no. 4, pp.
14–21, 2003.
[89] A. Basharat, N. Catbas, and M. Shah, “A framework for intelligent sensor network
with video camera for structural health monitoring of bridges,” in IEEE Intl. Conf.
on Pervasive Computing and Communications Workshops 2005, Mar. 2005, pp. 8–12.
[90] C. E. Perkins, E. M. Belding-Royer, and S. R. Das, “Ad hoc on-demand distance
vector routing,” in IEEE Workshops on Mobile Computing Systems and Applications
1999, Feb. 1999, pp. 90–100.
[91] K. Obraczka, R. Manduchi, and J. J. Garcia-Luna-Aveces, “Managing the information
flow in visual sensor networks,” in Proc. of 5th Intl. Symposium on Wireless Personal
Multimedia Communications, Oct. 2002, pp. 1177–1181.
[92] J. Chang and L. Tassiulas, “Energy conserving routing in wireless ad hoc networks,”
in Proc. INFOCOMM 2000, Mar. 2000.
[93] J. Pan, Y. T. Hou, L. Cai, Y. Shi, and S. X. Shen, “Topology control for wireless
sensor networks,” in Proc. of 9th Intl. Conf. on Mobile Computing and Networking,
San Jose, CA, USA, 2003, pp. 286–299.
[94] ——, “Locating base-stations for video sensor networks,” in IEEE Vehicular Technol-
ogy Conference 2003, Oct. 2003, pp. 3000–3004.
[95] F. Zhai, Y. Eisenberg, T. N. Pappas, R. Berry, and A. K. Katsaggelos, “Joint source-
channel coding and power adaptation for energy efficient wireless video communica-
tions,” Signal Processing: Image Communication 20, pp. 371–387, 2005.
[96] Y. Yu, V. K. Prasanna, and B. Krishnamachari, “Energy minimization for real-time
data gathering in wireless sensor networks,” IEEE Trans. on Wireless Communica-
tions, vol. 5, no. 11, pp. 3087–3096, Nov. 2006.
140
[97] W. H. Kuo and W. Liao, “Utility-based resource allocation in wireless networks,”
IEEE Trans. on Wireless Communications, vol. 6, no. 10, pp. 3600–3606, Oct. 2007.
[98] A. Farbod and T. D. Todd, “Resource allocation and outage control for solar-powered
wlan mesh networks,” IEEE Trans. on Mobile Computing, vol. 6, no. 8, pp. 960–970,
Aug. 2007.
[99] Z. He and D. Wu, “Resource allocation and performance analysis of wireless video
sensors,” IEEE Trans. on Circuits and System for Video Technology, vol. 16, no. 5,
pp. 590–599, May. 2006.
[100] P. Agrawal, J.-C. Chen, S. Kishore, P. Ramanathan, and K. Sivalingam, “Battery
power sensitive video processing in wireless networks,” in Proc. of IEEE PIMRC’98,
Boston, Sep. 1998.
[101] X. Lu, Y. Wang, and E. Erkip, “Power efficient h.263 video transmission over wireless
channels,” in Proc. of 2002 Intl. Conf. on Image Processing, Rochester, NY, Sep.
2002.
[102] I. Inc., “Intel xscale technology.” [Online]. Available:
www.intel.com/design/intelxscale
[103] Z. He, Y. Liang, L. Chen, I. Ahmad, and D. Wu, “Power-rate-distortion analysis for
wireless video communication under energy constraint,” IEEE Trans. on Circuits and
System for Video Technology, vol. 15, no. 5, pp. 645–659, May. 2005.
[104] R. Zhang, S. L. Regunathan, and K. Rose, “Video coding with optimal inter/intra-
mode switching for packet loss resilience,” IEEE Journal on Selected Areas in Com-
mun., vol. 18, no. 6, pp. 966–976, Jun. 2000.
[105] K. Stuhlmuller, N. Farber, M. Link, and B. Girod, “Analysis of video transmissiion
over lossy channel,” IEEE Journal on Selected Areas in Commun., vol. 18, no. 6, pp.
1012–1032, Jun. 2000.
141
[106] Z. He, J. Cai, and C. W. Chen, “Joint source channel rate-distortion analysis for
adaptive mode selection and rate control in wireless video coding,” IEEE Trans. on
Circuits and System for Video Technology, vol. 12, no. 6, pp. 511–523, Jun. 2002.
[107] B. Wang and Z. He, “Resource allocation and performance optimization over wire-
less video sensor networks,” in Proc. Intl. Symposium on Multimedia over Wireless
(ISMW) 2007, Honolulu, Hawaii, Aug. 2007.
[108] F. J. Solis and R. Wets, “Minimization by random search techniques,” Mathematics
of Operations Research 6, pp. 19–30, 1981.
[109] J. C. Taylor, An introduction to measure and probability. New York, NY: Springer,
1997.
[110] F. van den Bergh, An Analysis of Particle Swarm Optimizers. University of Pretoria
etd, Ph.D thesis, 2006.
[111] A. Slomson, An Introduction to Combinatorics. Chappman and Hall Mathematics
Series, 1991.
[112] G. F. Luger, Artificial Intelligence, Structures and Strategies for Complex Problem
Solving. Harlow, England: Addison-Wesley: 4th Edition, 2002.
[113] J. H. Holland, Adaptation in natural and artificial system. Ann Arbor: University
of Michigan Press, 1975.
[114] D. Whitley, A genetic algorithm tutorial. Computer Science Department, Colorado
State University, 1993.
[115] M. Avriel, Nonlinear Programming: Analysis and Methods. Dover Publishing, 2003.
[116] A. J. Soest and L. J. R. Casius, “The merits of a parallel genetic algorithm in solving
hard optimization problems,” Journal of Biomech. Eng., pp. 141–146, 2003.
142
[117] J. Schaffer, R. A. Caruana, L. J. Eshelman, and R. Das, “A study of control parameters
affecting online performance of genetic algorithms for function optimizing,” in Proc.
3rd Int. Conf. Genetic Alg. 1989, San Mateo, CA, 1989, pp. 51–60.
[118] A. Corana, M. Marchesi, C. Martini, and S. Ridella, “Minimizing multimodal func-
tions of continuous variables with the.”
[119] J. F. Schutte, B. Koh, J. A. Reinbolt, R. T. Haftka, A. D. George, and B. J. Fregly,
“Evaluation of a particle swarm algorithm for biomechanical optimization,” Journal
of Biomech. Eng., vol. 127, pp. 465–474, 2005.
143
VITA
Bo Wang was born in Urumchi, China in 1978. He received his bachelor degrees in both
Electrical Engineering and Business of Administration at Xi’an Jiaotong University, Xi’an,
China in 2000. And he received his master degree in Electrical Engineering at Southeast
University, Nanjing, China in 2003. At meantime, he was a Research Engineer in National
Mobile Communications Research Laboratory (NCRL) and Southeast University Commu-
nication Corporation Ltd. (SeuComm) for China Third Generation Mobile Communication
(C3G) system design. After that, he joined Zhong Xing Telecommunication Equipment
Company (ZTE) Shanghai Research Center as a Software Engineer in 2003. He will get his
Ph.d degree in Electrical and Computer Engineering at University of Missouri-Columbia
in 2007. He did his internship in Motorola (San Diego, CA) as a System Engineer in the
summer of 2006. Now he is a Senior Electrical Engineer at Motorola (San Diego, CA).
His research interests include multimedia communications, wireless communications, digital
design, video streaming, and signal processing.
144
